Abstract
Experimental psychopathology researchers are devoted to understanding the causes and mechanisms of psychological disorders, with the central goal of developing and improving effective interventions. Indeed, cognitive behavioural therapy (CBT) arose from experiments identifying key mechanisms that clinicians can target to reduce their clients’ symptoms. Despite the well-documented efficacy of CBT for diverse problems, many people continue to live with psychopathology, even after CBT. In this paper, we reflect on ways in which experimental psychopathology has advanced CBT (i.e. Where have we been?), areas that are fruitful for continued progress (i.e. Where are we now?), and crucial areas for improvement (i.e. Where are we going?). In particular, we contend that improving the science of psychopathology via open and transparent practices, prioritising diverse samples, testing the effects of specific CBT techniques, and focusing on problems that our clients identify will enable experimental psychopathology research to serve as the foundation for evidence-based practice.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
In 2019, Aaron T. Beck published a retrospective on his 60 years in the field of cognitive behavioural therapy (CBT), with a specific focus on the scientific method that guided his work developing, implementing, and disseminating an efficacious treatment for diverse psychological disorders. He highlighted what is now CBT lore—that his initial interest in validating psychoanalytic treatment for depression via rigorous scientific methods led him ultimately to leave psychoanalysis behind and revolutionise psychotherapy by focusing on the key role of automatic thoughts (Beck 2019). He never used the term experimental psychopathology; yet, Beck described a back-and-forth process between experiments in the lab and interventions in the clinic, advancing both cognitive theory and therapy. In so doing, he laid the groundwork for research that focuses not just on evidence-based practice, but also on practice-based research.
Researchers define experimental psychopathology (EPP) in a variety of ways (e.g. Waters et al. 2017; Zvolensky et al. 2001), though all agree that the overarching goal is to understand which factors contribute to the aetiology and maintenance of psychological disorders and how they do so. Forsyth and Zvolensky et al. (2001; see also Zvolensky et al. 2013) describe four research designs that fall under the EPP umbrella, each varying on participant type and experimental manipulation of independent variables. Boddez et al. (2017) lay out EPP core tenets: (1) psychopathological behaviour is caused by deviations in cognitive processes that operate on a continuum “shared by healthy people and patients alike” (p. 110); (2) proposed mechanisms (i.e. manipulated constructs) are well-known psychological phenomena rather than novel, previously unidentified constructs; and (3) the central goal is to contribute to improving evidence-based treatments. Despite the long history of clear links between EPP and CBT (e.g. van den Hout et al. 2017), the degree to which researchers consider clinical implications as primary varies considerably (e.g. Forsyth and Zvolensky 2001; Ouimet et al. 2019).
In July 2019, over 4000 researchers from more than 80 countries came together in Berlin to share and discuss CBT research and treatment at the World Congress of Cognitive Behavioural Therapy (WCBCT) – “CBT at the Crossroads”. The integral role of EPP in CBT research is apparent in the conference program (https://wcbct2019.org/Downloads/Congress-Programme.pdf). From basic science/experimental psychopathology poster sessions to symposia related to implementing behavioural experiments in treatment (Kube and Rief 2019), researchers and clinicians learned about applying evidence from new EPP research to their CBT practice.
Although CBT clinicians should feel comfortable pointing to the evidence base to support much of what they do with their clients, the pathway from the cognitive behavioural model to cognitive behavioural treatment is not often clear (Clark 2004). Waters et al. (2017) proposed a conceptual model of research programming to encourage better integration of EPP and clinical practice, highlighting the multi-directional links between theory, experiments, and practice. For example, high relapse rates observed in randomised controlled trials (RCTs) should inform EPP researchers’ efforts to identify additional potential mechanisms of change, which should then be tested via RCTs. The model is iterative: as researchers, we must move information from science to practice, and then use practice to inform our science (Waters et al. 2017).
There is a general structure that EPP/CBT research tends to follow:
-
(1)
Identify an existing phenomenon that appears exaggerated among people with psychopathology;
-
(2)
Induce that exaggeration among people without psychopathology to see whether “symptoms” appear;
-
(3)
Test whether changing that exaggeration (i.e. via an intervention) reduces psychopathology;
-
(4)
Modify and repeat.
Although this structure has led to improved treatment outcomes for people with psychopathology (e.g. Clark 2004), it is not uncommon for researchers to skip the second step—testing the role of the proposed mechanism, leaving open the question of whether that mechanism should be prioritised in an intervention. Moreover, psychological research is changing rapidly, with a focus on improving research practices and reproducibility via Open Science methods, increased transparency, more precise measurement, larger, a priori justified sample sizes, and inclusion of diverse samples to name a few. EPP has to change with it, particularly given its important role in developing and refining evidence-based treatments.
In this paper, we take a critical eye to EPP research and its role in CBT practice, focusing on where we have been, where we are now, and where we are going. This is by no means an exhaustive review. Rather, we aim to highlight research that has pushed therapeutic practices forward, current research that has the potential to do so, and to put forth a call for continued improvement of our research practices to ensure we help people with various forms of psychopathology. Moreover, we put a particular emphasis on the diverse crossroads that EPP and CBT researchers encounter, using the WCBCT convention as our inspiration.
Although there are numerous examples of exciting ways in which EPP has advanced CBT, we were able to include only a few. The choice was not always easy, and was certainly influenced by our own areas of expertise and familiarity. Rather than provide a complete account of how EPP shaped CBT for a given disorder, we highlighted themes we believed to be particularly important (e.g. Identifying mechanisms of change), using specific processes and disorders as exemplars.
Where Have We Been?
EPP has a long history dating back to Wundt, who advocated for experimental laboratories as the cornerstone of psychological science, James, who was perhaps the first to use experiments to highlight how psychopathology was a variant of normative processes, and Pavlov, whose experimental work on systematic desensitisation was the precursor to exposure therapy. A detailed review is beyond our paper’s scope; we refer interested readers to Olatunji et al. (2007) and Forsyth and Zvolensky (2001). Instead, we reflect on a few specific programs of research that have put EPP at the centre of treatment development. We provide examples of how researchers used innovative experimental manipulations to better test and measure cognitive and behavioural causes of symptoms, changing the way we understand and treat some disorders.
Developing Novel Interventions: Memory Confidence in OCD
Following a long history of being considered “untreatable”, obsessive-compulsive disorder (OCD) research leapt forward following strong, clinically focused cognitive models by Rachman (1997, 1998) and Salkovskis (1985). By focusing on people’s beliefs—that their thoughts meant they were mad, bad, or dangerous, and that they were responsible for preventing their feared events from occurring, respectively—Rachman and Salkovskis laid the foundation for experimental work focused on delineating the effects of specific beliefs on OCD symptoms. These experiments were critical to developing CBT for OCD, which has one of the highest efficacy rates among anxiety-related disorders (Carpenter et al. 2018). By exploring the memory confidence research in depth, we aim to highlight how well-designed experiments pave the road for specific CBT components.
Since Rachman’s (1997, 1998) and Salkovskis’ (1985) early models, researchers have induced OCD symptoms experimentally to better understand their causal and maintaining mechanisms (De Putter et al. 2017). For example, Rachman et al. (1996) demonstrated that asking healthy people high in thought-action fusion to write out a sentence wishing their loved one would die increased their anxiety and urge to neutralise. Rassin et al. (1999) informed healthy participants that every time they thought the word “apple” (assessed ostensibly via EEG), a participant in the next room would receive a shock. This “thought-action fusion” increased participants’ intrusions of the word “apple”, discomfort, efforts to suppress the word “apple”, self-directed anger, and even their evaluation of the valence of the word “apple”. Using a novel (deceptive) pill-sorting task, Bouchard et al. (1999) demonstrated that perceived high responsibility led to more delayed decision-making, checking, and accuracy, all of which likely reinforce symptoms and mechanisms such as perfectionism. These experiments, in tandem with others highlighting the important roles of mental contamination (e.g. Fairbrother et al. 2005), disgust propensity and sensitivity (e.g. Deacon and Olatunji 2007; Inozu et al. 2017), and doubt and inferential confusion (e.g. Wong and Grisham 2017) as key maintaining factors—and thus treatment targets—for OCD symptoms led to important refinements of CBT for OCD.
One debate in the OCD literature centred on suggestions that people engaged in compulsive checking because of memory deficits. Researchers hypothesised that they could not remember well enough whether they had checked something properly, so they checked again (e.g. Rubenstein et al. 1993; Sher et al. 1989). McNally and Kohlbeck (1993), however, offered preliminary evidence that repeated checking may arise more from poor memory confidence than from poor memory performance. In an early meta-analysis, Woods et al. (2002) concluded that although people with OCD exhibited impaired memory performance, those who checked compulsively had lower memory confidence than those who did not. It was not until van den Hout and Kindt (2003) tested the effects of repeated checking on memory confidence via experimental manipulation, however, that a clinically plausible theory began to take hold.
In an innovative experiment, van den Hout and Kindt (2003) asked healthy participants to repeatedly check either a virtual stove or a virtual light bulb, prior to performing a final check of the virtual stove. Across three experiments, participants who repeatedly checked the stove remembered the final check just as accurately as those who repeatedly checked the light bulb; however, they reported significantly lower vividness, detail, and confidence in their memory. The authors provided strong evidence that OCD symptoms and behaviours likely cause decreased memory confidence. Alcolado and Radomsky (2011) further demonstrated that experimentally reducing people’s memory confidence caused them to check even more. Researchers have replicated these findings across non-clinical and clinical samples (e.g. Boschen and Vuksanovic 2007), leading to a novel CBT intervention specifically targeting memory confidence (Alcolado and Radomsky 2016). Moreover, people undergoing cognitive therapy for compulsive checking reported increased memory confidence immediately after psychoeducation and behavioural experiments targeting that construct (Radomsky et al. 2020b). Crucially, increases in memory confidence predicted decreased reported checking time.
Taken together, people who check repeatedly do so not because they cannot remember whether they checked properly, but because they are not confident that their memory of the check is accurate. It is difficult to overstate the importance of this conclusion for CBT practice. A focus on memory impairment could lead to ineffective memory training (and potentially client beliefs that they are “untreatable”), whereas a focus on memory confidence has already led to efficacious cognitive interventions (and thus client symptom improvement) (Alcolado and Radomsky 2016; Radomsky et al. 2020a, b). Indeed, targeting memory confidence via behavioural experiments has been incorporated successfully into CBT treatment recommendations for compulsive checking (Radomsky et al. 2010; Shafran et al. 2013).
Identifying Mechanisms of Change: EMDR for PTSD
Multiple CBT interventions for posttraumatic stress disorder (PSTD) are effective, including prolonged exposure therapy, cognitive processing therapy, and trauma-focused CBT (see Watkins et al. 2018, for a review). We focus here, however, on eye-movement desensitisation and reprocessing to highlight EPP’s vital role not only in CBT development, but also in better understanding the mechanisms of efficacious treatments and improving their implementation. Shapiro (1989, 1995) developed eye-movement desensitisation and reprocessing (EMDR) as a novel treatment for posttraumatic stress disorder (PTSD), with little theoretical or empirical foundation. Shapiro herself stated “The basis for the effectiveness is, however, unclear in that the technique was not derived from a theoretical position and there is, at this time, insufficient evidence to justify conclusions.” (Shapiro 1989, p. 216). What ensued was a decades-long back-and-forth between researchers and clinicians declaring EMDR essentially a miracle cure (e.g. Marcus et al. 1997) and those insisting EMDR was either ineffective (e.g. McNally 1999) or effective exclusively because it included elements of evidence-based psychotherapy, such as exposure and empathy (e.g. Cusack and Spates 1999).
Despite the debatable mechanism of change, early reviews and meta-analyses suggested that EMDR was an efficacious treatment, equally so to cognitive behavioural and exposure-based therapies (e.g. Davidson and Parker 2001; Van Etten and Taylor 1998), though serious questions remained about the plausibility and clinical utility of the eye movements. The argument that the components above and beyond exposure therapy were likely unnecessary (e.g. Cusack and Spates 1999; Davidson and Parker 2001) was paving the way for ignoring or omitting the potential additive effect of eye movements altogether, despite some research suggesting they may have important impacts on memory and emotionality (e.g. Andrade et al. 1997).
Some EPP researchers, however, were not content to leave unanswered the question of whether and/or why eye movements might be important. van den Hout et al. (2001) extended hypotheses about eye movements taxing working memory during recollection (e.g. Andrade et al. 1997; Merckelbach et al. 1994), by showing experimentally that healthy participants who recalled negative and positive memories reported reduced vividness and negative valence of the negative memories only after engaging in eye movements. They suggested that eye movements were an active ingredient in EMDR, contrary to CBT assumptions at the time. Crucially, there was finally a mechanism to which researchers and clinicians could point: eye movements during recall tax working memory, which reduces memory vividness and emotionality, resulting in re-encoding a less distressing memory (van den Hout and Engelhard 2012).
In line with an alternate theory—that eye movements were effective because they increase communication between the two cerebral hemispheres (e.g. Christman et al. 2003)—many clinicians began substituting auditory tones that alternate from the left to right headphones for eye movements, despite a lack of evidence suggesting that this “binaural stimulation” was equivalent to eye movements or other working memory taxing behaviours (van den Hout et al. 2011). In four experiments, van den Hout et al. (2011) demonstrated that eye movements impair working memory and reduce negative memory vividness and emotionality significantly more than do binaural tones for healthy participants in the lab. Moreover, eye movements resulted in significantly larger reductions in memory emotionality and vividness than did tones among people with PTSD undergoing EMDR (van den Hout et al. 2012). These findings clarified that working memory taxation was the crucial mechanism of action over and above exposure to the memory in EMDR and was thus critical to ensuring its proper implementation.
Since then, experimental (e.g. Kearns and Engelhard 2015) and clinical research (e.g. Hurley 2018) related to eye movements and EMDR has increased substantially, including several symposium presentations at WCBCT (e.g., Bendall 2019; Dominguez 2019; Lee 2019). In a recent meta-analysis of mechanisms of change, Landin-Romero et al. (2018) supported the working memory taxation and psychophysiological change hypotheses, and clinical practice guidelines recommend EMDR as an effective treatment for PTSD (NICE 2018). As such, the innovation at the core of EPP allowed researchers to determine a potential mechanism of change to explain a previously implausible treatment effect, resulting hopefully, in more people receiving adequate care for their PTSD symptoms.
Perhaps even more important, EPP should not end where clinical research begins. There needs to be synergy between EPP and intervention science (Waters et al. 2017). Continued experimental investigations into eye movements resulted in findings that eye movements during initial recall may lead to increased false memories (Houben et al. 2018), though van Schie and Leer (2019) were unable to replicate the finding with a larger sample. van Veen et al. (2019) recently suggested that eye movements may actually function as a safety behaviour during exposure to traumatic memories by reducing distress and vividness in the moment, but ultimately resulting in rebound distress at follow-up. In other words, there is more work to be done, and EPP offers a context in which to do that work well.
Where Are We Now?
CBT is no longer (and has not been for quite a while!) the underdog that needs to prove itself. There are numerous meta-analyses, systematic reviews, Cochrane Reviews, and clinical guidelines attesting to its efficacy and effectiveness for multiple mental health problems (e.g. Hofmann et al. 2012). Nonetheless, many people continue to live with psychopathology, even after receiving high-quality CBT (e.g. Ali et al. 2017; Litt et al. 2018). Accessing evidence-based treatment remains elusive for many, whether due to living in remote locations, lack of public funding, or lack of trained clinicians (Gunter and Whittal 2010). The solutions to many of these questions rest in intervention science. For example, there is now strong evidence that CBT interventions delivered online are generally as efficacious as in-person CBT (e.g. Andersson et al. 2014). In some jurisdictions, publicly funded CBT is a government policy (e.g. Improving Access to Psychological Therapies, UK; The National Collaborating Centre for Mental Health 2020). Manualised training programs support paraprofessionals in administering CBT with good effectiveness rates (e.g. Bass et al. 2013). Thus, it may seem as if the role of EPP in CBT is dwindling (or should dwindle). Not so.
There remain several mental health issues that have been historically understudied, or for which maintaining mechanisms are poorly understood, hampering their treatment efficacy. Even for well-understood disorders, there is significant room for treatment improvement, likely because researchers have yet to identify key mechanisms and/or the relations among those mechanisms (i.e. moderators and mediators). EPP researchers continue to innovate, test, and manipulate (variables, not people) with the goal of improving treatment for people living with psychopathology. Indeed, it is exciting to survey the numerous EPP research programs currently ongoing that have a high potential to contribute to improvements in CBT. Below, we provide just a few examples, before moving on to our thoughts on how to advance the role of EPP for CBT.
Identifying Treatment Targets for Understudied Disorders: Body Dysmorphic Disorder
In a recent meta-analysis, Harrison et al. (2016) demonstrated that CBT was more efficacious for body dysmorphic disorder (BDD) than wait-list control and credible placebo treatments, with treatment gains maintained at 2–4 months. However, outside of controlled research trials, only a minority of people with BDD receive CBT, and remission rates are low (Phillips et al. 2013). Although CBT models of BDD focus on mechanisms such as information processing biases, overvaluation of appearance, and avoidance/safety behaviours (e.g., Wilhelm et al. 2010), evidence for the causal and maintaining roles of those factors is lacking.
More recently, EPP researchers have begun experimentally manipulating these proposed mechanisms. For example, Veale et al. (2016) expanded on previous research demonstrating that people experience more body dissatisfaction after staring into a mirror (e.g. Windheim et al. 2011). They manipulated the object of attention (self vs external) and low mood (self with negative mood induction vs without) among a large sample of unselected women. Whereas attentional direction had no effect on any of the key outcomes, people in the negative mood condition reported more body dissatisfaction and sadness after mirror gazing, suggesting that low mood plays an important role in how mirror gazing impacts BDD symptoms. These findings have important implications for mirror exposure interventions in CBT, and for targeting transdiagnostic factors such as low mood, which frequently co-occurs with psychopathology.
People with BDD engage in safety behaviours such as mirror and body/appearance checking, reassurance-seeking, hiding their disliked feature, and excessive grooming, to avoid their disliked feature being evaluated negatively and/or transiently reduce their anxiety (Wilhelm et al. 2010). Despite hypotheses that these safety behaviours contribute to people’s preoccupations with their own perceived physical flaw and to other important factors in BDD (e.g. catastrophic misinterpretations), experimental work testing the causal and maintaining role of safety behaviours in BDD is only just beginning. Using a 1-week, pre-post experiment, Summers and Cougle (2018) found that healthy participants who increased their safety behaviour use reported greater BDD symptoms, social anxiety symptoms, body dissatisfaction, appearance-, and social-evaluative-related cognitive biases at post-test, compared to participants in control conditions. Following an appearance-threat task (having full-body photos taken), participants in the increased safety behaviour condition also reported higher levels of fear, urges to seek reassurance, and symptoms of anxiety and depression. As such, safety behaviours can cause BDD and associated symptoms, providing solid evidence for the importance of targeting them in CBT. Indeed, Wilver et al. (2020) demonstrated that fading safety behaviours in response to text message reminders reduced BDD symptoms (compared to control) among young women with elevated appearance concerns—a compelling example of how quickly experimental work can translate to potential treatment improvements. Continued experimental research related to causal and maintaining factors of BDD are sorely needed to continue to improve CBT outcomes.
How EPP Advances CBT for Previously Overlooked Disorders: Paranoia
Given the importance of genetic and biological causes in psychotic disorders (and thus, the primacy of pharmaceutical therapies), researchers have focused less on cognitive and behavioural treatment targets. Nonetheless, CBT, as an adjunct to medication, prevents (e.g. Hutton and Taylor 2014) and reduces psychotic symptoms (e.g. Hazell et al. 2016; van der Gaag et al. 2014). However, effect sizes are relatively small, suggesting that more work is needed. EPP may seem particularly misplaced in psychosis research, given the presumed difficulty inherent in “manipulating” psychosis; however, hallucinations and delusions occur on a continuum just like any other symptom (Johns and van Os 2001). A fundamental tenet of EPP is that pathological behaviour is an extreme manifestation of universal cognitive processes that impact functioning for people with and without clinical disorders (e.g. Boddez et al. 2017).
Researchers have thus taken on the challenge of investigating cognitive and behavioural mechanisms relevant to paranoia, a broad symptom common to many presentations of psychotic disorders, using a well-established social exclusion paradigm—Cyberball (Williams et al. 2000). Cyberball is a virtual game in which participants toss a ball with two experimentally controlled “players”. Researchers vary the degree to which the computer-controlled players interact with the participant, i.e., the degree to which they are ostracised. Generally, the more people are excluded, the more they report negative affect, interpret the exclusion as intentional, and engage in maladaptive interpersonal behaviours (Hartgerink et al. 2015).
Kesting et al. (2013) used this paradigm with a sample of healthy undergraduate students to test whether social exclusion and criticism would lead to increases in paranoia symptoms, consistent with hypotheses that threatening interpretations (i.e. cognitive biases) of social and other stressors play a causal role in some psychotic symptoms (e.g. Freeman et al. 2002). They found that people who were excluded and received negative feedback reported a greater increase in state paranoid ideation than those in the control condition (included, received neutral feedback). Social exclusion and negative feedback led to reduced self-esteem, which in turn led to greater beliefs that others were out to get them. Kaltsi et al. (2018) showed experimentally that people who held positive beliefs about paranoia experienced greater paranoid ideation in response to Cyberball than people who held negative beliefs about paranoia. EPP researchers further demonstrated that criticism amplifies social exclusion’s effect on paranoia whereas self-esteem is protective (Butler et al. 2019). Social exclusion can trigger reductions in both explicit and implicit self-esteem, both associated with paranoia post-Cyberball (Stewart et al. 2017).
Despite the seeming difficulty using EPP to better understand psychotic symptoms, Brown et al. (2019) found over 40 experiments conducted in the last 20 years with the objective of determining causes of psychosis. They found that the majority of non-intervention experiments were conducted with non-clinical samples, and that studies involving manipulation of a purported mechanism provided the “most robust causal evidence” (p. 25). Paranoia was by far the most-studied symptom, leading Brown et al. (2019) to advocate for more EPP research aimed at investigating causal and maintaining factors of other symptoms.
Freeman (2016) reviewed the evidence for focusing on specific mechanisms, including worry, self-beliefs, and mistrust when treating paranoia and persecutory delusions. Although one meta-analysis shed doubt on the long-term efficacy of CBT for delusions (Mehl et al. 2015), the more recent interventions targeting mechanisms identified by EPP showed more positive effects on symptoms. In their meta-regression, Sitko et al. (2020) demonstrated that the efficacy of CBT for delusions has improved over time. Clearly, increased understanding of causes and maintaining factors of delusions has translated to increased efficacy of CBT for these symptoms.
The Importance of Theory and Clinical Relevance
We contend that “where we are” currently, in EPP, is at something of a crossroads, between needing to focus on specific mechanisms to better prioritise treatment targets and getting so caught up in minute details that we miss the forest for the trees. Indeed, one thing shared by the research programs discussed thus far is an attempt not just to identify some kind of phenomenological “uniqueness” specific to a disorder, but rather a focus on testing a theory based on experiences working with people living with mental health problems. Although EPP research typically begins when researchers identify human processes that seem exaggerated in people with psychopathology, that identification is most likely to translate into an important treatment target when the mechanism of change is theoretically and clinically plausible. Researchers began studying memory and memory confidence in OCD because clients often reported checking because they “could not remember” whether they had checked properly. Identifying the role of eye movements came about because researchers and clinicians alike were not satisfied with knowing that a treatment worked without knowing why it worked. The research on paranoia has progressed so much further than research on cognitive and behavioural causes of other psychotic symptoms because of its basis in Freeman’s (Freeman 2016; Freeman et al. 2002) plausible, complex, and evidence-based model of persecutory delusions, which focuses on clients’ real concerns about deciding who to trust, when, and on what basis. In other words, when we use EPP to validate experimentally what our clients already know, we seem to get much closer to understanding what thoughts and behaviours drive psychopathological symptoms. This perspective requires keeping clinical relevance at the heart of our research, rather than simply studying what we think might be interesting. It also requires testing theories, rather than relationships among variables. As researchers focused on improving treatment, we need to think deeply about what we want to know about people who experience mental health symptoms, and why we want to know it.
For example, in a previous review, we found that although cognitive science research had provided much knowledge about the phenomenology of OCD, it had contributed little to improving treatments for people living with OCD (Ouimet et al. 2019). We critiqued work on cognitive deficits and highlighted problems with paradigm specificity and failure to account for the alternative hypothesis—that participants’ beliefs and symptoms caused the so-called deficits. Indeed, the major advances in CBT have come not from documented brain differences, but from thoughts, behaviours, and emotions that people living with psychopathology describe. We must consider that some EPP research will be more useful than others for improving treatment.
In principle, experimental psychopathologists are more interested in core human processes that occur across the range of human functioning than in diagnostic categories (van den Hout et al. 2017). Nonetheless, much of the published research focuses on people who experience a specific type of psychopathology. We contend, however, that the programs of research that have been the most effective in developing and refining CBT have tended to focus on the causes of a particular symptom rather than of a particular disorder.
Although it may thus appear that EPP tends towards “splitting” rather than “clumping”, there is a strong connection between this kind of specificity and transdiagnostic treatments for various problems. For example, although Rachman (1984) originally described safety behaviours as they occurred for people with agoraphobia, we now have strong evidence that people use safety behaviours across many types of psychopathology. Similarly, we associate paranoia with psychotic disorders; however, it may play a role in other disorders where social/interpersonal functioning is affected, such as social anxiety disorder, borderline personality disorder, and posttraumatic stress disorder. This focus on the specific allows for strong experimental design and internal validity, which over time contributes to better understanding how a specific cognitive or behavioural process impacts symptoms across multiple presentations.
As such, we encourage EPP researchers to put theory and clinical relevance at the forefront of their research programs, and to prioritise studying human processes that fit logically as potential mechanisms. Essentially, we need to focus on the forest and the trees. One of the enormous advantages of participating in an international conference dedicated to CBT is the opportunity to get a bird’s eye view of the exciting EPP and CBT research going on worldwide.
Where Are We Going?
Looking back and looking forward, a few areas stand out as necessary to ensure EPP remains a solid and effective bridge between basic science and clinical practice. First, the push towards improving scientific practices and transparency in psychology needs to be more quickly and more forcefully integrated into EPP to ensure replicable methods and findings—and thus, reliable treatment targets for CBT. Second, we need more EPP research to identify the cognitive and behavioural causes of symptoms and disorders that remain poorly understood, or for which CBT produces suboptimal outcomes. Third, for therapies that we know are efficacious, we need to start testing which components are truly “active” in reducing people’s symptoms, with the goals of increasing CBT efficiency and access. Finally, we are particularly guilty of ignoring diversity in EPP and CBT research. The time for that to change is past due.
Improving the Science of Psychopathology
The “Open Science” movement is no longer novel. In 2015, the Open Science Collaboration (2015) published the results of their large-scale, multi-site replication project wherein 270 researchers conducted 100 direct replications of journal articles published in 2008 from three leading journals in psychology. They directly reproduced fewer than 40% of the significant findings of the original research. Their work served as a call to action for psychology researchers to critically examine and improve our research practices. Since then, we have witnessed important changes in publication practices, such as pre-registering methods, hypotheses, and analyses to reduce questionable research practices (e.g. p-hacking, HARKing) and making data and analysis code openly available alongside published articles for verification.
Nonetheless, EPP and CBT research have been slower to implement these practices. According to the Center for Open Science (n.d.), over 250 journals currently offer the option to submit articles as Registered Reports. Of these, 11 (4%) appear relevant to clinical psychology/psychiatry, CBT models and treatments, or psychopathology. For comparison, 24 (9%) appear relevant to cognitive psychology (excluding those specific to neuroscience); however, there are far fewer cognitive psychology journals in general (154; Scimago Journal and Country Rank 2019a) than clinical psychology journals (295; Scimago Journal and Country Rank 2019b). Hildebrandt and Prenoveau (2020) analysed the rigour and reproducibility of research design and data analyses in all articles published in 2018 in arguably the highest impact EPP journal, Behaviour Research and Therapy; none met all the criteria they outlined for statistical rigour and reproducibility, and only 14% met just less than half of the criteria. They noted particular areas for improvement, including pre-registering hypotheses, reporting statistical power, including data and analysis code and output, and linking hypotheses to planned analyses.
At the WCBCT, one of us (A.O.) chaired a well-attended panel discussion entitled Open science and reproducibility in CBT research: Where do we go from here? Panellists, including Andreas Burger (UK Leuven), Michelle Craske (UCLA), Peter McEvoy (Curtin University), and Bethany Teachman (University of Virginia), discussed reasons that CBT research has experienced slowed uptake of Open Science practices, and avenues for facilitating such practices moving forward. Improving the science of psychopathology has important implications for improving CBT and increasing access to treatment advances for those who need it most.
Improving CBT
Although CBT is the most empirically validated treatment for a wide range of clinical disorders, it produces suboptimal outcomes for some problems. The reasons for CBT’s underperformance are multifactorial and include lack of research in some areas and therapeutic elements that clients find difficult. Perhaps most importantly, researchers, at times, have skipped the crucial step of identifying and testing causal and maintaining mechanisms. Below, we focus on eating disorders research and treatment as our exemplar, because there is considerable room for improvement in CBT efficacy (particularly for anorexia nervosa) and a notable lack of experimental work attempting to test the core tenets of CBT models of eating disorders.
Historically, eating disorders, and the people who lived with them, were considered intransigent, a term that puts the onus of treatment failure on the client rather than where it belongs—on the researchers and clinicians who have not yet figured out how to help them. Pennesi and Wade (2016) noted that of 54 theoretical models of disordered eating identified in their systematic review, fewer than 20% translated to tests of interventions. They also identified overlapping core features of the models (e.g. emotion regulation, external pressure, maladaptive beliefs; Pennesi and Wade 2016); however, the evidence for any of these variables as symptom causes or perpetuators is minimal (Jansen 2016). The efficacy of CBT for eating disorders is modest at best (Bulik 2014; Galsworthy-Francis and Allan 2014; but see Linardon et al. 2017), and may be due to a lack of EPP work validating the theories that underlie them.
One of the most influential models of eating disorders—the transdiagnostic maintenance model of eating disorders (Fairburn et al. 2003)—holds that multiple mechanisms perpetuate both anorexia and bulimia nervosa symptoms (as well as those of other eating disorders). Targeting those mechanisms via CBT can reduce symptoms (NICE 2017). However, Murray et al. (2019) found that although specialised treatments led to better weight gain outcomes among people with anorexia nervosa at post-treatment than did treatment-as-usual, there were no differences on psychological factors, and weight outcomes were not maintained at follow-up.
To illustrate the shortage of experimental psychopathology work related to models and treatments of eating disorders, we focus on one important hypothesis: that dietary restriction reinforces anorexia and bulimia nervosa symptoms. People with anorexia who do not binge-eat ultimately experience something of a “starvation syndrome”, wherein they withdraw socially, become even more preoccupied with food and eating, but actually experience heightened fullness and decreased appetitive reward (Fairburn et al. 2003). To explain this relationship, some researchers, citing correlational data, proposed that anorexia symptoms stem from biological abnormalities in reward system functioning (e.g. Wierenga et al. 2015). However, Jansen (2016) highlighted a series of experiments that point the finger at restriction as the primary cause of altered reward system functioning. Essentially, people with anorexia expose themselves frequently to high-reward foods, but extinguish their reward expectancies by failing to actually eat these foods. These experimental findings suggest that treatments incorporating exposure to eating high-reward foods may be critical to improving CBT efficacy among the minority of people with eating disorders who engage in restriction without binge-eating. Moreover, inhibitory learning accounts of extinction suggest that ensuring exposures are varied, occur in multiple contexts, and prioritise new learning will help reduce relapse (Jansen 2016; Reilly et al. 2017). However, we need EPP to clarify these mechanisms and how best to target them in CBT for eating disorders.
With respect to bulimia nervosa (and anorexia nervosa, binge-purge type), researchers theorise that people who restrict their eating are more likely to eventually lose control of their eating, binge, purge to correct for the binge, and further exacerbate their maladaptive beliefs related to overvaluation of weight and shape (Fairburn et al. 2003). However, there is limited evidence supporting the causal role of dietary restriction on binge-eating. Cooper et al. (1993) tested the effects of maladaptive beliefs on bulimia symptoms by priming participants with bulimia nervosa with word pairs either related (experimental condition) or unrelated (control condition) to negative and positive interpretations of weight and shape. People in the experimental condition reported more negative thoughts after the cognitive induction and ate less food in a “taste-test” than the control condition; however, they also reported the same number of objective binges and fewer subjective binges in the following 24 h. Although negative thoughts about weight and shape activated dietary restriction, the restriction did not trigger more binge-eating (Cooper et al. 1993). In a longitudinal weight loss study, Anton et al. (2009) found no differences in appetite over 6 months between people engaged in various caloric restriction diets. Beliefs about restriction—manipulated by describing a shake as indulgent and high-calorie or low-fat and low-calorie—increased physiological but not subjective hunger cues, suggesting that perceptions of restriction may be more important than actual restriction (Crum et al. 2011).
As summarised eloquently by Jansen (2016), the bulk of the experimental evidence supporting dietary restriction as a cause of binge-eating does not include experimental manipulation of dietary restriction. Instead, researchers typically classified participants as “high-restrained” or “low-restrained” eaters, and then tested the impact of different variables on their consumption of high-reward food following an initial forced food load. Moreover, although researchers propose that the restrained eating model may explain bulimia nervosa (e.g. Fairburn et al. 2003), very little of the dietary restriction research is focused on eating disorders, blurring the lines among dieting, overeating, binge-eating, and health interventions for overweight or obese people. We are not arguing that dietary restraint is unimportant to understanding bulimia, or that clinicians should not target dietary restraint in their interventions. Rather, we are arguing that we do not know the exact role of dietary restraint in bulimia nervosa, which may be limiting our ability to successfully reduce symptoms in treatment. We need EPP to advance CBT for eating disorders and other forms of psychopathology for which treatment outcome is suboptimal.
Identifying Active Components of CBT
RCTs are the gold standard of intervention science, and typically feature tests of complete treatment packages. We also need to understand which elements of therapy actually lead to the most symptom change. Despite the historical debate regarding whether cognitive or behavioural change was the driving mechanism for treatment efficacy (e.g. Ougrin 2011), researchers and clinicians alike now appear to agree that symptom reduction occurs via diverse cognitive and behavioural mechanisms of change (Hofmann and Hayes 2019). Although it is no longer interesting or useful to pit the “B” against the “C”, it is important to test the efficacy of individual techniques, an objective for which EPP is especially well suited.
There are many important reasons for homing in on particular techniques, whether to add new targets to existing treatments or to validate the efficacy of an existing technique on a specific mechanism. Given the well-documented difficulty accessing evidence-based treatments (Gunter and Whittal 2010), understanding which techniques give us the “greatest bang for our buck” should help reduce the length of some treatments and work through waiting lists a little more quickly. Additionally, understanding which technique works for whom may help clinicians individualise their treatment administration and achieve symptom reduction earlier. Finally, “Mini”-intervention trials are experiments in their own right. They provide further evidence that a particular mechanism plays a role in maintaining symptoms, or identify factors that are not as crucial as we believed, allowing researchers to focus on the next candidate mechanism.
One particularly interesting EPP program exploring intrusive memories in PTSD demonstrated that healthy people who engage in some kind of visuospatial task (i.e. Tetris) after an analogue traumatic event actually experience fewer future trauma intrusions due to interference with encoding of the memory trace (Holmes et al. 2009). Moreover, although playing Tetris disrupts and reduces involuntary intrusions, it leaves voluntary memories intact—providing a selective interference effect (see James et al. 2016, for a review; Badawi et al. 2020 for a replication by a different research group). In a clever series of three experiments, Lau-Zhu et al. (2019) further demonstrated that the interference effect was not attributable to cue overlap, attentional capture, or working memory load during retrieval, providing further experimental support for Tetris post-trauma as a potential inoculation of sorts against traumatic intrusions.
Iyadurai et al. (2018) translated these laboratory-based findings to the clinic by conducting an RCT in an emergency room. Participants who had experienced a motor vehicle collision were randomly assigned to either a memory cue + 20 min of Tetris condition or a control activity log condition. Participants who played Tetris after being reminded of their collision reported substantially fewer intrusions during the week following the collision and significantly less distress related to the intrusions at 1-week follow-up. They overwhelmingly rated the intervention as helpful, easy to implement, and minimally burdensome. Horsch et al. (2017) demonstrated similar positive effects in women who had experienced an emergency caesarean section, suggesting that the intervention likely generalises to diverse types of traumatic experiences. Whether the effects on intrusions have positive downstream impacts on other PTSD symptoms and the degree to which other variables (e.g. negative affect) moderate the effect of the visuospatial task remain to be seen (Cristea et al. 2018). By testing specific techniques validated in EPP research, we get closer to understanding if, how, and why interfering with memory encoding may reduce PTSD incidence.
Typically, specific techniques are just one component that make up an intervention. For example, dialectical behavior therapy (DBT) (Linehan 2014) teaches a diverse variety of skills to improve emotion regulation, distress tolerance, mindfulness, and interpersonal effectiveness. Although DBT is an evidence-based treatment for borderline personality disorder (e.g. Cristea et al. 2017), it is not clear how each technique or skill targets a particular mechanism, or even whether each technique is effective.
The “half-smile” skill, for example, is a distress tolerance skill based on the facial feedback hypothesis (Strack et al. 1988)—that producing a smiling facial expression provides people with (unconscious) feedback that improves their affective experience. In a famous set of two studies, people who held a pen between their teeth (unknowingly simulating a smile) rated cartoons as funnier than people who held a pen with their lips (unknowingly simulating a frown) (Strack et al. 1988). According to Google Scholar, that article was cited over 1600 times until 2016, when Wagenmakers et al. (2016) conducted a Registered Replication Report of the original study, which included over 17 independent replication attempts. A meta-analysis of the findings concluded that there was no effect of smiling on cartoon ratings, shedding substantial doubt on the potential helpfulness of the “half-smile” technique. We recently explored smiling as an emotion regulation strategy in response to viewing negative images (Bahl and Ouimet 2020, manuscript submitted for publication) and during a conversation task (Bahl and Ouimet 2020, manuscript in preparation). Overall, smiling offered no advantages over maintaining a neutral expression or reacting as usual on cognitive, behavioural, subjective, or psychophysiological outcomes, providing further evidence that the “half-smile” technique may be inefficacious at best, and may thus consume more therapeutic time than it is worth.
Testing specific techniques also leads to improvements in their impact. In their classic CBT model of social anxiety disorder, Clark and Wells (1995) theorise that self-focused attention during exposure to a perceived social threat, coupled with excessive use of safety behaviours and biased post-event processing, maintains people’s symptoms. CBT based on their model leads to robust decreases in social anxiety disorder symptoms (e.g. Stangier et al. 2011). Moreover, video feedback—a set of behavioural experiments designed specifically to target self-focused attention and safety behaviours—leads to immediate decreases in social anxiety symptoms and in self-perception ratings (McManus et al. 2009; Schreiber et al. 2015). Some researchers reported equivocal findings on the efficacy of video feedback with clinical (Smits et al. 2006) and analogue (Rodebaugh 2004) samples. Following continued EPP to understand the disparate findings and clarify if, when, how, and for whom video feedback might be helpful (e.g. Moscovitch et al. 2013), Warnock-Parkes et al. (2017) refined and re-tested the technique. Their modifications resulted in greater effect sizes for reductions in negative self-perceptions and social anxiety compared to McManus et al. (2009). In other words, initial tests of a specific technique resulted in further EPP and subsequently, refinement of an already efficacious technique—hopefully providing more relief to a larger proportion of treatment seekers. These findings also bolster support for self-focused attention and safety behaviours as key maintaining mechanisms that must be targeted early in social anxiety treatment.
Mini intervention trials are also vital to determine which factors identified by EPP are actually causal or maintaining mechanisms that need targeting during CBT, rather than simply correlates or consequences of the symptoms. Cognitive bias modification interventions are targeted, computerised techniques that researchers developed to try to reverse attentional, interpretation, memory, and behavioural biases exhibited by people with psychopathology (initially identified via correlational and experimental research). For example, participants may be asked to complete ambiguous sentences with positive (vs negative) outcomes, to identify a target that always appears near a positive (vs negative) stimulus, to use a joystick to virtually approach a positive (vs negative) stimulus, or to actively forget negative (vs positive) stimuli (see Hertel and Mathews 2011). In an ingenious series of experiments, MacLeod et al. (2002) expanded on previous findings demonstrating that people with anxiety had an attention bias for threat-relevant information by modifying the dot-probe paradigm to actually manipulate attention bias. They demonstrated that threat-relevant attention biases could cause anxious reactions to stress. Rapid proliferation of research on attention bias and psychopathology followed, ultimately leading to attention bias modification paradigms as potential interventions, and an ongoing spirited debate about their usefulness (e.g. Emmelkamp 2012).
Despite the innovation in the original work by MacLeod et al. and some promising initial findings supporting attention bias modification as a low-cost, easily accessible intervention for psychopathology (e.g. Hakamata et al. 2010), several meta-analyses have failed to support its reliability in substantially reducing symptoms of anxiety and depression over the long term (e.g. Fodor et al. 2020; Mogoaşe et al. 2014). Moreover, in a meta-analysis of attention bias modification RCTs, Kruijt et al. (2019) estimated the mean threat-related attention bias on the dot-probe paradigm of over 1000 people with anxiety disorders prior to participating (i.e. at baseline). They found no evidence for a threat-relevant attention bias in people with anxiety disorders, shedding some doubt on a long-held belief in the literature. However, Kruijt et al. focused on bias towards threat (vs neutral or positive stimuli) among people with anxiety disorders, rather than bias compared to people with varying degrees of anxiety. Indeed, in a similar meta-analysis that focused on individual differences at a participant level, people who reported higher social anxiety also demonstrated greater attentional biases towards threat (Price et al. 2016), further supporting the general contention that when people are anxious, they preferentially attend to anxiety-relevant stimuli.
Unfortunately, despite the current evidence that attention bias modification is unlikely to lead to reliable improvements in psychopathology, the cart ran ahead of the horse. Building on the excitement and dissemination of the initial attention bias modification findings, app developers boast of the large evidence base backing up the ability of their numerous for-purchase bias modification apps to “cure” people of their mood and anxiety disorders with minimal effort, time, and money (see, e.g. BMI n.d.). Our objective of helping people may lead us to push interventions into the clinic before they are well understood. While this urge is understandable, it can cause more harm than good, potentially teaching clients that therapy is not helpful, or at least cannot help them. The role of EPP goes beyond studying what causes psychopathology; we have an ethical obligation and responsibility to ensure interventions work, understand how and for whom they work, and identify any iatrogenic impacts before they end up available for widespread public use. For example, Laurens et al. (2020) assessed the feasibility of an alcohol avoidance training app among a large sample of people experiencing problems with alcohol use. They found some evidence for the app’s ease of use and efficacy in reducing alcohol consumption. However, there was a large drop-out rate (over 60%), highlighting a need for improvement. Given the proliferation of mental health apps, we need controlled studies of their advantages and limitations to complement the EPP work that serves as their foundation.
To account for the degree to which translation from EPP to CBT can be delayed, researchers may want to consider Blackwell et al.’s (2019) leapfrog design, a version of adaptive rolling trials used previously in cancer treatment research, modified for psychological treatment development and implementation. Essentially, researchers can compare multiple versions of a standard treatment in a clinical trial, and use repeated Bayesian modelling to quickly identify treatment variants that are or are not likely to improve upon the standard treatment. Probably inefficacious arms are dropped earlier (and perhaps sent back to the lab for improvement) and new insights from EPP are translated to new arms to be tested in the already established treatment trial. This research design reduces the likelihood of false negatives and false positives, accelerates the inclusion of new EPP research insights into treatment trials, requires fewer participants, and increases efficiency and cost-effectiveness of clinical trials. Indeed, this design has enormous potential to strengthen and facilitate the EPP-CBT bridge.
Prioritising Diversity in CBT and Reducing Mental Health Disparities
There is a wealth of evidence that people from minority groups are underrepresented in CBT trials (Mak et al. 2007; Mendoza et al. 2012; Wetterneck et al. 2012; Williams et al. 2010), and that non-culturally adapted psychological interventions (including CBT) lead to insufficient improvement for people from underrepresented minority groups (Benish et al. 2011; Hall et al. 2016; Huey and Tilley 2018; Windsor et al. 2015). In some cases, there is not enough well-conducted research to determine the efficacy of interventions for members of minority groups (Mak et al. 2007; Steinka-Fry et al. 2017). We need adequately powered, methodologically rigorous experiments with underrepresented participants to understand which mechanisms to target within diverse populations (e.g. Gregory 2016). This issue is further complicated by researchers’ use of outcome measures that themselves are not culturally adapted, and do not take into consideration how people from different cultural groups interpret the questionnaire and interview items (Helms 2015). Rathod et al. (2018) succinctly summarised the impact of these limitations: “…the generalization of findings of the effectiveness of therapeutic interventions to many cultural groups may not be valid or even appropriate” (p. 165).
The lack of attention to cultural diversity is even more noticeable in EPP. We encountered considerable difficulty locating experiments wherein researchers had a primary goal of understanding how elements of diversity play a role in the cause and/or maintenance of psychopathology. There is strong evidence from correlational research that people from different cultural groups experience symptoms differently (see Chapman et al. 2014; Moleiro 2018 for reviews), and also differ in terms of risk, protective, and associated factors (e.g. Hall-Clark et al. 2016; Williams et al. 2012a, b, c). However, cultural diversity is rarely more than a statistical consideration in EPP research, and even in that case, we fall far short of APA recommendations for reporting and considering diversity in research (APA 2019, p. 79).
We quickly reviewed the sociodemographic reporting in the first two issues published in 2020 by two leading experimental psychopathology journals. Across 46 articles, there were 32 experiments or quasi-experiments, all of which we counted as experiments (excluding secondary analyses of data). Authors of the experiments consistently reported age and sex/gender; however, only 9 (less than 30%!) reported on any other sociodemographic factors (e.g. marital status, education level). Notably, only one EPP article described their sample’s self-reported race/ethnicity. Given the evidence for cultural differences in symptoms and potential mechanisms, we should expect that there are important cultural differences in causal/maintaining models that require not only adaptations in existing treatments but novel treatments that include culture as a core feature through which beliefs and behaviours develop. How can we hope to improve CBT effectiveness and access for all people who need it if we are not even measuring and reporting the diversity of the participants in our experiments?
It is thus incumbent upon EPP researchers to actively consider cultural diversity at the outset of their study design, including conceptualisation/theory, recruitment, methods and measurement, data analysis, interpretation, and conclusions. Obviously, we are not suggesting that EPP researchers should find “clever” ways to manipulate diversity. At a bare minimum, measuring and describing the sample’s sociodemographic characteristics are vital. But we can do much more, starting with intentionally recruiting diverse samples. The purported difficulty of including participants from underrepresented minority groups is not a valid excuse for using almost all-White samples. Williams and colleagues describe current barriers to participating in research for Black Americans and offer evidence-based and tangible solutions for improving both recruitment and retention of these participants (Williams et al. 2012b, d, 2013a, b). In a recent meta-analysis, Pieterse et al. (2012) demonstrated that Black Americans who reported more exposure to racism also reported greater mental distress and psychopathological symptoms. So, the question is not “Is being a member of an underrepresented minority group a risk factor for psychopathology?”, but instead “How do the experiences of people from underrepresented minority groups put them at greater risk for psychopathology?”, a question that EPP researchers are exceptionally poised to answer.
Conclusion
EPP has been instrumental to CBT since its inception. It has been especially useful when researchers used theoretically driven and rigorous research designs to focus on specific processes that align with the experiences of people living with psychopathology. To navigate the most important CBT crossroads, EPP researchers need to expand their efforts by including diverse populations, prioritising open and transparent research practices, and clarifying which therapeutic elements are the most important for whom. In so doing, they will ensure EPP serves as more than just a basic-applied science bridge, but as the framework for understanding core processes in psychopathology and translating them directly to efficacious treatments to reduce suffering and increase well-being.
References
Alcolado, G. M., & Radomsky, A. S. (2011). Believe in yourself: manipulating beliefs about memory causes checking. Behaviour Research and Therapy, 49(1), 42–49. https://doi.org/10.1016/j.brat.2010.10.001.
Alcolado, G. M., & Radomsky, A. S. (2016). A novel cognitive intervention for compulsive checking: targeting maladaptive beliefs about memory. Journal of Behavior Therapy and Experimental Psychiatry, 53, 75–83. https://doi.org/10.1016/j.jbtep.2015.02.009.
Ali, S., Rhodes, L., Moreea, O., McMillan, D., Gilbody, S., Leach, C., Lucock, M., Lutz, W., & Delgadillo, J. (2017). How durable is the effect of low intensity CBT for depression and anxiety? Remission and relapse in a longitudinal cohort study. Behaviour Research and Therapy, 94, 1–8. https://doi.org/10.1016/j.brat.2017.04.006.
Andersson, G., Cuijpers, P., Carlbring, P., Riper, H., & Hedman, E. (2014). Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry, 13(3), 288–295. https://doi.org/10.1002/wps.20151.
Andrade, J., Kavanagh, D., & Baddeley, A. (1997). Eye-movements and visual imagery: a working memory approach to the treatment of post-traumatic stress disorder. The British Journal of Clinical Psychology, 36(2), 209–223. https://doi.org/10.1111/j.2044-8260.1997.tb01408.x.
Anton, S. D., Han, H., York, E., Martin, C. K., Ravussin, E., & Williamson, D. A. (2009). Effect of calorie restriction on subjective ratings of appetite. Journal of Human Nutrition and Dietetics, 22(2), 141–147. https://doi.org/10.1111/j.1365-277X.2008.00943.x.
APA. (2019). Publication manual of the American Psychological Association (7th ed.). American Psychological Association. https://doi.org/10.1037/0000165-000.
Badawi, A., Berle, D., Rogers, K., & Steel, Z. (2020). Do cognitive tasks reduce intrusive-memory frequency after exposure to analogue trauma? An experimental replication. Clinical Psychological Science: A Journal of the Association for Psychological Science, 8(3), 569–583. https://doi.org/10.1177/2167702620906148.
Bass, J. K., Annan, J., McIvor Murray, S., Kaysen, D., Griffiths, S., Cetinoglu, T., Wachter, K., Murray, L. K., & Bolton, P. A. (2013). Controlled trial of psychotherapy for Congolese survivors of sexual violence. The New England Journal of Medicine, 368(23), 2182–2191. https://doi.org/10.1056/NEJMoa1211853.
Beck, A. T. (2019). A 60-year evolution of cognitive theory and therapy. Perspectives on Psychological Science, 14(1), 16–20. https://doi.org/10.1177/1745691618804187.
Bendall, S. (2019). Using trauma-focused therapies to treat posttraumatic symptoms in psychosis: what works, when, and for whom? [Symposium]. Berlin: World Congress of Behavioural & Cognitive Therapies. Retrieved September 20, 2020, from https://wcbct2019.org.
Benish, S. G., Quintana, S., & Wampold, B. E. (2011). Culturally adapted psychotherapy and the legitimacy of myth: a direct-comparison meta-analysis. Journal of Counseling Psychology, 58(3), 279–289. https://doi.org/10.1037/a0023626.
Blackwell, S. E., Woud, M. L., Margraf, J., & Schönbrodt, F. D. (2019). Introducing the leapfrog design: a simple Bayesian adaptive rolling trial design for accelerated treatment development and optimization. Clinical Psychological Science: A Journal of the Association for Psychological Science, 7(6), 1222–1243.
BMI. (n.d.). Cognitive Bias Modification Apps. Retrieved September 15, 2020, from http://www.biasmodification.com/.
Boddez, Y., Davey, G., & Vervliet, B. (2017). Experimental psychopathology: defining the field. Psychopathology Review, 4(2), 109–111. https://doi.org/10.1177/20438087170a400201.
Boschen, M. J., & Vuksanovic, D. (2007). Deteriorating memory confidence, responsibility perceptions and repeated checking: comparisons in OCD and control samples. Behaviour Research and Therapy, 45(9), 2098–2109. https://doi.org/10.1016/j.brat.2007.03.009.
Bouchard, C., Rhéaume, J., & Ladouceur, R. (1999). Responsibility and perfectionism in OCD: an experimental study. Behaviour Research and Therapy, 37(3), 239–248. https://doi.org/10.1016/S0005-7967(98)00141-7.
Brown, P., Waite, F., & Freeman, D. (2019). ‘Twisting the lion’s tail’: manipulationist tests of causation for psychological mechanisms in the occurrence of delusions and hallucinations. Clinical Psychology Review, 68, 25–37. https://doi.org/10.1016/j.cpr.2018.12.003.
Bulik, C. M. (2014). The challenges of treating anorexia nervosa. Lancet, 383(9912), 105–106. https://doi.org/10.1016/S0140-6736(13)61940-6.
Butler, R., Berry, K., Ellett, L., & Bucci, S. (2019). An experimental investigation of the impact of critical and warm comments on state paranoia in a non-clinical sample. Journal of Behavior Therapy and Experimental Psychiatry, 62, 30–37. https://doi.org/10.1016/j.jbtep.2018.08.008.
Carpenter, J. K., Andrews, L. A., Witcraft, S. M., Powers, M. B., Smits, J. A. J., & Hofmann, S. G. (2018). Cognitive behavioral therapy for anxiety and related disorders: a meta-analysis of randomized placebo-controlled trials. Depression and Anxiety, 35(6), 502–514. https://doi.org/10.1002/da.22728.
Center for Open Science. (n.d.). Registered reports: peer review before results are known to align scientific values and practices. Retrieved September 4, 2020, from https://www.cos.io/our-services/registered-reports.
Chapman, L. K., DeLapp, R. C. T., & Williams, M. T. (2014). Impact of race, ethnicity, and culture on the expression and assessment of psychopathology. In Adult psychopathology and diagnosis (7th ed., pp. 131–162). Hoboken: John Wiley & Sons Inc.
Christman, S. D., Garvey, K. J., Propper, R. E., & Phaneuf, K. A. (2003). Bilateral eye movements enhance the retrieval of episodic memories. Neuropsychology, 17(2), 221–229. https://doi.org/10.1037/0894-4105.17.2.221.
Clark, D. M. (2004). Developing new treatments: on the interplay between theories, experimental science and clinical innovation. Behaviour Research and Therapy, 42(9), 1089–1104. https://doi.org/10.1016/j.brat.2004.05.002.
Clark, D. M., & Wells, A. (1995). A cognitive model of social phobia. In M. R. Liebowitz, R. G. Heimberg, F. R. Schneier, & D. A. Hope (Eds.), Social phobia: Diagnosis, assessment, and treatment (pp. 69–93). New York: The Guilford Press.
Cooper, M. J., Clark, D. M., & Fairburn, C. G. (1993). An experimental study of the relationship between thoughts and eating behaviour in bulimia nervosa. Behaviour Research and Therapy, 31(8), 749–757. https://doi.org/10.1016/0005-7967(93)90005-F.
Cristea, I. A., Gentili, C., Cotet, C. D., Palomba, D., Barbui, C., & Cuijpers, P. (2017). Efficacy of psychotherapies for borderline personality disorder: a systematic review and meta-analysis. JAMA Psychiatry, 74(4), 319–328. https://doi.org/10.1001/jamapsychiatry.2016.4287.
Cristea, I. A., Naudet, F., Shanks, D. R., & Hardwicke, T. E. (2018). Post-retrieval Tetris should not be likened to a ‘cognitive vaccine.’ Molecular Psychiatry, 23(10), 1972–1973. https://doi.org/10.1038/mp.2017.222.
Crum, A. J., Corbin, W. R., Brownell, K. D., & Salovey, P. (2011). Mind over milkshakes: mindsets, not just nutrients, determine ghrelin response. Health Psychology, 30(4), 424–429. https://doi.org/10.1037/a0023467.
Cusack, K., & Spates, C. R. (1999). The cognitive dismantling of eye movement desensitization and reprocessing (EMDR) treatment of posttraumatic stress disorder (PTSD). Journal of Anxiety Disorders, 13(1–2), 87–99. https://doi.org/10.1016/S0887-6185(98)00041-3.
Davidson, P. R., & Parker, K. C. H. (2001). Eye movement desensitization and reprocessing (EMDR): a meta-analysis. Journal of Consulting and Clinical Psychology, 69(2), 305–316. https://doi.org/10.1037//0022-006x.69.2.305.
De Putter, L. M. S., Van Yper, L., & Koster, E. H. W. (2017). Obsessions and compulsions in the lab: a meta-analysis of procedures to induce symptoms of obsessive-compulsive disorder. Clinical Psychology Review, 52, 137–147. https://doi.org/10.1016/j.cpr.2017.01.001.
Deacon, B., & Olatunji, B. O. (2007). Specificity of disgust sensitivity in the prediction of behavioral avoidance in contamination fear. Behaviour Research and Therapy, 45(9), 2110–2120. https://doi.org/10.1016/j.brat.2007.03.008.
Dominguez, S. (2019). The impact of eye movement desensitization and reprocessing as an adjunct to group cognitive behavioral therapy for individuals with depression [Poster presentation]. Berlin: World Congress of Behavioural & Cognitive Therapies.
Emmelkamp, P. M. (2012). Attention bias modification: the Emperor’s new suit? BMC Medicine, 10(1), 63. https://doi.org/10.1186/1741-7015-10-63.
Fairbrother, N., Newth, S. J., & Rachman, S. (2005). Mental pollution: feelings of dirtiness without physical contact. Behaviour Research and Therapy, 43(1), 121–130. https://doi.org/10.1016/j.brat.2003.12.005.
Fairburn, C. G., Cooper, Z., & Shafran, R. (2003). Cognitive behaviour therapy for eating disorders: a “transdiagnostic” theory and treatment. Behaviour Research and Therapy, 41(5), 509–528. https://doi.org/10.1016/S0005-7967(02)00088-8.
Fodor, L. A., Georgescu, R., Cuijpers, P., Szamoskozi, Ş., David, D., Furukawa, T. A., & Cristea, I. A. (2020). Efficacy of cognitive bias modification interventions in anxiety and depressive disorders: a systematic review and network meta-analysis. Lancet Psychiatry, 7(6), 506–514. https://doi.org/10.1016/S2215-0366(20)30130-9.
Forsyth, J. P., & Zvolensky, M. J. (2001). Experimental psychopathology, clinical science, and practice: an irrelevant or indispensable alliance? Applied and Preventive Psychology, 10(4), 243–264. https://doi.org/10.1016/S0962-1849(01)80002-0.
Freeman, D. (2016). Persecutory delusions: a cognitive perspective on understanding and treatment. Lancet Psychiatry, 3(7), 685–692. https://doi.org/10.1016/S2215-0366(16)00066-3.
Freeman, D., Garety, P. A., Kuipers, E., Fowler, D., & Bebbington, P. E. (2002). A cognitive model of persecutory delusions. The British Journal of Clinical Psychology, 41(4), 331–347. https://doi.org/10.1348/014466502760387461.
van der Gaag, M., Valmaggia, L. R., & Smit, F. (2014). The effects of individually tailored formulation-based cognitive behavioural therapy in auditory hallucinations and delusions: a meta-analysis. Schizophrenia Research, 156(1), 30–37. https://doi.org/10.1016/j.schres.2014.03.016.
Galsworthy-Francis, L., & Allan, S. (2014). Cognitive behavioural therapy for anorexia nervosa: a systematic review. Clinical Psychology Review, 34(1), 54–72. https://doi.org/10.1016/j.cpr.2013.11.001.
Gregory, V. L. (2016). Cognitive-behavioral therapy for depressive symptoms in persons of African descent: a meta-analysis. Journal of Social Service Research, 42(1), 113–129. https://doi.org/10.1080/01488376.2015.1084973.
Gunter, R. W., & Whittal, M. L. (2010). Dissemination of cognitive-behavioral treatments for anxiety disorders: overcoming barriers and improving patient access. Clinical Psychology Review, 30(2), 194–202. https://doi.org/10.1016/j.cpr.2009.11.001.
Hakamata, Y., Lissek, S., Bar-Haim, Y., Britton, J. C., Fox, N. A., Leibenluft, E., Ernst, M., & Pine, D. S. (2010). Attention bias modification treatment: a meta-analysis toward the establishment of novel treatment for anxiety. Biological Psychiatry, 68(11), 982–990. https://doi.org/10.1016/j.biopsych.2010.07.021.
Hall, G. C. N., Ibaraki, A. Y., Huang, E. R., Marti, C. N., & Stice, E. (2016). A meta-analysis of cultural adaptations of psychological interventions. Behavior Therapy, 47(6), 993–1014. https://doi.org/10.1016/j.beth.2016.09.005.
Hall-Clark, B., Sawyer, B., Golik, A., & Asnaani, A. (2016). Racial/ethnic differences in symptoms of posttraumatic stress disorder. Current Psychiatry Reviews, 12(2), 124–138.
Harrison, A., Fernández de la Cruz, L., Enander, J., Radua, J., & Mataix-Cols, D. (2016). Cognitive-behavioral therapy for body dysmorphic disorder: a systematic review and meta-analysis of randomized controlled trials. Clinical Psychology Review, 48, 43–51. https://doi.org/10.1016/j.cpr.2016.05.007.
Hartgerink, C. H. J., van Beest, I., Wicherts, J. M., & Williams, K. D. (2015). The ordinal effects of ostracism: a meta-analysis of 120 Cyberball studies. PLoS ONE, 10(5), e0127002. https://doi.org/10.1371/journal.pone.0127002.
Hazell, C. M., Hayward, M., Cavanagh, K., & Strauss, C. (2016). A systematic review and meta-analysis of low intensity CBT for psychosis. Clinical Psychology Review, 45, 183–192. https://doi.org/10.1016/j.cpr.2016.03.004.
Helms, J. E. (2015). An examination of the evidence in culturally adapted evidence-based or empirically supported interventions. Transcultural Psychiatry, 52(2), 174–197. https://doi.org/10.1177/1363461514563642.
Hertel, P. T., & Mathews, A. (2011). Cognitive bias modification: past perspectives, current findings, and future applications. Perspectives on Psychological Science, 6(6), 521–536. https://doi.org/10.1177/1745691611421205.
Hildebrandt, T., & Prenoveau, J. M. (2020). Rigor and reproducibility for data analysis and design in the behavioral sciences. Behaviour Research and Therapy, 126, 103552. https://doi.org/10.1016/j.brat.2020.103552.
Hofmann, S. G., & Hayes, S. C. (2019). The future of intervention science: process-based therapy. Clinical Psychological Science: A Journal of the Association for Psychological Science, 7(1), 37–50. https://doi.org/10.1177/2167702618772296.
Hofmann, S. G., Asnaani, A., Vonk, I. J. J., Sawyer, A. T., & Fang, A. (2012). The efficacy of cognitive behavioral therapy: a review of meta-analyses. Cognitive Therapy and Research, 36(5), 427–440. https://doi.org/10.1007/s10608-012-9476-1.
Holmes, E. A., James, E. L., Coode-Bate, T., & Deeprose, C. (2009). Can playing the computer game “Tetris” reduce the build-up of flashbacks for trauma? A proposal from cognitive science. PLoS ONE, 4(1), e4153. https://doi.org/10.1371/journal.pone.0004153.
Horsch, A., Vial, Y., Favrod, C., Harari, M. M., Blackwell, S. E., Watson, P., Iyadurai, L., Bonsall, M. B., & Holmes, E. A. (2017). Reducing intrusive traumatic memories after emergency caesarean section: a proof-of-principle randomized controlled study. Behaviour Research and Therapy, 94, 36–47. https://doi.org/10.1016/j.brat.2017.03.018.
Houben, S. T. L., Otgaar, H., Roelofs, J., & Merckelbach, H. (2018). Lateral eye movements increase false memory rates. Clinical Psychological Science: A Journal of the Association for Psychological Science, 6(4), 610–616. https://doi.org/10.1177/2167702618757658.
van den Hout, M. A., & Engelhard, I. M. (2012). How does EMDR work? Journal of Experimental Psychopathology, 3(5), 724–738. https://doi.org/10.5127/jep.028212.
van den Hout, M. A., & Kindt, M. (2003). Repeated checking causes memory distrust. Behaviour Research and Therapy, 41(3), 301–316. https://doi.org/10.1016/S0005-7967(02)00012-8.
van den Hout, M. A., Muris, P., Salemink, E., & Kindt, M. (2001). Autobiographical memories become less vivid and emotional after eye movements. The British Journal of Clinical Psychology, 40(2), 121–130. https://doi.org/10.1348/014466501163571.
van den Hout, M. A., Engelhard, I. M., Rijkeboer, M. M., Koekebakker, J., Hornsveld, H., Leer, A., Toffolo, M. B. J., & Akse, N. (2011). EMDR: eye movements superior to beeps in taxing working memory and reducing vividness of recollections. Behaviour Research and Therapy, 49(2), 92–98. https://doi.org/10.1016/j.brat.2010.11.003.
van den Hout, M. A., Rijkeboer, M. M., Engelhard, I. M., Klugkist, I., Hornsveld, H., Toffolo, M. J. B., & Cath, D. C. (2012). Tones inferior to eye movements in the EMDR treatment of PTSD. Behaviour Research and Therapy, 50(5), 275–279. https://doi.org/10.1016/j.brat.2012.02.001.
van den Hout, M. A., Engelhard, I. M., & McNally, R. J. (2017). Thoughts on experimental psychopathology. Psychopathology Review, 4(2), 141–154. https://doi.org/10.5127/pr.045115.
Huey, S. J., & Tilley, J. L. (2018). Effects of mental health interventions with Asian Americans: a review and meta-analysis. Journal of Consulting and Clinical Psychology, 86(11), 915–930. https://doi.org/10.1037/ccp0000346.
Hurley, E. C. (2018). Effective treatment of veterans with PTSD: comparison between intensive daily and weekly EMDR approaches. Frontiers in Psychology, 9(1458), 1–10. https://doi.org/10.3389/fpsyg.2018.01458.
Hutton, P., & Taylor, P. J. (2014). Cognitive behavioural therapy for psychosis prevention: a systematic review and meta-analysis. Psychological Medicine, 44(3), 449–468. https://doi.org/10.1017/S0033291713000354.
Inozu, M., Eremsoy, E., Cicek, N. M., & Ozcanli, F. (2017). The association of scrupulosity with disgust propensity and contamination based obsessive compulsive symptoms: an experimental investigation using highly scrupulous muslims. Journal of Obsessive-Compulsive and Related Disorders, 15, 43–51. https://doi.org/10.1016/j.jocrd.2017.08.004.
Iyadurai, L., Blackwell, S. E., Meiser-Stedman, R., Watson, P. C., Bonsall, M. B., Geddes, J. R., Nobre, A. C., & Holmes, E. A. (2018). Preventing intrusive memories after trauma via a brief intervention involving Tetris computer game play in the emergency department: a proof-of-concept randomized controlled trial. Molecular Psychiatry, 23(3), 674–682. https://doi.org/10.1038/mp.2017.23.
James, E. L., Lau-Zhu, A., Clark, I. A., Visser, R. M., Hagenaars, M. A., & Holmes, E. A. (2016). The trauma film paradigm as an experimental psychopathology model of psychological trauma: intrusive memories and beyond. Clinical Psychology Review, 47, 106–142. https://doi.org/10.1016/j.cpr.2016.04.010.
Jansen, A. (2016). Eating disorders need more experimental psychopathology. Behaviour Research and Therapy, 86, 2–10. https://doi.org/10.1016/j.brat.2016.08.004.
Johns, L. C., & van Os, J. (2001). The continuity of psychotic experiences in the general population. Clinical Psychology Review, 21(8), 1125–1141. https://doi.org/10.1016/S0272-7358(01)00103-9.
Kaltsi, M., Bucci, S., & Morrison, A. P. (2018). Experimental manipulation of metacognitive beliefs and paranoia in a non-clinical population. Psychosis, 10(1), 1–10. https://doi.org/10.1080/17522439.2017.1413128.
Kearns, M., & Engelhard, I. M. (2015). Psychophysiological responsivity to script-driven imagery: an exploratory study of the effects of eye movements on public speaking flashforwards. Frontiers in Psychiatry, 6(115), 1–9. https://doi.org/10.3389/fpsyt.2015.00115.
Kesting, M.-L., Bredenpohl, M., Klenke, J., Westermann, S., & Lincoln, T. M. (2013). The impact of social stress on self-esteem and paranoid ideation. Journal of Behavior Therapy and Experimental Psychiatry, 44(1), 122–128. https://doi.org/10.1016/j.jbtep.2012.07.010.
Kruijt, A.-W., Parsons, S., & Fox, E. (2019). A meta-analysis of bias at baseline in RCTs of attention bias modification: no evidence for dot-probe bias towards threat in clinical anxiety and PTSD. Journal of Abnormal Psychology, 128(6), 563–573. https://doi.org/10.1037/abn0000406.
Kube, T., & Rief, W. (2019). Using behavioural experiments in the treatment of mental disorders—recent developments and future directions [Symposium]. Berlin: World Congress of Cognitive and Behavioural Therapies.
Landin-Romero, R., Moreno-Alcazar, A., Pagani, M., & Amann, B. L. (2018). How does eye movement desensitization and reprocessing therapy work? A systematic review on suggested mechanisms of action. Frontiers in Psychology, 9(1395), 1–23. https://doi.org/10.3389/fpsyg.2018.01395.
Laurens, M. C., Pieterse, M. E., Brusse-Keizer, M., Salemink, E., Ben Allouch, S., Bohlmeijer, E. T., & Postel, M. G. (2020). Alcohol avoidance training as a mobile app for problem drinkers: longitudinal feasibility study. JMIR mHealth and uHealth, 8(4), e16217. https://doi.org/10.2196/16217.
Lau-Zhu, A., Henson, R. N., & Holmes, E. A. (2019). Intrusive memories and voluntary memory of a trauma film: differential effects of a cognitive interference task after encoding. Journal of Experimental Psychology. General, 148(12), 2154–2180. https://doi.org/10.1037/xge0000598.
Lee, C. (2019). Imagery rescripting and eye movement desensitisation and reprocessing (IREM) for treatment of adults with childhood PTSD: an international randomized clinical trial [Symposium]. Berlin: World Congress of Behavioural & Cognitive Therapies.
Linardon, J., Wade, T. D., de la Piedad Garcia, X., & Brennan, L. (2017). The efficacy of cognitive-behavioral therapy for eating disorders: a systematic review and meta-analysis. Journal of Consulting and Clinical Psychology, 85(11), 1080–1094. https://doi.org/10.1037/ccp0000245.
Linehan, M. M. (2014). DBT® skills training manual (2nd ed.). New York: The Guilford Press.
Litt, M. D., Kadden, R. M., & Tennen, H. (2018). Treatment response and non-response in CBT and Network Support for alcohol disorders: targeted mechanisms and common factors. Addiction, 113(8), 1407–1417. https://doi.org/10.1111/add.14224.
Macleod, C., Rutherford, E., Campbell, L., Ebsworthy, G., & Holker, L. (2002). Selective attention and emotional vulnerability: assessing the causal basis of their association through the experimental manipulation of attentional bias. Journal of Abnormal Psychology, 111(1), 107–123. https://doi.org/10.1037/0021-843X.111.1.107.
Mak, W. W. S., Law, R. W., Alvidrez, J., & Pérez-Stable, E. J. (2007). Gender and ethnic diversity in NIMH-funded clinical trials: review of a decade of published research. Administration and Policy in Mental Health, 34(6), 497–503. https://doi.org/10.1007/s10488-007-0133-z.
Marcus, S. V., Marquis, P., & Sakai, C. (1997). Controlled study of treatment of PTSD using EMDR in an HMO setting. Psychotherapy: Theory, Research, Practice, Training, 34(3), 307–315. https://doi.org/10.1037/h0087791.
McManus, F., Clark, D. M., Grey, N., Wild, J., Hirsch, C., Fennell, M., Hackmann, A., Waddington, L., Liness, S., & Manley, J. (2009). A demonstration of the efficacy of two of the components of cognitive therapy for social phobia. Journal of Anxiety Disorders, 23(4), 496–503. https://doi.org/10.1016/j.janxdis.2008.10.010.
McNally, R. J. (1999). On eye movements and animal magnetism: a reply to Greenwald’s defense of EMDR. Journal of Anxiety Disorders, 13(6), 617–620. https://doi.org/10.1016/S0887-6185(99)00020-1.
McNally, R. J., & Kohlbeck, P. A. (1993). Reality monitoring in obsessive-compulsive disorder. Behaviour Research and Therapy, 31(3), 249–253. https://doi.org/10.1016/0005-7967(93)90023-N.
Mehl, S., Werner, D., & Lincoln, T. M. (2015). Does cognitive behavior therapy for psychosis (CBTp) show a sustainable effect on delusions? A meta-analysis. Frontiers in Psychology, 6. https://doi.org/10.3389/fpsyg.2015.01450.
Mendoza, D. B., Williams, M. T., Chapman, L. K., & Powers, M. (2012). Minority inclusion in randomized clinical trials of panic disorder. Journal of Anxiety Disorders, 26(5), 574–582. https://doi.org/10.1016/j.janxdis.2012.02.011.
Merckelbach, H., Hogervorst, E., Kampman, M., & de Jongh, A. (1994). Effects of “eye movement desensitization” on emotional processing in normal subjects. Behavioural and Cognitive Psychotherapy, 22(4), 331–335. https://doi.org/10.1017/S1352465800013217.
Mogoaşe, C., David, D., & Koster, E. H. W. (2014). Clinical efficacy of attentional bias modification procedures: an updated meta-analysis. Journal of Clinical Psychology, 70(12), 1133–1157. https://doi.org/10.1002/jclp.22081.
Moleiro, C. (2018). Culture and psychopathology: new perspectives on research, practice, and clinical training in a globalized world. Frontiers in Psychiatry, 9. https://doi.org/10.3389/fpsyt.2018.00366.
Moscovitch, D. A., Rowa, K., Paulitzki, J. R., Ierullo, M. D., Chiang, B., Antony, M. M., & McCabe, R. E. (2013). Self-portrayal concerns and their relation to safety behaviors and negative affect in social anxiety disorder. Behaviour Research and Therapy, 51(8), 476–486. https://doi.org/10.1016/j.brat.2013.05.002.
Murray, S. B., Quintana, D. S., Loeb, K. L., Griffiths, S., & Le Grange, D. (2019). Treatment outcomes for anorexia nervosa: a systematic review and meta-analysis of randomized controlled trials. Psychological Medicine, 49(4), 535–544. https://doi.org/10.1017/S0033291718002088.
NICE. (2017). Eating disorders: recognition and treatment. Retrieved September 9, 2020, from https://www.nice.org.uk/guidance/ng69.
NICE. (2018). NICE guideline: post-traumatic stress disorder. Retrieved August 20, 2020, from https://www.nice.org.uk/guidance/ng116/chapter/Recommendations#management-of-ptsd-in-children-young-people-and-adults.
Olatunji, B. O., Leen-Feldner, E. W., Feldner, M. T., & Forsyth, J. P. (2007). Experimental psychopathology. In D. McKay (Ed.), Handbook of research methods in abnormal and clinical psychology (pp. 47–60). Thousand Oaks: SAGE.
Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716–aac4716. https://doi.org/10.1126/science.aac4716.
Ougrin, D. (2011). Efficacy of exposure versus cognitive therapy in anxiety disorders: systematic review and meta-analysis. BMC Psychiatry, 11(1), 200. https://doi.org/10.1186/1471-244X-11-200.
Ouimet, A. J., Ashbaugh, A. R., & Radomsky, A. S. (2019). Hoping for more: how cognitive science has and hasn’t been helpful to the OCD clinician. Clinical Psychology Review, 69, 14–29. https://doi.org/10.1016/j.cpr.2018.04.003.
Pennesi, J.-L., & Wade, T. D. (2016). A systematic review of the existing models of disordered eating: do they inform the development of effective interventions? Clinical Psychology Review, 43, 175–192. https://doi.org/10.1016/j.cpr.2015.12.004.
Phillips, K. A., Menard, W., Quinn, E., Didie, E. R., & Stout, R. L. (2013). A 4-year prospective observational follow-up study of course and predictors of course in body dysmorphic disorder. Psychological Medicine, 43(5), 1109–1117. https://doi.org/10.1017/S0033291712001730.
Pieterse, A. L., Todd, N. R., Neville, H. A., & Carter, R. T. (2012). Perceived racism and mental health among Black American adults: a meta-analytic review. Journal of Counseling Psychology, 59(1), 1–9. https://doi.org/10.1037/a0026208.
Price, R. B., Wallace, M., Kuckertz, J. M., Amir, N., Graur, S., Cummings, L., Popa, P., Carlbring, P., & Bar-Haim, Y. (2016). Pooled patient-level meta-analysis of children and adults completing a computer-based anxiety intervention targeting attentional bias. Clinical Psychology Review, 50, 37–49. https://doi.org/10.1016/j.cpr.2016.09.009.
Rachman, S. (1984). Agoraphobia—a safety-signal perspective. Behaviour Research and Therapy, 22(1), 59–70. https://doi.org/10.1016/0005-7967(84)90033-0.
Rachman, S. (1997). A cognitive theory of obsessions. Behaviour Research and Therapy, 35(9), 793–802. https://doi.org/10.1016/s0005-7967(97)00040-5.
Rachman, S. (1998). A cognitive theory of obsessions: elaborations. Behaviour Research and Therapy, 36(4), 385–401. https://doi.org/10.1016/S0005-7967(97)10041-9.
Rachman, S., Shafran, R., Mitchell, D., Trant, J., & Teachman, B. (1996). How to remain neutral: an experimental analysis of neutralization. Behaviour Research and Therapy, 34(11), 889–898. https://doi.org/10.1016/S0005-7967(96)00051-4.
Radomsky, A. S., Shafran, R., Coughtrey, A. E., & Rachman, S. (2010). Cognitive-behavior therapy for compulsive checking in OCD. Cognitive and Behavioral Practice, 17(2), 119–131. https://doi.org/10.1016/j.cbpra.2009.10.002.
Radomsky, A. S., Giraldo-O’Meara, M., Wong, S. F., Dugas, M. J., Gelfand, L. A., Rachman, S., Schell, S., Senn, J. M., Shafran, R., & Whittal, M. L. (2020a). Cognitive therapy for compulsive checking in obsessive-compulsive disorder: a pilot trial. Psychiatry Research, 286, 112850. https://doi.org/10.1016/j.psychres.2020.112850.
Radomsky, A. S., Wong, S. F., Giraldo-O’Meara, M., Dugas, M. J., Gelfand, L. A., Myhr, G., Schell, S. E., Senn, J. M., Shafran, R., & Whittal, M. L. (2020b). When it’s at: an examination of when cognitive change occurs during cognitive therapy for compulsive checking in obsessive-compulsive disorder. Journal of Behavior Therapy and Experimental Psychiatry, 67, 101442. https://doi.org/10.1016/j.jbtep.2018.12.003.
Rassin, E., Merckelbach, H., Muris, P., & Spaan, V. (1999). Thought–action fusion as a causal factor in the development of intrusions. Behaviour Research and Therapy, 37(3), 231–237. https://doi.org/10.1016/S0005-7967(98)00140-5.
Rathod, S., Gega, L., Degnan, A., Pikard, J., Khan, T., Husain, N., Munshi, T., & Naeem, F. (2018). The current status of culturally adapted mental health interventions: a practice-focused review of meta-analyses. Neuropsychiatric Disease and Treatment, 14, 165–178. https://doi.org/10.2147/NDT.S138430.
Reilly, E. E., Anderson, L. M., Gorrell, S., Schaumberg, K., & Anderson, D. A. (2017). Expanding exposure-based interventions for eating disorders. The International Journal of Eating Disorders, 50(10), 1137–1141. https://doi.org/10.1002/eat.22761.
Rodebaugh, T. L. (2004). I might look OK, but I’m still doubtful, anxious, and avoidant: the mixed effects of enhanced video feedback on social anxiety symptoms. Behaviour Research and Therapy, 42(12), 1435–1451. https://doi.org/10.1016/j.brat.2003.10.004.
Rubenstein, C. S., Peynircioglu, Z. F., Chambless, D. L., & Pigott, T. A. (1993). Memory in sub-clinical obsessive-compulsive checkers. Behaviour Research and Therapy, 31(8), 759–765. https://doi.org/10.1016/0005-7967(93)90006-G.
Salkovskis, P. M. (1985). Obsessional-compulsive problems: a cognitive-behavioural analysis. Behaviour Research and Therapy, 23(5), 571–583. https://doi.org/10.1016/0005-7967(85)90105-6.
van Schie, K., & Leer, A. (2019). Lateral eye movements do not increase false-memory rates: a failed direct-replication study. Clinical Psychological Science: A Journal of the Association for Psychological Science, 7(5), 1159–1167. https://doi.org/10.1177/2167702619859335.
Schreiber, F., Heimlich, C., Schweitzer, C., & Stangier, U. (2015). Cognitive therapy for social anxiety disorder: the impact of the “self-focused attention and safety behaviours experiment” on the course of treatment. Behavioural and Cognitive Psychotherapy, 43(02), 158–166. https://doi.org/10.1017/S1352465813000672.
Scimago Journal & Country Rank. (2019a). Journal rankings on experimental and cognitive psychology. Retrieved September 4, 2020, from https://www.scimagojr.com/journalrank.php?area=3200&category=3205.
Scimago Journal & Country Rank. (2019b). Journal rankings on clinical psychology. Retrieved September 4, 2020, from https://www.scimagojr.com/journalrank.php?area=3200&category=3203.
Shafran, R., Radomsky, A. S., Coughtrey, A. E., & Rachman, S. (2013). Advances in the cognitive behavioural treatment of obsessive compulsive disorder. Cognitive Behaviour Therapy, 42(4), 265–274. https://doi.org/10.1080/16506073.2013.773061.
Shapiro, F. (1989). Eye movement desensitization: a new treatment for post-traumatic stress disorder. Journal of Behavior Therapy and Experimental Psychiatry, 20(3), 211–217. https://doi.org/10.1016/0005-7916(89)90025-6.
Shapiro, F. (1995). Eye movement desensitization and reprocessing: basic principles, protocols, and procedures. New York: The Guilford Press.
Sher, K. J., Frost, R. O., Kushner, M., Crews, T. M., & Alexander, J. E. (1989). Memory deficits in compulsive checkers: replication and extension in a clinical sample. Behaviour Research and Therapy, 27(1), 65–69. https://doi.org/10.1016/0005-7967(89)90121-6.
Sitko, K., Bewick, B. M., Owens, D., & Masterson, C. (2020). Meta-analysis and meta-regression of cognitive behavioral therapy for psychosis (CBTp) across time: the effectiveness of CBTp has improved for delusions. Schizophrenia Bulletin Open, 1(1), sgaa023. https://doi.org/10.1093/schizbullopen/sgaa023.
Smits, J., Powers, M., Buxkamper, R., & Telch, M. (2006). The efficacy of videotape feedback for enhancing the effects of exposure-based treatment for social anxiety disorder: a controlled investigation. Behaviour Research and Therapy, 44(12), 1773–1785. https://doi.org/10.1016/j.brat.2006.01.001.
Stangier, U., Schramm, E., Heidenreich, T., Berger, M., & Clark, D. M. (2011). Cognitive therapy vs interpersonal psychotherapy in social anxiety disorder: a randomized controlled trial. Archives of General Psychiatry, 68(7), 692–700. https://doi.org/10.1001/archgenpsychiatry.2011.67.
Steinka-Fry, K. T., Tanner-Smith, E. E., Dakof, G. A., & Henderson, C. (2017). Culturally sensitive substance use treatment for racial/ethnic minority youth: a meta-analytic review. Journal of Substance Abuse Treatment, 75, 22–37. https://doi.org/10.1016/j.jsat.2017.01.006.
Stewart, C., Rogers, F., Pilch, M., Stewart, I., Barnes-Holmes, Y., & Westermann, S. (2017). The effect of social exclusion on state paranoia and explicit and implicit self-esteem in a non-clinical sample. Journal of Behavior Therapy and Experimental Psychiatry, 57, 62–69. https://doi.org/10.1016/j.jbtep.2017.04.001.
Strack, F., Martin, L. L., & Stepper, S. (1988). Inhibiting and facilitating conditions of the human smile: a nonobtrusive test of the facial feedback hypothesis. Journal of Personality and Social Psychology, 54(5), 768–777. https://doi.org/10.1037/0022-3514.54.5.768.
Summers, B. J., & Cougle, J. R. (2018). An experimental test of the role of appearance-related safety behaviors in body dysmorphic disorder, social anxiety, and body dissatisfaction. Journal of Abnormal Psychology, 127(8), 770–780. https://doi.org/10.1037/abn0000387.
The National Collaborating Centre for Mental Health. (2020). The improving access to psychological therapies manual. National Health Service. Retrieved October 9, 2020, from https://www.england.nhs.uk/wp-content/uploads/2020/05/iapt-manual-v4.pdf.
Van Etten, M. L. V., & Taylor, S. (1998). Comparative efficacy of treatments for post-traumatic stress disorder: a meta-analysis. Clinical Psychology & Psychotherapy, 5, 126–144.
Veale, D., Miles, S., Valiallah, N., Butt, S., Anson, M., Eshkevari, E., Gledhill, L. J., & Baldock, E. (2016). The effect of self-focused attention and mood on appearance dissatisfaction after mirror-gazing: an experimental study. Journal of Behavior Therapy and Experimental Psychiatry, 52, 38–44. https://doi.org/10.1016/j.jbtep.2016.03.002.
van Veen, S. C., van Schie, K., van de Schoot, R., van den Hout, M. A., & Engelhard, I. M. (2019). Making eye movements during imaginal exposure leads to short-lived memory effects compared to imaginal exposure alone. Journal of Behavior Therapy and Experimental Psychiatry, 67, 101466. https://doi.org/10.1016/j.jbtep.2019.03.001.
Wagenmakers, E.-J., Beek, T., Dijkhoff, L., Gronau, Q. F., Acosta, A., Adams, R. B., Albohn, D. N., Allard, E. S., Benning, S. D., Blouin-Hudon, E.-M., Bulnes, L. C., Caldwell, T. L., Calin-Jageman, R. J., Capaldi, C. A., Carfagno, N. S., Chasten, K. T., Cleeremans, A., Connell, L., DeCicco, J. M., et al. (2016). Registered replication report: Strack, Martin, & Stepper (1988). Perspectives on Psychological Science, 11(6), 917–928. https://doi.org/10.1177/1745691616674458.
Warnock-Parkes, E., Wild, J., Stott, R., Grey, N., Ehlers, A., & Clark, D. M. (2017). Seeing is believing: using video feedback in cognitive therapy for social anxiety disorder. Cognitive and Behavioral Practice, 24(2), 245–255. https://doi.org/10.1016/j.cbpra.2016.03.007.
Waters, A. M., LeBeau, R. T., & Craske, M. G. (2017). Experimental psychopathology and clinical psychology: an integrative model to guide clinical science and practice. Psychopathology Review, 4(2), 112–128. https://doi.org/10.5127/pr.038015.
Watkins, L. E., Sprang, K. R., & Rothbaum, B. O. (2018). Treating PTSD: a review of evidence-based psychotherapy interventions. Frontiers in Behavioral Neuroscience, 12. https://doi.org/10.3389/fnbeh.2018.00258.
Wetterneck, C. T., Little, T. E., Rinehart, K. L., Cervantes, M. E., Hyde, E., & Williams, M. (2012). Latinos with obsessive-compulsive disorder: mental healthcare utilization and inclusion in clinical trials. Journal of Obsessive-Compulsive and Related Disorders, 1(2), 85–97. https://doi.org/10.1016/j.jocrd.2011.12.001.
Wierenga, C. E., Bischoff-Grethe, A., Melrose, A. J., Irvine, Z., Torres, L., Bailer, U. F., Simmons, A., Fudge, J. L., McClure, S. M., Ely, A., & Kaye, W. H. (2015). Hunger does not motivate reward in women remitted from anorexia nervosa. Biological Psychiatry, 77(7), 642–652. https://doi.org/10.1016/j.biopsych.2014.09.024.
Wilhelm, S., Buhlmann, U., Hayward, L. C., Greenberg, J. L., & Dimaite, R. (2010). A cognitive-behavioral treatment approach for body dysmorphic disorder. Cognitive and Behavioral Practice, 17(3), 241–247. https://doi.org/10.1016/j.cbpra.2010.02.001.
Williams, K. D., Cheung, C. K. T., & Choi, W. (2000). Cyberostracism: effects of being ignored over the Internet. Journal of Personality and Social Psychology, 79(5), 748–762. https://doi.org/10.1037/0022-3514.79.5.748.
Williams, M. T., Powers, M., Yeo-Gin, Y., & Foa, E. (2010). Minority participation in randomized controlled trials for obsessive-compulsive disorder. Journal of Anxiety Disorders, 24(2), 171–177. https://doi.org/10.1016/j.janxdis.2009.11.004.
Williams, M. T., Abramowitz, J. S., & Olatunji, B. O. (2012a). The relationship between contamination cognitions, anxiety, and disgust in two ethnic groups. Journal of Behavior Therapy and Experimental Psychiatry, 43(1), 632–637. https://doi.org/10.1016/j.jbtep.2011.09.003.
Williams, M. T., Chapman, L. K., Wong, J., & Turkheimer, E. (2012b). The role of ethnic identity in symptoms of anxiety and depression in African Americans. Psychiatry Research, 199(1), 31–36. https://doi.org/10.1016/j.psychres.2012.03.049.
Williams, M. T., Jayawickreme, N., Sposato, R., & Foa, E. B. (2012c). Race-specific associations between trauma cognitions and symptoms of alcohol dependence in individuals with comorbid PTSD and alcohol dependence. Addictive Behaviors, 37(1), 47–52. https://doi.org/10.1016/j.addbeh.2011.08.008.
Williams, M. T., Proetto, D., Casiano, D., & Franklin, M. (2012d). Recruitment of a hidden population: African Americans with obsessive-compulsive disorder. Contemporary Clinical Trials, 33(1), 67–75. https://doi.org/10.1016/j.cct.2011.09.001.
Williams, M. T., Beckmann-Mendez, D. A., & Turkheimer, E. (2013a). Cultural barriers to African American participation in anxiety disorders research. Journal of the National Medical Association, 105(1), 33–41.
Williams, M. T., Tellawi, G., Wetterneck, C. T., & Chapman, L. K. (2013b). Recruitment of ethnoracial minorities for mental health research. Behavior Therapy, 36(6), 151–156.
Wilver, N. L., Summers, B. J., & Cougle, J. R. (2020). Effects of safety behavior fading on appearance concerns and related symptoms. Journal of Consulting and Clinical Psychology, 88(1), 65–74. https://doi.org/10.1037/ccp0000453.
Windheim, K., Veale, D., & Anson, M. (2011). Mirror gazing in body dysmorphic disorder and healthy controls: effects of duration of gazing. Behaviour Research and Therapy, 49(9), 555–564. https://doi.org/10.1016/j.brat.2011.05.003.
Windsor, L. C., Jemal, A., & Alessi, E. J. (2015). Cognitive behavioral therapy: a meta-analysis of race and substance use outcomes. Cultural Diversity and Ethnic Minority Psychology, 21(2), 300–313. https://doi.org/10.1037/a0037929.
Wong, S. F., & Grisham, J. R. (2017). Causal role for inverse reasoning on obsessive-compulsive symptoms: preliminary evidence from a cognitive bias modification for interpretation bias study. Journal of Behavior Therapy and Experimental Psychiatry, 57, 143–155. https://doi.org/10.1016/j.jbtep.2017.06.001.
Woods, C. M., Vevea, J. L., Chambless, D. L., & Bayen, U. J. (2002). Are compulsive checkers impaired in memory? A meta-analytic review. Clinical Psychology: Science and Practice, 9(4), 353–366. https://doi.org/10.1093/clipsy.9.4.353.
Zvolensky, M. J., Lejuez, C. W., Stuart, G. L., & Curtin, J. J. (2001). Experimental psychopathology in psychological science. Review of General Psychology, 5(4), 371–381. https://doi.org/10.1037/1089-2680.5.4.371.
Zvolensky, M. J., Forsyth, J. P., & Johnson, K. (2013). Laboratory methods in experimental psychopathology. In J. S. Comer & P. C. Kendall (Eds.), The Oxford handbook of research strategies for clinical psychology (pp. 7–23). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199793549.013.0002.
Acknowledgements
We are grateful to Olivia Gardam for her help with reference management.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Ouimet, A.J., Dixon-Luinenburg, T. & Rooyakkers, M. Experimental Psychopathology at the Crossroads: Reflections on Past, Present, and Future Contributions to Cognitive Behavioural Therapy. J Cogn Ther 14, 133–159 (2021). https://doi.org/10.1007/s41811-020-00093-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41811-020-00093-4