Abstract
Clinicians’ judgments about clients can be influenced by the causal context (e.g., life events) preceding behavioral symptoms. However, it is unclear whether this influence extends to diagnosis judgments. In diagnosing Posttraumatic Stress Disorder (PTSD), traumatic event context must be present, and severe immediate reaction context was formerly required for many years. In a vignette study, we systematically examined whether event and reaction severity influence clinicians’ open-ended diagnoses of PTSD behaviors, Major Depressive Disorder (MDD) behaviors, and nondisordered behaviors. Clinicians made more diagnoses of PTSD for all three types of behaviors (PTSD, MDD, distressed) given a traumatic event than a mildly stressful event but simultaneously found the behaviors to be less abnormal. We found no evidence that reaction context influenced diagnoses. Future directions and the role of causal context in clinical diagnosis are discussed.
What factors affect clinicians’ diagnoses? Symptomology, distress, and functional impairment clearly play central roles in influencing clinical diagnosis decisions. In the current work, we systematically examine the role of an additional factor, causal context (e.g., stressful life events preceding the client’s current behaviors). Current nosologies of mental disorder have not yet clearly articulated a prescribed model across disorders for exactly how clinicians should take stressful life events into account in general (Whooley, 2014). An important question is therefore whether, and exactly how, clinicians do take them into account. For example, if clinicians’ diagnosis decisions are systematically influenced by stressful life events, we should be able to predict when their judgments would or would not depart from the recommendations of formal nosologies such as the Diagnostic and Statistical Manual of Mental Disorders (5th ed., DSM-5, American Psychiatric Association [APA], 2013). Our understanding of the clinical utility of such recommendations may be informed, in part, by the extent to which clinicians’ judgments reliably depart from them.
We suggest that core cognitive science research on categorization and concepts may lend critical insights into better understanding this issue. Such research asks, among other key questions, how categories such as animals (e.g., humans) and artifacts (e.g., cars, computers) are mentally represented and how this representation influences the classification of new exemplars. In recent years, this work has also been directly applied to understanding how we represent and reason about undesirable states of category members (e.g., a person suffering from anorexia nervosa or a melanoma, a car with a coolant leak, a computer with a malware infection). For example, evidence suggests that we mentally represent our concepts of disorders (e.g., anorexia nervosa) as causal networks of nodes (e.g., distorted body image causes a fear of being fat) and that this representation influences how we diagnose a new patient as suffering from one or more of those disorders (e.g., taking distorted body image as stronger evidence for anorexia than a fear of being fat; Kim & Ahn, 2002).
People need not be explicitly told that an event caused certain behaviors to infer that it did. There are a number of well-documented cues-to-causality that reasoners tend to take as evidence to support a causal attribution (Einhorn & Hogarth, 1986). For example, when a candidate cause occurs before its candidate effect closely together in time and/or when a candidate cause and its candidate effect are similar or proportionate to one another (i.e., both are strong or both are weak), then we are likely to infer that there is a causal relationship between them. If we are told that Betty was fired from her longtime job (strongly negative candidate cause) shortly before she experienced a major depressive episode (strongly negative candidate effect), inferring a causal relationship is highly likely. In contrast, people are less likely to infer a causal link between a strong cause and weak effect or between a weak cause and strong effect. For example, people initially found it extremely difficult to accept the germ theory of disease because germs seem so disproportionately tiny next to their deadly effects (Medin, 1989).
Evidence suggests that the presence of causal context (i.e., a stressful life event) for disordered behaviors tends to lead clinicians to judge those behaviors as more prevalent, normal, psychologically healthy, and less in need of treatment than when not explained (Ahn, Novick, & Kim, 2003; Kim & LoSavio, 2009; Kim, Paulus, Gonzalez, & Khalife, 2012; Meehl, 1973; Weine, Kim, & Lincoln, 2016; see also Kahneman & Tversky, 1982). Ahn et al. (2003) presented clinicians with causal life-event explanations (e.g., a severe car accident) for hypothetical clients with atypical combinations of DSM (4th ed., text rev; DSM-IV-TR; APA, 2000) mental disorder symptoms (e.g., overwhelming anxiety and difficulty concentrating). They asked clinicians to rate their first impressions of each client’s psychological health on a 1–9 Likert-type scale, from very good (normal) to very poor (severely disturbed). Clinicians rated hypothetical clients as more psychologically healthy when they were presented in causal context than when they were not (Ahn et al., 2003). Following this work, Kim and LoSavio (2009) asked lay people to judge treatment need for hypothetical clients with single disorder symptoms from the DSM-IV-TR (APA, 2000). Clients’ symptoms were either presented in the context of a plausible or implausible causal event, which was either under the client’s own control (e.g., enlisting in the army) or outside the client’s control (e.g., being drafted into the army). Clients presented with a plausible, externally controlled cause received lower ratings of treatment need than the exact same clients presented with a cause that was implausible and/or internally controlled.
Kim, Paulus, Gonzalez et al. (2012) further showed that the subjective difficulty with which clinicians attribute behavioral symptoms to an external cause predicts the degree to which they perceive those behaviors as problematic. Clinicians in their study read about disordered behaviors (i.e., DSM-IV-TR symptoms of PTSD or depression) or unaffected behaviors (i.e., nondisordered behaviors) that were placed in the context of a traumatic life event (e.g., experiencing a school shooting) or a mildly distressing life event (e.g., reading a news article on a school shooting across the country). Cases in which a traumatic life event led to disordered behaviors were judged by clinicians to be easier to understand and less psychologically abnormal than cases in which a mildly distressing life event led to the exact same disordered behaviors. Furthermore, clinicians judged cases where a mildly distressing life event led to unaffected behaviors to be easier to understand and less psychologically abnormal than cases where a traumatic life event led to those exact same unaffected behaviors. In sum, clinicians found behaviors that were proportionate to preceding events in strength and in valence to be both easier to understand and less psychologically abnormal 1 than the same behaviors when they were disproportionate to preceding events (Kim, Paulus, Gonzalez et al., 2012).
One possibility, which we sought to explore in the current work, was that context plays a disorder-specific role in diagnostic judgment. For example, Kim, Paulus, Nguyen, and Gonzalez (2012) asked clinicians to make judgments of the likelihood of a depression diagnosis for sets of depression symptoms presented in either a life-event causal context, filler noncausal context, or no context. In this work, life-event causal context led clinicians to make lower judgments of depression likelihood compared to both the filler context and no context conditions. Thus, causal context influenced clinicians’ diagnostic judgments for depression behaviors, even though causal context is not part of the diagnostic criteria for this disorder. Whether and how causal context influences clinicians’ diagnostic judgments for other disordered behaviors remained an open question.
Current Work
The overarching goal of the current work was therefore to gain a clearer understanding of when and how individual clinicians take causal context into account when making diagnostic judgments. To do so, we selected two disorders to examine: one that has traditionally required causal context for formal diagnosis, and one that has not. For the first disorder, we considered Posttraumatic Stress Disorder (PTSD), Acute Stress Disorder, Adjustment Disorder, and Specific Phobia, all of which include life-event causal context in their respective DSM-5 (APA, 2013) diagnostic criteria. Specifically, PTSD, Acute Stress Disorder, and Adjustment Disorder are generally characterized as responses to a stressor in the client’s life and Specific Phobia as a negative conditioning experience with an item, person, or experience that later elicits phobic responses (DSM-5, APA, 2013).
Of these, PTSD presents the strongest case in which to investigate the influence of context on diagnostic decisions because of the unique role that context has played in formal PTSD diagnosis. Some form of a life-event stressor has been required for PTSD diagnosis from 1980 (DSM, 3rd ed.; DSM-III; APA, 1980) up through the current day (DSM-5, APA, 2013). Specifically, the DSM-5 (APA, 2013) asks clinicians to diagnose PTSD only with the following causal context (in addition to behavioral symptoms): The person was exposed to actual or threatened death, serious injury, or sexual violence (the A1 criterion). The previous DSM-IV-TR (APA, 2000) also included the causal context criterion of an immediate reaction 2 : The person’s exposure to a traumatic event was accompanied by an intense immediate reaction of fear, helplessness, or horror (the A2 criterion). Although the immediate reaction criterion was removed from the DSM-5 (see Friedman, Resick, Bryant, & Brewin, 2011), the context of a stressor (the A1 criterion) remains an absolute requirement for PTSD diagnosis in the DSM-5 (APA, 2013). 3 Thus, focusing on PTSD also enabled us to investigate for the first time whether immediate reaction context, in addition to life-event context, influences clinicians’ diagnostic judgments for disordered behaviors.
For the second disorder, we needed to ask clinicians about a disorder that traditionally does not necessarily include context (e.g., among its formal diagnostic criteria). We asked participants to reason about Major Depressive Disorder (MDD) behaviors as a comparison disorder to PTSD, as the DSM-5 does not include event or reaction context in the MDD diagnostic criteria (APA, 2013).
In addition, the literature on causal context in clinical judgment has focused on somewhat abstract judgments to date, using Likert-type scales to measure clinicians’ assessments. Doing so has allowed numerical judgments made for hypothetical clients in causal context to be compared systematically with those made in noncausal context or no context at all. Yet research using more ecologically valid measurements is needed to more closely reflect the judgments clinicians might make when assessing a client in real life. In the current work, we measured clinicians’ diagnostic judgments by asking them to provide open-ended diagnoses. In making these open-ended diagnoses, clinicians had the opportunity to provide any diagnosis of their choosing, multiple diagnoses, or no diagnoses as they saw fit for hypothetical client cases. In the diagnosis likelihood ratings measured in previous work, one might argue that clinicians may have been primed to consider particular diagnoses as candidate disorders simply because they were offered by the experimenter; this issue does not apply to the open-ended diagnoses measured in the current study. In addition, open-ended diagnoses may more closely approximate the manner in which clinicians make their diagnoses independently (i.e., before consulting with other clinicians or the client).
Thus, we investigated open-ended diagnoses of two disorders with radically differing instructions in the DSM regarding how to use causal life-event context. Taken together with past work, such measures should provide a fuller picture of how such causal context influences clinical diagnosis. We did not aim to determine whether clinicians appropriately apply DSM diagnoses to hypothetical client cases, as this latter issue has already been rigorously examined in field trials. Instead, our central goal was to gain a clearer understanding of when and how individual clinicians take causal context into account when making diagnostic judgments.
Research Questions
Our specific research questions were threefold. First, we examined whether life-event context (e.g., the A1 criterion), intense reaction context (e.g., the A2 criterion), both contexts, or neither context influence clinicians’ open-ended diagnostic decisions. This research question merits a full investigation independent of the previously discussed work documenting the effects of causal context on clinical judgments because it is unclear whether both types of causal context, events and reactions, influence clinical judgments. In past work, it was found that clinicians took events into account in diagnosis and treatment judgments (Kim & LoSavio, 2009; Kim, Paulus, Gonzalez et al., 2012; Kim, Paulus, Nguyen et al., 2012). However, whether and how clinicians make use of information about reactions, separately from and interactively with information about events, has yet to be systematically examined.
One hypothesis is that life-event context, but not immediate reaction context, influences clinical judgments of disordered behaviors. Clinicians may find behaviors to be better understood in the context of a deeper cause (e.g., event) rather than an intermediate cause (e.g., reaction) of those behaviors. Studies suggest that people are unlikely to discount the importance of a deeper cause if they view it as a root cause of the disorder (Ahn, Proctor, & Flanagan, 2009).
A plausible alternative hypothesis is that a life event and an immediate reaction together influence clinicians’ diagnoses of disordered behaviors. If this is the case, we might find that clinicians give a diagnosis only when all three elements in this causal structure are proportionately strong and negative (Einhorn & Hogarth, 1986)—that is, when they perceive the event, reaction, and behavior as causally linked.
Yet another alternative hypothesis is that intense reaction context, but not traumatic event context, influences clinicians’ diagnoses of disordered behaviors. Clinicians may find intense immediate psychological (emotional) reactions to be the most salient and useful kind of causal context when assessing behavioral symptoms. That is, they may reason that it matters most whether the client reacted to the event badly and not whether other people (e.g., the clinician) considered the event to be traumatic. Also, people’s reactions more immediately temporally precede behavioral symptoms (compared to events) and might thereby be judged more responsible for bringing the symptoms about (Einhorn & Hogarth, 1986).
And finally, it is possible that neither life-event nor immediate reaction context may influence clinicians’ diagnoses of disordered behaviors. Clinicians may simply not take contextual factors into account when providing open-ended diagnoses. Clinicians may focus primarily on the behavioral symptoms and pay little heed to context, or clinicians may attend to the context but simply not deem it relevant when choosing disorders to name.
Second, we asked whether the above effects of event and/or reaction context on open-ended diagnoses are disorder-specific. One possibility is that strongly negative causal context may increase the number of PTSD diagnoses but not affect MDD diagnoses, in line with the DSM-5 (APA, 2013) criteria for these disorders. Alternatively, causal context may affect clinicians’ diagnostic judgments of PTSD and MDD similarly. Causal context might increase diagnoses for both disorders, because a severely negative stressor may have been seen as consistent with disorder development in general (see also De Los Reyes & Marsh, 2011). Alternatively, causal context might help clinicians explain away the need for a diagnosis for any disordered behaviors, given that such context may render the behaviors less difficult to understand (Kim, Paulus, Gonzalez et al., 2012).
Third, we asked whether the proportionate-response effect would be replicated in psychological abnormality judgments and difficulty-of-understanding judgments. We expected to replicate past findings, in which PTSD and MDD behavioral symptoms were shown to seem less psychologically abnormal and less difficult to understand in proportionate than in disproportionate causal context (Kim, Paulus, Gonzalez et al., 2012), regardless of the open-ended diagnosis findings.
Method
The Northeastern University Internal Review Board approved all experiment protocols.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Ethical approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Participants
We aimed to recruit a sample of clinicians with a wide range of expertise to increase the generalizability of our findings. Thus, we recruited practicing clinical psychologists and clinical trainees separately. Given past work on this issue (e.g., Ahn et al., 2003), we did not expect to find expertise effects (although we conducted analyses with expertise as a factor to be thorough; see Results).
Practicing clinical psychologists
Fifty-six licensed practicing clinical psychologists with Ph.D. degrees (36 female) participated. A random sample of 240 clinicians was taken from 47 U.S. states via psychologytoday.com, a publicly available online directory of U.S. mental health professionals. We separately verified that each clinician had a state license in good standing. Because Alabama, Arkansas, and Minnesota did not offer free clinician licensure information online at the time of recruitment, we could not recruit from those states. To further increase our sample’s diversity, we identified 38 additional clinicians listed in the Directory of Ethnic Minority Professionals in Psychology (4th ed.; American Psychological Association, 2001). All were mailed a postcard invitation to an online study and were offered a $25 gift card to an online retailer. Those who did not respond to the first mailing were sent a follow-up postcard 2 to 4 weeks later. Out of the 278 practicing clinicians mailed postcards, 11 clinicians’ cards were returned because of a changed address, yielding a 21.0% response rate (comparable to previous work; e.g., Rottman, Ahn, Sanislow, & Kim, 2009).
Practicing clinicians reported a mean age of 50 years (range: 26–71). Most identified as White (95%). The remaining participants identified as Asian (3%) and Black (2%). In addition, 7% identified their ethnicity as Hispanic. Eighty-nine percent held a private practice. They had a median of 19 years of clinical experience (interquartile range: 12 years, 28 years) and held a range of primary clinical orientations (47% Cognitive or Cognitive-Behavioral, 18% Psychoanalytic, 18% Eclectic, 5% Humanistic, 5% Family Systems, 2% Behavioral, and 5% Other). All clinicians participated between 9 and 13 months following the DSM-5 release date.
Clinical psychology graduate trainees
Eighty clinical trainees (60 female), enrolled in APA-accredited clinical psychology graduate programs in the United States, were also recruited. Trainees were contacted by email using publicly available addresses on department websites. Participants were offered a raffle for one of several $25 gift cards to an online retailer.
Clinical trainees reported a mean age of 27 years (range: 23–35) and self-identified as White (88%), Asian (5%), Black (1%), and multiple races (4%; the remaining participants declined to report race). In addition, 8% self-identified as Hispanic. They provided psychotherapy services in psychology department clinics (39%), hospitals (18%), community clinics (11%), and other places (16%; 16% had no service experience). They had a median of 3 years of clinical experience (interquartile range: 0.9 years, 4 years) and a range of clinical orientations (69% cognitive or cognitive-behavioral, 14% behavioral, 6% eclectic, 5% psychoanalytic, 6% other). All trainees participated 9 months after the DSM-5 release.
Materials
We adapted five vignettes from Kim, Paulus, Gonzalez et al. (2012) and created a sixth; each described a hypothetical person (see Table 1 for a partial sample vignette). Each vignette contained three types of content—the event, the reaction, and the behaviors—divided into two paragraphs. The first paragraph, labeled “Recent Experiences,” included the event and the reaction descriptions, which were equated for length. These were included in the same paragraph to meet the DSM-IV-TR (APA, 2000) specification that the reaction (A2) is an immediate response to the event (A1). The second paragraph, labeled “Subsequent Behaviors,” described PTSD behavioral symptoms (excluding the A1 and A2 criteria), MDD behavioral symptoms, or distressed behaviors.
Partial Sample Vignette—“Recent Past”
Note: This is a partial vignette for illustrative purposes. The actual vignettes described 4–5 times more information. Reaction text, italicized here to differentiate it from event text, was not italicized in the experiment.
The topics of the events described in the vignettes were modified from (a) Kim, Paulus, Gonzalez et al.’s (2012) materials, (b) the DSM-IV-TR casebook (Spitzer, Gibbon, Skodol, Williams, & First, 1994), and (c) published studies of PTSD-triggering experiences (Matsakis, 1996; Rothschild, 2000). Each event was three to five sentences long and included a brief description of the person (e.g., “Lucy teaches at an elementary school”), details about a particular occurrence (e.g., “Lucy and her students were pinned down by sniper fire”), the person’s whereabouts (e.g., “on the playground”), actions (e.g., “she tried to hide”), witnessed details (e.g., “Lucy saw a child get shot”), and a brief resolution (e.g., “after the gunfire ceased”). The resolution indicated that the event was a single finite occurrence that did not continue.
The events were either traumatic (as in the example above) or everyday (nontraumatic). Traumatic events strictly adhered to the A1 criterion from both the DSM-IV-TR and DSM-5, by describing actual, threatened, or witnessed death or serious injury (e.g., experiencing a severe car accident; DSM-IV-TR, APA, 2000; DSM-5, APA, 2013). Everyday events did not describe a traumatic event and were designed to fall markedly short of meeting the A1 criterion. Four described a less severe form of a traumatic event (e.g., experiencing a fender-bender); two described a secondhand experience of a traumatic event (e.g., reading about a severe car accident in the newspaper).
The reactions were described as being either intense or mild and were all written for the current study. Reactions were tailored to the events, such that each of the two reaction vignettes was interwoven coherently with each of the two event vignettes (see Table 1). The reactions included the person’s thoughts (e.g., “she realized that safety was an illusion”), feelings (e.g., “she felt like she had failed her students”), and emotions (e.g., “she experienced an extremely heightened sense of fear”) at the time of the event. The reactions never described a disordered behavioral symptom from the DSM. For example, a person could not be described to “feel jumpy” in the reaction because an exaggerated startle reflex is a PTSD behavioral symptom (Criterion E; DSM-5, APA, 2013).
Intense reactions were defined by strict adherence to the A2 criterion in the DSM-IV-TR (APA, 2000), as a severe internal reaction of fear, helplessness, or horror encompassing thoughts, feelings, and emotions during an event (DSM-IV-TR, APA, 2000). Mild reactions did not meet the DSM-IV-TR A2 criterion but described someone’s mildly negative or neutral thoughts and feelings during the event (e.g., feelings of surprise and thoughts of concern; APA, 2000).
The final paragraph in each vignette, “Subsequent Behaviors,” appeared after “Recent Experiences” and described five symptoms or behaviors. These were either PTSD or MDD symptoms or distressed behaviors. The PTSD and MDD behavior descriptions satisfied both DSM-IV-TR (APA, 2000) and DSM-5 (APA, 2013) diagnostic criteria for behaviors (see again Note 3). We designed the PTSD vignettes such that none of the PTSD behavioral symptoms described negative alterations of cognitions and mood (Criterion D; DSM-5, APA, 2013). This criterion encompassed the intense reaction criterion (A2) in the DSM-IV-TR (APA, 2000). For MDD, we indicated that all of the behavioral symptoms continued for several months past the precipitating event to prevent the MDD vignettes from qualifying for the bereavement exclusion. Distressed behaviors were mild, nondisordered behaviors that did not meet diagnostic criteria for any disorder. The time course of all behavioral symptoms was sufficient to differentiate them from Adjustment Disorder and Acute Stress Disorder (that is, they lasted 6 months or more; DSM-IV-TR, APA, 2000; DSM-5, APA, 2013).
Manipulation check
Sixteen undergraduate psychology students (10 female) independently rated each event for how traumatic it seemed on a Likert-type scale of 1–9, where 1 = not at all traumatic and 9 = extremely traumatic, and each reaction for how severe it seemed on a Likert-type scale of 1–9, where 1 = not at all severe and 9 = extremely severe, in randomized order. In the main study, the two events and two reactions for each vignette were labeled with only one hypothetical person’s name (e.g., Lucy). Because these participants were rating four items for each vignette, we substituted three of the names with other names matched for age, gender, and country of origin (e.g., Megan, Stephanie, Heather; Social Security Administration, 2010). Paired-samples t tests revealed that traumatic events (M = 7.52, SD = .77) were perceived as more traumatic than everyday events (M = 3.67, SD = 1.34), t(15) = 9.15, p < .001. Intense reactions (M = 7.17, SD = 1.04) were perceived as more severe than mild reactions (M = 3.55, SD = .96), t(15) = 12.14, p < .001. All four means differed from the midpoint of 5, indicating, for example, that traumatic events were viewed as sufficiently traumatic; traumatic events, t(15) = 13.06, p < .001; everyday events, t(15) = −3.97, p = .001; intense reactions, t(15) = 8.36, p < .001; mild reactions, t(15) = −6.05, p < .001. Further, all means ran in the same direction across vignettes.
Design and procedure
Event type and reaction type were manipulated between-subjects such that participants read one of the two event types (traumatic or everyday) combined with one of the two reaction types (intense or mild), for all six vignettes. Behavior type was manipulated within-subjects such that participants read two vignettes describing PTSD behavioral symptoms, two describing MDD behavioral symptoms, and two describing distressed behaviors, for a total of six vignettes.
Clinicians assessed the “Subsequent Behaviors” across three judgments: open-ended diagnoses, judgments of psychological abnormality, and judgments of difficulty understanding (see complete descriptions in Table 2). The open-ended diagnosis question allowed up to three diagnoses. At least one was explicitly required, but it could be “no disorder” or “none.” The likelihood of each diagnosis was judged on a 1–9 scale, where 1 = very unlikely and 9 = very likely. Psychological abnormality judgments were on a 1–9 scale, where 1 = very psychologically normal and 9 = very psychologically abnormal. Difficulty understanding judgments were also on a 1–9 scale, where 1 = very easy to understand and 9 = very difficult to understand. Finally, an open-ended question asked, “What, if any, is your experience with the DSM-5?”
Dependent Variables
Open-ended diagnoses instructions continued: If you believe no diagnosis is warranted, please type “none” into Diagnosis Box 1 and leave the other boxes blank. If there are any other diagnoses you think are likely, please type them into the subsequent Diagnosis Boxes 2 and 3 (one diagnosis per box). Then, please rate the likelihood of each diagnosis on a scale of 1–9, where 1 = very unlikely and 9 = very likely, by selecting a rating on the scale below each diagnosis box. If you believe no diagnosis is warranted, please also rate the likelihood of this answer on the scale of 1–9, where 1 = very unlikely and 9 = very likely.
Participants received a consent form and basic instructions. They answered the same question for all six vignettes before answering another for all six, until all three questions were answered for each vignette. The order of the three questions was randomized; the order of vignettes within each dependent variable block was also randomized. Finally, participants answered questions about demographical information, provided an email address for compensation (optional), and were debriefed.
Results and Discussion
DSM-5 experience coding
To determine whether DSM-5 exposure affected diagnoses, participants’ open-ended responses to the question “What, if any, is your experience with the DSM-5 ?” were coded into three categories: no experience (e.g., “haven’t looked at it”), minimal experience (e.g., “a little”), and some experience (e.g., “couple of workshops”). Two independent coders agreed 100%. Overall, 58.93% of practicing clinicians and 45.0% of clinical trainees reported either no experience or minimal experience with the DSM-5. Reporting some experience were 41.07% of practicing clinicians and 55.0% of clinical trainees.
Judgments about PTSD behaviors
Again, our central questions were whether clinicians are influenced by event context, reaction context, or both contexts when making open-ended diagnoses and whether any such effects are disorder specific. We also checked for any effects of clinical expertise and level of DSM-5 experience. We ran four chi-square analyses (event context, reaction context, expertise, and DSM-5 experience) on each of the following: PTSD diagnoses for (1) PTSD behaviors, (2) MDD behaviors, and (3) distressed behaviors; (4) depression diagnoses for MDD behaviors; and (5) “no diagnosis” responses for distressed behaviors. Out of three diagnoses possible per vignette, we analyzed only those rated the most likely, tied for most likely, or cases where participants made a single diagnosis.
PTSD diagnosis responses for PTSD behaviors
Participants made more PTSD diagnoses for PTSD behaviors in the context of a traumatic event (75.74%) than in the context of an everyday event (47.06%), X 2(2, N = 136) = 21.43, p < .001 (see Fig. 1). However, reaction context did not affect these diagnoses, X 2(2, N = 136) = .79, p = .704; neither did expertise, X 2(2, N = 136) = 1.64, p = .440, nor DSM-5 experience, X 2(4, N = 136) = 2.75, p = .601. We also ran the above analyses using only the data from when a PTSD behavior vignette was read first. Participants still diagnosed PTSD more frequently in traumatic event context than in everyday event context, X 2(2, N = 47) = 7.88, p = .019, and were unaffected by reaction context, expertise, and DSM-5 experience (all ps ≥ .338).
Proportion of diagnoses made in traumatic event context and everyday event context by diagnosis and behavior type (PTSD diagnoses for PTSD behaviors, depression diagnoses for MDD behaviors, “none” diagnoses for distressed behaviors). Each column displays the proportion of diagnoses of interest (given for a total of 136 cases). Sixty-eight participants were in the traumatic event condition and 68 in the everyday event condition. Each participant provided diagnoses for two vignettes describing PTSD behaviors, two describing MDD behaviors, and two describing distressed behaviors.
PTSD diagnoses for PTSD behaviors were therefore affected solely by event context. Proportionately strongly negative traumatic event context, and not intense reaction context, increased the frequency of PTSD diagnoses and did not explain away diagnosis need. We speculate that clinicians may have treated the event as a deeper cause of other features of the disorder (Ahn et al., 2009), increasing its influence in diagnosis decisions. Practicing clinicians and clinical trainees differed in years of clinical experience and orientation distribution, yet these factors did not influence how frequently they gave diagnoses of PTSD for PTSD behaviors. There were also no effects of reported DSM-5 experience. Many participants were unfamiliar with the DSM-5 (APA, 2013) criteria. Interestingly, their judgments generally aligned with the DSM-5 (APA, 2013) diagnostic criteria for PTSD (an APA-defined traumatic event must be present) but did not strictly align with the DSM-IV-TR (both a traumatic event and an intense reaction must be present; APA, 2000), even though many had little to no experience with the DSM-5 (APA, 2013).
The proportionate-response effect for PTSD behaviors
We next asked whether the proportionate-response effect was replicated in psychological abnormality and difficulty understanding judgments, the dependent measures for which that effect was originally found (our third question; Kim, Paulus, Gonzalez et al., 2012). For each variable, we ran a 2 (Event Type) × 2 (Reaction Type) × 2 (Expertise) ANOVA separately for judgments about PTSD, MDD, and distressed behaviors (DSM-5 experience was not included, as it was not theoretically relevant to these judgments).
First, PTSD behaviors were judged less psychologically abnormal in proportionate severely negative traumatic event context (M = 5.10, SD = 2.12) than in disproportionate mildly negative everyday event context (M = 6.60, SD = 1.60), F(1, 128) = 22.88, p < .001, ηp2 = .15 (Table 3). We found no main effects of reaction context ( p = .788, ηp2 = .01), expertise ( p = .161, ηp2 = .02), or any interactions (all ps > .215, all ηp2 < .02).
Mean Ratings for Psychological Abnormality Judgments and Difficulty Understanding Judgments by Context and Behavior Type, With Statistical Significance of Event Type Main Effects From 2 × 2 ANOVAs
Note: Psychological abnormality judgments were made on a scale of 1–9, where 1 = very psychologically normal and 9 = very psychologically abnormal. Difficulty understanding judgments were made on a scale of 1–9, where 1 = very easy to understand and 9 = very difficult to understand.
In addition, PTSD behaviors were rated less difficult to understand in proportionate traumatic event context (M = 2.50, SD = 1.18) than in disproportionate everyday event context (M = 4.99, SD = 1.98), F(1, 128) = 78.12, p < .001, ηp2 = .38. There were no main effects of reaction context ( p = .886, ηp2 < .01) or expertise (p = .704, ηp2 < .01) and no interactions (all ps > .130, all ηp2 < .02). Taking these results together, the proportionate-response effect (Kim, Paulus, Gonzalez et al., 2012) was replicated for PTSD behaviors.
Judgments about MDD behaviors
Depression diagnosis responses for MDD behaviors
We next asked whether any context effects on diagnoses are disorder-specific or whether there are different effects on diagnoses of other disorders. To this end, we examined clinicians’ depression diagnoses for MDD disorder behaviors, as per our second research question. We did find a disorder-specific context effect: Participants’ diagnoses of depression (the appropriate diagnosis) for MDD behaviors did not differ by event context, X 2(2, N = 136) = .90, p = .637, reaction context, X 2(2, N = 136) = 4.38, p = .112, expertise, X 2(2, N = 136) = 2.39, p = .302, or DSM-5 experience, X 2(4, N = 136) = 2.57, p = .631 (see Fig. 1). There were also no effects of context, expertise, or DSM-5 experience examining only the data from when participants diagnosed a MDD vignette first (all ps ≥ .440). Participants diagnosed depression 82.35% of the time for MDD vignettes. Thus, proportionately severely negative context neither explained away depression diagnoses for MDD behaviors nor increased the likelihood that they would be given.
In past work, clinicians advocated a depression diagnosis for MDD behaviors less strongly (on a Likert-type scale) in negative life-event context than in neutral life event or no life event context (Kim, Paulus, Nguyen et al., 2012). We might not have replicated this because the control conditions differed too much across studies. The mildly negative everyday event context in this experiment (e.g., witnessing a fender bender) was arguably more negative than the neutral life events used in past work (e.g., taking one’s family to the community center; Kim, Paulus, Nguyen et al., 2012). If so, it is reasonable to expect any difference between the experimental and control conditions here to be smaller than that found in this past work. It is also possible that clinicians take context into account differently when making open-ended diagnoses as opposed to simply rating the likelihood of a depression diagnosis. Additional work is needed to explore these possibilities directly.
PTSD diagnosis responses for MDD behaviors
We also examined whether proportionate causal context similarly influences the frequency of (mistaken) PTSD diagnoses for other disordered behaviors (i.e., MDD behaviors). Participants made more PTSD diagnoses for these non-PTSD behaviors in traumatic event context (18.38%) than everyday event context (4.41%), X 2(2, N = 136) = 10.73, p = .005. There was also a marginal effect of reaction context (severe reaction = 16.91%, mild reaction = 5.88%), X 2(2, N = 136) = 5.58, p = .061, but no effects for expertise, X 2(2, N = 136) = .79, p = .704, or DSM-5 experience, X 2(4, N = 136) = 2.75, p = .601. Although it is very important to note that these percentages are considerably lower than the percentage of PTSD diagnoses for PTSD behaviors across all PTSD vignettes (75.74%), it is very interesting that both severely negative contexts (event and reaction), compared to mildly negative ones, tended to elicit more PTSD diagnoses for MDD behaviors.
The proportionate-response effect for MDD behaviors
MDD behaviors, too, were rated less psychologically abnormal in proportionate traumatic event context (M = 4.76, SD = 2.10) than in disproportionate everyday event context (M = 6.75, SD = 1.45), F(1, 52) = 40.30, p < .001, ηp2 = .24 (Table 3). There was no main effect of reaction context ( p = .486, ηp2 < .01), no main effect of expertise ( p = .410, ηp2 < .01), and no interactions (all ps > .525, all ηp2 < .01).
MDD behaviors were also rated as less difficult to understand in proportionate traumatic event context (M = 2.53, SD = 1.26) than in disproportionate everyday event context (M = 5.13, SD = 2.14), F(1, 128) = 70.97, p < .001, ηp2 = .36. We found no main effect of reaction context ( p = .627, ηp2 < .01), no main effect of expertise (p = .595, ηp2 < .01), and no interactions (all ps > .475, all ηp2 < .01). In sum, the proportionate-response effect was replicated for MDD behaviors.
Judgments about distressed behaviors
No-diagnosis responses for distressed behaviors
Overall, participants indicated “none” (i.e., no diagnosis) for distressed behavior vignettes 84.93% of the time. Participants made fewer indications that no diagnosis was necessary for distressed behaviors in the context of a traumatic event (79.41%) than in the context of an everyday event (90.44%), X 2(2, N = 136) = 12.86, p = .002 (see Fig. 1). In other words, they diagnosed a disorder more often for merely distressed behaviors when they were placed in traumatic event context than in everyday event context. (The most common diagnoses for distressed behaviors in traumatic event context were PTSD and Adjustment Disorder, both diagnosed 8% of the time.) The number of times clinicians indicated “no diagnosis” did not differ by reaction context, X 2(2, N = 136) = 1.07, p = .587, expertise, X 2(2, N = 136) = .406, p = .816, or DSM-5 experience, X 2(4, N = 136) = 4.29, p = .368. The effect of event context was also found examining only the data when participants diagnosed a distressed vignette first—event context, X 2(2, N = 45) = 6.48, p = .039 (reaction context, expertise, and DSM-5 experience: all ps ≥ .224). More work is needed to examine whether diagnoses for other disorders and other nondisordered behaviors are increased in traumatic event context. Additionally, finding that distressed behaviors are less often judged as disordered in proportionate everyday context compared to disproportionate traumatic event context is consistent with the proportionate-response effect (in which behaviors seem less psychologically abnormal and less difficult to understand in proportionate context; Kim, Paulus, Gonzalez et al., 2012).
PTSD diagnosis responses for distressed behaviors
Participants were marginally more likely to make PTSD diagnoses for distressed behaviors in traumatic event context (4.41%) than in everyday event context (0.73%), X 2(1, N = 136) = 3.76, p = .052, although this occurred very infrequently overall. Neither reaction context, X 2(1, N = 136) = .95, p = .698, expertise, X 2(1, N = 136) = .01, p = .926, nor DSM-5 experience, X 2(2, N = 136) = .63, p = .731, affected these responses. Similarly, as we saw, PTSD behaviors and MDD behaviors were also given PTSD diagnoses more often in traumatic event context. Overall, these findings may suggest that the causal context of a traumatic event may activate, in part, clinicians’ concept of PTSD, increasing the likelihood that they consider it as a possible diagnosis across behavior types.
The proportionate-response effect for distressed behaviors
Finally, distressed behaviors were seen as more psychologically abnormal in the disproportionate context of a traumatic event (M = 2.71, SD = 1.39) than in the proportionate context of an everyday event (M = 2.20, SD = 1.17), F(1, 128) = 4.21, p = .042, ηp2 = .03 (Table 3), also replicating past work. There were no main effects of reaction context ( p = .711, ηp2 < .01) or expertise ( p = .544, ηp2 < .01) and no interactions (all ps > .198, all ηp2 < .01).
In addition, distressed behaviors were rated as more difficult to understand in disproportionate traumatic event context (M = 3.03, SD = 1.82) than in proportionate everyday event context (M = 2.26, SD = 1.27), F(1, 128) = 7.61, p = .007, ηp2 = .06. There were no main effects of reaction context ( p = .455, ηp2 < .01) or expertise ( p = .696, ηp2 < .01) and no interactions (all ps > .147, all ηp2 < .01). Thus, the proportionate-response effect was replicated for distressed behaviors, just as it was for PTSD and MDD behaviors in the current study.
Other diagnosis responses
Given space to provide up to three disorder responses, participants also sometimes indicated Adjustment Disorder (78 times across all 816 vignette presentations in this study, or 10% of the time), Acute Stress Disorder (37 times, 5%), and Anxiety Disorder (38 times, 5%). Despite our efforts to restrict the time course of behavioral symptom presentation to exclude Adjustment Disorder and Acute Stress Disorder, it appears that clinicians still judged that these two other diagnoses were possible for some vignettes. It is likely that such judgments were cued by the recent past information described for each vignette, as a recent traumatic stressor is part of the DSM-IV-TR (APA, 2000) and DSM-5 (APA, 2013) criteria for Adjustment Disorder and Acute Stress Disorder. At the very least, the everyday event information described a stressor that might plausibly lead to distressed behaviors, so clinicians may have judged even the everyday events to be stressful enough to qualify someone for these diagnoses. Acute Stress Disorder shares many symptoms with PTSD (e.g., avoidance of trauma reminders, heightened startle reflex). Adjustment Disorder, PTSD, and MDD share common symptoms of distress and impairment. In sum, although these alternative diagnoses were not implausible, they were made quite infrequently, unlike diagnoses of PTSD or MDD.
General Discussion
Our findings are consistent with the notion that clinicians tend to take events, but not reactions, into account in making open-ended diagnostic judgments. Clinicians were significantly more likely to make diagnoses of PTSD in the context of a traumatic event than an everyday event, regardless of whether the vignette described PTSD, MDD, or merely distressed behaviors. We uncovered no evidence that reaction context independently or interactively influenced clinicians’ judgments of interest regarding disordered behaviors (PTSD, MDD) or distressed behaviors. Furthermore, these findings did not differ between practicing clinicians and clinical trainees.
We believe it to be unlikely that our mostly null effects of reaction information occurred because reactions in our vignettes were less salient than events; we carefully equated reactions and events for length and interspersed both throughout the “Recent Past” paragraphs. It may be that clinicians placed the most weight on the deeper cause (i.e., the event) as in prior research (Ahn et al., 2009; Yopchick & Kim, 2009), relative to the less central cause (i.e., the reaction to the event). However, we might still have expected some intermediate-strength influence of the secondary factor if this was the case, but we found no influence instead. Another possibility is that clinicians did not deem reactions to be relevant for the judgments made for PTSD behaviors because reactions are not included in the newest PTSD criteria (DSM-5, APA, 2013). However, PTSD diagnosis judgments did not differ between those who reported differing levels of DSM-5 experience. Furthermore, this logic does not apply to judgments about MDD behaviors and distressed behaviors. Yet another possibility is that clinicians’ memory is more error-prone for reactions than for events. If memories of cases are reconstructed upon retrieval from piecemeal information, clinicians may retrieve a distorted memory of the reaction (cf. Brainerd & Reyna, 2002), especially if the client’s reaction was disproportionate to the event (and/or behaviors). Some recent follow-up work strongly suggests that this may be the case (Weine & Kim, 2017).
Reconciling clinicians’ abnormality and diagnosis judgments: Future directions
In the current study, we fully replicated past work demonstrating a proportionate-response effect (Kim, Paulus, Gonzalez et al., 2012). Cases of life events and behaviors proportionate in strength and valence (e.g., a traumatic event and PTSD behaviors) were rated as less psychologically abnormal and less difficult to understand, compared to the same behaviors preceded by life events disproportionate to them in strength and valence (e.g., a nontraumatic event and PTSD behaviors).
Taken together with our open-ended diagnosis results, this finding raises the question of whether mental disorder, in clinicians’ minds, is primarily about psychological abnormality. In fact, the clinicians in our study did not necessarily seem to think so. Interestingly, clinicians diagnosed PTSD more often, and diagnosed depression no less often, in the same type of context that made these disordered behaviors seem relatively psychologically normal (i.e., traumatic event context), compared to control context (e.g., everyday event context). These findings suggest that clinicians do not base diagnoses solely upon intuitions about psychological abnormality and that they may make such diagnoses for cases they believe to be relatively normal behaviors given the type of event that occurred. The finding that clinicians are more likely to apply a PTSD diagnosis to cases they deem less psychologically abnormal is fascinating and raises a number of questions for future work that are important for both basic and applied clinical science.
In particular, future research will be needed to more comprehensively uncover whether and when clinicians are placed in the difficult position of making mental disorder diagnoses for conditions they judge to be relatively psychologically normal (i.e., given the causal context). It is important for understanding the clinical utility of such diagnostic criteria to uncover whether or not this makes the task of diagnosis subjectively more difficult for clinicians and whether this subjective difficulty influences their decisions. In the case of current mental disorders (e.g., PTSD, Acute Stress Disorder, Adjustment Disorder) and any possible future DSM mental disorders with causal context as a diagnostic criterion, it will be critical to pinpoint how well clinicians feel they are able to disregard considerations of abnormality when making diagnoses and whether they think they ought to do so.
Limitations of the current work
Our findings should be interpreted in the context of the following limitations. One is that the practicing clinical psychologists and clinical trainees in our study were self-selected to participate. Although we took considerable pains to sample clinicians from across the country and have no reason to suppose that our sample differed radically from the population of clinicians at large, they might, for example, be more accepting of experimental research than the average.
Clinicians in this study judged hypothetical others described in vignettes, albeit as realistically as possible, rather than learning about and diagnosing actual people in person, as in clinical practice. It was necessary to conduct this research with vignettes so that we could systematically control and manipulate the events, reactions, and behaviors described and draw cause-effect conclusions about their influence on clinicians’ judgments. Our open-ended diagnostic judgment measure was intended to be a more ecologically valid method than employed in past work with Likert-type scales, but additional work is needed to test for effects of context in real-time clinical situations and to determine the degree to which actual practice differs from the kinds of diagnostic decisions carried out in experimental work.
Although we found clear effects of context on judgments of psychological abnormality, study participants may have held differing ideas about the exact meaning of the term. The construct of psychological abnormality has no singular, explicit definition but rather denotes the general idea that psychologically abnormal behaviors are those that merit assessment and treatment (Bennett, 2011). We measured participants’ judgments of psychological abnormality not under the assumption that there is one correct and widely used definition but because of the common framework it denotes in clinical psychology, within which many clinicians may cultivate their own meaning through experience.
We did not measure the past diagnosis experiences of our clinicians in the current study. In future studies, such information may be helpful in determining whether clinical specialists in PTSD or MDD are influenced by causal context differently than those with more general diagnostic experiences or other disorder specialties. Furthermore, our use of six total vignettes necessarily limited the total number of judgments measured, so as to prevent attrition rates due to a lengthy overall study time. We suggest that in future work, the dependent variables employed could be expanded to include questions such as suggested treatment, other expected symptoms, and perceived symptom severity.
We did not employ vignettes describing non-A1 criterion events that have been shown to lead to PTSD behavioral symptoms (e.g., divorce; Helzer, Robins, & McEvoy, 1987; burglary without confrontation with the burglar, unemployment; Mol et al., 2005; and heterosexist discrimination; Syzmanski & Balsam, 2011). Whether clinicians make similar judgments for vignettes describing traumatic events that are excluded from DSM-5 is a possibility that can be explored directly in future work.
Conclusion
The current work allowed us to examine the case wherein context is a formal diagnostic criterion rather than background information about a client that professionals might consider. Our work suggests that including life-event context in the diagnostic criteria, as is the case for PTSD, leads to quite radically different clinical and diagnostic judgments than when it is not (e.g., in the case of MDD). Severely negative life-event context appears to be treated as having substantial causal force leading to PTSD, whereas emotional reaction context does not. Understanding the conditions under which different types of context are likely to influence their judgments may be valuable for clinicians to consider when making mental disorder diagnoses.
Footnotes
Acknowledgements
We are grateful to clinical psychologists Dawn Cisewski and Susan Nissen for valuable feedback on the wording of our vignettes and dependent measures. We thank Tara Armor, Sara Atlas, Marianna Ferris, Rachel Foulkes, Dunia Kassay, Brittney Leibert, Allison Leibold, Jordan Nissen, Katharine Quain, Alicia Rogers, Angela Saulsbery, Dillon Smith, and Cara Vincelette for help with checking materials, programming, data collection, and coding.
Author Contributions
E. R. Weine came up with the study idea. E. R. Weine and N. S. Kim developed the study concept and design. E. R. Weine carried out data collection, programming, data analysis, and interpretation under the supervision of N. S. Kim. E. R. Weine wrote the initial draft, and N. S. Kim provided critical revisions. Both authors approved the final version of the manuscript for submission.
Declaration of Conflicting Interests
The author(s) declared that there were no conflicts of interest with respect to the authorship or the publication of this article.