Abstract
Distress tolerance (DT), or the ability to withstand psychological distress, is a popular construct in the psychological literature. However, research has not specified the nomological network of DT across self-report measures. The purpose of the current investigation was to understand what personality features, environmental stressors, current affective states, and behaviors contribute to DT in two different samples: college students and those in residential substance use treatment. Correlations revealed that self-reported DT was most strongly associated with trait negative emotionality, state negative affect, impulsivity, and perceived stress. In comparisons across samples, self-harm exhibited a stronger relationship with self-reported DT in the drug treatment than in the student sample, whereas perceived stress had a stronger association in the student sample. Correlations between self-report and behavioral measures of DT were nonsignificant. To understand this lack of associations, associations of outcomes with behavioral measures were assessed. In contrast to self-reported DT, behavioral DT was more closely related to achievement orientation, state negative affect, and state positive affect, but was not significantly related to psychopathology and maladaptive behaviors. It is necessary to continue investigating the construct validity of behavioral DT measures via the use of incremental utility analyses and experimental approaches.
Keywords
Building distress tolerance (DT), or the ability to withstand aversive psychological states, is central to several modern therapeutic modalities, especially dialectical behavior therapy (Linehan, 1993). The focus on DT comes in recognition of its empirically established role in a variety of maladaptive behaviors and psychological disorders (Leyro, Zvolensky, & Bernstein, 2010). Indeed, low DT is associated with coping motives for alcohol use (Howell, Leyro, Hogan, Buckner, & Zvolensky, 2010) and problematic alcohol and marijuana use (Buckner, Keough, & Schmidt, 2007), and distinguishes substance-dependent patients from those with affective disorders (McHugh & Otto, 2012). Additionally, low DT is related to anxiety disorder symptoms (Cougle, Timpano, & Goetz, 2012; Keough, Riccardi, Timpano, Mitchell, & Schmidt, 2010; Vujanovic, Bonn-Miller, Potter, Marshall, & Zvolensky, 2011), depression symptoms (Bernstein, Zvolensky, Vujanovic, & Moos, 2009; Mahon, Yarcheski, Yarcheski, & Hanks, 2007), and borderline personality disorder features (Bornovalova, Gratz, et al., 2008; Bornovalova, Matusiewicz, & Rojas, 2008; Kiselica, Webber, & Bornovalova, 2014).
In spite of this wealth of research, our understanding of DT is limited by several factors. First, the nomological network of DT is not well established (Leyro et al., 2010; Zvolensky, Vujanovic, Bernstein, & Leyro, 2010). Individual studies often report associations of DT with specific trait and state factors, including neuroticism (Marshall-Berenz, Vujanovic, Bonn-Miller, Bernstein, & Zvolensky, 2010), impulsivity (Anestis et al., 2012), state negative affect (Simons, Gaher, Oliver, Bush, & Palmer, 2005), stressful life events (O’Cleirigh, Ironson, & Smits, 2007), and psychopathology (as discussed above). However, no study has simultaneously assessed convergent and discriminant relationships of DT with these and other potentially related constructs. Additionally, virtually no study has examined the relationship of DT with more established normal personality constructs, such as the Big 5 (Costa & McCrae, 1992) or the Big 3 (Tellegen, 1982). Thus, there is a clear need for a study that comprehensively establishes these relationships with DT.
Second, it is unclear what “makes up” DT. If DT is a normal personality feature, it is likely a compound trait, that is, one composed of several different items and factors. For instance, individuals low in DT will choose the immediate reward of relief from stress, instead of the potential long-term rewards stemming from persisting through negative affective states. Such decisions are analogous to the impulsive choices seen in delay discounting (an index of impulsivity). And indeed, a previous study reported that DT is negatively related to delay discounting, such that individuals higher in distress tolerance exhibit less delay discounting (Dennhardt & Murphy, 2011). Moreover, the ability to persist in goal-directed activity is thought to be reduced by (and therefore to be intimately related to) situational stress and negative affect (Baumeister & Alquist, 2009), an assertion that has been borne out in the experimental literature (Chan & Wan, 2012; Denson, Pedersen, Friese, Hahm, & Roberts, 2011). Thus, at the very least, DT is probably made up of impulsivity and negative affect, among a range of other possible variables. This heterogeneity is not inherently problematic. Many traits are compound (a vast array of examples can be listed, but some include resilience [Garcia-Dia, DiNapoli, Garcia-Ona, Jakubowski, & O’Flaherty, 2013], depression [Diagnostic and Statistical Manual of Mental Disorders, 4th ed., text revision, DSM-IV-TR; American Psychiatric Association, 2000], and expectancies [Reich, Below, & Goldman, 2010]), yet they are useful as clinical predictors, descriptors, and outcome measures. Indeed, a compound construct may be more useful than several unitary constructs in many cases because it accounts for a number of relevant factors simultaneously. Regardless, it is important from a scientific perspective to gain a full understanding of DT if researchers are to continue to use it as an indicator of psychological vulnerability.
A third limitation to our understanding of DT is that there is a great deal of variability in its measurement. Currently, DT can be measured with at least four behavioral tasks and three self-report measures. The behavioral measures assess persistence in difficult, negative affect–inducing tasks to earn a potential reward. These measures include the Paced Auditory Serial Addition Task (PASAT; Lejuez, Kahler, & Brown, 2003), the Mirror Tracing Persistence Task (Strong et al., 2003), the Anagram Persistence Task (Eisenberger & Leonard, 1980), and the Distress Tolerance Test (Nock & Mendes, 2008). Self-Report measures of DT, on the other hand, assess the perceived ability to tolerate stressful life circumstances or emotions, and include the Distress Tolerance Scale (DTS; Simons & Gaher, 2005), which focuses on the tolerance of general distress; the Frustration Discomfort Scale (FDS; Harrington, 2005a), which concentrates specifically on the tolerance of frustration; and the Tolerance of Negative Affective States scale (TNAS; Bernstein & Brantz, 2013), which asks about tolerance of a variety of negative emotions, including fear, social discomfort, anxiety, sadness, anger, and disgust.
A fourth issue with the DT literature is that DT has been studied in very specific samples (e.g., recovering smokers, drug abusers, college students, etc.), often using different measures to assess DT. Consequently, research has been unable to identify similar relationships of DT with other variables in two samples simultaneously. Completing this step can permit cross-validation of the DT nomological network and reduce the likelihood of chance or sample-specific findings. In other words, there is a need to be certain that a given configuration of correlations with DT is not merely due to sampling differences, measurement differences, or statistical aberration. Moreover, it is possible that DT operates differently for different populations. For instance, relationships between DT and psychopathology may be stronger in clinical than normal adult samples. In this vein, it is important to identify constructs related to DT in both normal and clinical samples.
Current Study
The current study was designed to address these limitations to our understanding of DT by examining and consolidating across three widely used self-reported DT measures. This allowed us to examine both what is common and unique to self-reported DT. We then sought to establish relationships of DT with a variety of personality, affective (state positive affect, state negative affect, and perceived stress), psychopathology, and behavioral variables. Finally, we sought to establish convergent validity with the behavioral tasks. In performing these analyses, we used two different samples: individuals in residential substance use treatment and college students. Using these different samples allowed for a highly representative demographic breakdown, a wide distribution of psychopathology, and generalization of findings. Moreover, because the groups represent extremes (i.e., one high functioning college sample with little psychopathology and another lower functioning, drug treatment sample with more frequent and severe psychopathology), consistency across samples would ensure confidence in the fact that results are not local phenomena.
Method
Participants
Participants Were Recruited From Two Distinct Populations
Sample 1
Sample 1 consisted of 227 adults (52.40% female) in residential treatment, who reported their races/ethnicities as White/Caucasian (68.30%), Black/African American (13.20%), and Latino/a (16.70%). They ranged in age from 18 to 53 years (M = 30.04, SD = 8.40), and were from a variety of family income levels, 1 including $0 to $40,000 (70.90%), $40,000 to $80,000 (11.40%), and ≥$80,000 (9.30%). Rates of individuals meeting full criteria for Axis I psychopathology in this sample were as follows (see the “Measures” section for details on how diagnoses were made): 32% met diagnostic criteria for current alcohol dependence, 88% for drug dependence, 47% for anxiety disorders, 19% for major depressive disorder. Recruitment occurred at the Drug Abuse and Comprehensive Coordinating Office, Inc., a residential treatment facility in the Tampa Bay Metropolitan Area. This office has three residential treatment facilities (one female, two male) with 57 to 80 beds per facility (a total of 205 beds). Treatment consists of a broad residential drug and alcohol rehabilitation program. During the program, participants are offered group therapy, educational programs, Bible study, literacy education, medical screening, anger management, and relapse prevention. Group treatments occur from 9 a.m. to 8 p.m. Monday through Friday, with breaks for meals and recreation. The weekends are used for free time, and often group retreats take place on Saturdays and Sundays. Drug testing is implemented throughout treatment, and a positive screen is grounds for dismissal. A typical stay lasts 6 months.
Sample 2
Sample 2 consisted of 233 students (72.5% female) from the University of South Florida, who reported their races/ethnicities as White/Caucasian (49.80%), Black/African American (17.60%), Hispanic/Latino (18.00%), Asian/Southeast Asian (12.90%), and Native American (0.90%). They ranged in age from 18 to 52 years (M = 20.28, SD = 4.09), and were from a variety of family/household income levels, including $0 to $40,000 (30.40%), $40,000 to $80,000 (37.30%), and ≥$80,000 (29.10%). Rates of individuals meeting full criteria for Axis I psychopathology in this sample were as follows (see the “Measures” section for details on how diagnoses were made): 14.60% for anxiety disorders, 7.30% for major depressive disorder, 9% for alcohol dependence, and 3% for drug dependence. These participants were recruited through the psychology department research pool.
Measures 2
Demographics
A short self-report questionnaire was administered to obtain age, sex, race/ethnicity, and total household income.
Measures of Distress Tolerance
Self-report measures
We used the FDS, DTS, and the TNAS to measure DT. The FDS consists of 35 items, which yield a total score, as well as four 7-item subscales (only the total score was used in our analyses). Individuals were asked to rate the strength of their ability to tolerate frustration on a 5-point scale with the following anchors: 1 = absent, 2 = mild, 3 = moderate, 4 = strong, 5 = very strong. The FDS differentiates individuals on the basis of self-esteem and self-control, displaying construct validity (Harrington, 2005a, 2005b). Internal consistency of the total FDS scale was .94 in both samples.
The DTS is a 15-item measure that evaluates an individual’s ability to experience and endure negative emotional states by answering on a 5-point scale (1 = strongly agree, 5 = strongly disagree). This scale has convergence with other self-report ratings of affective distress and regulation (Simons & Gaher, 2005). In addition, the DTS has demonstrated adequate 6-month test–retest reliability in previous research (r = .61; Simons & Gaher, 2005). Internal consistency was .89 in the drug using sample and .85 in the student sample.
The TNAS is made up of 25 items that ask about respondents’ ability to tolerate different negative emotions, including fear, social discomfort, anxiety, sadness, anger, and disgust. Participants rate their tolerance levels on a 5-point scale of 1 (very intolerant) to 5 (very tolerant). The TNAS has high internal consistency (α = .92), good convergent validity with other measures of DT, and good discriminant validity from measures of simple negative affect (Bernstein & Brantz, 2013). Internal consistency was .93 in the student sample and .96 in the drug treatment sample.
Behavioral measures
We assessed behavioral DT via the Paced Auditory Serial Addition Test–Computerized Version (PASAT; Lejuez et al., 2003) and the computerized Mirror Tracing Persistence Task (Strong et al., 2003). In the PASAT, numbers sequentially flashed on a computer screen and participants were asked to add the presented number to the previous, before the subsequent number appeared (the numbers only ranged from 0 to 20 with no sum > 20 to limit the confounding role of mathematical skill in distress tolerance). There were three levels with varying latencies between number presentations: one practice level (2 minutes) and two actual levels (each with a 10-minute maximum, of which the participants were uninformed). The two actual levels increased in difficulty by titrating to the average value from the practice level (e.g., at 75% titrated value if the participant’s average is 2 seconds, latency will be 1.5 seconds). For the Mirror Tracing task, participants were required to trace a red dot along the lines of a star using the computer’s mouse. The mouse moved the red dot in the reverse direction (e.g., if an individual moved the mouse to the left, the red dot moved to the right). To further increase difficulty level and frustration, if the participant stalled for more than 2 seconds or moved the red dot outside the lines of the star, a loud buzz sounded and the red dot returned to the starting position. There were two practice levels and then one actual level with a star shape.
For both tasks, once the actual levels began, participants were told they could terminate exposure to the task at any time by selecting the “quit task” button, but that the amount of money they would make by the end of the session depended on their performance on the task. Distress tolerance was indexed as latency (in seconds) to task termination. Due to potential skill differences and the potential of ceiling effects across samples, the possible maximum time allowed on the tasks varied for those in substance use treatment and those in the student sample. For the substance treatment sample, the possible quit time ranged from 0 to 300 seconds for both PASAT and Mirror Tracing; for the student sample, the possible quit time ranged from 0 to 1,200 seconds for the PASAT and 0 to 900 seconds for Mirror Tracing. This difference in administration between samples was a result of pilot testing, which suggested that most college students did not quit the task in the typically allotted amount of time. Consequently, the latency to quit was allowed to be larger in the student sample to ensure that variability in DT level was obtained.
To ensure that the tasks measured distress tolerance, rather than individual distress prior to the task, levels of anxiety, frustration, irritability, and discomfort (on a scale from 0 [none] to 100 [extreme]) were assessed via the Positive and Negative Affect Schedule (Watson, Clark, & Tellegen, 1988) pre- and midfinal level of the task for the PASAT and pre- and posttask for Mirror Tracing to confirm that the task increased levels of distress. Next, we performed four paired t-tests on negative affect before and after the PASAT and Mirror Tracing tasks. In the drug treatment sample, negative affect significantly increased on both the PASAT, t(225) = 9.90, p < .001, d = .48, and the Mirror Tracing, t(225) = 9.90, p < .001, d = .48, tasks. Similarly, in the undergraduate sample, the PASAT yielded significant increases in negative affect, t(228) = 19.28, p < .001, d = 1.39, though results were not significant with Mirror Tracing, t(228) = .180, p = .857. The order of presentation of the PASAT and Mirror Tracing tasks was counterbalanced.
Personality
To explore normative personality dimensions, we chose the Multidimensional Personality Questionnaire–Brief Form (MPQ-BF; Patrick, Curtin, & Tellegen, 2002). The MPQ-BF is a 155-item true–false scale composed of 11 subscales, including well-being (optimistic, enjoying activities), social potency (decisive, enjoy leadership), achievement (hard working, ambitious), social closeness (sociable, warm and affectionate), stress reaction (tense, nervous, easily upset), aggression (physically aggressive, victimizes others), alienation (feeling pushed around, feeling betrayed and deceived), control (cautious, planful), harm avoidance (prefers safe activities and experiences), traditionalism (high moral standards, values a good reputation), and absorption (becomes immersed in own thoughts and feelings, responsive to evocative sensory experiences). The MPQ-BF has shown strong reliability when compared with the original MPQ (Patrick et al., 2002), with coefficients ranging from .75 to .84 (Tellegen, 1982). Internal consistencies of the subscales were adequate across both samples (alphas ranged from .66 to .83), with the exception of traditionalism in the drug treatment sample (α = .41). Although we included this scale in the analyses, results for this particular scale in the drug treatment sample should be interpreted with caution.
We measured impulsivity using the UPPS-P Impulsive Behavior Scale (Cyders et al., 2007; Whiteside & Lynam, 2001). The UPPS-P measures five distinct dimensions of impulsivity: negative urgency (rash responding to negative affect), positive urgency (rash responding to positive affect), premeditation (planfulness), perseverance (willingness to see tasks through), and sensation seeking (pursuit of thrills). The UPPS correlates with measures of externalizing behaviors in college samples (J. Miller, Flory, Lynam, & Leukefeld, 2003) and effectively differentiates individuals with a range of common impulsive pathologies (e.g., pathological gambling, ADHD/conduct problems) from control groups (D. J. Miller, Derefinko, Lynam, Milich, & Fillmore, 2010; Whiteside, Lynam, Miller, & Reynolds, 2005). Because the premeditation subscale of the UPPS-P has many of the same items as the control subscale of the MPQ (five items), premeditation was not used for the analyses. Internal consistencies for the total and subscales ranged from .74 to .92 across samples.
State Affect and Stress
We used the Positive and Negative Affect Schedule (Watson et al., 1988) to measure state affect over the past week. Participants rated their current feelings for a variety of moods (e.g., “joyful” and “hostile”) from 1 (very slightly or not at all) to 5 (extremely). Overall positive and negative affect factors can be derived from summed scores on this measure. This scale demonstrates adequate reliability, as well convergent and divergent validity, across a variety of studies (Watson & Clark, 1994). Internal consistencies of both scales were high in the two samples (α ≤ .83).
Next, we chose the Perceived Stress Scale (Cohen, Kamarck, & Mermelstein, 1983) to measure perceived stress. This 14-item scale assesses the degree to which an individual perceived stress during the past month of his or her life. Participants respond to questions about how often they experience stress under different circumstances on a 5-point scale (0 = Never, 2 = Sometimes, and 4 = Very Often). Scores on the Perceived Stress Scale are validated by their associations with failure to quit smoking, inability of those afflicted with diabetes to control their sugar levels, and prevalence of symptoms of depression and health issues (Cohen et al., 1983). The Perceived Stress Scale also has good test–retest reliability (r = .86; Reis, Hino, & Rodriguez-Añez, 2010). It was internally consistent in the student (α = .85) and drug treatment samples (α = .72).
Psychopathology
We used the Personality Assessment Inventory–Borderline Features Scale (PAI-BOR; Morey, 1991) to assess severe BPD traits. It consists of 24 items that are rated on a 4-point scale, and possible total ranges from (0-3; false, slightly true, mainly true, very true), with higher total scores indicating more borderline features. The PAI-BOR is consistent with clinical interviews in identifying the presence of BPD and is concurrently valid, exhibiting correlations with suicidality, hospitalization, trauma, drug and alcohol use, and arrest (Slavin-Mulford et al., 2012; Stein, Pinsker-Aspen, & Hilsenroth, 2007). Finally, the PAI-BOR is measurement invariant across sex and age, making it useful for diverse populations (De Moor, Distel, Trull, & Boomsma, 2009). Internal consistency of this scale was adequate in the student (α = .71) and drug treatment (α = .86) samples.
Next, we administered the Mini-International Neuropsychiatric Interview (MINI; Sheehan et al., 1998), a structured diagnostic interview for DSM-IV disorders, to obtain symptom counts for the depression, anxiety disorders (general anxiety disorder, posttraumatic stress disorder, panic disorder), 3 and drug and alcohol dependence. The MINI is well validated and reliable, and is used often in both clinical and research settings (Lecrubier et al., 1997; Pinninti, Madison, Musser, & Rissmiller, 2003; Sheehan et al., 1997). All MINIs were recorded and 25% were double coded by a second rater to ensure integrity of symptom counts. Kappa values for interrater reliability ranged from .79 to 1.00 for disorder diagnoses.
Observable Behaviors
Several observable risky behaviors, including deliberate self-harm, suicide attempts, and physical fighting were assessed via the National College Health Risk Behavior Survey (Douglas et al., 1997). This measure has been shown to accurately assess risk behaviors of college students in the past year in national epidemiological studies (Douglas et al., 1997). Because of the low base rates of these behaviors in both our samples, they were coded as dichotomous (i.e., 1 if the individual reported engaging in the behavior at all in the past year and 0 if not).
Procedure
Participants from the university population were recruited via the psychology department participant pool. These participants were given extra credit toward a course grade, in addition to $20 for their participation. They were administered the MINI in person in the Psychology Department laboratory and completed surveys in writing in a private room with no research assistant present to encourage honest participation. Participants from the drug offending population were recruited via announcements at the residential addictions treatment facility. Research staff visited the site to consent participants and administer interviews and surveys. Participants were paid $20 and participation lasted 2 hours. All procedures were approved by the University of South Florida Institutional Review Board.
Results
Distress Tolerance Factor Results
We first performed a principal components analysis on the three self-report measures of DT. All scales loaded onto the same principal component, with high factor loadings in the drug treatment (DTS = .76, TNAS = .58, FDS = .56) and undergraduate samples (DTS = .73, TNAS = .78, FDS = .80). From these analyses, we extracted a DT factor, on which higher scores indicated greater ability to tolerate distress. For the two behavioral DT measures, we calculated a mean z-score for both groups to create a DT mean score (factor analysis can only be reliably used with three or more variables, so that approach was not useful for the behavioral measures). We then examined the relationships of each individual DT scale, as well the self-reported DT factor with personality, affective, psychopathology, and behavioral variables to begin establishing the nomological network of DT.
Descriptive Statistics and Cross-Sample Comparisons
Means and standard deviations for study variables are in Table 1, along with sample differences and corresponding effect sizes. The drug treatment sample exhibited significantly lower distress tolerance than the student sample across all measures, with the exception of the behavioral DT composite, on which there were no sample differences. In terms of personality, the drug treatment sample was higher than the student sample on social closeness, stress reaction, achievement, and aggression. In contrast, college students scored higher on control, well-being, and absorption. Next, individuals in substance use treatment reported being more impulsive than those from the student sample, as indexed by low perseverance and high positive and negative urgency. The drug treatment sample reported on average higher positive and negative affect, as well as perceived stress, at the time of data collection than did those in the student sample. Individuals in substance use treatment also experienced more symptoms of every form of psychopathology than the student sample. Finally, they were more likely to have been in a physical fight and engaged in a suicide attempt in the past year. Thus, in general those in the drug treatment sample represented a more stressed, alienated, emotional, and impulsive group, and were more likely to be experiencing clinically significant psychological distress and risky behavior.
Means (Standard Deviations) or Frequencies for Study Variables.
Note. TNASS = Tolerance of Negative Affective States Scale; FDS = Frustration Discomfort Scale; DTS = Distress Tolerance Scale; PASAT = Paced Auditory Serial Addition Task; MPQ = Multidimensional Personality Questionnaire. The drug treatment sample is coded as 0 and the undergraduate sample as 1; thus, negative values for T indicate that the drug treatment sample is higher.
Cells are blank because percentages serve as effect size differences for the chi-square test.
p < .05. **p < .01. ***p < .001.
Correlational Analyses
Correlations between DT measures are presented in Table 2. The self-report measures of DT were moderately correlated in the undergraduate sample and weakly correlated in the drug treatment sample, with the exception of the TNAS and FDS, which were unrelated. The behavioral measures were moderately to strongly correlated across samples, suggesting convergent validity. However, there were no significant correlations between self-reported DT and behavioral DT in either sample.
Correlations Between DT Measures by Sample.
Note. DT = distress tolerance; TNASS = Tolerance of Negative Affective States Scale; FDS = Frustrational Discomfort Scale; DTS = Distress Tolerance Scale; PASAT = Paced Auditory Serial Addition Task. Drug treatment sample results are presented above the diagonal in the gray shaded region and undergraduate sample results are presented below the diagonal in the unshaded region.
p < .05. **p < .01. ***p < .001.
Self-reported DT exhibited similar associations across samples with regard to personality, affect, and perceived stress (see Tables 3 and 4). In terms of personality, DT was moderately correlated with stress reaction and alienation across samples. Similarly, negative urgency emerged as moderately associated with DT in both the drug treatment and student samples. Perceived stress and negative affect were also related to low DT in both samples. Finally, BPD and anxiety were significantly negatively associated with DT in both samples at a moderate-to-large level. These results imply that negative affect, stress, isolation, and related forms of psychopathology are particularly relevant constructs for understanding self-reported DT.
Correlations With Self-Reported DT.
Note. DT = distress tolerance; TNASS = Tolerance of Negative Affective States Scale; FDS = Frustration Discomfort Scale; DTS = Distress Tolerance Scale; PASAT = Paced Auditory Serial Addition Task.
Denotes a significant difference in the size of the correlation between the samples.
Because only one person in this sample reported a past year suicide attempt, this variable was not used in the correlations.
p < .05. **p < .01. ***p < .001.
Correlations With Behavioral DT.
Note. DT = distress tolerance; TNASS = Tolerance of Negative Affective States Scale; FDS = Frustration Discomfort Scale; DTS = Distress Tolerance Scale; PASAT = Paced Auditory Serial Addition Task.
Denotes a significant difference in the size of the correlation between the samples.
Because only one person in this sample reported a past year suicide attempt, this variable was not used in the correlations.
p < .05. **p < .01. ***p < .001.
Because there were no significant correlations between self-reported and behavioral DT, correlations between behavioral DT and outcomes were examined. This step was designed to elucidate the lack of associations between the measurement types and better define differences between self-reported and behavioral DT. The behavioral measures of DT exhibited far fewer significant relationships than the self-report measures (see Table 4). Across samples, state negative affect was weakly but significantly correlated with DT. In the drug treatment sample, positive affect, achievement, and sensation seeking also emerged as consistent, weak correlates of behavioral DT. In contrast, in the undergraduate sample, traditionalism was the only other variable significantly associated with DT. Finally, the only significant correlation between DT and the observable outcomes was between the TNAS and deliberate self-harm in the drug treatment sample. Differences in the strength of correlations are also reported in Tables 3 and 4. The association between DT and deliberate self-harm appeared stronger in the drug treatment than the student sample; in contrast, the relationship between DT and perceived stress was stronger in the student sample.
In addition to these correlations, we examined associations of each DT measure with demographics to determine if age and sex should be included as controls in subsequent analyses. In the drug treatment sample, none of the measures of DT correlated significantly with demographics, with the exception of the FDS with sex (rpb = −.20, p = .003; female = 0, male = 1). In the undergraduate sample, age was not significantly correlated with any variables, while sex was significantly correlated with FDS (rpb = .16, p = .016), TNAS (rpb = .24, p < .001), and the DT factor (rpb = .21, p = .001). Thus, there appears to be a trend for males to have higher DT than females, but only among undergraduates.
Agreement Across Measures
We analyzed the level of agreement for DT’s relationship with other variables across measures by correlating each column of relationships with the other columns. For instance, the r values for the DTS were correlated with the r values for the FDS, TNAS, and DT factor, then this process was repeated for each measure to examine all possible relationships. In the drug treatment sample, the average agreement across self-report measures was .33 (range = .09 to .51). However, in the student sample, the average agreement was .37 (range = .12 to .83). For the behavioral measures, average agreement was .45 (range = .15 to .68) in the drug treatment sample and .54 (range = .21 to .82) in the student sample. Thus, agreement in both samples was moderate for the self-report measures and high for the behavioral measures.
Discussion
The current study aimed to examine the nomological network of self-reported DT. This study had several strengths that support the validity of our findings. First, in contrast to previous studies that focus on one measure of DT at any given time, we were able to examine the relationship of what is common rather than simply unique across DT measures with personality and psychopathology. Second, this study used two large, diverse samples that captured a wide range of demographic factors and psychopathology levels and allowed for cross-validation of findings between a clinical and normal sample. This design also ensured that our results were not an artifact of any specific sample but represent the “true state of affairs.” Third, we used a comprehensive set of demographic, personality, affective, psychopathology, and behavioral variables to examine relationships with DT. Fourth, we examined both behavioral and self-report measures of DT to gain a full understanding of the DT construct and potential differences across measurement types. Several interesting findings emerged.
First, the three different self-report measures of DT loaded highly onto the same principal component, yielding a DT factor. Despite the fact that each measure purportedly assesses a different “type” of DT (or tolerance of different types of aversive psychological states), there are commonalities across measures that can be statistically isolated. This finding is bolstered by the moderate level of agreement across the measures in terms of their relationships with other variables. This result points to the fact that there may be a latent self-report DT factor, which assesses the perceived ability to tolerate mental distress. Additionally, it evinces the utility of using multiple measures of the DT construct. Relatedly, the behavioral measures of DT were moderately intercorrelated and had moderate levels of agreement in terms of their relationships with nomological network constructs, suggesting the convergent validity and overlap of the tasks.
Second, we were able to begin to establish relationships of DT with personality, affect, psychopathology, and maladaptive behaviors. In both samples, correlational analyses suggested that self-reported DT is particularly associated with the tendency to experience stress and negative emotions (on both an immediate state level and a trait level) and to be impulsive (particularly negative urgency, or rash action in response to negative affect). Self-reported DT was also related to several forms of psychopathology across samples, including BPD features, depressive symptoms, and anxiety symptoms. These findings are consistent with previous work (Bernstein et al., 2009; Bornovalova et al., 2008; Bornovalova, Gratz, et al., 2008; Cougle et al., 2012; Keough et al., 2010; Mahon et al., 2007; Vujanovic et al., 2011). However, DT was typically not significantly related to alcohol and drug use disorder symptoms. Though previous research has found relations of DT with problematic marijuana and alcohol use (Buckner et al., 2007), some studies have failed to find an association of DT with problematic drinking (Gorka, Ali, & Daughters, 2012; Howell et al., 2010). Our finding is consistent with the latter set of studies and may suggest that DT is not an important contributor to drug and alcohol use problems. This finding may also be explained by sample characteristics. Our samples consisted of those at the low and high ends of the drug and alcohol dependence continuums; consequently, there may have been a restriction in the range of drug and alcohol severity in both samples, which precluded the discovery of systematic relationships with DT.
The results with respect to self-reported DT make sense conceptually and fit well with DT theory. Individuals who are low in DT may experience more negative affect (state negative affect) and stress (perceived stress) and react to it with aversion (stress reaction) and impulsive choices (negative urgency; Leyro et al., 2010; Zvolensky et al., 2010). These associations are also consistent with prior work (Anestis et al., 2012; Marshall-Berenz et al., 2010; Simons et al., 2005). Indeed, impulsive tendencies are thought to be caused in part by an inability to tolerate aversive psychological states (i.e., low distress tolerance), as in the negative urgency dimension of the UPPS (Whiteside & Lynam, 2001). Furthermore, those who experience persistent negative affect tend to have a reduced capacity to withstand stress (Rozanski & Kubzansky, 2005). This reduction in coping ability may be explained by the limited resource model of self-control (Baumeister & Alquist, 2009): When more resources are directed to controlling negative affect and dealing with perceived stressors, they are less available for inhibiting behavior.
The self-report measures of DT did not correlate significantly with the behavioral measures, a finding that is consistent with a number of previous studies (Anestis et al., 2012; Marshall-Berenz et al., 2010; McHugh, Daughters, et al., 2011; Schloss & Haaga, 2011). Consistent with the self-report measures, state negative affect was inversely related to quit time on the tasks across the two samples. Diverging from the self-report measures, however, the behavioral measures were related to being in a state of positive affect and having an achievement orientation (at least among those in residential substance use treatment). This finding is consistent with the “task persistence” nature of the measures, in that a belief in one’s ability to endure distress and motivation to receive a reward likely aid in continuing the tasks. In line with the differential relationships of behavioral and self-report DT with personality, affect, and psychopathology, measures from the different methods did not correlate significantly with one another. The lack of correlations across measurement types and the different nomological networks obtained for self-report and behavioral DT suggest that self-report and behavioral techniques are assessing different constructs. The behavioral tasks likely measure persistence through difficult and frustrating situations or emotions with the possibility of a later reward (Brandon et al., 2003). In contrast, self-report measures assess the perceived ability to withstand distress, without a goal-directed or reward component (Leyro et al., 2010; McHugh, et al., 2011; Zvolensky et al., 2010). To date, it is unclear which conceptualization is more accurate and has more utility. Indeed, the DT self-report measures are consistent with both the name and the theories of DT. However, it is possible that the personality correlates of behavioral DT measures in the current study are telling us something important about the nature of DT, namely, that the most effective method of tolerating distress is via a combination of positive affect and achievement orientation. As such, a clear avenue for future research is (a) to understand the relative utility of negative affect, positive affect, and achievement orientation in the conceptualization of DT; and (b) the development of self-report measures that conceptually correspond to DT behavioral tasks, and vice versa.
While acknowledging the fact that the DT behavioral and self-report measures may measure different constructs, it is also important to discuss the somewhat surprising finding that behavioral measures of DT exhibited far fewer significant correlations with personality, affective, and psychopathology variables than did the self-report measures. This may be explained in part by the problem of method variance. Indeed, all nomological network variables were measured via self-report and/or interview measures in our study, which could have lessened the likelihood of obtaining significance with a behavioral measure of DT. As a consequence, our test of the nomological network of behavioral DT may not have been as “fair” as we would have liked. An ideal test would use a methodologically balanced approach. In such an approach, self-report measures are compared with self-report measures (as was done here), and behavioral measures are tested against other behavioral measures (e.g., behavioral measures of perceived stress or impulsivity) and/or actual behaviors (e.g., treatment dropout; relapse; recidivism, suicidal behavior collected from agency, state, or hospital records, respectively).
Thus, the validity and utility of the behavioral DT tasks is ambiguous at the moment, and there is a clear need to continue investigating this question. The simplest test is incremental utility—an approach that produced some success in other bodies of literature where variables putatively measuring the same construct are uncorrelated (e.g., impulsivity, Sharma, Markon, & Clark, 2014). In other words, it is possible that, while uncorrelated, behavioral and self-report measures of DT simultaneously and incrementally predict real-world outcomes. And, indeed, in previous work predicting impulsive behaviors and posttraumatic stress disorder symptom severity, the behavioral DT tasks were significant predictors independent of self-reported DT (Anestis et al., 2012; Marshall-Berenz et al., 2010). The possibility of incremental prediction is supported by our findings, which suggest that the behavioral measures appear to be tapping into elements of extraversion (i.e., positive affect, achievement) that the self-reported tasks do not assess.
Additionally, supplements to typical correlational studies may be useful for assessing the validity of DT measures. Indeed, recent work in psychometrics has emphasized that though correlational analyses investigating the validity of measures are important, validity evidence may also be gathered through careful experimentation with the processes thought to underlie performance on a measure (Bornstein, 2011). To test the validity of DT measures, for example, a researcher might provide DT training to one group and no training to another, with the expectation that increasing one’s ability to withstand psychological distress would result in a corresponding increase in performance on the DT tasks. Likewise, to test if state negative affect partially composes DT, one might induce negative affect in one group and compare effects on DT with a control group. The behavioral DT tasks may be more suited to validity evidence of this type, as opposed to traditional correlational methods.
Although interesting, our findings need to be viewed in light of the study’s limitations. First, it was beyond the scope of this study to assess DT in relation to a number of other potentially related constructs (e.g., hardiness [Funk, 1992], grit [Duckworth, Peterson, Matthews, & Kelly, 2007], resilience [Luthar, Cicchetti, & Becker, 2000], or the five-factor personality traits [Costa & McCrae, 1992]). It is important to examine if DT is unique from these related variables to establish its independent value for psychological research. Second, as mentioned above, outcome variables were measured entirely via self-report. Use of behavioral or observable outcome measures, such as behavioral risk–taking tasks, treatment dropout, relapse, or recidivism, may represent more effective means of assessing the nomological network of behavioral DT. Third, in this study, we used a relatively high functioning college sample and a sample of drug users in residential addictions treatment, findings from which may not apply to the “general” community. Finally, the scope of this study was limited to understanding psychological DT; however, physical DT overlaps with psychological DT both empirically and conceptually and may fit into the nomological network of psychological DT (Bernstein et al., 2009; Mitchell, Riccardi, Keough, Timpano, & Schmidt, 2013).
Future research might address these limitations in several ways. First, a meta-analysis that comprehensively assesses the relationships between DT and other overlapping constructs could help enhance our understanding of DT and delineate it from other existing, well-established constructs in the literature. Second, research might examine the nomological networks of self-report and behavioral DT via multiple methods (e.g., interview, behavioral tasks, self-reports, observations) to avoid the problem of method variance. Third, assessing incremental utility of the behavioral tasks over self-report measures may elucidate their independent value. Fourth, experimental methods might be used to investigate the validity of DT. Fifth, our study could be repeated with a community sample. Finally, the study could be duplicated with measures of pain tolerance. These steps could enhance our understanding of the construct of DT, and ultimately, reduce rates of psychopathology and maladaptive behavior by informing clinical research and interventions.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.