The Utility of MMPI-2-RF Substantive Scales in Prediction of Negative Treatment Outcomes in a Community Mental Health Center

Abstract

This study examined the utility of the Minnesota Multiphasic Personality Inventory–2–Restructured Form (MMPI-2-RF) substantive scales in the prediction of premature termination and therapy no-shows while controlling for other relevant predictors in a university-based community mental health center, a sample at high risk of both premature termination and no-show appointments. Participants included 457 individuals seeking services from a university-based psychology clinic. Results indicated that Juvenile Conduct Problems (JCP) predicted premature termination and Behavioral/Externalizing Dysfunction and JCP predicted number of no-shows, when accounting for initial severity of illness, personality disorder diagnosis, therapist experience, and other related MMPI-2-RF scales. The MMPI-2-RF Aesthetic-Literary Interests scale also predicted number of no-shows. Recommendations for applying these findings in clinical practice are discussed.

Keywords

personality assessment MMPI-2-RF outpatients termination therapy no-shows

Despite the many recent advances in psychotherapy research and delivery, the problem of treatment retention remains, limiting the ultimate effectiveness of these interventions. For example, it has recently been estimated that roughly 20% of clients terminate mental health treatment prior to obtaining the full “dose” of therapy (Swift & Greenberg, 2012). Relatedly, estimates of the percentage of scheduled mental health sessions to which clients “no-show,” or who receive an irregular and inconsistent “dose” of therapy, range from 10% to 60% (e.g., Lefforge, Donohue, & Strada, 2007). Both premature termination and “no-show” appointments negatively affect the client (Klein, Stone, Hicks, & Pritchard, 2003; Knox et al., 2011; Kokotovic & Tracey, 1987; Pekarik, 1992), the provider (Klein et al., 2003; Pekarik, 1985; Piselli, Halgin, & MacEwan, 2011), and the agency (e.g., loss of productivity credits). The present study strove to contribute to the literature on both premature termination and therapy no-shows, by examining the ability of the Minnesota Multiphasic Personality Inventory–2–Restructured Form (MMPI-2-RF) to predict these outcomes while controlling for other relevant predictors in a university-based community mental health center, a sample at high risk of both premature termination and no-show appointments.

Premature termination is typically defined as the termination of treatment prior to the resolution of the presenting problem (e.g., Swift & Greenberg, 2012). A recent meta-analysis of 669 adult psychotherapy outcome studies (n = 83,834) reported a weighted mean dropout rate of 19.7%, although the authors noted that the rate of premature termination varied widely across studies, ranging from 0% to 74% (Swift & Greenberg, 2012). These authors identified a number of demographic, clinical, and treatment variables that are related to premature discontinuation of therapy (Swift & Greenberg, 2012). The more robust predictors (i.e., were significant across all levels of analysis) included clients being of younger age; clients of university-based clinics; clients with trainee therapists; clients with a diagnosis of a personality, eating, or “other” disorder (“other” disorder meaning either the diagnosis was not specific or the diagnosis did not fit into a diagnostic category in use); and utilization of a treatment that is not manualized or time-limited. Although not examined in this meta-analysis, several studies also note the importance of severity of psychopathology (e.g., lower Global Assessment of Functioning [GAF] rating) in prediction of premature termination (e.g., Klein et al., 2003; McMurran, Huband, & Overton, 2010; Minnix et al., 2005).

Fewer empirical data have been published regarding predictors of therapy session no-shows, and the studies that do exist seem to be largely found in the medical and psychiatry literature. Despite this paucity of research, evidence suggests that irregular attendance also results in negative outcomes. For example, there is a noteworthy relationship between therapy appointments spaced out over time and worse client outcome (Reardon, Cukrowicz, Reeves, & Joiner, 2002). In fact, missed appointments have been shown to be positively related to premature termination in a university psychological services center (Berrigan & Garfield, 1981), and many of the same predictors of premature termination have been implicated for “no-show” appointments, including clients of younger age (Coodin, Staley, Cortens, Desrochers, & McLandress, 2004; Fenger, Mortensen, Poulsen, & Lau, 2011; Gudjonsson et al., 2004; Kruse, Rohland, & Wu, 2002; Sharp & Hamilton, 2001) and severity of psychopathology/functional impairment (Coodin et al., 2004; Fenger et al., 2011; Gudjonsson et al., 2004; Killaspy, 2000).

Given the literature identifying robust predictors of premature termination, and to a lesser extent, no-show appointments, the current study sought to determine if the MMPI-2-RF substantive scales incrementally predicted poor treatment attendance, relative to other predictors. Harkness and Lilienfeld (1997) argued that personality assessment provides essential information for treatment planning; thus, the present study sought to determine if the personality and psychopathology information present in the MMPI-2-RF could be used as tools to identify and then strategically thwart poor treatment attendance. The MMPI and MMPI-2 are the most commonly used measures of personality and psychopathology by clinical psychologists (Camara, Nathan, & Puente, 2000), and a number of studies have reported inconsistent findings on the relationship between MMPI and MMPI-2 clinical and content scales and treatment attendance/termination in a variety of settings. In examining Clinical Scales, for example, Craig (1984) found that only Scale 2 (Depression) was predictive of premature termination in an inpatient drug abuse sample. Conversely, Walters, Solomon, and Walden (1982) found that only Scale 9 (Hypomania) was predictive of low therapy persistence (i.e., attended fewer than six sessions) in a university-based clinic. Minnix et al. (2005) reported that the total number of Clinical Scale elevations incrementally added to the prediction of premature termination in a university-based community mental health center, whereas Negative Treatment Indicator (TRT) scores did not. Chisholm, Crowther, and Ben-Porath (1997) found that only the Anxiety (ANX) and TRT Content Scales demonstrated significant relationships with progress toward therapy goals and global improvement, respectively. Hilsenroth, Handler, Toman, and Padawer (1995) found no significant mean differences between premature terminators and mutual terminators on all Clinical Scales and select Validity Scales: Lie (L), Infrequency (F), Correction (K), and Content (Ego Strength [Es], TRT).

The past decade has seen remarkable changes in the MMPI family of instruments, as researchers have attempted to address some of the shortcomings of the MMPI-2. The Restructured Clinical (RC) Scales were developed to ameliorate psychometric limitations and structural problems of the Clinical Scales (Tellegen et al., 2003). Two studies have examined treatment outcome using the RC Scales for the MMPI-2. Sellbom, Ben-Porath, Baum, Erez, and Gregory (2008) reported a relationship between dismissal from a domestic violence intervention program and both RC4 (Antisocial Behavior) and RC9 (Hypomanic Activation). Arbisi, Rusch, Polusny, Thuras, and Erbes (2013) examined the role of cynicism in veterans using an eight-item RC3 (Cynicism) proxy measure. Results indicated that predeployment cynicism was a barrier to seeking mental health treatment postdeployment for veterans diagnosed with a mental illness (Arbisi et al., 2013). The MMPI-2-RF, which replaces the Clinical Scales with the RC scales and consists of a new hierarchical structure and fewer items, was published in 2008. Only one published study to date has examined treatment outcome with the MMPI-2-RF. Mattson, Powers, Halfaker, Akeson, and Ben-Porath (2012) recently reported that scales related to antisocial behavior (BXD, RC4 [Antisocial Behavior], Juvenile Conduct Problems [JCP], Aggression [AGG], Disconstraint-Revised [DISC-r]), as well as RC8 (Aberrant Experiences), were related to noncompletion of drug court treatment at a zero-order level. In multivariate analyses when accounting for the other externalizing scales, JCP was the only significant predictor of noncompletion.

Despite the small amount of research examining the MMPI-2-RF predictors of treatment outcomes, hypotheses about treatment outcome are also found in MMPI-2-RF interpretive texts (e.g., Ben-Porath, 2012; Graham, 2012). It is important to emphasize that these are hypotheses made by the authors based on their extensive conceptual knowledge of the constructs. In both Graham (2012) and Ben-Porath (2012), statements regarding treatment outcome are made about scale elevations on numerous scales. Consistent with the findings of Mattson et al. (2012) and Sellbom et al. (2008), externalizing spectrum psychopathology has been linked to poor treatment adherence in the general psychopathology literature (e.g., de Haan, Boon, de Jong, Hoeve, & Vermeiren, 2013; Kazdin, Mazurick, & Bass, 1993; Marlowe, Kirby, Festinger, Husband, & Platt, 1997). As such, MMPI-2-RF scales assessing various aspects of externalizing traits and behaviors are theorized to also be related to lack of motivation for treatment (BXD, DISC-r), poor compliance and poor rapport (RC4), and poor treatment engagement (JCP, Activation [ACT]). Internalizing psychopathology and personality traits associated with low positive emotions, anhedonia, malaise, indecisiveness, and avoidance are conceptualized as hindrances to treatment (e.g., RC2 [Low Positive Emotions], Introversion/Low Positive Emotionality–Revised [INTR-r], Malaise [MLS], Inefficacy [NFC]), whereas those scales associated with significant distress and negative emotionality are considered to be motivators for treatment attendance (e.g., Emotional/Internalizing Dysfunction [EID], RCd [Demoralization], RC7 [Dysfunctional Negative Emotions], Negative Emotionality/Neuroticism–Revised [NEGE-r]). Scales indicative of overall poor social engagement, negative views of other people, and/or a suspicious view of others are considered harbingers of difficulties in developing therapeutic alliances (RC3, RC6 [Ideas of Persecution], Disaffiliativeness [DSF]). Likewise, elevations on scales related to disordered thought processes or excessive behavioral activation are considered to be related to a difficulty engaging in treatment (RC8, RC9 [Hypomanic Activation]).

Based on the findings in the existing literature, as well as the predictions made by interpretative texts, we hypothesized that the following 15 scales would be related to both premature termination and increased numbers of no-show appointments: BXD, RC2, RC3, RC4, RC6, RC8, RC9, MLS, NFC, JCP, AGG, ACT, DSF, DISC-r, and INTR-r. We further predicted that these relationships would be demonstrated both at a univariate level and at the multivariate level, when accounting for clinical and demographic predictors of premature termination (client age, therapist experience, severity of client psychopathology at intake, personality disorder diagnosis at intake). We sought to test these hypotheses in a setting at high risk for premature termination, according to the findings of Swift and Greenberg (2012): a university-based psychology clinic staffed by trainee therapists, seeing a clientele of relative young age. It is arguably of particular importance to identify personality and psychopathology predictors in a sample for which the risk of premature termination and no-show appointments is high across all clients. Relying on personality assessment in such samples provides time for trainee therapists to strategically implement methods shown to decrease poor treatment attendance. Furthermore, previous studies with this instrument/scales have been limited to veteran (Arbisi et al., 2013) and forensic (Mattson et al., 2012; Sellbom et al., 2008) samples, necessitating the need for examination in other settings.

Method

Participants

The current sample is an expanded version of the Minnix et al. (2005) sample. The initial sample was composed of 1,116 individuals who presented for treatment at the outpatient psychology training clinic of a large southeastern university and completed the MMPI-2 (see description of setting below). Of this sample, individuals were excluded from the analyses if they had been court-ordered for treatment (n = 104), were missing termination data (n = 482), or produced invalid MMPI-2-RF profiles (n = 169). Note that these numbers add up to a number greater than the initial sample size because of some individuals being excluded for more than one reason. Invalid MMPI-2-RF profiles were determined based on the following criteria: CNS ≥raw score 15; VRIN-r, TRIN-r, and L-r ≥80T; K-r ≥70T; F-r ≥120T; Fp-r, Fs, FBS-r ≥100T. Excluded and included participants did not significantly differ on age or marital status. Excluded clients had significantly higher GAF scores, F(1, 803) = 19.36, p < .001, η² = 0.02; attended fewer therapy sessions, F(1, 729) = 10.22, p < .001, η² = 0.01; had significantly lower levels of education, χ²(6) = 16.36, p = .012, Φ = 0.12; and were significantly more likely to be African American, χ²(5) = 19.11, p = .002, Φ = 0.14 or male, χ²(1) = 7.64, p = .006, Φ = −0.09.

The resulting sample used for analyses (n = 457) was largely female (61.7%), Caucasian (75.1%), never married (74.8%), and had completed some college education (54.9%). The sample was relatively young (mean age = 26.86 years [SD = 9.67], range = 18-63 years). Mood (46.2%) and anxiety (38.1%) disorders were the most common diagnostic categories. In regards to personality disorders, 17.4% were diagnosed with at least one disorder. Most were exhibiting “mild symptoms” or “some difficulty” in functioning (American Psychiatric Association [APA], 2000, p. 34). The range of the GAF score was 25 to 95, with mean GAF score being 61.68 (SD = 10.63).

Setting and Therapists

The clinic uses minimal exclusionary criteria. Only individuals with medically untreated psychosis or mania requiring hospitalization and/or individuals who are considered dangerous to self and/or others requiring hospitalization are referred out. The clinic is part of the Clinical Psychology doctoral training program at a large southeastern university. The Clinical Psychology program is a member of the Academy of Psychological Clinical Science (APCS). As such, only empirically supported treatment (EST) protocols are provided. These treatments are manualized, time-limited, and selected based on each individual’s diagnosis, following an intake that consists of a life history interview, a semistructured diagnostic interview (e.g., SCID-I, SCID-II), and self-report measures of personality and psychopathology (e.g., MMPI-2). All clients sign an IRB-approved informed consent form prior to intake at the clinic allowing for the research use of all treatment and assessment-related materials.

Trainees begin working in the clinic during their second year in the program. The therapists in this sample were relatively inexperienced. The median number of months worked in the clinic at time of intake was 10 (M = 11.66 months, SD = 8.49). Each trainee is supervised by a clinical psychologist and attends (at a minimum) a weekly 2-hour group supervision meeting and a weekly 1-hour didactic.

Measures

Descriptive statistics and Cronbach’s alpha (when available) for all variables are reported in Table 1. Inter-rater reliability for therapist-rated variables were unavailable; however, previous studies in this setting have reported adequate inter-rater reliability for the variables used (Minnix et al., 2005; Reardon et al., 2002).

Table 1.

Descriptive Statistics, Internal Consistency, and Correlations for Client Age, Therapist Experience, GAF, and MMPI-2-RF Scales.

	M (SD)	α	Termination type (r_pb)	No-shows (r)
PD^a	— (—)	—	0.15**	0.10*
Client age	26.86 (9.67)	—	0.01	0.05
Experience	11.66 (8.49)	—	−0.12**	−0.12*
GAF	62.85 (10.22)	—	−0.25**	−0.19**
EID	63.75 (14.10)	0.93	0.12*	0.06
THD	52.23 (10.32)	0.69	0.08	0.18**
BXD	53.56 (10.74)	0.79	0.10*	0.17**
RCd	65.42 (12.63)	0.92	0.14**	0.08
RC1	59.32 (12.41)	0.83	0.12**	0.09
RC2	61.69 (14.62)	0.81	0.12**	−0.00
RC3	53.19 (10.65)	0.80	0.06	0.11*
RC4	56.61 (11.03)	0.77	0.12*	0.17**
RC6	56.14 (10.72)	0.68	0.10*	0.12*
RC7	58.94 (12.39)	0.85	0.09*	0.05
RC8	53.87 (101.84)	0.72	0.09	0.16**
RC9	51.20 (10.95)	0.80	0.06	0.13**
MLS	62.08 (12.63)	0.75	0.12*	0.01
GIC	57.16 (11.92)	0.73	0.12*	0.04
HPC	56.51 (11.88)	0.75	0.13**	0.07
NUC	57.11 (11.92)	0.64	0.11*	0.14**
COG	63.42 (12.74)	0.74	0.09	0.08
SUI	54.63 (13.02)	0.67	0.12*	0.14**
HLP	55.04 (12.84)	0.65	0.05	0.01
SFD	60.55 (12.06)	0.83	0.09	0.05
NFC	59.05 (11.54)	0.75	0.10*	0.02
STW	60.55 (11.89)	0.65	0.13**	0.04
AXY	62.25 (13.23)	0.62	0.13**	0.04
ANP	55.40 (10.93)	0.72	0.12**	0.05
BRF	54.19 (10.95)	0.53	0.13**	0.02
MSF	46.43 (7.42)	0.65	0.02	−0.06
JCP	52.95 (11.25)	0.69	0.12*	0.17**
SUB	56.15 (11.99)	0.70	0.07	0.05
AGG	52.48 (11.04)	0.68	0.06	0.13**
ACT	50.93 (11.60)	0.68	0.06	0.07
FML	55.30 (12.18)	0.73	0.09*	0.15**
IPP	52.35 (10.78)	0.71	−0.02	−0.09
SAV	53.63 (12.89)	0.86	0.07	−0.00
SHY	53.61 (11.18)	0.79	0.06	−0.01
DSF	54.20 (10.74)	0.52	0.09	0.02
AES	47.29 (9.76)	0.56	0.02	0.13**
MEC	48.07 (9.07)	0.68	−0.07	0.08
AGGR-r	47.66 (9.92)	0.74	0.01	0.13**
PSYC-r	53.23 (10.19)	0.67	0.08	0.14**
DISC-r	53.57 (10.56)	0.75	0.05	0.14**
NEGE-r	60.54 (12.51)	0.81	0.13**	0.06
INTR-r	55.85 (13.66)	0.84	0.06	−0.02

Note. MMPI-2-RF = Minnesota Multiphasic Personality Inventory–2–Restructured Form; M (SD) = mean (standard deviation; α = Cronbach’s alpha; Termination type (r_pb) = point-biserial correlation between termination type (Premature = 1, Mutual = 0) and each variable; No-shows (r) = zero-order correlation between number of no-show appointments and each variable; PD = Personality Disorder diagnosis at intake; Experience = Number of months of therapist experience in the clinic at the time of intake; GAF = Global Assessment of Functioning rating at client intake; EID = Emotional/Internalizing Dysfunction; THD = Thought Dysfunction; BXD = Behavioral/Externalizing Dysfunction; RC = Restructured Clinical; RCd = Demoralization; RC1 = Somatic Complaints; RC2 = Low Positive Emotions; RC3 = Cynicism; RC4 = Antisocial Behavior; RC6 = Paranoia; RC7 = Dysfunctional Negative Emotion; RC8 = Aberrant Experiences; RC9 = Hypomanic Activation; MLS = Malaise; GIC = Gastrointestinal Complaints; HPC = Head Pain Complaints; NUC = Neurological Complaints; COG = Cognitive Complains; SUI = Suicidal/Death Ideation; HLP = Helplessness/Hopelessness; SFD = Self-doubt; NFC = Inefficacy; STW = Stress and Worry; AXY = Anxiety; ANP = Anger Proneness; BRF = Behavior Restricting Fears; MSF = Multiple Specific Fears; JCP = Juvenile Conduct Problems; SUB = Substance Abuse; AGG = Aggression; ACT = Activation; FML = Family Problems; IPP = Interpersonal Passivity; SAV = Social Avoidance; SHY = Shyness; DSF = Disaffiliativeness; AES = Aesthetic-Literary Interests; MEC = Mechanical-Physical Interests; AGGR-r = Aggressiveness–Revised; PSYC-r = Psychoticism–Revised; DISC-r = Disconstraint–Revised; NEGE-r = Negative Emotionality–Revised; INTR-r = Introversion/Low Positive Emotionality–Revised.

Variable is dichotomous (0,1); therefore, reporting of mean (standard deviation) is not considered useful.

p < .05. **p < .01. ***p < .001.

Predictor Variables

MMPI-2-RF

The MMPI-2-RF (Ben-Porath & Tellegen, 2008) comprises 338 items available in the MMPI-2 item pool. For the current study, participants were administered the MMPI-2 after presenting for treatment, and the MMPI-2 protocols were rescored into MMPI-2-RF scores. Previous studies have demonstrated the validity of this strategy (Tellegen & Ben-Porath, 2008; Van Der Heijden, Egger, & Derksen, 2010). Psychometric properties and external correlates for all scales are available in the MMPI-2-RF technical manual (Tellegen & Ben-Porath, 2008). Unless otherwise noted, raw scores of the MMPI-2-RF scales were used in all analyses. The MMPI-2-RF consists of 9 Validity Scales, 3 Higher-Order Scales, 9 RC Scales, 23 Specific Problems Scales, 2 Interest Scales, as well as a revised version of the Personality Psychopathology Five (PSY-5; Harkness, McNulty, & Ben-Porath, 1995). These scales provide information about an individual’s personality, psychopathology, and specific behaviors endorsed (for more information, see Ben-Porath & Tellegen, 2008). Internal consistency reliability for each scale is reported in Table 1. Cronbach’s alpha ranged from .52 (DSF) to .93 (EID).

Outcome Variables

Termination

The graduate student therapists determined termination type (Premature = 1, Mutual = 0) at the conclusion of therapy and included this information in the termination report. Fifty-seven percent (n = 260) of the sample prematurely terminated from therapy. Termination was coded as being premature when the client ended treatment against therapist recommendation because treatment goals had not been met or when a client stopped attending therapy sessions despite having a future session scheduled. Mutual termination was defined as a planned termination from treatment by both the therapist and client. Clients who terminated for an external reason, such as moving away, were coded as mutual termination.

No-show appointments

The graduate student therapists also reported number of no-show appointments in the termination report. In this sample, number of no-show appointments ranged from 0 to 10 (M = 0.91, SD = 1.56).

Control Variables

Client age

Clients provide demographic information (including age) as part of the application for treatment.

Therapist experience

Clinic staff determines the number of months of therapist experience for each client. It is defined as the number of months the therapist has been in the clinic at the time of intake. Therapists for the clients in this sample had an average of 11.66 months of experience in the clinic (range = 0-35).

Global Assessment of Functioning

The GAF is a numeric scale (0-100) indicating a client’s level of psychological, social, and occupational functioning (APA, 2000). The graduate student therapists determined GAF scores as part of the intake report, based on information gathered in the intake interview. The reliability and concurrent validity of GAF scores in this setting has been established in previous studies (Reardon et al., 2002).

Structured Clinical Interview for the DSM-IV-TR (Diagnostic and Statistical Manual of Mental Disorders [4th ed., Text Rev.]) Axis II Personality Disorders (SCID-II)

The SCID-II (First, Spitzer, Gibbon, Williams, & Smith Benjamin, 2002) is a structured clinical interview assessing the presence (=1) or absence (=0) of DSM-IV-TR personality disorders. In this setting, therapists administered modules of the SCID-II based on clients’ presenting complaints. Seventeen percent (n = 79) of the sample was diagnosed with at least one personality disorder at intake.

Data Analytic Procedures

First, to determine which MMPI-2-RF scales were related to the outcomes, correlations (point-biserial or zero-order) were calculated between all variables. Only variables significantly correlated with each outcome were retained for further analyses. Logistic regression analyses tested the hypothesis that MMPI-2-RF scales would predict premature termination, while controlling for relevant covariates. Using a hierarchical model of logistic regression, covariates were entered into Step 1 of the regression, and MMPI-2-RF scales were entered into Step 2. Termination type served as the dependent variable. Results from the second step indicate the importance of MMPI-2-RF scales in predicting premature termination. Four logistic regressions were conducted for each level of MMPI-2-RF scales: Higher Order Scales, RC Scales, Specific Problem/Interest Scales, and PSY-5 Scales. Four negative binomial regression analyses for each level of MMPI-2-RF scales were used to determine the ability of RF scales to predict number of no-show appointments (a count variable), controlling for covariates. Number of no-shows served as the dependent variable. MMPI-2-RF scales served as independent variables. Results indicate the importance of MMPI-2-RF scales in predicting no-show appointments.

Results

Premature Termination

Point-biserial correlations between premature termination and all variables are found in Table 1. Premature termination was significantly related to fewer months of therapist experience, lower GAF scores (i.e., more severe psychopathology), and diagnosis of a personality disorder. As client age was not significantly correlated to premature termination in this sample, the variable was not retained in further analyses. Twenty-one MMPI-2-RF scales were significantly, positively correlated with premature termination, including 7 of the 15 predicted scales (BXD, RC2, RC4, RC6, MLS, NFC, JCP).

To assess the extent to which these 21 MMPI-2-RF scales predicted premature termination when accounting for other predictors of premature termination, four logistic regressions were conducted (see Table 2).¹ For the Specific Problems/Interest Scales, addition of these scales in the second step did not contribute significantly to the model, Δχ²(12) = 11.26, p = .450, though the overall model was significant, χ²(15) = 52.50, p < .001. Controlling for severity at intake, personality disorder diagnosis, therapist experience, and the other correlated Specific Problems/Interest scales, JCP significantly predicted premature termination, β = 0.16, p = .025, Exp(β) = 1.18. For every one-unit increase in JCP, the log odds of premature termination increased by .18. Logistic regressions involving Higher Order, RC, and PSY-5 Scales indicated that these scales did not significantly predict premature termination, when controlling for GAF, personality disorder diagnosis, therapist experience, and the other scales. In each of the four logistic regressions, lower GAF scores, a personality disorder at intake, and fewer months of therapist experience significantly predicted premature termination.

Table 2.

MMPI-2-RF Substantive Scales Predicting Type of Termination.

Higher-Order scales
	−2 LL		528.63
	χ²(5)		43.85***
	Δχ²(2)		3.30
	B (SE)	Wald (1)	Exp(β)	95% CI
GAF	−0.05 (0.01)	16.15***	0.95	0.93-0.98
Exp.	−0.03 (0.01)	5.52*	0.97	0.95-0.99
PD	0.67 (0.29)	5.40*	1.95	1.11-3.43
EID	−0.00 (0.01)	0.06	1.00	0.97-1.02
BXD	0.05 (0.03)	3.25	1.05	1.00-1.10
Restructured Clinical scales
	−2 LL		528.71
	χ² (9)		43.77***
	Δχ²(6)		3.22
	B (SE)	Wald (1)	Exp(β)	95% CI
GAF	−0.04 (0.01)	12.76***	0.96	0.94-0.98
Exp.	−0.03 (0.01)	5.73*	0.97	0.95-0.99
PD	0.69 (0.29)	5.57*	1.99	1.12-3.54
RCd	−0.01 (0.03)	0.06	0.99	0.94-1.05
RC1	0.02 (0.03)	0.57	1.02	0.97-1.08
RC2	0.01 (0.04)	0.14	1.02	0.94-1.10
RC4	0.04 (0.03)	1.95	1.04	0.98-1.10
RC6	0.02 (0.06)	0.11	1.02	0.90-1.16
RC7	−0.01 (0.03)	0.21	0.99	0.93-1.05
Specific Problem scales
	−2 LL		519.98
	χ²(15)		52.50***
	Δχ²(12)		11.95
	B (SE)	Wald (1)	Exp(β)	95% CI
GAF	−0.04 (0.01)	11.26***	0.96	0.94-0.98
Exp.	−0.03 (0.01)	5.56*	0.97	0.95-0.99
PD	0.68 (0.30)	5.18*	1.98	1.10-3.56
MLS	−0.05 (0.06)	0.62	0.95	0.85-1.08
GIC	0.04 (0.09)	0.23	1.05	0.87-1.25
HPC	0.05 (0.08)	0.49	1.05	0.91-1.22
NUC	−0.01 (0.07)	0.01	0.99	0.87-1.14
SUI	0.07 (0.12)	0.35	1.07	0.85-1.35
NFC	−0.02 (0.05)	0.13	0.98	0.88-1.09
STW	0.08 (0.08)	0.86	1.08	0.92-1.27
AXY	0.07 (0.12)	0.00	1.01	0.80-1.26
ANP	0.02 (0.06)	0.17	1.02	0.91-1.15
BRF	0.13 (0.10)	1.81	1.14	0.94-1.38
JCP	0.16 (0.07)	5.00*	1.18	1.02-1.36
FML	−0.06 (0.05)	1.44	0.94	0.85-1.04
Personality Psychopathology Five scales
	−2 LL		530.62
	χ²(4)		41.85***
	Δχ²(1)		1.31
	B (SE)	Wald (1)	Exp(β)	95% CI
GAF	−0.05 (0.01)	16.11***	0.96	0.94-0.98
Exp.	−0.03 (0.01)	5.79*	0.97	0.95-0.99
PD	0.68 (0.29)	5.55*	1.97	1.12-3.45
NEGE-r	0.03 (0.03)	1.31	1.03	0.98-1.08

Note. These are data from Step 2 of 4 separate logistic regressions. GAF and therapist experience were entered in Step 1. MMPI-2-RF = Minnesota Multiphasic Personality Inventory–2–Restructured Form; MMPI-2-RF scales were entered in Step 2. −2LL = −2 log likelihood; χ² = overall chi-square for the model; B(SE) = coefficient (standard error) for predicting the dependent variable from each independent variable; Wald (1) = Wald chi-square (degrees of freedom); Exp(β) = odds ratio for each predictor; 95% CI = 95% confidence interval for each odds ratio; GAF = Global Assessment of Functioning rating at client intake; Exp. = number of months of therapist experience in the clinic at the time of intake; PD = Presence (=1) or Absence (=0) of a DSM-IV personality disorder diagnosis; DSM-IV = Diagnostic and Statistical Manual of Mental Disorders (4th ed.); EID = Emotional/Internalizing Dysfunction, BXD = Behavioral/Externalizing Dysfunction; RC = Restructured Clinical; RCd = Demoralization; RC1 = Somatic Complaints; RC2 = Low Positive Emotions; RC4 = Antisocial Behavior; RC6 = Paranoia; RC7 = Dysfunctional Negative Emotion; MLS = Malaise; GIC = Gastrointestinal Complaints; HPC = Head Pain Complaints; NUC = Neurological Complaints; SUI = Suicidal/Death Ideation; NFC = Inefficacy; STW = Stress and Worry; AXY = Anxiety; ANP = Anger Proneness; BRF = Behavior Restricting Fears; JCP = Juvenile Conduct Problems; FML = Family Problems; NEGE-r = Negative Emotionality–Revised.

p < .05. **p < .01. ***p < .001.

In order to make these findings more clinically useful, we calculated relative risk ratios (RRs) to identify the increased risk associated with elevated scores on the scales that were significantly correlated with premature termination (Table 3). We conducted RR analyses using three separate T-score cutoffs: 60, 65, and 75. RRs are considered significant if the 95% confidence interval does not contain 1. Significant RRs at varying T-score cutoffs were observed for numerous internalizing scales (EID, SUI, NFC, STW, AXY, BRF, and NEGE-r), externalizing (RC4 and JCP), and somatic (RC1, GIC, HPC, and NUC) scales, as well as RCD and RC6.

Table 3.

Relative Risk Ratios of Significantly Correlated MMPI-2-RF scales (r ≥ .10) Predicting Premature Termination.

Scale	T-score cutoff	Selection ratio (%)	Risk if elevated (%)	Risk if not elevated (%)	Relative risk	95% CI
EID	60	60.90	62.36	48.28	1.29*	1.08-1.54
	65	50.98	62.23	51.34	1.21*	1.03-1.42
	75	27.57	57.14	56.80	1.01	0.84-1.20
BXD	60	22.76	62.50	54.06	1.16	0.97-1.38
	65	18.16	65.06	55.08	1.18	0.98-1.42
	75	3.94	66.67	56.49	1.18	0.84-1.65
RCD	60	65.01	62.85	46.45	1.35*	1.12-1.63
	65	54.49	63.45	49.04	1.29*	1.10-1.52
	75	26.91	62.60	54.79	1.14	0.97-1.35
RC1	60	44.64	64.71	50.59	1.28*	1.10-1.49
	65	31.95	64.38	53.38	1.21*	1.03-1.41
	75	10.94	62.00	56.27	1.10	0.87-1.39
RC2	60	51.64	60.17	53.39	1.13	0.96-1.32
	65	43.54	60.30	54.26	1.11	0.95-1.30
	75	21.01	63.54	55.12	1.15	0.97-1.38
RC4	60	30.85	63.12	54.11	1.17	0.99-1.37
	65	18.16	66.27	54.81	1.21*	1.01-1.44
	75	7.66	68.57	55.92	1.23	0.97-1.56
RC6	60	37.20	64.12	52.61	1.22*	1.04-1.42
	65	13.79	60.32	56.35	1.07	0.86-1.33
	75	5.47	72.00	56.02	1.29	1.00-1.66
MLS	60	52.95	60.74	52.56	1.15	0.98-1.36
	65	39.17	60.89	54.32	1.12	0.96-1.31
	75	15.10	62.32	55.93	1.11	0.91-1.37
GIC	60	46.17	61.61	52.85	1.17	1.00-1.36
	65	29.76	63.97	53.89	1.19*	1.01-1.39
	75	8.32	73.68	55.37	1.33*	1.08-1.63
HPC	60	31.95	64.38	53.38	1.21*	1.03-1.31
	65	21.88	68.00	53.78	1.26*	1.08-1.49
	75	11.38	73.08	54.81	1.33*	1.11-1.60
NUC	60	32.17	63.95	53.55	1.19*	1.02-1.40
	65	19.91	65.93	54.64	1.21*	1.02-1.43
	75	7.00	59.38	56.71	1.05	0.78-1.41
SUI	60	31.73	64.14	53.53	1.20*	1.02-1.40
	65	14.00	71.88	54.45	1.32*	1.11-1.57
	75	7.79	75.00	55.34	1.36*	1.11-1.66
NFC	60	43.11	62.94	52.31	1.20*	1.03-1.41
	65	30.64	63.57	53.94	1.18*	1.01-1.38
	75	7.88	66.67	56.06	1.19	0.93-1.52
STW	60	50.11	62.88	50.88	1.24*	1.05-1.45
	65	30.42	62.59	54.40	1.15	0.98-1.35
	75	9.63	75.00	54.96	1.36*	1.13-1.64
AXY	60	44.20	64.36	50.98	1.26*	1.08-1.47
	65	44.20	64.36	50.98	1.26*	1.08-1.47
	75	23.41	64.49	54.57	1.18	1.00-1.40
BRF	60	29.32	65.67	53.25	1.23*	1.05-1.44
	65	13.35	68.85	55.05	1.25*	1.04-1.51
	75	5.91	77.78	55.58	1.40*	1.13-1.73
JCP	60	24.73	67.26	53.49	1.26*	1.07-1.47
	65	14.00	68.75	54.96	1.25*	1.04-1.51
	75	7.22	75.76	55.42	1.37*	1.11-1.68
NEGE-r	60	49.67	61.23	52.61	1.16	0.99-1.37
	65	40.04	61.20	54.01	1.13	0.97-1.33
	75	15.75	66.67	55.06	1.21*	1.01-1.46

Note. CI = confidence interval; EID = Emotional/Internalizing Dysfunction; BXD = Behavioral/Externalizing Dysfunction; RC = Restructured Clinical; RCd = Demoralization; RC1 = Somatic Complaints; RC2 = Low Positive Emotions; RC4 = Antisocial Behavior; RC6 = Paranoia; RC7 = Dysfunctional Negative Emotion; MLS = Malaise; GIC = Gastrointestinal Complaints; HPC = Head Pain Complaints; NUC = Neurological Complaints; SUI = Suicidal/Death Ideation; JCP = Juvenile Conduct Problems; AGG = Aggression; FML = Family Problems; AES = Aesthetic-Literary Interests; AGGR-r = Aggressiveness–Revised; PSYC-r = Psychoticism–Revised; DISC-r = Disconstraint–Revised.

95% CI does not include 1.00; therefore, p < .05.

No-show Appointments

Zero-order correlations between no-show appointments and all variables are in Table 1. Higher numbers of no-show appointments were associated with fewer months of therapist experience, lower GAF scores (i.e., more severe psychopathology), and diagnosis of a personality disorder. Client age was not significantly correlated with number of no-show appointments and thus was not retained in subsequent analyses. Sixteen MMPI-2-RF scales were significantly, positively correlated with number of no-show appointments, including 9 of the 15 predicted scales (BXD, RC3, RC4, RC6, RC8, RC9, JCP, DISC-r).

Four negative binomial regression analyses were conducted, with MMPI-2-RF scales predicting no-show appointments while controlling for covariates (results are presented in Table 4).² For each regression, the 95% confidence interval for the dispersion coefficient did not contain zero, indicating that the negative binomial model was more appropriate than Poisson model. For the Higher-Order Scales, BXD significantly predicted number of no-show appointments, accounting for GAF, therapist experience, personality disorder diagnosis, and THD, B(SE) = 0.04(0.02), Incident rate ratio (IRR) = 1.04. The IRR is the exponentiation of the regression coefficient, Exp(B), representing the multiplicative effect of a 1-unit change in substantive scale score on number of no-show appointments. An individual with BXD raw score of 12, for example, is expected to no-show 1.04 times as often as an individual with a raw score of 11. For the Specific Problems/Interests Scales, both JCP, B(SE) = 0.13(0.05), IRR = 1.13, and AES, B(SE) = 0.13 (0.04), IRR = 1.14, significantly predicted no-show appointments, when accounting for GAF, therapist experience, personality disorder diagnosis, and the other significantly correlated Specific Problems/Interests scales. Regression analyses regarding RC and PSY-5 scales indicated that these scales did not significantly predict number of no-show appointments when accounting for GAF, therapist experience, personality disorder diagnosis, and the other scales. In each of the four negative binomial regressions, lower GAF scores and fewer months of therapist experience, but not personality disorder diagnosis at intake, significantly predicted number of no-show appointments.

Table 4.

MMPI-2-RF Scales Predicting No-show Appointments.

	B	SE	IRR	95% CI
Higher order scales
GAF	−0.03***	0.01	0.97	0.96-0.99
Exp.	−0.02*	1.00	0.98	0.96-1.00
PD	0.20	0.19	1.22	0.83-1.78
THD	0.04	0.03	1.04	0.98-1.11
BXD	0.04*	0.02	1.04	1.00-1.08
RC scales
GAF	−0.03***	0.01	0.97	0.96-0.99
Exp.	−0.02*	0.01	0.98	0.96-1.00
PD	0.18	0.19	1.20	0.82-1.75
RC3	0.01	0.02	1.01	0.96-1.05
RC4	0.03	0.02	1.03	0.99-1.07
RC6	−0.00	0.05	1.00	0.91-1.09
RC8	0.03	0.04	1.03	0.96-1.11
RC9	0.01	0.02	1.01	0.97-1.05
Specific problems/interests scales
GAF	−0.02**	0.58	0.98	0.97-1.00
Exp.	−0.02*	0.01	0.98	0.96-1.00
PD	0.11	0.01	1.12	0.78-1.60
NUC	0.05	0.18	1.05	0.97-1.14
SUI	0.07	0.07	1.07	0.92-1.23
JCP	0.13*	0.05	1.13	1.03-1.25
AGG	−0.01	0.05	0.99	0.90-1.09
FML	0.02	0.03	1.02	0.96-1.09
AES	0.13***	0.04	1.14	1.06-1.23
PSY-5 scales
GAF	−0.03***	0.01	0.97	0.96-0.99
Exp.	−0.02*	0.01	0.98	0.96-1.00
PD	0.23	0.20	1.26	0.85-1.86
AGGR-r	0.04	0.03	1.04	0.99-1.09
PSYC-r	0.03	0.03	1.03	0.96-1.09
DISC-r	0.03	0.02	1.03	0.99-1.08

Note. These are data from 4 separate negative binomial regressions. MMPI-2-RF = Minnesota Multiphasic Personality Inventory–2–Restructured Form; B (SE) = coefficient (standard error) for predicting the dependent variable from each independent variable; IRR = incident rate ratio; 95% CI = 95% confidence interval for each IRR. GAF = Global Assessment of Functioning rating at client intake; Exp. = number of months of therapist experience in the clinic at the time of intake; PD = Presence (=1) or absence (=0) of a Diagnostic and Statistical Manual of Mental Disorders (4th ed., [DSM-IV]) personality disorder diagnosis; THD = Thought Dysfunction; BXD = Behavioral/Externalizing Dysfunction; RC = Restructured Clinical; RC3 = Cynicism; RC4 = Antisocial Behavior; RC6 = Paranoia; RC8 = Aberrant Experiences; RC9 = Hypomanic Activation, NUC = Neurological Complaints; SUI = Suicidal/Death Ideation; JCP = Juvenile Conduct Problems; AGG = Aggression; FML = Family Problems; AES = Aesthetic-Literary Interests; AGGR-r = Aggressiveness–Revised; PSYC-r = Psychoticism–Revised; DISC-r = Disconstraint–Revised.

p < .05. **p < .01. ***p < .001.

We calculated IRRs to identify the increased risk associated with elevated scores on the substantive scales that were significantly correlated with number of no-show appointments (Table 5). IRRs were calculated using three separate T-score cutoffs: 60, 65, and 75. Significant IRRs were observed for numerous externalizing scales (BXD, RC4, RC9, JCP, AGGR-r, DISC-r) and scales indicating possible thought dysfunction (THD, RC6, RC8, PSYC-r) as well as NUC, SUI, FML, and AES.

Table 5.

Incident Rate Ratios of Significantly Correlated MMPI-2-RF Scales (r ≥ .10) Predicting No-show Appointments.

Scale	T-score cutoff	Selection ratio (%)	Incident rate ratio	95% CI
THD	60	15.9	1.78*	1.14-2.77
	65	11.3	2.21*	1.41-3.47
	75	3.1	3.05*	1.43-6.51
BXD	60	25.4	1.86*	1.31-2.62
	65	18.6	1.63*	1.11-2.39
	75	4.2	2.25*	1.12-4.52
RC3	60	25.4	1.43	0.99-2.07
	65	20.7	1.48	0.99-2.22
	75	3.1	1.28	0.49-3.32
RC4	60	32.0	1.70*	1.23-2.35
	65	18.4	1.61*	1.12-2.33
	75	7.8	1.97*	1.16-3.34
RC6	60	37.2	1.32	0.95-1.85
	65	13.9	1.42	0.87-2.31
	75	5.4	1.90*	1.03-3.52
RC8	60	25.4	1.56*	1.06-2.29
	65	14.8	1.27	0.79-2.04
	75	4.5	2.19*	1.06-4.52
RC9	60	22.1	1.42	0.98-2.05
	65	14.1	1.66*	1.08-2.56
	75	0.0	—	—
NUC	60	32.7	1.44*	1.03-2.01
	65	20.2	1.61*	1.08-2.38
	75	6.8	1.57	0.84-2.94
SUI	60	32.0	1.70*	1.22-2.37
	65	14.1	1.69*	1.09-2.61
	75	8.2	1.47	0.81-2.66
JCP	60	24.7	1.99*	1.42-2.81
	65	14.4	1.86*	1.21-2.87
	75	7.8	1.93*	1.11-3.36
AGG	60	27.1	1.35	0.94-1.92
	65	15.3	1.47	0.98-2.20
	75	3.5	2.38	1.10-5.16
FML	60	32.0	1.61*	1.15-2.26
	65	21.9	1.55*	1.05-2.29
	75	6.8	1.07	0.61-1.87
AES	60	11.8	1.63*	1.04-2.55
	65	4.0	1.52	0.81-2.83
	75	0.0	—	—
AGGR-r	60	8.7	1.84*	1.08-3.14
	65	6.5	1.82	0.97-3.42
	75	1.9	2.10	0.64-6.88
PSYC-r	60	21.7	1.54*	1.01-2.37
	65	13.2	1.74*	1.10-2.77
	75	4.5	1.99	0.89-4.45
DISC-r	60	25.8	1.75*	1.23-2.49
	65	18.1	1.62*	1.10-2.38
	75	5.9	2.00*	1.13-3.54

Note. MMPI-2-RF = Minnesota Multiphasic Personality Inventory–2–Restructured Form; CI = confidence interval. THD = Thought Dysfunction; BXD = Behavioral/Externalizing Dysfunction; RC = Restructured Clinical; RC3 = Cynicism; RC4 = Antisocial Behavior; RC6 = Paranoia; RC8 = Aberrant Experiences; RC9 = Hypomanic Activation; NUC = Neurological Complaints; SUI = Suicidal/Death Ideation; JCP = Juvenile Conduct Problems; AGG = Aggression; FML = Family Problems; AES = Aesthetic-Literary Interests; AGGR-r = Aggressiveness–Revised; PSYC-r = Psychoticism–Revised; DISC-r = Disconstraint–Revised.

95% CI does not include 1.00; therefore, p < .05.

Discussion

The present study explored the relationship between MMPI-2-RF substantive scales and negative treatment outcomes, focusing on univariate relationships in addition to the predictive utility of these scales while controlling for other relevant predictors in a university-based community mental health center, a sample at high risk of both premature termination and no-show appointments. Results indicated that JCP significantly predicted premature termination and BXD, JCP, and AES significantly predicted number of no-show appointments, when accounting for initial severity of illness, personality disorder diagnosis, therapist experience, and other related MMPI-2-RF scales. In addition to these predictive findings, a number of scales were univariately correlated with one or both of the treatment outcomes examined.

Regarding the multivariate analyses, the present findings regarding the relationship between externalizing scales (JCP and BXD) and treatment-related variables corroborate the findings of Mattson et al. (2012), who found that the externalizing scales, and JCP in particular, were related to noncompletion of a drug court treatment, as well as Sellbom et al.’s (2008) findings that RC4 and RC9 predicted dismissal from a domestic violence intervention. The present study represents generalization of these findings in a university-based community mental health clinic providing ESTs. These current findings also align with the broader treatment literature that finds antisocial behavior in childhood (de Haan et al., 2013; Kazdin et al., 1993) and adulthood (Marlowe et al., 1997) predict premature termination from therapy.

On a conceptual level, these findings are consistent with the definitions of BXD and JCP found in interpretive manuals. Elevated scores on BXD have been reported to indicate significant externalizing and acting-out behaviors with empirical correlation including substance abuse, criminal offending and arrest histories, violence, abusive behavior, and impaired impulse control (Ben-Porath, 2012; Graham, 2012; Tellegen & Ben-Porath, 2008). Interpretive statements noting that individuals who elevate the BXD scale are less likely than those who do not elevate this scale to be motivated for treatment and are at risk for being noncompliant with treatment are supported by the present data (Ben-Porath, 2012; Graham, 2012). While BXD is a considered a Higher-Order scale, JCP refers to more specific externalizing behaviors. Elevated scores on JCP indicate that the individual has endorsed a history of juvenile conduct problems, which could include juvenile delinquency and truancy (Ben-Porath, 2012; Graham, 2012). Empirical correlates of the JCP scale include a history of criminal offending, problematic interpersonal relationships, that are marked with conflict, problems with authority figures, and difficulty with trust (Ben-Porath, 2012; Graham, 2012). As with BXD, interpretive statements asserting that individuals with JCP elevations are unlikely to be motivated for treatment are supported by the present data (Graham, 2012).

Unexpectedly, the AES scale was significantly predictive of number of no-shows, when accounting for other variables. Empirical correlations of AES elevations include empathy and sentience (Ben-Porath, 2012; Tellegen & Ben-Porath, 2008). A possible explanation for this unexpected finding could be that individuals high on AES are more likely to make emotion-based decisions about attending sessions or more likely to dislike more structured approaches to therapy and, thus, attend irregularly. It is possible that these individuals view some of the techniques in ESTs as threatening or as critiques and may prematurely terminate due to perceptions of therapy. Given the novel nature of this finding, replication in other samples is needed.

Other scales (RC2, RC3, RC4, RC6, RC8, RC8, RC9, MLS, AGG, ACT, DSF, DISC-r, and INTR-r) predicted to be related to treatment outcome were not predictive of either outcome, when accounting for robust predictors of treatment outcome. Controlling for established predictors of treatment outcome appears paramount for identifying the incremental validity of personality assessment in predicting treatment outcome. JCP, BXD, and AES appear to have unique relationships with treatment outcomes in an outpatient mental health setting, over and above other factors that have been associated with these outcomes. Given the inconsistencies apparent when examining the past literature on treatment and the MMPI and MMPI-2 scales (e.g., Chisholm et al., 1997; Craig, 1984; Hilsenroth et al., 1995; Minnix et al., 2005), the consistent relationship of externalizing scales on the MMPI-2-RF and treatment outcome is notable.

Given that the MMPI-2-RF scales are interpreted individually (as opposed to taking other covariates into consideration), results from the univariate analyses have implications for the application of these findings to clinical practice. Notably, three externalizing scales (BXD, RC4, JCP) as well as RC6, NUC, SUI, and FML were significantly correlated with both premature termination and number of no-show appointments. Ten internalizing (EID, RCd, RC2, RC7, NFC, STW, AXY, ANP, BRF, NEGE-r) and four somatic (RC1, MLS, GIC, HPC) scales demonstrated a statistically significant relationship with premature termination at a univariate level. In somewhat of a contrast, three scales related to thought dysfunction (THD, RC8, PSYC-r), two externalizing scales (AGG, AGG-r) as well as RC3, RC9, NUC, and AES were significant correlates of no-show appointments. These findings are of further practical interest to clinicians, as scores on many of these scales were related to an increased risk of a negative treatment outcome across three T-score cutoffs. Although these associations should not be interpreted as indicating that these MMPI-2-RF scales are independent predictors of premature termination or the number of no-show appointments, they can be useful in the context of assessing a patient during intake while taking other factors (e.g., GAF score) into consideration.

As predicted, level of the therapists’ experience, client functioning, and personality disorder diagnosis were related to both premature termination and number of no-shows. The relationship between therapist experience and treatment outcome is particularly notable in this sample, where all therapists had less than 3 years of experience. Unexpectedly, client age was found to be unrelated to either outcome in the present study. It is possible that there was less variability in the ages of the participants in the present study, given that the setting was a university training clinic, although other MMPI studies examining treatment outcome have not found a significant relationship between age and treatment outcome (e.g., Mattson et al., 2012).

The present study provides information that may aid trainee therapists working in university-based clinics, settings in which the risk of premature termination is already elevated, to identify individuals at even higher risk for premature termination and then strategically implement interventions demonstrated to prevent negative treatment outcomes. Given that resources are typically low in training clinics and the learning curve for trainee therapists is typically steep, it may be too burdensome (either financially or time-wise) to implement such strategies across the board to all clients. Selective implementation, however, based on information gleaned from the MMPI-2-RF may be a successful way to provide strategies, such as reminder letters and phone calls (Conduit, Byrne, Court, & Stefanovic, 2004; Lefforge, Donohue, & Strada, 2007; Ogrodniczuk, Joyce, & Piper, 2005; Reda, Rowett, & Makhoul, 2001), positive reinforcement (Lefforge et al., 2007), treatment contracts (Lefforge et al., 2007; Ogrodniczuk et al., 2005), and motivational interviewing (Ogrodniczuk et al., 2005; Swartz et al., 2007) to those who need it most. Such use of the MMPI-2-RF would be consistent with the arguments set forth by Harkness and Lilienfeld (1997) that personality assessment is a valuable tool in treatment planning.

Results of the current study should be interpreted in light of some limitations. In this sample, the premature termination rate of 56.9% is much higher than the overall rate reported in the Swift and Greenberg (2012) meta-analysis (19.7%); however, the rate was within the range reported by Swift and Greenberg (0% to74.23%) and consistent with, or lower than, rates reported in other training clinics (e.g., 77.5%; Callahan, Aubuchon-Endsley, Borja, & Swift, 2009). This is likely due to the presence of multiple risk factors for premature termination in this sample: university training clinic employing minimal exclusionary criteria (i.e., admitting clients with relatively severe psychopathology), admitting relatively young clients, and employing relatively inexperienced clinicians. The manner in which premature termination was determined may also have increased the rate of premature termination in this sample. Rates of premature termination are significantly higher when determined by therapist judgment (Swift & Greenberg, 2012), as was done in this study. Furthermore, the specific reasons for premature termination were not available with the current data. Future studies would benefit from more specific information about premature termination, such as examining reasons/types of premature termination (e.g., not continuing with treatment despite having another session scheduled vs. terminating treatment against the advice of the therapist). Additionally, we were unable to attain interrater reliability for the clinician-rated variables, although previous studies in this setting have reported adequate inter-rater reliability for the variables used (Minnix et al., 2005; Reardon et al., 2002). Finally, although the current study employed a large sample size, the observed effect sizes were small. Given the complex nature of a therapeutic relationship, the additional prediction of variance is meaningful. In fact, small effects are common both in the psychological assessment and medical literatures (Meyer et al., 2001). Future research would benefit by testing these hypotheses in clinics that have more experienced therapists than the student trainees in the current sample.

In sum, results from the present study indicate that elevations on JCP are related to premature termination from treatment and elevations on BXD, JCP, and AES predict the number of client no-shows in therapy, even when accounting for the impact of therapist experience, overall client functioning, personality disorder diagnosis, and other related MMPI-2-RF scales. Other predictions regarding specific MMPI-2-RF scales were not supported, suggesting that these three scales have specificity in predicting treatment outcomes, above and beyond other factors. The ability to predict which clients could be at risk for premature termination from treatment or often no-showing to therapy appointments would benefit clients, therapists, and treatment settings alike.

Footnotes

Acknowledgements

We would like to thank the following for their advice and assistance in completing this study: Michael Anestis, Paul Arbisi, Jacob Finn, Kristin Fitch, Bradley Green, and Edward Selby.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Notes

References

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., Text Rev.). Washington, DC: American Psychiatric Association.

Arbisi

P. A.

Rusch

Polusny

M. A.

Thuras

Erbes

C. R.

(2013). Does cynicism play a role in failure to obtain needed care? Mental health service utilization among returning U.S. National Guard soldiers. Psychological Assessment, 29(3), 991-996. doi:10.1037/a0032225.

Ben-Porath

Y. S.

(2012). Interpreting the MMPI-2-RF. Minneapolis: University of Minnesota Press.

Ben-Porath

Tellegen

(2008). MMPI-2-RF (Minnesota Multiphasic Personality Inventory-2 Restructured Form): Manual for administration, scoring, and interpretation. Minneapolis: University of Minnesota Press.

Berrigan

L. P.

Garfield

S. L.

(1981). Relationship of missed psychotherapy appointments to premature termination and social class. British Journal of Clinical Psychology, 20, 239-242. doi:10.1111/j.2044-8260.1981.tb00524.x

Callahan

J. L.

Aubuchon-Endsley

Borja

S. E.

Swift

J. K.

(2009). Pretreatment expectancies and premature termination in a training clinic environment. Training and Education in Professional Psychology, 3, 111-119. doi:10.1037/a0012901

Camara

W. J.

Nathan

J. S.

Puente

A. E.

(2000). Psychological test usage: Implications in professional psychology. Professional Psychology: Research and Practice, 31, 141-154. doi:10.1037/0735-7028.31.2.141

Chisholm

S. M.

Crowther

J. H.

Ben-Porath

Y. S.

(1997). Selected MMPI-2 scales’ ability to predict premature termination and outcome from psychotherapy. Journal of Personality Assessment, 69, 127-144. doi:10.1207/s15327752jpa6901_7

Conduit

Byrne

Court

Stefanovic

(2004). Non-attendance at a university-based psychology clinic: Telephone appointment reminders versus no reminders. Australian Psychologist, 39, 68-75. doi:10.1080/00050060410001660362

10.

Coodin

Staley

Cortens

Desrochers

McLandress

(2004). Patient factors associated with missed appointments in persons with schizophrenia. Canadian Journal of Psychiatry, 49, 145-148.

11.

Craig

R. J.

(1984). Can personality tests predict treatment dropouts? International Journal of Addictions, 19, 665-674.

12.

de Haan

A. M.

Boon

A. E.

de Jong

J. T. V. M.

Hoeve

Vermeiren

R. R. J. M.

(2013). A meta-analytic review on treatment dropout in child and adolescent outpatient mental health care. Clinical Psychology Review, 33, 698-711. doi:10.1016/j.cpr.2013.04.005

13.

Fenger

Mortensen

E. L.

Poulsen

Lau

(2011). No-shows, drop-outs and completers in psychotherapeutic treatment: Demographic and clinical predictors in a large sample of non-psychotic patients. Nordic Journal of Psychiatry, 65, 183-191. doi:10.3109/08039488.2010.515687

14.

First

M. B.

Gibbon

Spitzer

R. L.

Williams

J. B.

Smith Benjamin

(1997). Structured clinical interview for DSM-IV axis II personality disorders (SCID-II). Washington, DC: American Psychiatric Association.

15.

Graham

J. R.

(2012). MMPI-2: Assessing personality and psychopathology (5th ed.). New York, NY: Oxford University Press.

16.

Gudjonsson

G. H.

Hannesdottir

Agustsson

Sigurdsson

J. F.

Gudmundsdottir

Þordardottir

Þ.

… Petursson

(2004). Personality predictors of the failure of alcoholics to come for follow-up assessment. Personality and Individual Differences, 37, 805-813. doi:10.1016/j.paid.2003.11.006

17.

Harkness

A. R.

Lilienfeld

S. O.

(1997). Individual differences science for treatment planning: Personality traits. Psychological Assessment, 9, 349-360. doi:10.1037/1040-3590.9.4.349

18.

Harkness

A. R.

McNulty

J. L.

Ben-Porath

Y. S.

(1995). The Personality Psychopathology Five (PSY-5): Constructs and MMPI-2 scales. Psychological Assessment, 7, 104-114. doi:10.1037/1040-3590.7.1.104

19.

Hilsenroth

M. J.

Handler

Toman

K. M.

Padawer

J. R.

(1995). Rorschach and MMPI-2 indices of early psychotherapy termination. Journal of Consulting and Clinical Psychology, 63, 956-965.

20.

Kazdin

A. E.

Mazurick

J. L.

Bass

(1993). Risk for attrition in treatment of antisocial children and families. Journal of Clinical Child Psychology, 22, 2-16. doi:10.1207/s15374424jccp2201_1

21.

Killaspy

(2000). Prospective controlled study of psychiatric out-patient non-attendance: Characteristics and outcome. British Journal of Psychiatry, 176, 160-165. doi:10.1192/bjp.176.2.160

22.

Klein

E. B.

Stone

W. N.

Hicks

M. W.

Pritchard

I. L.

(2003). Understanding dropouts. Journal of Mental Health Counseling, 25, 89-100.

23.

Knox

Adrians

Everson

Hess

Hill

Crook-Lyon

(2011). Clients’ perspectives on therapy termination. Psychotherapy Research, 21, 154-167. doi:10.1080/10503307.2010.534509

24.

Kokotovic

A. M.

Tracey

T. J.

(1987). Premature termination at a university counseling center. Journal of Counseling Psychology, 34, 80-82. doi:10.1037/0022-0167.34.1.80

25.

Kruse

G. R.

Rohland

B. M.

(2002). Factors associated with missed first appointments at a psychiatric clinic. Psychiatric Services, 53, 1173-1176. doi:10.1176/appi.ps.53.9.1173

26.

Lefforge

N. L.

Donohue

Strada

M. J.

(2007). Improving session attendance in mental health and substance abuse settings: A review of controlled studies. Behavior Therapy, 38, 1-22. doi:10.1016/j.beth.2006.02.009

27.

Marlowe

D. B.

Kirby

K. C.

Festinger

D. S.

Husband

S. D.

Platt

J. J.

(1997). Impact of comorbid personality disorders and personality disorder symptoms on outcomes of behavioral treatment for cocaine dependence. Journal of Nervous and Mental Disease, 185, 483-490.

28.

Mattson

Powers

Halfaker

Akeson

Ben-Porath

(2012). Predicting drug court treatment completion using the MMPI-2-RF. Psychological Assessment, 24, 937-943. doi:10.1037/a0028267

29.

McMurran

Huband

Overton

(2010). Non-completion of personality disorder treatments: A systematic review of correlates, consequences, and interventions. Clinical Psychology Review, 30, 277-287. doi:10.1016/j.cpr.2009.12.002

30.

Meyer

G. J.

Finn

S. E.

Eyde

L. D.

Kay

G. G.

Moreland

K. L.

Dies

R. R.

. . .Reed

G. M.

(2001). Psychological testing and psychological assessment: A review of evidence and issues. American Psychologist, 56, 128-165. doi:10.1037//OOO3-O66X.56.2.128

31.

Minnix

J. A.

Reitzel

L. R.

Repper

K. A.

Burns

A. B.

Williams

Lima

E. N.

. . .Joiner

T. E.

(2005). Total number of MMPI-2 clinical scale elevations predicts premature termination after controlling for intake symptom severity and personality disorder diagnosis. Personality and Individual Differences, 38, 1745-1755. doi:10.1016/j.paid.2004.11.006

32.

Ogrodniczuk

J. S.

Joyce

A. S.

Piper

W. E.

(2005). Strategies for reducing patient-initiated premature termination of psychotherapy. Harvard Review of Psychiatry, 13(2), 57-70. doi:10.1080/10673220590956429

33.

Pekarik

(1985). Coping with dropouts. Professional Psychology: Research and Practice, 16, 114-123. doi:10.1037/0735-7028.16.1.114

34.

Pekarik

(1992). Posttreatment adjustment of clients who drop out early vs. late in treatment. Journal of Clinical Psychology, 48, 379-387. doi:10.1002/1097-4679(199205)48:3<379::AID-JCLP2270480317>3.0.CO;2-P

35.

Piselli

Halgin

R. P.

MacEwan

G. H.

(2011). What went wrong? Therapists’ reflections on their role in premature termination. Psychotherapy Research, 21, 400-415. doi:10.1080/10503307.2011.573819

36.

Reardon

M. L.

Cukrowicz

K. C.

Reeves

M. D.

Joiner

T. E.

(2002). Duration and regularity of therapy attendance as predictors of treatment outcome in an adult outpatient population. Psychotherapy Research, 12(3), 273-285. doi:10.1080/713664390

37.

Reda

Rowett

Makhoul

(2001). Prompts to encourage appointment attendance for people with serious mental illness. Cochrane Database of Systematic Reviews, (2), CD002085. Retrieved from http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD002085/pdf

38.

Sellbom

Ben-Porath

Y. S.

Baum

L. J.

Erez

Gregory

(2008). Predictive validity of the MMPI-2 Restructured Clinical (RC) Scales in a batterers’ intervention program. Journal of Personality Assessment, 90, 129-135. doi:10.1080/00223890701845153

39.

Sharp

D. J.

Hamilton

(2001). Non-attendance at general practices and outpatient clinics. British Medical Journal, 323(7321), 1081-1082. doi:10.1136/bmj.323.7321.1081

40.

Swartz

H. A.

Zuckoff

Grote

N. K.

Spielvogle

H. N.

Bledsoe

S. E.

Shear

M. K.

Frank

(2007). Engaging depressed patients in psychotherapy: Integrating techniques from motivational interviewing and ethnographic interviewing to improve treatment participation. Professional Psychology: Research and Practice, 38, 430-439. doi:10.1037/0735-7028.38.4.430

41.

Swift

J. K.

Greenberg

R. P.

(2012). Premature discontinuation in adult psychotherapy: A meta-analysis. Journal of Consulting and Clinical Psychology, 80, 547-559. doi:10.1037/a0028226

42.

Tellegen

Ben-Porath

(2008). Minnesota Multiphasic Personality Inventory-2-Restructured Form: Technical manual. Minneapolis: University of Minnesota Press.

43.

Tellegen

Ben-Porath

McNulty

J. L.

Arbisi

Graham

J. R.

Kaemmer

(2003). MMPI-2 Restructured Clinical (RC) scales: Development, validation, and interpretation. Form: Technical manual. Minneapolis: University of Minnesota Press.

44.

Van Der Heijden

P. T.

Egger

J. I. M.

Derksen

J. J. L.

(2010). Comparability of scores on the MMPI-2-RF scales generated with the MMPI-2 and MMPI-2-RF booklets. Journal of Personality Assessment, 92(3), 254-259. doi:10.1080/00223891003670208

45.

Walters

G. D.

Solomon

G. S.

Walden

V. R.

(1982). Use of the MMPI in predicting psychotherapeutic persistence in groups of male and female outpatients. Journal of Clinical Psychology, 38, 80-83. doi:10.1002/1097-4679(198201)38:1<80::AID-JCLP2-270380110>3.0.CO;2-D