Associations Between MMPI-2-RF Validity Scale Scores and Extra-Test Measures of Personality and Psychopathology

Abstract

The current study explored associations between two potentially invalidating self-report styles detected by the Validity scales of the Minnesota Multiphasic Personality Inventory–2–Restructured Form (MMPI-2-RF), over-reporting and under-reporting, and scores on the MMPI-2-RF substantive, as well as eight collateral self-report measures administered either at the same time or within 1 to 10 days of MMPI-2-RF administration. Analyses were conducted with data provided by college students, male prisoners, and male psychiatric outpatients from a Veterans Administration facility. Results indicated that if either an over- or under-reporting response style was suggested by the MMPI-2-RF Validity scales, scores on the majority of the MMPI-2-RF substantive scales, as well as a number of collateral measures, were significantly affected in all three groups in the expected directions. Test takers who were identified as potentially engaging in an over- or under-reporting response style by the MMPI-2-RF Validity scales appeared to approach extra-test measures similarly regardless of when these measures were administered in relation to the MMPI-2-RF. Limitations and suggestions for future study are discussed.

Keywords

personality assessment validity interpretability college outpatients veterans

It has long been recognized that the validity of self-report–based personality assessment is limited by the degree to which the individual assessed accurately communicates his or her inner experiences and perceptions. Whether this communication occurs via interview or psychological testing, issues such as limited insight or purposeful distortion can lead to inaccurate results on personality inventories. Unlike interview data, which rely on the interviewer’s subjective assessment of the veracity of the interviewee’s self-report, in personality-based psychological testing, two potentially biasing response styles, Noncontent-Based and Content-Based Invalid Responding (CBIR), can be assessed through empirically validated objective means (i.e., scales designed to measure such response styles; Ben-Porath, 2013). Individuals who engage in Noncontent-Based Invalid Responding (NCIR) fail to respond or provide random or fixed responses to test items regardless of their content. CBIR occurs when the test taker portrays him or herself as functioning better (under-reporting) or worse (over-reporting) than would be indicated by an objective assessment of his or her functioning.

Given the potential costs of misinterpreting psychological test results (e.g., in outpatient and/or inpatient settings where an individual is prescribed medication or forensic settings where an individual is deemed incompetent to proceed to trial because of psychological difficulties), once NCIR has been ruled out, being able to effectively detect CBIR is of paramount importance. Three questions need to be considered when selecting a psychological test and considering its ability to detect CBIR. First, does the instrument include scales designed to assess CBIR? If so, have these scales been empirically validated? Finally, does any CBIR they detect generalize to other psychological tests taken at or near the same time?

With regard to the first two questions just listed, the Minnesota Multiphasic Personality Inventory–2–Restructured Form (MMPI-2-RF; Ben-Porath & Tellegen, 2008; Tellegen & Ben-Porath, 2008/2011) contains well-validated measures of both NCIR and CBIR (Ben-Porath, 2013). In terms of NCIR, the MMPI-2-RF retains the use of the Cannot Say Scale (CNS) Index, which is a count of omitted or double-answered items. The MMPI-2-RF also includes restructured versions of the MMPI-2 Variable Response Inconsistency (VRIN) and True Response Inconsistency (TRIN) scales, VRIN-r and TRIN-r, which assess random and fixed responding, respectively. In terms of CBIR, the MMPI-2-RF contains five primary scales designed to detect over-reporting. Three of the CBIR over-reporting indicators are revised versions of MMPI-2 scales, including Infrequent Responses (F-r), Infrequent Psychopathology Responses (F_P-r), and Symptom Validity (FBS-r). Infrequent Somatic Responses (F_S) and the Response Bias Scale (RBS) were added to the MMPI-2-RF to assess the report of unusual somatic symptoms and noncredible memory complaints, respectively. The MMPI-2-RF also contains two revised MMPI-2 scales designed to detect under-reporting, Uncommon Virtues (L-r), and Adjustment Validity (K-r).

A number of studies have supported the ability of the over-reporting scales of the MMPI-2-RF to detect invalid response styles. For example, two studies (Marion, Sellbom, & Bagby, 2011; Sellbom & Bagby, 2010) reported that the MMPI-2-RF over-reporting scales were able to differentiate between college students coached to feign mental illness and psychiatric patients. Furthermore, Wygant et al. (2011) reported that the MMPI-2-RF over-reporting scales were able to accurately identify individuals undergoing compensation-seeking evaluations who were classified as probable/definite malingerers based on structured criteria for malingered neurocognitive dysfunction (Slick, Hopp, Strauss, & Thompson, 1997) or malingered pain-related disability (Bianchini, Greve, & Glynn, 2005). All the studies just mentioned demonstrated a large effect size for the ability of the MMPI-2-RF CBIR over-reporting scales in differentiating between groups.

With respect to under-reporting, Sellbom and Bagby (2008) reported that L-r and K-r were able to differentiate between both college students and psychiatric patients instructed to under-report problems and individuals instructed to take the test under standard instructions. Sellbom and Bagby (2008) concluded that their results support the utility of the MMPI-2-RF under-reporting indicators and noted that the effect sizes they obtained were similar to the large effect sizes reported in Baer and Miller’s (2002) meta-analysis of the MMPI-2 measures of under-reporting, from which L-r and K-r were derived. Data reported in the MMPI-2-RF Technical Manual indicate that scores on these scales are commonly elevated in samples of individuals tested under circumstances that are likely to motivate under-reporting (e.g., personnel screening and child custody litigation).

With respect to the third question listed earlier concerning the generalizability of MMPI-2-RF CBIR findings to other measures, only limited MMPI-2 research has been conducted to date. Garcia, Franklin, and Chambliss (2010) examined whether invalid response styles detected by the MMPI-2 Validity scales were correlated with both MMPI-2 substantive scale scores and conjointly administered collateral measures in a sample of veterans diagnosed with posttraumatic stress disorder (PTSD). Garcia et al. (2010) demonstrated that scores on MMPI-2 Clinical scales and conjointly administered measures of depression and PTSD symptoms were significantly higher for individuals with elevated scores on the F (Infrequency) scale. Although this study had some methodological shortcomings that may have artificially increased the detected over-reporting rate (i.e., using nonstandard cut scores for some validity scales), overall, results indicated that MMPI-2-detected over-reporting generalized to the other conjointly administered measures.

Forbey and Lee (2011) attempted to further clarify the association between under- and over-reporting response styles scores on the MMPI-2 and conjointly administered measures of social, behavioral, and psychological difficulties. The authors used data provided by 1,112 undergraduates who took the MMPI-2 and a number of collateral measures together as part of a larger study. After excluding NCIR individuals, three groups of participants were identified based on their Validity scale scores: CBIR over-reporters (CBIR-OR), CBIR under-reporters (CBIR-UR), and “within normal limits” (WNL) responders on the validity scales. In terms of MMPI-2 substantive scales, results mirrored those of previous coaching and simulation research, which has long established that scores on MMPI-2 substantive scales vary substantially as a function of validity scale scores. More important, with respect to the conjointly administered collateral measures, Forbey and Lee (2011) found 27 of 35 collateral measures’ total and/or subscale scores in the CBIR-OR group were significantly different from the WNL responders (with small to large median effect sizes), and 15 of 35 collateral total scale and/or subscale scores were significantly different in the CBIR-UR group when compared with the WNL responders (with medium to large median effect sizes). Overall, these results indicated that individuals who were identified as potentially engaging in CBIR-OR by the MMPI-2 Validity scales reported significantly higher levels of psychopathology on the MMPI-2 substantive scales and on conjointly administered collateral measures when compared with individuals with WNL validity scores. Individuals identified as potentially engaging in CBIR-UR by the MMPI-2 Validity scales generally reported substantially lower (than the WNL group) levels of psychopathology on both the MMPI-2 substantive scales and collateral measures administered at the same time.

The study by Forbey and Lee (2011) indicated that MMPI-2 detected CBIR-OR and CBIR-UR likely reflects a more generalized response style that affects conjointly administered collateral measures. This finding is of critical importance because the MMPI-2 or MMPI-2-RF are often administered as part of a battery that includes other tests that lack empirically validated CBIR measures. Being able to rely on MMPI Validity scale findings to guide interpretation of these additional measures may increase their utility considerably. However, Forbey and Lee’s (2011) study was limited in generalizability, as it only included college students who were administered all the measures conjointly (i.e., on the same occasion). Furthermore, this study used the MMPI-2, rather than the MMPI-2-RF, though use of the MMPI-2-RF may confer some advantages, including the ability to examine additional over-reporting scales and the reduced item overlap among the validity scales (in particular the removal of the item overlap among the under- and over-reporting scales that existed on the MMPI-2). Moreover, because it is approximately 40% shorter than the MMPI-2, the MMPI-2-RF is more likely to be administered along with other self-report measures.

The current study was conducted to build on Forbey and Lee’s (2011) findings. The goals of the study were to examine associations between CBIR, as detected by MMPI-2-RF Validity scales and scores on the substantive scales of the test (to serve as an overall procedural evaluation of the methodology) and, more important, to examine associations between the CBIR indicators and conjointly administered self-report measures (which lack such validity indicators) in different populations and during varying time frames. Specifically, using a criterion groups design, we examined MMPI-2-RF and criterion measure scale score differences between subgroups of individuals who were classified as engaging in CBIR-OR or CBIR-UR and individuals who produced interpretable (i.e., WNL on all the validity scales) MMPI-2-RF protocols in male and female college students, males undergoing intake to a correctional facility, and male Veterans Administration mental health outpatients.

Based on previous research, we hypothesized that when compared with individuals classified as responding candidly, individuals engaging in an CBIR-OR or CBIR-UR would score differently on MMPI-2-RF substantive scales and collateral measures administered conjointly or within a specified amount of time. Specifically, we hypothesized that individuals identified as potentially engaging in CBIR-OR based on MMPI-2-RF Validity scale cut scores (i.e., elevations on F-r, F_P-r, FS, FBS-r, and/or RBS) would score higher than those assigned to the WNL group on substantive measures. We further hypothesized that individuals who were identified as potentially engaging in CBIR-UR based on their cut -scores on MMPI-2-RF-specific Validity scales (i.e., elevations on L-r and/or K-r) would score lower than those on the WNL group on substantive measures.

Method

Participants

Three archival samples of participants were used in the current study, with data originally being collected from these participants as part of a larger series of unpublished studies examining the comparability between conventional and several modularized¹ versions of the MMPI-2–computerized adaptive version (MMPI-2-CA; Forbey & Ben-Porath, 2007). The first sample, composed of college participants, included 1,194 (487 men and 707 women) undergraduate students from a Midwestern U.S. university enrolled in Introductory Psychology classes. The second sample, made up of correctional participants, included 632 male inmates from a large Midwestern U.S. intake correctional facility who volunteered to participate in the original study. The third sample included 181 men who were recruited from outpatient psychiatric care Veterans Affairs facility at a large tertiary care medical center.

To reduce error variance in our analyses, individuals in each of the samples who provided NCIR MMPI-2-RF profiles, defined by CNS scores ≥15 and/or T scores ≥80 on VRIN-r or TRIN-r based on the recommendations in the MMPI-2-RF administration manual (Ben-Porath & Tellegen, 2008), were removed from the current study. This resulted in a total of 129 (10.8%) college, 19 (3.0%) correctional, and 17 (9.4%) psychiatric participants being excluded from analyses. Among the college participants, no statistically significant differences between included and excluded participants were evident in terms of age or ethnicity/racial group. However, a statistically significant differences of a small effect size (Cohen, 1988) was evident for gender, χ²(1) = 26.986, p < .001, Φ = .15) with men being slightly more likely to produce NCIR MMPI-2-RF profiles. Among the correctional and psychiatric participants, no statistically significant differences between included and excluded were evident in terms of age. However, a statistically significant difference of a small effect size (Cohen, 1988) was evident for ethnicity/racial group in both sets of participants, χ²(2) = 8.526, p < .05, Φ = .12 and χ²(2) = 14.474, p < .001, Φ = .28, respectively, with minority participants being slightly more likely to produce NCIR MMPI-2-RF profiles.

After NCIR exclusions, the final sample of 1,065 college participants included 407 men and 658 women (age range = 18-48 years, M = 19.60 years, SD = 3.14) who were Caucasian (n = 947, 88.9%), African American (n = 84, 7.9%), or of another or unidentified ethnicity (n = 34, 3.2%). The final sample of correctional participants included 613 men (age range = 18-66 years, M = 32.37 years, SD = 9.94) who were Caucasian (n = 310, 50.6%), African American (n = 173, 28.2%), or of another or unidentified ethnicity (n = 130, 21.2%). The final sample of psychiatric participants included 164 men (age range = 26-85 years, M = 55.44 years, SD = 11.90) who were Caucasian (n = 138, 84.7%), African American (n = 21, 12.9%), or of another or unidentified ethnicity (n = 5, 3.1%).

Measures

Minnesota Multiphasic Personality Inventory–2–Restructured Form

Revised from the MMPI-2 (Butcher et al., 2001), the MMPI-2-RF (Ben-Porath & Tellegen, 2008; Tellegen & Ben-Porath, 2008/2011) is a 338-item, true/false, self-report inventory that assesses an individual’s psychological functioning in a number of domains (i.e., personality, psychopathology, and social/ behavioral functioning). The MMPI-2-RF Technical Manual (Tellegen & Ben-Porath, 2008/2011) provides extensive evidence supporting reliability and validity for this instrument. For the current study, the MMPI-2-RF was rescored from a conventional administration of the MMPI-2 (see Procedure section below). Tellegen and Ben-Porath (2008/2011) and Van der Heijden, Egger, and Derksen (2010) have demonstrated that MMPI-2-RF scale scores generated from an MMPI-2 administration are interchangeable with those generated from the MMPI-2-RF booklet.

Collateral measures

Eight self-report based collateral measures, originally selected for the purpose of examining the comparative validity of modularized versions of the MMPI-2-CA and conventional administrations of the MMPI-2 with conceptually relevant personality and psychopathology constructs were used in the current study. Depending on setting, the collateral measures were either divided into two administration packets (i.e., the college and psychiatric participants) or all measures were administered in a single packet (i.e., the correctional participants). All eight collateral measures were administered via paper and pencil. Each of the eight self-report collateral measures is described below.

Barratt Impulsivity Scale–Version 10

The Barratt Impulsivity Scale (BIS, Barratt, 1985) is a 34-item measure of impulsivity rated on a 4-point Likert-type scale (1 = rarely/never, 4 = almost always/always) that provides a total score as well as scores on three dimensions of impulsivity: Nonplanning (12 items), Motor (11 items), and Cognitive (11 items). In the current study, only the total scale score was uses, which had an estimated internal consistency (α) of .80 for college, .87 for correctional, and .86 for psychiatric participants.

Beck Depression Inventory

The Beck Depression Inventory (BDI; Beck, Ward, Mendelson, Mock, & Erbaugh, 1961) is a 21-item measure of depressive symptomatology rated on a 4-point Likert-type scale, with higher ratings generally indicating higher levels of psychological distress. In the current study, estimated internal consistency (α) for the BDI was .88 for the college, .89 for the correctional, and .93 for the psychiatric participants.

Drug Abuse Screening Test

The Drug Abuse Screening Test (DAST; Skinner, 1982) is a 20-item measure rated dichotomously (i.e., yes or no) that assesses an individual’s self-reported consumption of prescription, over the counter, and other illicit drugs. In the current study, estimated internal consistency (α) for the DAST was estimated to be .81 for the college, .93 for the correctional, and .94 for the psychiatric participants.

Magical Ideation Scale

The Magical Ideation Scale (MIS; Eckblad & Chapman, 1983) is a 30-item true/false inventory that assesses beliefs about causal relations between events that are unconventional and which are commonly associated with thought disorders. In the current study, estimated internal consistency (α) for the MIS was .80 for the college, .81 for the correctional, and .86 for the psychiatric participants.

Michigan Alcohol Screening Test

The Michigan Alcohol Screening Test (MAST; Selzer, 1971) is a 24-item measure rated dichotomously (i.e., yes or no) that assesses an individual’s self-reported level of problematic alcohol consumption. In the current study, internal consistency (α) for the MAST was estimated to be .68 for the college, .80 for the correctional, and .84 for the psychiatric participants.

Perceptual Aberration Scale

The Perceptual Aberration Scale (PAS; Chapman, Chapman, & Raulin, 1978) is a 35-item true/false inventory that assesses physical and other perceptual distortions related to thought disorders. In the current study, internal consistency (α) for the PAS was estimated to be .96 for the college, .97 for the correctional, and .92 for the psychiatric participants.

Screener for Somatoform Disorders

The Screener for Somatoform Disorders (SSD; Janca et al., 1995) is a 12-item measure designed to assess diffuse somatic complaints related to somatoform disorders as defined by the International Classification of Diseases (ICD-10; World Health Organization, 1992) and the fourth edition of the American Psychiatric Association’s (APA) Diagnostic and Statistical Manual of Mental Disorders (DSM-IV; APA, 1994). In the current study, internal consistency (α) for the SSD was estimated to be .80 for the college, .84 for the correctional, and .86 for the psychiatric participants.

State-Trait Personality Inventory

The State-Trait Personality Inventory (STPI; Spielberger, 1979) is a 60-item measure of anxiety and anger rated on a 4-point Likert-type scale (1 = almost never, 4 = almost always). The current study relied on a modified 10-item version of the instrument in which only the Trait Anger items were administered. In the current study, the estimated internal consistency (α) for Trait Anger subscale was .69 for the college, .88 for the correctional, and .88 for the psychiatric participants.

Procedure

As indicated, all participants in the current study were selected from three larger unpublished studies examining various modularized versions of the computerized adaptive version of the MMPI-2 (i.e., the MMPI-2-CA). Although the measures used in each study were consistent across settings, because of various institutional considerations/constraints, each of the data collection sites had slightly different data collection procedures. However, all participants were assessed in accordance with procedures approved by the institutional review board at the facilities where the data were collected, received compensation in line with those procedures dictated by the facilities and their respective institutional review boards, and were free to withdraw their participation at any time.

For the college participants, after agreeing to participate, individuals were assigned to complete the MMPI-2, as well as one of two packets of selected collateral measures, and then return in 1 week to complete another MMPI-2 administration and the remaining collateral measures. For the psychiatric participants, letters were sent prior to initial clinic appointments introducing the study and individuals were asked to call and set up an appointment to participate in the study. However, the actual data collection in the psychiatric participants was similar to that of the college participants, with the exception that the time frame for the second MMPI-2 administration ranged from 6 to 10 days. For both college and psychiatric participants, administration of the MMPI-2 and collateral measures was counterbalanced across the two time frames (i.e., either the adaptive or conventional version of the MMPI-2 was administered followed by the collateral measures or vice versa).

For the correctional participants, all individuals completed a standard paper and pencil audio-taped version of the MMPI-2 on intake to the correctional facility as part of standard institutional screening procedures. On completion of the paper-and-pencil audio MMPI-2, participants were recruited for participation in the larger study. Those who volunteered for the study were required to have a sixth-grade reading level, which was verified using facility records.² Administration of the collateral measures (as well as either a computerized conventional or adaptive MMPI-2) occurred in a group format typically between 1 and 5 days after administration of the intake MMPI-2, with 87.8% (N = 555) of participants completing the collateral measures 1 day after completing the initial MMPI-2. The administration order of the second MMPI-2 and collateral measures was counterbalanced.

In the current study, across the samples, only the first conventional MMPI-2 administered was used in the rescoring of the MMPI-2 to the MMPI-2-RF. In the case of the correctional participants, the rescored MMPI-2 selected for use in the current study was always the first MMPI-2 administered during the original intake procedure. For the college and psychiatric participants, the rescored MMPI-2 used in the current study could have been administered either approximately a week before or after the administration of some of the collateral measures because of counterbalancing of administration formats. Across settings, all collateral measures were administered via a Latin square design and were considered invalid for the current study’s analyses if 10% or more of items were not answered.

Response Style Group Assignment

After removing content nonresponsive MMPI-2-RF profiles (as described in the Participants section above), participants in each sample were assigned to one of three groups: CBIR-OR, CBIR-UR, and WNL reporting on the validity scales, based on their T scores on standard MMPI-2-RF validity. The MMPI-2-RF Validity scale T score cutoffs used for group assignment in the current study were consistent with those indicated in the MMPI-2-RF Manual for Administration, Scoring, and Interpretation (Ben-Porath & Tellegen, 2008). CBIR-OR groups were identified by T scores of 100 or greater on the Infrequent Responses (F-r), Infrequent Somatic Responses (F_S), Symptom Validity (FBS-r), or Response Bias (RBS) Scales, and/or 80 or more on the Infrequent Psychopathology Responses (F_P-r) Scale. CBIR-UR was identified by T scores of 70 or greater on the Uncommon Virtues (L-r) scale and/or an Adjustment Validity (K-r) scale score of 66 or higher. In addition, no individuals in the CBIR-OR group could have elevations on the under-reporting scales and vice versa. Finally, the WNL group consisted of individuals who had no elevations on any of the over-reporting and/or under-reporting MMPI-2-RF scales.

Using this group assignment procedure for the 1,065 included college participants, 100 were assigned to CBIR-OR, 59 were assigned to CBIR-UR, 903 were assigned to WNL, and 3 were removed from subsequent analyses because of having elevations on both CBIR-OR and CBIR-UR Scales. For the 613 included correctional participants, 39 were assigned to CBIR-OR, 163 were assigned to CBIR-UR, 406 were assigned to WNL, and 5 were removed from subsequent analyses because of having elevations on both CBIR-OR and CBIR-UR Scales. For the 164 included psychiatric participants, 44 were assigned to CBIR-OR, 16 were assigned to CBIR-UR, and 104 were assigned to WNL.

Comparative demographic analyses in group assignments revealed a few statistically significant differences for each of the samples. For college participants, no differences emerged in terms of gender or ethnic/racial identity. However, statistically significant differences of a small effect size (Cohen, 1988) was demonstrated for age, F(2, 1059) = 3.409, p < .033, η² = .01. Post hoc comparisons for age using Tukey’s HSD test indicated that the mean age in the CBIR-UR group (M = 20.61 years, SD = 5.275) was significantly older than the WNL (M = 19.57 years, SD = 3.052) and CBIR-OR groups (M = 19.35 years, SD = 2.037) groups. For both correctional and psychiatric participants, no differences were found between groups in terms of age or ethnicity.

Data Analyses

To examine potential differences in MMPI-2-RF scale scores between the three response style groups that were created, a series of t tests were computed comparing mean T scores on MMPI-2-RF substantive scales for each of the three samples of participants separately. To reduce potential Type I error, a Bonferroni correction was applied and the critical alpha for analyses within each of these three samples was set to .001 (.05/42). Cohen’s d (1988) effect sizes, with .3, .5, and .8 reflecting small, medium, and large effects, respectively, for the differences between means were reported for all analyses, regardless of significance.

To examine for potential differences on collateral measure scores between response style groups, a series of t tests were computed comparing mean scores on the collateral measures for each of the three samples. As each of the eight collateral measures have unique scale score ranges, means, and standard deviations, the descriptive statistics reported for the collateral measure analyses were converted (within each group) to a z-score metric to facilitate interpretation of the results. As with the MMPI-2-RF substantive scale analyses, to reduce potential Type I error, a Bonferroni correction was applied, setting the critical alpha for these analyses at .006 (.05/8) for each of the samples. Effect sizes (d; Cohen, 1988) were reported for all analyses, regardless of significance.

Results

The first set of analyses explored the differences on substantive scale scores between the WNL and CBIR-OR response style group as well as for the WNL and CBIR-UR response style groups, across the three samples. Tables 1, 2, and 3 contain the results of these analyses for the college, correctional, and psychiatric participants, respectively.

Table 1.

Minnesota Multiphasic Personality Inventory–2 (MMPI-2) Scale Mean T Score Differences Between MMPI-2 Within Normal Limits (WNL) Versus Content-Responsive Invalidity: Over-Reporting (CBIR-OR) and Content Responsive Invalidity: Under-Reporting (CBIR-UR): College Participants.

	WNL (n = 903)		CBIR-OR, (n = 100)						CBIR-UR, (n = 59)
MMPI scales	Mean	SD	Mean	SD	t	df	p	ES (d)	Mean	SD	t	df	p	ES (d)
Higher order
EID	51.04	10.74	62.50	12.34	−9.97	1001	<.001	1.05	40.08	7.11	7.72	960	<.001	−1.04
THD	52.52	9.92	68.85	14.17	−14.87	1001	<.001	1.57	47.58	9.46	3.72	960	<.001	−0.50
BXD	54.09	9.79	63.04	10.06	−8.65	1001	<.001	0.91	47.80	8.95	4.81	960	<.001	-0.65
Restructured clinical
RCd	54.54	10.07	65.51	11.60	−10.17	1001	<.001	1.07	43.44	6.42	8.36	960	<.001	−1.12
RC1	53.17	10.29	64.56	11.70	−10.35	1001	<.001	1.09	47.17	7.55	4.40	960	<.001	−0.59
RC2	48.03	9.85	55.32	13.15	−6.77	1001	<.001	0.71	45.25	8.32	2.11	960	ns	−0.28
RC3	56.14	9.40	63.80	9.40	−7.73	1001	<.001	0.81	46.66	9.08	7.52	960	<.001	−1.01
RC4	52.87	9.63	62.00	10.59	−8.90	1001	<.001	0.94	47.07	8.75	4.51	960	<.001	−0.61
RC6	55.39	10.64	70.12	13.42	−12.77	1001	<.001	1.35	49.04	7.71	4.52	960	<.001	−0.61
RC7	53.94	10.88	68.58	12.83	−12.53	1001	<.001	1.32	41.83	7.00	8.44	960	<.001	−1.13
RC8	54.10	10.47	70.08	13.05	−14.10	1001	<.001	1.49	48.05	10.60	4.29	960	<.001	−0.58
RC9	57.09	10.90	67.21	10.95	−8.80	1001	<.001	0.93	46.59	8.62	7.25	960	<.001	−0.97
Somatic/cognitive
MLS	51.38	9.62	59.24	11.38	−7.61	1001	<.001	0.80	44.64	6.17	5.31	960	<.001	−0.71
GIC	52.55	11.65	57.98	15.19	−4.28	1001	<.001	0.45	48.10	6.59	2.90	960	ns	−0.39
HPC	52.20	10.65	59.64	11.56	−6.57	1001	<.001	0.69	46.56	7.15	4.01	960	<.001	−0.54
NUC	54.14	10.84	67.04	11.82	−11.18	1001	<.001	1.18	48.73	10.07	3.73	960	<.001	−0.50
COG	55.92	11.63	71.06	13.39	−12.15	1001	<.001	1.28	46.93	8.79	5.83	960	<.001	−0.78
Internalizing
SUI	49.32	10.72	60.85	17.78	−9.43	1001	<.001	0.99	45.58	4.43	2.67	960	ns	−0.36
HLP	50.22	10.66	59.49	12.53	−8.09	1001	<.001	0.85	44.88	6.60	3.80	960	<.001	−0.51
SFD	53.01	11.65	60.75	13.00	−6.23	1001	<.001	0.66	44.58	5.95	5.51	960	<.001	−0.74
NFC	54.03	10.61	61.50	10.83	−6.66	1001	<.001	0.70	43.95	6.79	7.20	960	<.001	−0.97
STW	52.88	10.89	61.08	11.86	−7.08	1001	<.001	0.75	43.53	5.40	6.54	960	<.001	−0.88
AXY	55.60	12.70	73.63	14.24	−13.30	1001	<.001	1.40	47.34	7.59	4.94	960	<.001	−0.66
ANP	53.24	10.74	63.24	10.85	−8.83	1001	<.001	0.93	43.97	7.58	6.52	960	<.001	−0.88
BRF	52.70	10.93	69.14	14.38	−13.78	1001	<.001	1.45	47.42	8.21	3.64	960	<.001	−0.49
MSF	48.40	8.60	51.54	10.22	−3.39	1001	<.001	0.36	44.75	7.07	3.20	960	<.001	−0.43
Externalizing
JCP	50.61	10.12	57.10	11.11	−6.02	1001	<.001	0.63	46.97	9.07	2.69	960	ns	−0.36
SUB	52.66	11.29	64.16	13.89	−9.42	1001	<.001	0.99	49.22	9.39	2.29	960	ns	−0.31
AGG	52.27	11.60	62.13	12.31	−8.01	1001	<.001	0.84	43.29	7.08	5.88	960	<.001	−0.79
ACT	53.74	9.96	60.54	10.32	−6.46	1001	<.001	0.68	46.76	8.94	5.24	960	<.001	−0.70
Interpersonal
FML	51.56	10.18	59.77	11.35	−7.56	1001	<.001	0.80	44.00	6.94	5.62	960	<.001	−0.75
IPP	45.48	8.11	45.69	6.57	−0.25	1001	ns	0.03	47.08	6.56	−1.48	960	ns	0.20
SAV	45.46	9.87	47.11	11.10	−1.56	1001	ns	0.17	44.93	9.19	0.40	960	ns	−0.05
SHY	50.83	11.10	55.18	10.55	−4.35	1001	<.001	0.39	43.12	6.16	5.28	960	<.001	−0.71
DSF	50.87	9.73	57.18	12.67	−6.31	1001	<.001	0.63	50.17	8.84	0.54	960	ns	−0.07
Interest
AES	44.80	9.48	45.75	10.28	−0.95	1001	ns	0.10	45.27	9.53	−0.37	960	ns	0.05
MEC	49.10	9.82	50.37	9.41	−1.27	1001	ns	0.13	48.86	10.34	0.18	960	ns	−0.02
Personality Psychopathology Five (PSY-5)
AGGR-r	51.43	10.18	54.39	9.61	−2.96	1001	ns	0.29	47.81	6.92	2.69	960	ns	−0.36
PSYC-r	52.90	10.02	69.95	13.10	−17.05	1001	<.001	1.64	47.88	9.66	3.74	960	<.001	−0.50
DISC-r	54.30	10.00	60.15	9.63	−5.85	1001	<.001	0.59	49.98	10.09	3.21	960	<.001	−0.43
NEGE-r	54.16	11.19	66.43	13.76	−12.27	1001	<.001	1.07	41.75	6.83	8.42	960	<.001	−1.13
INTR-r	44.28	9.56	46.66	11.07	−2.37	1001	ns	0.24	45.58	9.98	−1.01	960	ns	0.14

Note. ES = effect size; Higher Order: EID = Emotional/Internalizing Dysfunction, THD = Thought Dysfunction, BXD = Behavioral/Externalizing Dysfunction; Restructured Clinical (RC): RCd = Demoralization, RC1 = Somatic Complaints, RC2 = Low Positive Emotions, RC3 = Cynicism, RC4 = Antisocial Behavior, RC6 = Ideas of Persecution, RC7 = Dysfunctional Negative Emotions, RC8 = Aberrant Experiences, RC9 = Hypomanic Activation; Somatic/Cognitive: MLS = Malaise, GIC = Gastrointestinal Complaints, HPC = Head Pain Complaints, NUC = Neurological Complaints, COG = Cognitive Complaints; Internalizing Scales: SUI = Suicidal/Death Ideation, HLP = Helplessness/Hopelessness, SFD = Self-Doubt, NFC = Inefficacy, STW = Stress/Worry, AXY = Anxiety, ANP = Anger Proneness, BRF = Behavior-Restricting Fears, MSF = Multiple Specific Fears; Externalizing: JCP = Juvenile Conduct Problems, SUB = Substance Abuse, AGG = Aggression, ACT = Activation; Interpersonal Scales: FML = Family Problems, IPP = Interpersonal Passivity, SAV = Social Avoidance, SHY = Shyness, DSF = Disaffiliativeness; Interest Scales: AES = Aesthetic-Literary Interests, MEC = Mechanical-Physical Interests; Personality Psychopathology Five (PSY-5): AGGR-r = Aggressiveness-Revised, PSYC-r = Psychoticism–Revised, DISC-r = Disconstraint–Revised, NEGE-r = Negative Emotionality/Neuroticism–Revised, INTR-r = Introversion/Low Positive Emotionality–Revised.

Table 2.

	WNL (n = 406)		CBIR-OR (n = 39)						CBIR-UR (n = 163)
MMPI scales	Mean	SD	Mean	SD	t	df	p	ES (d)	Mean	SD	t	df	p	ES (d)
Higher order
EID	47.80	10.46	70.31	14.08	−12.41	443	<.001	2.08	38.75	6.06	10.34	567	<.001	−0.96
THD	51.17	9.34	75.51	14.53	−14.67	443	<.001	2.46	46.63	8.82	5.33	567	<.001	−0.49
BXD	64.47	10.92	71.38	11.16	−3.77	443	<.001	0.63	52.90	9.82	11.76	567	<.001	−1.09
Restructured clinical
RCd	51.10	9.64	72.00	11.86	−12.66	443	<.001	2.12	42.55	5.74	10.56	567	<.001	−0.98
RC1	48.05	10.14	70.10	14.29	−12.50	443	<.001	2.09	44.34	8.18	4.15	567	<.001	−0.39
RC2	47.32	10.01	68.56	16.02	−11.89	443	<.001	1.99	41.98	6.66	6.27	567	<.001	−0.58
RC3	56.99	11.18	65.21	10.87	−4.40	443	<.001	0.74	48.82	11.08	8.00	567	<.001	−0.73
RC4	66.69	11.33	76.87	9.90	−5.41	443	<.001	0.91	54.92	10.85	11.34	567	<.001	−1.05
RC6	56.44	11.74	81.82	12.78	−12.80	443	<.001	2.14	50.25	10.10	5.91	567	<.001	−0.55
RC7	46.84	9.24	66.74	12.86	−12.36	443	<.001	2.07	39.20	5.78	9.81	567	<.001	−0.91
RC8	49.81	8.33	71.15	15.16	−13.96	443	<.001	2.34	44.70	7.73	6.75	567	<.001	−0.63
RC9	50.82	9.44	57.82	12.05	−4.31	443	<.001	0.72	42.68	6.71	10.03	567	<.001	−0.93
Somatic/cognitive
MLS	50.93	9.36	48.78	7.96	−9.19	443	<.001	−0.23	45.44	6.91	6.79	567	<.001	−0.63
GIC	48.78	7.96	61.28	15.71	−8.38	443	<.001	1.41	47.36	5.24	2.10	567	ns	−0.19
HPC	48.48	9.21	65.26	11.64	−10.60	443	<.001	1.78	45.48	6.66	3.78	567	<.001	−0.35
NUC	50.45	10.69	70.21	16.97	−10.37	443	<.001	1.74	46.00	7.51	4.86	567	<.001	−0.45
COG	49.88	11.24	84.41	14.45	−13.19	443	<.001	2.99	42.81	5.94	7.62	567	<.001	−0.71
Internalizing
SUI	46.87	6.42	65.46	17.88	−13.74	443	<.001	2.30	45.39	2.83	2.83	567	ns	−0.26
HLP	46.07	9.27	62.10	13.32	−9.88	443	<.001	1.66	42.01	4.71	5.32	567	<.001	−0.49
SFD	48.97	9.14	64.64	12.43	−10.50	443	<.001	1.66	42.79	2.82	8.48	567	<.001	−0.79
NFC	48.27	8.80	64.05	10.99	−10.45	443	<.001	1.75	41.88	5.93	8.52	567	<.001	−0.79
STW	49.50	9.72	63.31	12.39	−8.26	443	<.001	1.38	41.80	7.09	9.17	567	<.001	−0.85
AXY	50.90	11.09	72.82	13.85	−11.52	443	<.001	1.93	45.66	4.72	5.82	567	<.001	−0.54
ANP	49.20	10.61	60.64	11.67	−6.37	443	<.001	1.07	42.49	5.92	7.61	567	<.001	−0.71
BRF	46.67	7.06	57.64	12.14	−8.59	443	<.001	1.44	45.42	5.87	1.99	567	ns	−0.19
MSF	45.92	7.87	46.08	8.51	−0.12	443	ns	0.02	45.48	6.80	0.63	567	ns	−0.06
Externalizing
JCP	67.18	12.31	74.72	8.71	−3.73	443	<.001	0.63	57.00	12.21	8.94	567	<.001	−0.83
SUB	62.71	13.90	69.10	13.55	−2.73	443	ns	0.46	53.38	13.09	7.36	567	<.001	−0.68
AGG	51.24	11.84	60.97	11.47	−4.92	443	<.001	0.82	41.70	7.92	9.47	567	<.001	−0.88
ACT	47.03	9.78	56.28	12.88	−5.47	443	<.001	0.92	41.75	7.60	6.19	567	<.001	−0.57
Interpersonal
FML	48.69	10.30	65.67	14.33	−9.46	443	<.001	1.59	42.28	6.53	7.38	567	<.001	−0.68
IPP	43.84	7.09	52.64	14.49	−6.57	443	<.001	1.10	45.45	5.90	−2.58	567	ns	0.24
SAV	48.05	10.15	58.64	15.42	−5.90	443	<.001	0.99	47.79	8.41	0.29	567	ns	−0.03
SHY	45.99	9.02	59.64	13.20	−8.62	443	<.001	1.44	41.24	6.41	6.13	567	<.001	−0.57
DSF	48.15	8.83	64.72	17.99	−9.93	443	<.001	1.66	46.74	7.40	1.81	567	ns	−0.17
Interest
AES	44.52	8.05	47.05	10.44	−1.82	443	ns	0.31	45.35	9.83	−1.04	567	ns	0.10
MEC	63.85	9.05	61.00	9.69	1.87	443	ns	−0.31	62.54	9.53	1.54	567	ns	−0.14
Personality Psychopathology Five (PSY-5)
AGGR-r	56.56	9.23	52.44	15.08	2.35	443	ns	−0.42	52.25	7.49	5.00	567	<.001	−0.49
PSYC-r	49.62	9.40	74.33	14.37	−14.85	443	<.001	2.49	45.05	8.61	5.63	567	<.001	−0.50
DISC-r	65.54	10.71	71.28	11.09	−3.19	443	ns	0.53	55.34	10.17	10.43	567	<.001	−0.97
NEGE-r	49.47	9.90	66.05	12.19	−9.78	443	<.001	1.64	40.68	7.05	10.33	567	<.001	−0.96
INTR-r	48.12	9.83	61.31	16.03	−7.49	443	<.001	1.26	47.48	7.36	0.75	567	ns	−0.07

Note. ES = effect size; Higher Order: EID = Emotional/Internalizing Dysfunction, THD = Thought Dysfunction, BXD = Behavioral/Externalizing Dysfunction; Restructured Clinical (RC): RCd = Demoralization, RC1 = Somatic Complaints, RC2 = Low Positive Emotions, RC3 = Cynicism, RC4 = Antisocial Behavior, RC6 = Ideas of Persecution, RC7 = Dysfunctional Negative Emotions, RC8 = Aberrant Experiences, RC9 = Hypomanic Activation; Somatic/Cognitive: MLS = Malaise, GIC = Gastrointestinal Complaints, HPC = Head Pain Complaints, NUC = Neurological Complaints, COG = Cognitive Complaints; Internalizing Scales: SUI = Suicidal/Death Ideation, HLP = Helplessness/Hopelessness, SFD = Self-Doubt, NFC = Inefficacy, STW = Stress/Worry, AXY = Anxiety, ANP = Anger Proneness, BRF = Behavior-Restricting Fears, MSF = Multiple Specific Fears; Externalizing: JCP = Juvenile Conduct Problems, SUB = Substance Abuse, AGG = Aggression, ACT = Activation; Interpersonal Scales: FML = Family Problems, IPP = Interpersonal Passivity, SAV = Social Avoidance, SHY = Shyness, DSF = Disaffiliativeness; Interest Scales: AES = Aesthetic-Literary Interests, MEC = Mechanical-Physical Interests; Personality Psychopathology Five (PSY-5): AGGR-r = Aggressiveness–Revised, PSYC-r = Psychoticism–Revised, DISC-r = Disconstraint–Revised, NEGE-r = Negative Emotionality/Neuroticism–Revised, INTR-r = Introversion/Low Positive Emotionality–Revised.

Table 3.

Minnesota Multiphasic Personality Inventory–2 (MMPI-2) Scale Mean T Score Differences between MMPI-2 Within Normal Limits (WNL) Versus Content-Responsive Invalidity: Over-Reporting (CBIR-OR) and Content-Responsive Invalidity: Under-Reporting (CBIR-UR): Psychiatric Participants.

	WNL (n = 104)		CBIR-OR (n = 44)						CBIR-UR (n = 16)
MMPI scales	Mean	SD	Mean	SD	t	df	p	ES (d)	Mean	SD	t	df	p	ES (d)
Higher order
EID	58.21	12.12	76.45	10.73	−8.65	146	<.001	1.56	39.44	6.11	6.06	118	<.001	−1.63
THD	52.67	10.53	77.20	18.01	−10.35	146	<.001	1.86	46.88	7.14	2.13	118	ns	−0.57
BXD	56.54	11.36	64.95	10.95	−4.16	146	<.001	0.75	48.38	8.75	2.75	118	ns	−0.74
Restructured clinical
RCd	59.04	11.93	75.84	9.81	−8.42	146	<.001	1.48	43.00	5.44	5.28	118	<.001	−1.42
RC1	59.97	12.38	76.93	12.37	−7.62	146	<.001	1.37	50.38	9.05	2.98	118	ns	−0.80
RC2	59.68	13.70	73.25	14.95	−5.34	146	<.001	0.96	47.44	6.37	3.51	118	<.001	−0.94
RC3	53.93	11.66	66.14	12.19	−5.74	146	<.001	1.03	41.81	8.72	3.98	118	<.001	−1.07
RC4	58.86	11.15	68.41	12.56	−4.59	146	<.001	0.82	48.44	12.72	3.42	118	<.001	−0.92
RC6	54.54	11.32	78.34	15.71	−10.37	146	<.001	1.86	50.44	7.88	1.40	118	ns	−0.37
RC7	52.36	10.53	73.70	10.90	−4.93	146	<.001	2.01	38.94	5.42	4.98	118	<.001	−1.34
RC8	54.22	11.49	77.36	13.72	−4.57	146	<.001	1.90	46.50	7.27	2.60	118	ns	−0.70
RC9	50.49	10.26	59.95	11.59	−4.17	146	<.001	0.89	42.13	5.94	3.17	118	ns	−0.85
Somatic/cognitive
MLS	65.52	12.41	75.48	11.43	−4.57	146	<.001	0.82	52.94	14.86	3.68	118	<.001	−0.99
GIC	54.50	13.37	64.82	14.66	−4.17	146	<.001	0.75	49.38	7.26	1.50	118	ns	−0.40
HPC	55.11	11.33	69.93	12.44	−7.07	146	<.001	1.27	49.69	7.69	1.85	118	ns	−0.50
NUC	63.78	14.70	79.18	12.82	−6.04	146	<.001	1.09	53.50	11.11	2.68	118	ns	−0.72
COG	59.99	14.38	82.18	9.67	−9.37	146	<.001	1.69	44.25	7.22	4.28	118	<.001	−1.15
Internalizing
SUI	57.97	13.34	78.05	19.32	−8.03	146	<.001	1.31	46.31	5.25	2.38	118	ns	−0.93
HLP	57.02	13.14	72.12	11.43	−6.63	146	<.001	1.19	44.75	7.62	3.63	118	<.001	−0.98
SFD	56.21	11.74	68.05	9.28	−5.94	146	<.001	1.07	42.63	2.50	4.60	118	<.001	−1.23
NFC	54.65	11.88	67.91	8.70	−6.68	146	<.001	1.20	42.56	4.93	4.01	118	<.001	−1.08
STW	51.45	11.16	66.68	9.89	−7.84	146	<.001	1.41	40.06	4.95	4.01	118	<.001	−1.08
AXY	54.67	12.88	80.41	15.74	−10.38	146	<.001	1.87	45.88	5.12	2.69	118	ns	−0.72
ANP	52.06	11.10	65.52	10.03	−6.93	146	<.001	1.25	40.50	3.22	4.12	118	<.001	−1.11
BRF	48.67	8.23	63.64	11.88	−8.81	146	<.001	1.58	46.69	6.78	0.918	118	ns	−0.25
MSF	47.68	8.44	49.95	9.17	−1.46	146	<.001	0.26	43.00	10.51	2.00	118	ns	−0.54
Externalizing
JCP	57.96	12.74	65.84	12.76	−3.44	146	<.001	0.62	51.13	14.82	1.96	118	ns	−0.52
SUB	55.27	10.91	61.39	15.24	−2.76	146	ns	0.50	50.50	11.64	1.61	118	ns	−0.43
AGG	50.95	11.74	64.36	14.23	−5.96	146	<.001	1.07	40.88	4.76	3.38	118	<.001	−0.91
ACT	48.92	9.15	55.32	10.75	−3.69	146	<.001	0.66	41.75	9.33	2.91	118	ns	−0.78
Interpersonal
FML	51.61	10.85	68.52	10.80	−8.68	146	<.001	1.56	41.56	5.78	3.62	118	<.001	−0.97
IPP	48.17	9.37	50.05	10.51	−1.07	146	ns	0.19	45.56	4.93	1.09	118	ns	−0.29
SAV	55.03	11.92	63.18	14.03	−3.60	146	<.001	0.65	46.81	4.74	2.72	118	ns	−0.73
SHY	51.64	11.38	58.95	11.44	−3.57	146	<.001	0.64	43.00	6.40	2.96	118	ns	−0.79
DSF	51.72	11.54	67.68	17.67	−6.51	146	<.001	1.17	45.75	4.78	2.03	118	ns	−0.55
Interest
AES	46.87	10.06	46.34	9.03	0.30	146	ns	−0.05	50.31	8.51	−1.30	118	ns	0.35
MEC	56.90	9.75	56.61	9.11	0.17	146	ns	−0.03	60.06	8.66	−1.22	118	ns	0.33
Personality Psychopathology Five (PSY-5)
AGGR-r	49.48	9.49	51.84	11.03	−1.32	146	ns	0.24	49.88	9.49	−0.16	118	ns	0.04
PSYC-r	52.20	10.62	76.52	18.55	−10.05	146	<.001	1.81	47.56	6.83	1.69	118	ns	−0.45
DISC-r	56.35	10.44	61.98	10.67	−2.98	146	<.001	0.54	51.81	9.11	1.64	118	ns	−0.44
NEGE-r	53.90	10.78	73.36	9.77	−10.31	146	ns	1.85	39.50	6.36	5.20	118	<.001	−1.39
INTR-r	56.78	13.82	64.77	15.91	−3.07	146	ns	0.55	48.50	5.88	2.36	118	ns	−0.63

Note. ES = effect size; Higher Order: EID = Emotional/Internalizing Dysfunction, THD = Thought Dysfunction, BXD = Behavioral/Externalizing Dysfunction; Restructured Clinical (RC): RCd = Demoralization, RC1 = Somatic Complaints, RC2 = Low Positive Emotions, RC3 = Cynicism, RC4 = Antisocial Behavior, RC6 = Ideas of Persecution, RC7 = Dysfunctional Negative Emotions, RC8 = Aberrant Experiences, RC9 = Hypomanic Activation; Somatic/Cognitive: MLS = Malaise, GIC = Gastrointestinal Complaints, HPC = Head Pain Complaints, NUC = Neurological Complaints, COG = Cognitive Complaints; Internalizing Scales: SUI = Suicidal/Death Ideation, HLP = Helplessness/Hopelessness, SFD = Self-Doubt, NFC = Inefficacy, STW = Stress/Worry, AXY = Anxiety, ANP = Anger Proneness, BRF = Behavior-Restricting Fears, MSF = Multiple Specific Fears; Externalizing: JCP = Juvenile Conduct Problems, SUB = Substance Abuse, AGG = Aggression, ACT = Activation; Interpersonal Scales: FML = Family Problems, IPP = Interpersonal Passivity, SAV = Social Avoidance, SHY = Shyness, DSF = Disaffiliativeness; Interest Scales: AES = Aesthetic-Literary Interests, MEC = Mechanical–Physical Interests; Personality Psychopathology Five (PSY-5): AGGR-r = Aggressiveness–Revised, PSYC-r = Psychoticism–Revised, DISC-r = Disconstraint–Revised, NEGE-r = Negative Emotionality/Neuroticism–Revised, INTR-r = Introversion/Low Positive Emotionality–Revised.

After a Bonferroni correction, t-test results for the 42 comparisons of MMPI-2-RF substantive scale scores for the CBIR-OR and WNL groups indicated that 36 scale scores for college participants, 36 scale scores for correctional participants, and 35 scale scores for psychiatric participants were significantly different. In all cases, mean MMPI-2-RF substantive scale scores for the CBIR-OR groups were significantly higher than the WNL groups. Median effect sizes for the CBIR-OR and WNL mean scale score comparisons for the Higher Order (H-O) scales were 1.05 for college participants (range = 0.91 to 1.57), 2.08 for correctional participants (range = 0.63 to 2.46), and 1.56 for the psychiatric participants (range = 0.75 to 1.86). Median effect sizes for the CBIR-OR and WNL mean scale score comparisons for the Restructured Clinical scales were 1.07 for college participants (range = 0.71 to 1.49), 2.07 for correctional participants (range = 0.72 to 2.34), and 1.37 for the psychiatric participants (range = 0.82 to 2.01). For the Specific Problems and Interest scales, median effect sizes for the CBIR-OR and WNL scale score comparisons were .70 for college participants (range = 0.10 to 1.45), 1.41 for correctional participants (range = −0.31 to 2.99), and 1.07 for the psychiatric participants (range = −0.05 to 1.87). Finally, median effect sizes for the CBIR-OR and WNL comparisons for scores on the PSY-5-r scales were 0.59 for college participants (range = 0.24 to 1.64), 1.26 for correctional participants (range = −0.42 to 2.49), and 0.55 for the psychiatric participants (range = 0.24 to 1.85). For the CBIR-OR and WNL effect size analyses, nonsignificant results were included in median calculations.

For the comparison of mean MMPI-2-RF substantive scale scores between CBIR-UR and WNL response style groups for each of the three samples, after a Bonferroni correction, t-test results for the 42 comparisons indicated that 30 scale scores for college participants, 32 scale scores for correctional participants, and 16 scale scores for psychiatric participants were significantly different. When significant differences existed, mean scores for the CBIR-UR groups were significantly lower than the WNL responders’ mean scores. Median effect sizes for the CBIR-UR and WNL mean scale score comparisons for the H-O scales were −0.65 for college participants (range = −0.50 to −1.04), −0.96 for correctional participants (range = −.49 to −1.09), and −0.74 for the psychiatric participants (range = −0.57 to −1.63). Median effect sizes for the CBIR-UR and WNL mean scale score comparisons for the Restructured Clinical scales were −0.61 for college participants (range = −0.28 to −1.13), −0.73 for correctional participants (range = −0.39 to −1.05), and −0.92 for the psychiatric participants (range = −0.37 to −1.42). For the Specific Problems and Interest scales, median effect sizes for the CBIR-UR and WNL scale score comparisons were −0.51 for college participants (range = 0.20 to −0.97), −0.54 for correctional participants (range = 0.24 to −0.85), and −0.73 for the psychiatric participants (range = 0.35 to −1.23). Finally, median effect sizes for the CBIR-UR and WNL comparisons for scores on the PSY-5-r scales were −0.43 for college participants (range = 0.14 to −1.13), −0.50 for correctional participants (range = −0.07 to −0.97), and −0.45 for the psychiatric participants (range = 0.04 to −1.39). As with the CBIR-OR analyses, for the CBIR-UR and WNL effect size analyses nonsignificant results were included in median calculations.

Tables 4, 5, and 6 report the scores on the collateral measures for the CBIR-OR and WNL response style groups, as well as CBIR-UR and WNL response style groups for the college, correctional, and psychiatric participants, respectively. Within the tables, total scores for collateral measures are reported in z scores, and the measures are divided into three broad categorizations (i.e., externalizing, internalizing, and thought dysfunction) based on the general content and/or the construct examined by each collateral measure.

Table 4.

Comparison of Collateral Measure Mean Scores for Within Normal Limits (WNL) Group With Content Responsive Invalidity: Over-Reporting (CBIR-OR) and Content Responsive Invalidity: Under-Reporting (CBIR-UR): College Participants.

Criterion measure (total and/or subscale scores)	WNL (n = 877 to 901)		CBIR-OR (n = 96 to 100)						CBIR-UR (n = 57 to 59)
Criterion measure (total and/or subscale scores)	Mean Z	SD	Mean Z	SD	t	df	p	ES (d)	Mean Z	SD	t	df	p	ES (d)
Externalizing
Barratt Impulsivity Scale
Total	−0.09	0.96	0.73	0.91	−8.13	997	<.001	−0.86	−0.58	0.94	3.731	956	<.001	0.51
Drug Abuse Screening Test
Total	−0.10	0.85	0.65	1.52	−7.62	995	<.001	−0.80	−0.27	0.64	1.437	955	ns	0.19
Michigan Alcohol Screening Test
Total	−0.09	0.88	0.53	1.45	−6.11	989	<.001	−0.65	−0.33	0.62	2.066	948	ns	0.26
State Anger Inventory
Total	−0.06	0.96	0.47	1.19	−5.04	990	<.001	−0.54	−0.41	0.74	2.816	951	ns	0.36
Internalizing
Beck Depression Inventory
Total	−0.08	0.87	0.85	1.44	−9.07	971	<.001	−0.99	−0.60	0.42	4.497	932	<.001	0.57
Screener for Somatoform Disorders
Total	−0.06	0.94	0.86	1.17	−9.11	995	<.001	−0.95	−0.71	0.53	5.235	954	<.001	0.68
Thought
Magical Ideation Scale
Total	−0.12	0.88	0.99	1.29	−11.41	997	<.001	−1.20	−0.55	0.87	3.591	955	<.001	0.47
Perceptual Aberration Scale
Total	−0.15	0.78	1.05	1.62	−12.70	999	<.001	−1.33	−0.31	0.99	1.429	958	ns	0.19

Table 5.

Criterion measure (total and/or subscale scores	WNL (n = 391 to 404)		CBIR-OR (n = 37 to 39)						CBIR-UR (n = 157 to 163)
Criterion measure (total and/or subscale scores	Mean Z	SD	Mean Z	SD	t	df	p	ES (d)	Mean Z	SD	t	df	p	ES (d)
Externalizing
Barratt Impulsivity Scale
Total	0.15	0.88	1.01	0.74	−5.73	439	<.001	−0.99	−0.63	0.93	9.186	559	<.001	0.87
Drug Abuse Screening Test
Total	0.08	1.03	0.44	1.03	−2.04	437	ns	−0.35	−0.31	0.83	4.278	559	<.001	0.40
Michigan Alcohol Screening Test
Total	0.10	1.04	0.31	1.19	−1.18	439	ns	−0.20	−0.32	0.76	4.723	559	<.001	0.43
State Anger Inventory
Total	0.02	0.81	0.73	1.01	−5.04	427	<.001	−0.86	−0.49	0.68	7.132	550	<.001	0.66
Internalizing
Beck Depression Inventory
Total	0.01	0.80	1.79	1.54	−11.94	431	<.001	−2.00	−0.40	0.82	5.518	551	<.001	0.51
Screener for Somatoform Disorders
Total	−0.02	0.88	1.54	1.44	−9.77	439	<.001	−1.66	−0.36	0.72	4.385	564	<.001	0.41
Thought
Magical Ideation Scale
Total	−0.05	0.85	1.11	1.53	−7.44	440	<.001	−1.25	−0.26	0.93	2.592	562	ns	0.24
Perceptual Aberration Scale
Total	−0.10	0.19	0.26	0.55	−8.95	438	<.001	−1.49	−0.11	0.21	0.300	563	ns	0.05

Table 6.

Criterion measure (total and/or subscale scores	WNL (n = 94 to 104)		CBIR-OR (n = 38 to 44)						CBIR-UR (n = 15 to 16)
Criterion measure (total and/or subscale scores	Mean Z	SD	Mean Z	SD	t	df	p	ES (d)	Mean Z	SD	t	df	p	ES (d)
Externalizing
Barratt Impulsivity Scale
Total	0.14	0.83	1.25	0.93	−7.06	141	<.001	−1.29	−0.88	0.75	4.607	114	<.001	1.24
Drug Abuse Screening Test
Total	0.20	1.08	0.71	1.57	−2.26	144	<.001	−0.41	0.09	0.99	.382	116	ns	0.10
Michigan Alcohol Screening Test
Total	0.35	1.16	0.67	1.22	−1.52	139	ns	−0.27	−0.28	0.89	2.047	112	ns	0.56
State Anger Inventory
Total	0.08	0.95	1.15	1.11	−5.87	141	<.001	−1.07	−0.83	0.41	3.725	114	<.001	1.01
Internalizing
Beck Depression Inventory
Total	0.01	0.75	1.57	1.06	−9.54	130	<.001	−1.84	−0.79	0.33	4.159	108	<.001	1.13
Screener for Somatoform Disorders
Total	0.05	0.93	1.02	0.92	−5.79	145	<.001	−1.05	−0.59	0.71	2.642	118	ns	0.71
Thought
Magical Ideation Scale
Total	−0.19	0.75	0.96	1.28	−6.75	144	<.001	−1.22	−0.57	0.70	1.928	116	ns	0.51
Perceptual Aberration Scale
Total	−0.17	0.71	1.09	1.46	−7.06	145	<.001	−1.28	−0.47	0.33	1.642	118	ns	0.45

For the CBIR-OR response style groups, results indicated that the CBIR-OR response style groups demonstrated significantly different mean scores on the majority of the eight measures when compared with the WNL groups after applying a Bonferroni correction. For the college participants, all the externalizing, internalizing, and thought disorders measures differed significantly compared with the WNL group (median effect size [d] = −0.91, range = −0.54 to −1.33). For the correctional participants, two of four externalizing, all internalizing, and all thought disorders measures differed significantly compared with the WNL group (median effect size [d] = −1.12, range = −0.20 to −2.00). Finally, for the psychiatric participants, three of four externalizing, both internalizing, and both thought disorders measures were significantly different (median effect size [d] = −1.15, range = −0.27 to −1.84). The median effect sizes include all results (i.e., significant and nonsignificant) in their calculations. For the scales that demonstrated a statistically significant difference across the three samples, inspection of the mean scores indicated the CBIR-OR group reported increased negative functioning on the criterion measures when compared with the WNL group.

Finally, with respect to the CBIR-UR group, after applying a Bonferroni correction, the CBIR-UR response style group demonstrated significantly different mean scores on a number of the eight measures when compared with the WNL groups. For the college participants, one of four externalizing, all internalizing, and one of two thought disorders measures differed significantly compared with the WNL group (median effect size [d] = 0.42, range = 0.19 to 0.68). For the correctional participants, all externalizing, all internalizing, but no thought disorders measures differed significantly compared with the WNL group (median effect size [d] = 0.42, range = 0.05 to 0.87). Finally, for the psychiatric participants, two of four of the externalizing, one of two internalizing, and no thought disorders measures were significantly different (median effect size [d] = 0.64, range = 0.10 to 1.24). The median effect sizes include all results (i.e., significant and nonsignificant) in their calculations. For the scales that demonstrated a statistically significant difference, inspection of the mean scores indicated the CBIR-UR groups reported increased positive functioning or fewer experiences of psychopathology-related symptoms on the criterion measures compared with the WNL groups across the three samples.

Discussion

The current study examined associations between CBIR response styles, as determined by MMPI-2-RF Validity scale cut scores, and scores on the MMPI-2-RF substantive scales and collateral measures in three groups of participants. As hypothesized, results indicated that individuals identified as potentially engaging in CBIR scored significantly higher on MMPI-2-RF and collateral measures of dysfunction than did participants who responded to the MMPI-2-RF in a valid manner. Specifically, individuals identified as potentially over-reporting were significantly more likely to report experiencing higher levels of social, behavioral, and psychological difficulties on both the MMPI-2-RF substantive scales and non-MMPI-2-RF collateral measures. Those identified as possibly engaging in an under-reporting response style were more likely to report having fewer social, behavioral, and psychological difficulties indicated by their scores on both MMPI-2-RF substantive scales and collateral measures, though the effect sizes of these differences were not as large as those demonstrated for over-reporting. One exception to these trends was that a smaller number of MMPI-2-RF substantive scale scores were lower for the under-reporting group in the psychiatric sample. This likely reflects the reduced power available for analyses conducted with this smaller sample.

Most germane to the purpose of the current study, individuals identified as engaging in a potential CBIR response style by MMPI-2-RF Validity scales demonstrated numerous statistically significant differences on the eight self-report collateral measures when compared with individuals believed to be responding honestly. This result was demonstrated for collateral measures that were administered either at the same time as the MMPI-2-RF or within a range of 1 to 10 days prior to or afterward (depending on the participant sample). For the CBIR-OR groups, all eight measures for the College participants, six of eight measures for the Correctional participants, and seven of eight measures for the Psychiatric participants were significantly different compared with the WNL group. The pattern of differences observed for these groups indicated that MMPI-2-RF-detected over-reporters endorsed significantly higher levels of internalizing, externalizing, and/or thought dysfunction on the collateral measures. Conversely, for the CBIR-UR groups, of the eight analyzed collateral measures, scores on four measures in the College participants, six of the eight measures in the correctional participants, and three of the measures in the psychiatric participants reflected significantly lower levels of externalizing, internalizing, and/or thought dysfunction when compared with the WNL group.

Our ability to use three quite different samples in which the collateral measures were administered with different timing points to a robust finding that invalid responding as detected by the MMPI-2-RF validity indicators generalizes to other measures administered along with the test. As noted earlier, this finding has important implications for clinical and correctional assessments in which the MMPI-2-RF is administered as part of a battery of tests that includes measures that lack validity scales. The implications indicated by validity scale results for interpretation of scores on the MMPI-2-RF substantive scales generalize to other measures and should be applied to their interpretation as well.

Although the current study has a number of strengths, use of “naturally occurring” (rather than simulated) invalid responding, replication in three distinct settings, and varying time frames for the administration of the collateral measures, a number of limitations must be acknowledged as well. Foremost among these is our broad classification of over-reporting or under-reporting based on any elevation on a single validity indicator. In practice, interpretive guidelines for the MMPI-2-RF indicate a need to consider scores on some validity indicators conjointly. For the present investigation, we opted not to do so in order to first explore the generalizability of validity scale findings at the broadest level. This approach led to roughly 9% of college, 6% of correctional, and 27% of psychiatric participants being identified as potentially engaging in CBIR-OR. Conversely, roughly 6% of college, 27% of correctional, and 10% of psychiatric participants were identified as potentially engaging in CBIR-UR. In examining the comparative CBIR-OR percentages, it is possible that some individuals in the psychiatric group who had legitimate psychological distress were misidentified as over-reporting because they had an elevated score on F-r. On the other hand, when examining the percentages of the CBIR-UR classifications across samples, it is not as likely that the discrepancy between the percentage of correctional and other groups reflected a genuinely overly virtuous or high level of psychological adjustment in the correctional participants. Furthermore, in a very limited number of cases, individuals were identified as members of both groups. This finding most likely was the result of using MMPI-2-RF cut scores that were slightly below those of more definitive invalidity as stated in the manual for several of the scales. In any event, although the overall results of the current study are similar to previous findings for the MMPI-2 and MMPI-2-RF substantive scales and collateral measures, future studies might consider using a different method of CBIR group assignment, such as symptom validity testing in combination with MMPI-2-RF Validity scale scores.

A second limitation of the current study involved the inability to examine the potential differences in substantive and collateral measure scores that specific MMPI-2-RF over- and/or under-reporting scales might suggest due to relatively small sample sizes for the CBIR-OR and UR groups. For example, individuals who have elevated scores on Fp-r (a measure of over-reporting severe psychopathology) might have substantially different scores on the MMPI-2-RF substantive scale and/or collateral measures compared with individuals who elevated on Fs (a measure of over-reporting somatic problems). On the other hand, different under-reporting strategies (as reflected by L-r and K-r) might also lead to differential patters on both the MMPI-2-RF substantive scales as well as collateral measures. We encourage researchers with access to considerably larger databases to apply the methodology of the current study to further explore the impact of more specifically delineated response styles.

A third limitation of the current study is that we focused only on self-report collateral measures to explore the impact of CBIR response styles. Therefore, we were not able to examine the potential impact on other information gathered via different means. Future studies should examine whether other forms of client-provided information (e.g., structured and unstructured interviews) might be affected by response biases detected by the MMPI-2-RF validity indicators. A final limitation of the current study involves the fact that only men were examined in the psychiatric and correctional groups. Future studies should attempt to include women from these populations as well as to explore the impact of CBIR styles in additional populations with both genders (e.g., other legal settings, employee screening settings, etc.).

The limitations just noted notwithstanding, our findings suggest that if the MMPI-2-RF Validity scale scores identify an individual as potentially engaging in an exaggerated or over-reporting response style, scores on collateral measures administered either conjointly or within a brief period of time are also likely to reflect an attempt to present one’s self in an overly negative or psychologically dysfunctional manner. Although not as strong as the results for the CBIR-OR analyses, if individuals are suspected of under-reporting on the MMPI-2-RF, they are also likely to suppress scores on collateral measures, reflecting an attempt to present one’s self in an overly positive or psychologically healthy fashion. These findings have both clinical and research implications. Clinically, when the MMPI-2-RF is administered as part of a battery of instruments that includes other self-report scales that lack validity indicators, the current study suggests that the cautions indicated for MMPI-2-RF interpretation should also be considered when interpreting these collateral measures. In addition, the current study suggests that researchers who rely on self-report measures that lack validity scales should consider including measures that include such scales, such as the MMPI-2-RF, in their designs.

Footnotes

Declaration of Conflicting Interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Yossef S. Ben-Porath is a paid consultant to the MMPI publisher, the University of Minnesota, and distributor, Pearson. As coauthor of the MMPI-2-RF, Dr. Ben-Porath receives royalties on sales of the test.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported in part by a grant to Yossef S. Ben-Porath from the University of Minnesota Press, publisher of the MMPI-2.

Notes

References

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental health disorders (4th ed). Washington DC: Author.

Baer

R. A.

Miller

(2002). Underreporting of psychopathology on the MMPI-2: A meta-analytic review. Psychological Assessment, 14, 16-26.

Barratt

E. S.

(1985). Impulsiveness subtraits: Arousal and information processing. In Spence

J. T.

Izard

C. E

(Eds.), Motivation, emotion, and personality (Vol. 5, pp. 137-146). New York, NY: North-Holland.

Beck

A. T.

Ward

C. H.

Mendelson

Mock

Erbaugh

(1961). An inventory for measuring depression. Archives of General Psychiatry, 12, 57-62.

Ben-Porath

Y. S.

(2013). Self-report inventories: Assessing personality and psychopathology. In Graham

J. R.

Naglieri

J. A.

(Eds.), Handbook of psychology: Vol. 10. Assessment psychology (2nd ed., pp. 622-641). New York, NY: Wiley.

Ben-Porath

Y. S.

Tellegen

(2008). MMPI-2-RF (Minnesota Multiphasic Personality Inventory-2 Restructured Form): Manual for administration, scoring, and interpretation. Minneapolis: University of Minnesota Press.

Bianchini

K. J.

Greve

K. W.

Glynn

(2005). Review article: on the diagnosis of malingered pain-related disability: Lessons from cognitive malingering research. Spine Journal, 5, 404-417.

Butcher

J. N.

Graham

J. R.

Ben-Porath

Y. S.

Tellegen

Dahlstrom

W. G.

Kaemmer

(2001). MMPI-2 (Minnesota Multiphasic Personality Inventory-2): Manual for administration and scoring (2nd ed.). Minneapolis: University of Minnesota Press.

Chapman

L. J.

Chapman

J. P.

Raulin

M. L.

(1978). Body-image aberration in schizophrenia. Journal of Abnormal Psychology, 87, 399-407.

10.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

11.

Eckblad

M. L.

Chapman

L. J.

(1983). Magical ideation as an indicator of schizotypy. Journal of Consulting and Clinical and Psychology, 51, 215-225.

12.

Forbey

J. D.

Ben-Porath

Y. S.

(2007). Computerized adaptive personality testing: A review and illustration with the MMPI-2 computerized adaptive version (MMPI-2-CA). Psychological Assessment, 19, 14-24.

13.

Forbey

J. D.

Lee

T. T. C.

(2011). An exploration of the impact of invalid MMPI-2 protocols on collateral self-report measure scores. Journal of Personality Assessment, 93, 556-565.

14.

Garcia

H. A.

Franklin

C. L.

Chambliss

(2010). Examining MMPI-2 F-family scales in PTSD-diagnosed veterans of Operation Enduring Freedom and Operation Iraqi Freedom. Psychological Trauma: Theory, Research, Practice, and Policy, 2, 126-129.

15.

Janca

Burke

J. D.

Isaac

Burke

K. C.

Costa e Silva

J. A.

Acuda

S. W.

Tacchini

(1995). The World Health Organization Somatoform Disorders Schedule: A preliminary report on design and reliability. European Psychiatry, 10, 373-378.

16.

Marion

B. E.

Sellbom

Bagby

R. M.

(2011). The Detection of Feigned Psychiatric Disorders Using the MMPI-2-RF Overreporting Validity Scales: An analog investigation. Psychological Injury and Law, 4, 1-12.

17.

Sellbom

Bagby

R. M.

(2008). Validity of the MMPI-2-RF (Restructured Form) L-r and K-r scales in detecting underreporting in clinical and nonclinical samples. Psychological Assessment, 20, 370-376.

18.

Sellbom

Bagby

R. M.

(2010). Detection of overreported psychopathological with the MMPI-2-RF form validity scales. Psychological Assessment, 22, 757-767.

19.

Selzer

(1971). The Michigan Alcoholism Screening Test: The quest for a new diagnostic instrument. American Journal of Psychiatry, 127, 1653-1658.

20.

Skinner

H. A.

(1982). The Drug Abuse Screening Test. Addictive Behaviors, 7, 363-371.

21.

Slick

Hopp

Strauss

Thompson

G. B.

(1997). Victoria Symptom Validity Test Version 1.0 professional manual. Odessa, FL: Psychological Assessment Resources.

22.

Spielberger

C. D.

(1979). Preliminary manual for the State-Trait Personality Inventory (STPI). Tampa: University of South Florida.

23.

Tellegen

Ben-Porath

Y. S.

(2008/2011). MMPI-2-RF (Minnesota Multiphasic Personality Inventory-2 Restructured Form): Technical manual. Minneapolis: University of Minnesota Press.

24.

Van der Heijden

P. T.

Egger

J. I.

Derksen

J. J.

(2010). Comparability of scores on the MMPI-2-RF scales generated with the MMPI-2 and MMPI-2-RF booklets. Journal of Personality Assessment, 92, 254-259. doi:10.1080/00223891003670208

25.

World Health Organization. (1992). The ICD-10 international classification of mental and behavioural disorders. Clinical descriptions and diagnostic guidelines. Geneva, Switzerland: Author.

26.

Wygant

D. B.

Anderson

J. L.

Sellbom

Rapier

J. L.

Allgeier

L. M.

Granacher

R. P.

(2011). Association of the MMPI-2 Restructured Form (MMPI-2-RF) Validity Scales with structured malingering criteria. Psychological Injury and Law, 4, 13-23.