Abstract
The present study evaluated the validity of Minnesota Multiphasic Personality Inventory-3 (MMPI-3) scores among police (n = 1,294), correctional officer (n = 190), dispatcher (n = 205), and firefighter (n = 237) candidates using psychosocial history data collected with the Psychological History Questionnaire (PsyQ) at a private practice in the Northwestern United States. MMPI-3 scale elevations at T score cutoffs specified in the MMPI-3 User’s Guide for the Public Safety Candidate Interpretive Reports (Corey & Ben-Porath, 2022) were examined. Consistent with previous research using the MMPI-2-RF, MMPI-3 T score means were lower and less variable in this public safety preemployment context relative to the normative sample. In addition, MMPI-3 scores were meaningfully associated with a number of aggregated scale scores derived from PsyQ data, particularly in the behavioral/externalizing domain. To address limited research on preemployment personality testing among female police candidates and the absence of research among nonpolice public safety occupations, Cohen’s q was used to compare validity coefficients across male and female police candidates and across police and correctional officer, dispatcher, and firefighter candidates. Differences were minimal, with all statistically significant effects being small in magnitude, indicating the MMPI-3 correlates identified with police candidates replicate to other public safety positions.
Most states in the United States require police officer candidates to undergo preemployment psychological evaluations after a conditional offer of employment has been tendered (Corey & Borum, 2013), and a growing number of public safety agencies require these evaluations for correctional officers, dispatchers, and firefighters (Corey & Detrick, 2022). These evaluations are intended to ensure the psychological suitability of individuals being hired for these high-stress and high-risk jobs (Chopko et al., 2015; Edwards & Kotera, 2021) in which mental health and interpersonal factors have the potential to negatively impact officer behavior and the safety of the communities they serve (Price, 2017; Price & Pinals, 2018). Indeed, research has demonstrated that inferences drawn from preemployment psychological testing are valid predictors of problematic job performance among police officers (Lough & Ryan, 2005; Lowmaster & Morey, 2012; Roberts et al., 2019; Tarescavage, Brewster et al., 2015; Weiss et al., 2005). One commonly used assessment tool that has been the subject of a sizable volume of research is the Minnesota Multiphasic Personality Inventory-2 Restructured Form (MMPI-2-RF; Corey & Ben-Porath, 2018; Tellegen & Ben-Porath, 2008/2011).
MMPI-2-RF Scores in Preemployment Contexts
Specifically, the validity and utility of MMPI-2-RF scores have been studied predominantly in the context of preemployment evaluations of police candidates (e.g., Corey et al., 2018; Detrick et al., 2016; Roberts et al., 2019; Sellbom et al., 2007; Tarescavage, Brewster et al., 2015; Tarescavage, Corey, & Ben-Porath, 2015; Tarescavage, Corey, Gupton et al., 2015; Tarescavage, Fischler et al., 2015). These studies have evaluated the validity of inferences drawn from MMPI-2-RF scores using other self-report measures, such as the Inwald Personality Inventory (IPI; Detrick et al., 2016; Inwald et al., 1982; Tarescavage, Fischler et al., 2015) and California Psychological Inventory (CPI; Gough & Bradley, 2002; Roberts et al., 2019; Tarescavage, Fischler et al., 2015), as well as supervisor ratings of posthire probationary period performance (Corey et al., 2018; Roberts et al., 2019; Sellbom et al., 2007; Tarescavage, Brewster et al., 2015; Tarescavage, Corey, & Ben-Porath, 2015; Tarescavage, Corey, Gupton et al., 2015; Tarescavage, Fischler et al., 2015), and records of civilian complaints (Sellbom et al., 2007; Tarescavage, Fischler et al., 2015). Collectively, these studies have found lower and less variable MMPI-2-RF scores in police candidate samples relative to the normative sample as well as consistent evidence supporting the criterion validity of MMPI-2-RF scores, especially within the emotional/internalizing domain.
However, in nearly all of these MMPI-2-RF studies, the sample used for validity analyses was limited to male police candidates owing to the relatively small number of female candidates for whom data were available. Therefore, additional research is needed to determine whether the inferences drawn from preemployment MMPI scores can generalize to female police candidates. Moreover, studies using MMPI-2-RF scores (as well as other psychological tests) in preemployment evaluations of nonpolice candidates, such as correctional officer, dispatcher, and firefighter candidates, are very limited. Consequently, and given the recent addition of the MMPI-3, more research is needed to investigate the validity of inferences drawn from MMPI-3 scores in nonpolice and female public safety candidate populations.
MMPI-3 Scores in Preemployment Contexts
An update to the MMPI-2-RF, the MMPI-3 (Ben-Porath & Tellegen, 2020a, 2020b) was released in 2020, along with an updated Police Candidate Interpretive Report (PCIR), public safety preemployment comparison groups, and empirical correlates reported in the MMPI-3 Technical Manual (Ben-Porath & Tellegen, 2020b). The MMPI-3 also includes an updated normative sample (replacing the MMPI-2-RF normative sample, which was collected in the 1980s) and several new and revised scales. New scales include Combined Response Inconsistency (CRIN), Eating Concerns (EAT), Compulsivity (CMP), Impulsivity (IMP), and Self-Importance (SFI). In addition, the MMPI-2-RF Anxiety (AXY) scale was significantly expanded to include a variety of anxiety-related content, and it was, therefore, renamed Anxiety-Related Experiences (ARX) on the MMPI-3. Finally, the MMPI-2-RF Stress/Worry (STW) scale was divided into Stress (STR) and Worry (WRY) scales, and each includes three new items.
Several of these new and revised scales are conceptually relevant in preemployment contexts, particularly CMP, IMP, SFI, ARX, and STR. For example, CMP is related to conscientiousness, which Detrick et al. (2010) demonstrated is a demand characteristic in the context of preemployment evaluations of police candidates. The California Commission on Peace Officer Standards and Training (Spilberg & Corey, 2014/2022) also requires all police candidates to be evaluated on 10 dimensions (the POST-10 psychological screening dimensions), which includes a Conscientiousness and Dependability dimension (related to CMP), an Impulse Control and Attention to Safety dimension (related to IMP), and an Emotional Regulation and Stress Tolerance dimension (related to ARX and STR). SFI scores, which measure beliefs that one has positive attributes and abilities, have been shown to be negatively associated with interpersonal engagement (Whitman et al., 2021; Whitman, Rice et al., 2022), which relates to the POST-10 Assertiveness, Social Competence, and Teamwork dimensions. Accumulating data to evaluate the validity of inferences drawn from these new and revised MMPI-3 scales in preemployment evaluations is especially important given the recent release of the revision.
To date, two peer-reviewed studies investigated the validity and clinical utility of MMPI-3 scores in police and other public safety candidates (Whitman et al., 2021; Whitman, Corey & Ben-Porath, 2022). The first, Whitman et al. (2021), evaluated the validity of inferences drawn from MMPI-3 scores, but also the generalizability of these inferences across male and female police candidates and across police candidates and an aggregate sample of nonpolice public safety candidates. This study involved a record review of preemployment psychological reports provided to the hiring agencies. Coders rated the POST-10 dimensions from “no problems” to “definite problems” based on non-MMPI information contained in the written psychological reports. The authors found that MMPI-3 scores yielded conceptually consistent and relatively large correlations with coder-rated problems in the POST-10 dimensions. This included evidence supporting new MMPI-3 scales. ARX and STR scores yielded large associations with the Emotional Regulation and Stress Tolerance dimension (r = .37 and .41, respectively), and CMP was also meaningfully associated with this dimension (r = .22), but not the Conscientiousness and Dependability dimension (r = .06). IMP scores were associated with problems in the Emotional Regulation and Stress Tolerance and the Teamwork dimensions, but not the IMP and Attention to Safety dimension. Low SFI scores (<39T) were associated with increased risk of problematic performance in domains related to interpersonal engagement.
Importantly, Whitman et al. (2021) also investigated differences in the magnitude of these correlations across men and women and across police and an aggregate sample of nonpolice public safety candidates. The authors reported finding relatively few significant differences, which were generally of small-to-moderate magnitudes. This latter finding was important given the relative paucity of research on preemployment personality testing in female police candidates and the lack of research on nonpolice public safety candidates and lends support for the generalizability of inferences drawn from MMPI-3 scores in male police candidates to other populations.
The second peer-reviewed study on the MMPI-3 in a preemployment context evaluated whether scores were meaningfully different across police candidates with no prior law enforcement experience, up to 5 years of previous experience, and 5 or more years of previous experience (Whitman, Corey & Ben-Porath, 2022). The authors found no practically meaningful differences across groups and concluded that the validity of inferences drawn from MMPI-3 scores could be generalized across police candidates regardless of whether they have prior law enforcement experience. Interestingly, both this study and Whitman et al. (2021) found that CMP scores were within the normal range (i.e., within five T-score points of the normative sample mean of 50T), whereas most other substantive scales were meaningfully lower than the normative sample. This finding is consistent with the Detrick et al. (2010) finding that conscientious (related to CMP) is a demand characteristic in preemployment evaluations of public safety candidates.
PsyQ Biodata Scales in Preemployment Contexts
In addition to personality testing with measures like the MMPI instruments, preemployment psychological evaluations typically involve collection of personal history data. These data can be collected by way of a clinical interview or standardized questionnaires, like the Psychological History Questionnaire (PsyQ; Johnson, Roberts, & Associates, Inc., 2011). The PsyQ is a proprietary tool used to collect psychosocial history data in public safety preemployment psychological evaluations. It consists of core items that assess job-relevant history in several domains, including education, employment, military experience, law enforcement experience, driving record, financial history, legal history, substance use, general information, developmental history, adult relationships, parental responsibilities, psychological treatment and evaluation history, and job-relevant sexual history. If candidates report problems in these domains, they may trigger additional “required explanations” items that collect clarifying information (i.e., who, what, when, where, and why).
The PsyQ has also been used in research on preemployment psychological evaluations; for example, Corey et al. (2018) administered the PsyQ to a large sample (n = 1,945) of police candidates and derived 11 biodata clusters related to externalizing problems using PsyQ items with endorsement frequencies of at least 3%. They then calculated correlations between these biodata clusters and MMPI-2-RF externalizing scales, finding that MMPI-2-RF measures of disinhibition (e.g., Behavioral/Externalizing Dysfunction [BXD], Antisocial Behavior [RC4], Juvenile Conduct Problems [JCP], Substance Abuse [SUB], and Disconstraint [DISC-r]) were more strongly correlated with the biodata clusters than measures of antagonism (e.g., Hypomanic Activation [RC9], Aggression [AGG], Activation [ACT], and Aggressiveness [AGGR-r]). Corey et al. (2018) also calculated correlations between the biodata clusters and posthire outcome variables, finding conceptually expected associations, such as a negative correlation between biodata Physical Aggression Problems and an Assertiveness outcome variable (r = −.12).
In another study, Sellbom et al. (2021) examined associations between the PsyQ biodata cluster scores and Multidimensional Personality Questionnaire (MPQ; Tellegen & Waller, 2008) scores in police candidates. They found conceptually expected associations between biodata clusters and MPQ scores, such as small-to-moderate correlations between biodata Social Competence & Teamwork Concerns and MPQ Negative Emotionality (r = .25), Alienation (r = .17), and Aggression (r = .17). Biodata Alcohol- and Drug-Related Problems were most strongly associated with Constraint in the negative direction (r = −.16 and −.10, respectively). Scores on biodata Work-Related Integrity Problems were associated with MPQ Stress Reactivity (r = .17), Negative Emotionality (r = .15), and Control (r = −.15).
Lastly, Sellbom et al. (2022) expanded on their 2021 study by evaluating the incremental validity of PsyQ biodata cluster and MMPI-2-RF scores for predicting posthire performance ratings among police officers. This study found that scores on the PsyQ biodata clusters and MMPI-2-RF meaningfully augmented each other for predicting performance, supporting the utility of both sources of information in combination. Corey et al. (2018) and Sellbom et al. (2021, 2022) used the PsyQ biodata scores to evaluate the validity of inferences drawn from police preemployment test scores. No research to date has used these biodata cluster scores to investigate the validity of inferences drawn from MMPI-3 scores in this context, or the validity of test scores among nonpolice public safety candidates, such as correctional officer, dispatcher, and firefighter candidates, or female police candidates.
Present Study
The goal of the present study was to investigate the validity of inferences drawn from MMPI-3 scores in preemployment psychological evaluations of public safety candidates. Specifically, we evaluated the convergent and discriminant validity of MMPI-3 scores using psychosocial history data derived from the PsyQ. Based on results from the Sellbom et al. (2022) study using the MMPI-2-RF, we expected that MMPI-3 scales related to low positive emotionality would be negatively associated with biodata Mental Health Concerns and Social Competence & Teamwork Concerns, MMPI-3 scales related to negative emotionality would be meaningfully associated with biodata Adverse Childhood Experiences (ACE), Social Competence & Teamwork concerns, and Work-Related Integrity Problems, and that MMPI-3 scales related to thought disorder would be associated with biodata Work-Related Commitment Problems and Disciplinary Problems.
The present study also sought to offer evaluator data to inform inferences from MMPI-3 scores in preemployment evaluations of understudied public safety applicants, including correctional officer, dispatcher, and firefighter candidates, as well as female police candidates. To do so, we evaluated whether the validity of inferences drawn from MMPI-3 scores can generalize across male and female police candidates and across police candidates and correctional officer, dispatcher, and firefighter candidates. Like Whitman et al. (2021), we compared the strength of validity evidence (i.e., correlations) across male and female police candidates. We also compared the strength of validity evidence across police candidates and nonpolice public safety candidates; however, rather than using an aggregate sample of all nonpolice candidates as did Whitman et al. (2021), we compared correlations across police candidates and subsamples of correctional officer, dispatcher, and firefighter candidates. In addition, inferences drawn from MMPI-3 scores are impacted by consideration of means and standard deviations (SDs) of these scores in preemployment evaluation contexts as well as the frequency with which scales are elevated at particular T score cutoffs. Therefore, we also added to the accumulating literature on MMPI-3 descriptive statistics (means, SDs) and the frequency of MMPI-3 scale elevations at various cutoffs specified in the new MMPI-3 User’s Guide for the Public Safety Candidate Interpretive Reports (Corey & Ben-Porath, 2022) in the total sample of public safety candidates and for each subgroup.
Method
Participants and Procedures
The present sample consisted of 1,928 candidates for public safety positions who were tested during preemployment psychological evaluations in the Northwestern United States. Two candidates were excluded because they produced invalid MMPI-3 protocols. The final sample consisted of 1,926 (21.4% women) candidates for the position of police officer (67.2%; 15.4% women), correctional officer (9.9%; 27.4% women), dispatcher (10.6%; 71.7% women), and firefighter (12.3%; 5.9% women). The average age was 31.29 (standard deviation (SD) = 7.91) years. Most (87.0%) were white, followed by Hispanic/Latinx (9.3%), Black/African American (3.5%), Asian (3.5%), American Indian/Alaska Native (2.3%), Hawaiian/Pacific Islander (2.1%), or another race/ethnicity (0.7%). Half (52.5%) were married, whereas the rest were never married (40.3%), divorced (6.0%), separated (1.1%), or widowed (0.2%).
All data came from an archival dataset, a subset of which was previously analyzed (Menton et al., 2022; Sellbom et al., 2021; Whitman, Corey & Ben-Porath, 2022), and subsets of which is included in the MMPI-3 Police Officer, Correction Officer, Dispatcher, and Firefighter Comparison Groups. Public safety candidates were referred by 154 hiring agencies to the second author’s private practice for psychological evaluations following the conditional offer of employment. All candidates were administered the MMPI-2-RF-Expanded Form (MMPI-2-RF-EX) as a routine part of the assessment battery, which also included the PsyQ. The MMPI-3 was scored from the MMPI-2-RF-EX. The psychometric equivalence of MMPI-3 scores derived from MMPI-3 and MMPI-2-RF-EX administrations has been established (Hall et al., 2022). All testing was proctored and conducted immediately prior to a clinical interview by one of two psychologists.
Data collection was approved by the Kent State University Institutional Review Board [#17-324]. This study was not preregistered. We report how we determined our sample size, all data exclusions, and all measures in the study. Data and study materials that support the findings in this article are available from corresponding author upon reasonable request.
Measures
MMPI-3
The MMPI-3 is a 335-item measure of personality and psychopathology. Its items are scored on 10 Validity Scales, and hierarchically on three Higher-Order (H-O) Scales, eight Restructured Clinical (RC) Scales, and 26 Specific Problems (SP) Scales. Five Personality Psychopathology Five (PSY-5) scales measure a dimensional model of personality psychopathology (Harkness et al., 1995, 2012). The MMPI-3 is an updated version of the MMPI-2-RF, which is frequently used in preemployment evaluations (Corey & Zelig, 2020).
PsyQ
The PsyQ is a self-report questionnaire designed to collect psychological and behavioral history data in the context of public safety preemployment evaluations. It consists of 340 items covering 14 content domains, including education, employment, military experience, law enforcement experience, driving record, financial history, legal history, substance use, general information, developmental history, adult relationships, parental responsibilities, psychological treatment and evaluation history, and job-relevant sexual history. Candidates’ responses to core items may trigger additional clarifying questions.
Sellbom et al. (2021) developed a set of 14 rationally derived content scales for the PsyQ called Mental Health Concerns, Developmental Risk Factors, Social Competence & Teamwork Concerns, Theft-Related Problems, Work-Related Integrity Problems, Past Violations of the Law, Juvenile Conduct Problems, Work-Related Commitment Problems, Disciplinary Problems, Failure to Adhere to Personal Obligations, Alcohol-Related Problems, Drug-Related Problems, Impulsive Driving, and Physical Aggression Problems. They used only fixed items that were answered in the dysfunctional direction by at least 3% of participants and grouped these items into conceptual clusters independently then collaboratively until reaching consensus. For this study, we recognized that all items on the Sellbom et al. (2021) Developmental Risk Factors biodata cluster related to ACE (e.g., experiencing emotional, physical, or sexual abuse or witnessing a caretaker hit another caretaker as a child) except for one item relating to experiencing intimate partner violence as an adult. Therefore, to increase unidimensionality of measurement, we revised the Developmental Risk Factors biodata cluster by deleting the item about adulthood intimate partner violence and renamed the resulting scale ACE.
Data Analyses
We conducted three sets of analyses, each of which yield results that are commonly considered when making inferences from MMPI-3 scores in preemployment contexts: elevation frequencies at interpretive cutoffs, descriptive statistics (i.e., means and SDs), and correlations. For each set of analyses, we compared results across subgroups (police, correctional officer, dispatcher, and firefighter candidates, and male and female police candidates) to investigate whether inferences drawn from MMPI-3 scores can generalize. We also focused on the new or substantially revised MMPI-3 scales that we identified above as having particular relevance in preemployment contexts (CMP, IMP, SFI, ARX, and STR).
Specifically, we first calculated the frequency at which candidates produced elevated MMPI-3 scales at the cutoffs specified in the MMPI-3 User’s Guide for the Public Safety Candidate Interpretive Reports (Corey & Ben-Porath, 2022). Next, we calculated means and SDs for MMPI-3 scales among the total sample, male and female police candidates separately, and each of the four occupations separately. Consistent with previous research, we considered any mean differences of at least half a SD (5T) to be practically meaningful. Contrary to null hypothesis significance testing, focusing on a practically meaningful effect size criterion is directly relevant to inferences drawn from MMPI-3 scores in preemployment settings. We expected most scales to have lower and less variable scores in all subsamples; however, based on results reported by Whitman et al. (2021), we expected CMP scores to be in the normal range.
Third, we calculated correlation coefficients between the 14 biodata scales developed by Sellbom et al. (2021) and MMPI-3 scale scores among the total sample and each of the subsamples. Based on recommendations by Funder and Ozer (2019) and consistent with past MMPI-3 research in a public safety preemployment evaluation context (Whitman et al., 2021), we considered correlations of r ≥ │.20│ to be practically meaningful. We also compared the magnitude of correlations using Cohen’s q (Cohen, 1992), or the difference between two Fisher’s z-converted correlations, to investigate whether the strength of validity coefficients differed across subsamples. Specifically, we compared correlations across male and female police candidates, and across police candidates and each of the three other public safety occupations for any meaningful correlations among the total sample or among all correctional officer, dispatcher, and firefighter candidates. We used police candidates as a reference group because this group has been the primary focus of the MMPI-2-RF literature in public safety settings (see above summary), whereas less research has accumulated on personality testing for other occupations. We reported statistically significant (p < .05) differences across subgroups and focused our interpretation on Cohen’s q effect sizes, with q values of .10, .30, and .50 as indicative of small, medium, and large effects, respectively. We did not have sufficient sample sizes (≥100 each gender) to calculate elevation frequencies, means, SDs, and correlations for male and female candidates separately within the other public safety occupations (e.g., we did not have at least 100 male and 100 female correctional officers to compare results across these subsamples).
Results
MMPI-3 Elevations and Descriptive Statistics
Table 1 displays the frequency of MMPI-3 scale elevations at T score cutoffs specified by the MMPI-3 User’s Guide for the Public Safety Candidate Interpretive Reports (Corey & Ben-Porath, 2022) for each subgroup, and Supplementary Table 1 shows elevation frequencies in the total sample. The total sample produced few clinical elevations (≥65T), ranging from none on several MMPI-3 scales to 5.2% on CMP, and less than 1.0% of the sample producing clinical elevations on most scales. To account for such low elevation rates, which are consistent with previous findings with the MMPI-2-RF, the authors of the MMPI-3 User’s Guide for the Public Safety Candidate Interpretive Reports (Corey & Ben-Porath, 2022) also specified lower T score cutoffs, associated empirically with negative findings in police candidates, for interpretation in public safety preemployment evaluations. The frequency of elevations at these lower cutoffs ranged from 0.1% (Dysfunctional Negative Emotions [RC7]) to 5.1% (Social Avoidance [SAV]), with a median of 1.6%. Low Scores (≤40T) on SFI and Dominance (DOM) are also interpretable, with 5.1% and 4.2% of the sample meeting these criteria, respectively.
MMPI-3 Scale Score Elevation Frequencies in Each Subsample.
Note. MMPI-3 = Minnesota Multiphasic Personality Inventory-3; PO = police officer candidate; CO = correctional officer candidate; DIS = dispatcher candidate; FF = firefighter candidate; PO-M = male police officer candidate; PO-F = female police officer candidate.
The same general pattern was apparent for each of the subgroups, with very low proportions of the samples producing clinical elevations, but slightly larger proportions having interpretable scores at lower cutoffs. There were generally small differences in elevation frequencies when comparing correction officer, dispatcher, and firefighter candidates to police candidates (for whom there is more research available). The median difference across police and correctional officer candidates was −0.1% (with correctional officer candidates higher), across police and dispatcher candidates was −0.3% (with dispatcher candidates scoring higher), and across police and firefighter candidates was 0.2% (with police candidates scoring higher). The largest position-based differences were observed across police and dispatcher candidates, with a larger proportion of dispatcher candidates producing elevated scores. For example, 2.6% of police candidates produced Shyness (SHY) scores ≥55T, whereas 12.2% of dispatcher candidates produced SHY scores ≥55T. Of note, relatively small differences in elevation frequencies across positions on the new and revised MMPI-3 scales identified as particularly relevant in preemployment contexts.
Across male and female police candidates, the median difference in elevation frequency was 0.0%, and the largest difference was 3.8%. A larger proportion of male police candidates (5.8%) produced clinical SFI elevations than female police candidates (2.0%). Regarding the new and revised MMPI-3 scales, differences were generally small, with the exception of clinically elevated SFI scores.
Table 2 shows means and SDs for the total sample, for male and female police candidates, and for each of the four occupations separately. Consistent with previous research, the current samples scored meaningfully higher than the normative sample on the two underreporting scales, Uncommon Virtues (L) and Adjustment Validity (K), with the exception of dispatcher candidates and female police candidates, who produced normal-range mean scores on L. Mean K scores were particularly high in the total sample (66.37T), with each subsample producing a mean K score of at least 64.38T. All of the subsamples scored within four T score points of one another on L and K.
Means and SDs.
Note. PC = police candidate; CO = correctional officer candidate; DIS = dispatcher candidate; FF = firefighter candidate; CRIN = Combined Response Inconsistency; VRIN = Variable Response Inconsistency; TRIN = True Response Inconsistency; F = Infrequent Responses; Fp = Infrequent Psychopathology Responses; Fs = Infrequent Somatic Responses; FBS = Symptom Validity; RBS = Response Bias Scale; L = Uncommon Virtues; K = Adjustment Validity; EID = Emotional/Internalizing Dysfunction; THD = Thought Dysfunction; BXD = Behavioral/Externalizing Dysfunction; RCd = Demoralization; RC1 = Somatic Complaints; RC2 = Low Positive Emotions; RC4 = Antisocial Behavior; RC6 = Ideas of Persecution; RC7 = Dysfunctional Negative Emotions; RC8 = Aberrant Experiences; RC9 = Hypomanic Activation; MLS = Malaise; NUC = Neurological Complaints; EAT = Eating Concerns; COG = Cognitive Complaints; SUI = Suicide/Death Ideation; HLP = Helplessness/Hopelessness; SFD = Self-Doubt; NFC = Inefficacy; STR = Stress; WRY = Worry; ARX = Anxiety-Related Experiences; ANP = Anger Proneness; BRF = Behavior-Restricting Fears; FML = Family Problems; JCP = Juvenile Conduct Problems; SUB = Substance Abuse; IMP = Impulsivity; ACT = Activation; AGG = Aggression; CYN = Cynicism; SFI = Self-Importance; DOM = Dominance; DSF = Disaffiliativeness; SAV = Social Avoidance; SHY = Shyness; AGGR = Aggressiveness; PSYC = Psychoticism; DISC = Disconstraint; NEGE = Negative Emotionality/Neuroticism; INTR = Introversion/Low Positive Emotionality.
Consistent with previous research, the current sample produced means and SDs on substantive scales that are lower and less variable than the normative sample. In the total sample, only mean scores on CMP (45.51T), SFI (49.31T), DOM (48.30T), AGGR (46.28T), and Introversion/Low Positive Emotionality (INTR; 45.97T) were >45T. All of the subsamples produced similar mean T scores on substantive scales with the greatest difference being 3.80 T score points on SHY across dispatcher and firefighter candidates. Scores were also less variable relative to the normative sample. Median SDs for the total sample and police and firefighter candidates were all less than half that of the normative sample (<5T), whereas median SDs were 5.25T and 5.43T for correctional officer and dispatcher candidates, respectively. In contrast and by definition, SDs on all MMPI-3 scales equal 10T for the normative sample.
Correlations
Correlations between MMPI-3 and biodata scores are presented for the total sample in Table 3, and for male and female police candidates in Supplementary Tables 2 and 3, and for police officer, correctional officer, dispatcher, and firefighter candidates separately in Supplementary Tables 4–7, respectively. In the total sample, we found meaningful associations (r ≥ │.20│) between MMPI-3 scores and several biodata scores, particularly in the behavioral/externalizing domain. For example, MMPI-3 BXD was associated with Theft-Related Problems, Past Violations of Law, Juvenile Conduct Problems, Physical Aggression Problems, and Alcohol- and Drug-Related Problems. Scores on RC4 yielded meaningful associations with the same variables, except for Drug-Related Problems. MMPI-3 JCP and DISC scale scores also yielded generally similar patterns of associations, with both scales yielding the strongest correlation with biodata Juvenile Conduct Problems (r = .61 with JCP and r = .54 with DISC). Scores on the SUB scale, which was expanded on the MMPI-3 to include more content related to opioid and other drug use, yielded associations that were strongest with the biodata scales Alcohol- and Drug-Related Problems (r = .38 for each).
Correlations Between MMPI-3 and PsyQ Scale Scores in the Total Sample.
Note. Correlations ≥ |.20|
ACE scores yielded meaningful positive associations with MMPI-3 ARX and several externalizing scales, including BXD, RC4, FML, JCP, and DISC. Some biodata scales (e.g., Mental Health Concerns, Social Competence & Teamwork Concerns, Failure to Adhere to Personal Obligations, Disciplinary Problems, Work-Related Commitment Problems, Work-Related Integrity Problems, and Impulsive Driving) showed no meaningful associations with MMPI-3 scales in the total sample, although some of these biodata scales did yield meaningful associations with MMPI-3 scores in particular subsamples (e.g., Social Competence & Teamwork Concerns was meaningfully associated with several MMPI-3 internalizing scales among correctional officer, dispatcher, and firefighter candidates).
Scores on the new and substantially revised MMPI-3 scales of interest in this study were not meaningfully associated with any of the biodata scales in the total sample. However, CMP scores were meaningfully associated with Work-Related Integrity Problems among female police, dispatcher, and firefighter candidates. IMP scores were meaningfully associated with Theft-Related Problems and Impulsive Driving among dispatcher candidates. SFI scores yielded meaningful associations with Work-Related Integrity problems among female police and dispatcher candidates and with Drug-Related Problems among correctional officer candidates. ARX scores were meaningfully associated with ACE among dispatcher and correctional officer candidates, with Impulsive Driving among dispatcher candidates, and with Social Competence and Teamwork Concerns among firefighter candidates. Finally, STR scores were meaningfully associated with ACE, as well as less Failure to Adhere to Personal Obligation and Work-Related Commitment Problems among female police candidates, as well as Social Competence and Teamwork Concerns and Work-Related Integrity Problems among correctional officer and firefighter candidates, and with Impulsive Driving among dispatcher candidates.
To directly compare validity evidence across male and female police candidates and across police candidates and correctional officer, dispatcher, and firefighter candidates, we calculated Cohen’s q for variables that yielded meaningful correlations in the total sample or among all correctional officer, dispatcher, and firefighter candidates (see Table 4). Across male and female police candidates, validity coefficients were generally of comparable magnitude, with only 3/40 (7.5%) statistically significant differences, which is just above the expected Type I error rate, and all were small effects. Across police candidates and candidates for other public safety positions, 13/120 (10.8%) q values were statistically significant, and all effect sizes were small. Most (7/13) meaningful differences in the magnitude of validity coefficients occurred for comparisons across police and dispatcher candidates. Correlations between MMPI-3 Negative Emotionality/Neuroticism (NEGE) and Social Competence & Teamwork Concerns, Malaise (MLS) and Work-Related Integrity Problems, and Cognitive Complaints (COG) and Theft-Related Problems were slightly larger among dispatcher than police candidates, whereas correlations between MMPI-3 and biodata scales measuring externalizing tendencies tended to be slightly stronger among police candidates relative to dispatcher candidates. Nevertheless, associations between these externalizing scales were still relatively large among dispatcher candidates; for example, MMPI-3 BXD and the Juvenile Conduct Problems biodata scale had a correlation of r = .52 among police candidates and .40 among dispatcher candidates.
Comparisons of Validity Findings Across Genders and Positions.
Note. q = Cohen’s q; — indicates that Cohen’s q is not statistically significant (p < .05). MMPI-3 = Minnesota Multiphasic Personality Inventory-3; PC = Police Candidate; CO = Correctional Officer Candidate; DIS = Dispatcher Candidate; FF = Firefighter Candidate; L = Uncommon Virtues; K = Adjustment Validity; BXD = Behavioral/Externalizing Dysfunction; RC4 = Antisocial Behavior; FML = Family Problems; ARX = Anxiety-Related Experiences; JCP = Juvenile Conduct Problems; COG = Cognitive Complaints; DISC = Disconstraint; EID = Emotional/Internalizing Dysfunction; STR = Stress; NEGE = Negative Emotionality/Neuroticism; MLS = Malaise; SUB = Substance Abuse.
Discussion
The present study adds to the literature available to guide interpretation of MMPI-3 scores in preemployment psychological evaluations by reporting frequencies of interpretable elevations, as well as means and SDs and validity coefficients for MMPI-3 scores in a large sample of public safety candidates. Notably, these results supported generalizability in the validity of inferences drawn from MMPI-3 scores across subsamples of police and correctional officer, dispatcher, and firefighter candidates, as well as male and female police candidates. Results from the present study are generally consistent with published studies on MMPI-2-RF scores in public safety preemployment evaluations, which predominantly used samples consisting of male police candidates, as well as two studies that investigated MMPI-3 scores in a similar context. Evidence also supported the validity of inferences to be drawn from some new and substantially revised scales on the MMPI-3.
Police candidates consistently produced lower and less variable scale scores on the MMPI-2-RF (Detrick et al., 2016; Roberts et al., 2019; Tarescavage, Brewster et al., 2015; Tarescavage, Corey, & Ben-Porath, 2015; Tarescavage, Corey, Gupton et al., 2015; Tarescavage, Fischler et al., 2015), MMPI-3 (Whitman et al., 2021; Whitman, Corey & Ben-Porath, 2022) and other tests of personality and psychopathology, such as the Personality Assessment Inventory (e.g., Lowmaster & Morey, 2012) and the California Psychological Inventory (in this case, higher scores indicated fewer reported problems; e.g., Roberts et al., 2019). Although Whitman et al. (2021) reported similar findings for MMPI-3 in an aggregate sample of nonpolice public safety candidates, no previous peer-reviewed study reported these findings for MMPI-3 scores in position-specific samples. The present findings indicate similar factors may contribute to low and less variable scores in correctional officer, dispatcher, and firefighter candidates, which influences interpretations of nonclinical elevations among these populations. Factors contributing to lower and less variable scores likely include underreporting in the context of preemployment evaluation (as indicated by high L and K scores), as well as being preselected for good adjustment relative to the general population. In employment law, preemployment psychological evaluations are considered “medical tests” and can only be conducted following the tender of a conditional offer of employment (Equal Employment Opportunity Commission [EEOC], 1995); thus, these candidates have already undergone extensive interviews and background checks, and have been judged otherwise suitable for hiring, and, therefore, the subjects in this study contain comparatively few candidates with problematic behavioral and psychological histories.
Lower and less variable test scores in preemployment contexts also indicate a need for lower interpretive cutoffs than traditionally used in other settings, such as mental health treatment clinics. Indeed, the frequency of scale elevations at the lower cutoffs specified in the MMPI-3 User’s Guide for the Public Safety Candidate Interpretive Reports (Corey & Ben-Porath, 2022) was higher than the largely zero-inflated elevation frequencies at the traditional 65T cutoff. Overall, elevations at specified cutoffs, which trigger interpretive, or inferential, statements, occur at similar rates across positions and across male and female police candidates. However, relative to police candidates, higher proportions of dispatcher candidates may produce interpretable scores on select scales, such as SHY, SAV, and SFD. The User’s Guide for the MMPI-3 Public Safety Candidate Interpretive Reports (Corey & Ben-Porath, 2022) accounts for these differences by specifying lower T score cutoffs to trigger interpretive statements among police, correctional officer, and firefighter candidates on SHY and SAV, but not among dispatcher candidates. Rather than a 57T elevation on RC2 triggering interpretive statements, as is the case with most public safety interpretive reports, the Dispatcher Candidate Interpretive Report requires a 60T elevation to trigger interpretive statements. Evaluators should ensure that they are using the proper cutoffs and consider the most appropriate, position-specific base rates when interpreting MMPI-3 scale elevations, and especially when making inferences among emergency communications dispatcher candidates where research supporting these statements has predominantly used police candidates.
In the present sample, scores on some MMPI-3 scales, including two new scales (CMP and SFI), as well as DOM, AGGR, and INTR, were within the normal range (45–55T). This finding was similar to the one reported by Whitman et al. (2021), who found that scores on CMP, SFI, DOM, and AGGR, as well as JCP and ACT were within the normal range in a different public safety candidate sample. Taken together, these studies indicate that public safety candidates may not yield range-restricted scores on these scales. This finding is not unexpected given the context. For example, CMP measures a tendency to repeatedly check things and be perfectionistic, both qualities related to conscientiousness, which is a demand characteristic in preemployment contexts (Detrick et al., 2010). Notably, means on CMP were within the normal range (45.51T in the present sample and 46.7T in Whitman et al., 2021), indicating that individuals have normal, rather than problematically high or exceptionally low, standings on these traits. Similarly, SFI measures individuals’ beliefs that they have special attributes and abilities, with low scores indicating a lack of such beliefs. In a preemployment context, one would not expect candidates to report lacking positive abilities and attributes. Finally, low scores on DOM and AGGR indicate interpersonal passivity, or difficulty asserting oneself in the service of obtaining a goal. In a public safety context, candidates would be expected to be assertive; therefore, normal range standing (neither low nor high scores) is likely important for suitability (Spilberg & Corey, 2014/2022).
The present study also investigated the convergent and discriminant validity of MMPI-3 scores using biodata scales derived from the PsyQ, an inventory of job-relevant psychological and behavioral history. We found evidence supporting the validity of inferences from MMPI-3 scores in the total sample, particularly within the behavioral/externalizing domain. Like Corey et al. (2018), we found stronger associations between biodata and externalizing scales that measure disinhibition (e.g., BXD, RC4, JCP, SUB, and DISC) relative to antagonism (e.g., RC9, ACT, AGG, and AGGR). Some biodata scale scores, including Impulsive Driving, Work-Related Integrity Problems, Work-Related Commitment Problems, Disciplinary Problems, Failure to adhere to Personal Obligations, Social Competence & Teamwork Concerns, and Mental Health Concerns, were not meaningfully associated with any MMPI-3 scale scores among the total sample, which was inconsistent with our hypotheses based on results reported by Sellbom et al. (2022). In previous research, Impulsive Driving was not meaningfully associated with any MMPI-2-RF externalizing scale scores (Corey et al., 2018) and Impulsive Driving and Failure to Adhere to Personal Obligations were also nonmeaningfully associated with MPQ scores. However, findings that MMPI-3 scales are not meaningfully associated with some of these constructs are inconsistent with past research. For example, Social Competence & Teamwork Concerns and Mental Health Concerns were not associated with MMPI-3 scores in the present study, but Whitman et al. (2021) found particularly robust correlations between MMPI-3 scores and ratings of these POST-10 dimensions from psychological reports and MMPI-3 scale scores. The lower frequency of MMPI-3 and biodata correlations considered meaningful (r ≥ .20), particularly in the police candidate sample, may also reflect the broad use of a “bifurcated model” to screen candidates in the present sample, which involved the removal of candidates whose backgrounds and scores on normal-range personality testing were judged disqualifying (see Corey & Zelig, 2020; Jones et al., 2010 for further discussion of the bifurcated model of preemployment screening). More research is needed to determine whether these findings fail to replicate, or whether the lack of meaningful associations with these particular biodata scales is due to problems with the operationalization of these biodata criteria.
The Sellbom et al. (2021) biodata scale, Developmental Risk Factors (also derived from the PsyQ), was revised to focus solely on abuse and maltreatment during childhood and adolescence. Meaningful associations were found between the newly derived ACE and MMPI-3 scales measuring ARX, which was expected, as well as externalizing scales (BXD, RC4, FML, JCP, and DISC). This latter finding is consistent with a large body of literature (see, e.g., Hamby et al., 2021) on the relationships between ACE, both single-trauma and cumulative, on a wide array of biopsychosocial outcomes, including negative legal and behavioral health consequences (Portwood et al., 2021).
Importantly, the present study also compared correlations with biodata criteria across male and female police candidates separately, and across police candidates and position-specific subsamples, including correctional officer, dispatcher, and firefighter candidates. Overall, differences across gender and positions were minimal, with just above a chance level of significant differences, and with those significant differences having exclusively small effect sizes. Moreover, in many cases where the strength of associations was significantly different, the correlation was strong for both groups. For example, although the biodata Juvenile Conduct Problems scale was more strongly associated scores on the MMPI-3 JCP scale among male (r = .61) than female (r = .46) police candidates, both correlations are relatively large compared with associations typically observed in these contexts. Therefore, these findings indicate that the validity of inferences drawn from MMPI-3 scores is comparable across these four public safety positions and across male and female police candidates; however, based on elevation frequency analyses, evaluators can expect to see different rates of elevations triggering interpretive statements, especially across dispatcher and police candidates. Altogether, these findings contribute to a growing literature on the validity of MMPI-3 scores in nonpolice public safety occupations and among female police candidates, indicating that the empirical correlates identified in the extensive MMPI-2-RF and emerging MMPI-3 literature on preemployment assessment of police candidates is likely to generalize across gender and public safety position.
The present study is not without limitations. For one, the proportion of women in the present sample was small (21.4% in the total sample and as low as 5.9% of firefighter candidates). However, these numbers are not necessarily unrepresentative of the proportions of women in these positions; for example, according to the International Association of Women in Fire and Emergency Services, of over 300,000 paid firefighters, only 3.7% are women (Hulett et al., 2008). Owing to small subsamples (<100 individuals) of female correctional officer and firefighter candidates, and a small subsample of male dispatcher candidates, we were unable to report findings across genders within each of these positions. Similarly, the present sample overrepresented white public safety candidates (87.0%), while underrepresenting Black/African American (3.5%) and Hispanic/Latinx (9.3%) public safety candidates. Castaneda and Ridgeway (2010) estimated the proportions of Black and Hispanic police recruits to be 18% and 14%, respectively. Future research using more diverse samples in terms of gender, racial, and ethnic composition is needed, particularly to address whether preemployment test scores demonstrate differential predictive validity.
Second, the present study relied on concurrently administered psychosocial history criteria. Future research on the predictive validity of MMPI-3 scores among police candidates and candidates for other public safety positions is of interest. Several studies using prospective designs have investigated the utility of preemployment MMPI-2-RF scores for predicting problematic behaviors of police candidate during a posthire probationary period (Corey et al., 2018; Roberts et al., 2019; Sellbom et al., 2007; Tarescavage, Brewster et al., 2015; Tarescavage, Corey, & Ben-Porath, 2015; Tarescavage, Corey, Gupton et al., 2015; Tarescavage, Fischler et al., 2015); however, nearly all of these studies excluded female police candidates, and no published studies have used similar prospective designs to predict posthire behaviors for other public safety positions.
Overall, the present study offered data to guide evaluator inferences drawn from MMPI-3 scores by reporting elevation frequencies at clinically relevant cutoffs, descriptive statistics, and validity coefficients among candidates for four public safety occupations and among male and female police candidates separately. Convergent and discriminant validity evidence for MMPI-3 scores was investigated using psychosocial history variables derived from the PsyQ.
Supplemental Material
sj-docx-1-asm-10.1177_10731911221138931 – Supplemental material for Associations Between MMPI-3 and Psychosocial History Findings Obtained in Preemployment Evaluations of Public Safety Candidates
Supplemental material, sj-docx-1-asm-10.1177_10731911221138931 for Associations Between MMPI-3 and Psychosocial History Findings Obtained in Preemployment Evaluations of Public Safety Candidates by Megan R. Whitman, David M. Corey and Yossef S. Ben-Porath in Assessment
Footnotes
Authors’ Note
The statements and opinions in this article are those of the authors. The data that support the findings of this study are available from the corresponding author upon reasonable request.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Regarding conflicts of interest, D.M.C. and Y.B.-P. receive research funding from the MMPI-3 test publisher, the University of Minnesota Press. As co-author of the MMPI-3, Y.B.-P. receives royalties on sales of the test. D.M.C. and Y.B.-P. receive royalties on sales of the MMPI-3 Public Safety Candidate Interpretive Reports and associated materials.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research is grant funded by the University of Minnesota Press.
Supplemental Material
Supplemental material for this article is available online.
Public Significance Statement
The present study evaluated the convergent and discriminant validity of MMPI-3 scores using psychosocial history data collected in the context of preemployment evaluations of public safety candidates. We found comparable validity of scores across male and female police candidates and across police and correctional officer, dispatcher, and firefighter candidates.
