Assessment

In this study, we explored the temporal stability of the Personality Assessment Inventory (PAI), which has not been comprehensively reexamined since it was first published. Our three specific aims were to determine the utility of PAI indicators of basic protocol validity (inconsistent responses [ICN] and highly unusual/unlikely responses [INF]) in identifying suspect responding; calculate the stability coefficients for each PAI scale and subscale across 3-, 6-, and 9-week spans; and explore whether profile stability across four measurements could be prospectively predicted. We administered the PAI to a sample of undergraduates (N = 579) at four separate timepoints. ICN and INF effectively identified likely attriters and inconsistent responders. All PAI full scales and subscales evidenced good test–retest reliability, with some small exceptions. Finally, all PAI clinical scales were correlated with profile instability although many of these correlations were no longer significant when controlling for mean clinical elevation of the profile. We interpreted these results as evidence for the utility of PAI validity scales, the temporal reliability of the PAI, and the role of psychopathology in response variability over time. We also discussed some preliminary evidence that this variability can be prospectively predicted, suggesting that it in part reflects substantive changes rather than random error variance.

Bifactor models are increasingly being utilized to study latent constructs such as psychopathology and cognition, which change over the lifespan. Although longitudinal measurement invariance (MI) testing helps ensure valid interpretation of change in a construct over time, this is rarely and inconsistently performed in bifactor models. Our review of MI simulation literature revealed that only one study assessed MI in bifactor models under limited conditions. Recommendations for how to assess MI in bifactor models are suggested based on existing simulation studies of related models. Estimator choice and influence of missing data on MI are also discussed. An empirical example based on a model of the general psychopathology factor (p) elucidates our recommendations, with the present model of p being the first to exhibit residual MI across gender and time. Thus, changes in the ordered-categorical indicators can be attributed to changes in the latent factors. However, further work is needed to clarify MI guidelines for bifactor models, including considering the impact of model complexity and number of indicators. Nonetheless, using the guidelines justified herein to establish MI allows findings from bifactor models to be more confidently interpreted, increasing their comparability and utility.

Intolerance of uncertainty (IU) is a risk factor for poor mental health. Acquired brain injury (ABI; for example, stroke, traumatic brain injury) often brings considerable uncertainty and increased mood disorder vulnerability. The Intolerance of Uncertainty Scale–Short Form (IUS-12) is a brief, well-validated IU measure in non-ABI samples, comprising two subscales, namely, Prospective Anxiety and Inhibitory Anxiety. Here, for the first time, we investigated its reliability and validity (N = 118), and factor structure (N = 176), in ABI. Both subscales had high test–retest reliability (intraclass correlation coefficients [ICCs] of .75 and .86) and were significantly associated with mood disorder symptoms. The two-factor model was superior to a one-factor IU model fit. Some fit statistics were less than optimal (standardized root mean square residual [SRMR] = 0.06, root mean square error of approximation [RMSEA] = 0.09); hence, exploration of other factor structures in other ABI samples may be warranted. Nonetheless, the IUS-12 appears suitable in ABI.

Cognitive reserve (CR) represents the adaptive response of the cognitive system responsible for preserving normal functioning in the face of brain damage. Experiential factors such as education, occupation, and leisure activities influence the development of CR. Theoretically, such factors build up from childhood and across adulthood. Thus, appropriate tools to define and measure CR as early as adolescence are essential to understand its developmental processes. To this aim, we introduce the construct of “Cognitive Reserve Potential” (CRP) and its corresponding index of experiential factors tailored to youth. We investigated prototypical youth exposures potentially associated with the lifelong development of CR (e.g., sport practice, musical experiences, cultural activities, and relationships with peers and family). Principal component analysis and confirmatory factor analysis identified and replicated the CRP factor structure on two independent samples of Italian students: N = 585 (295 F) and N = 351 (201 F), ages 11 to 20. CRP was associated mainly with family socio-cultural status (i.e., socioeconomic status [SES], Home Possessions, and Books at Home). Results confirmed the strength of the factorial model and warranted the proposal of the CRP-questionnaire as an innovative tool for understanding CR evolutionary dynamics.

Despite problematic internet use being especially high among adolescents, there are no screening instruments in Spain specifically for adolescents that would facilitate early detection of this problem. The main goal of this study was to validate the Compulsive Internet Use Scale (CIUS) in the adolescent Spanish population as well as to analyze the discriminative capacity of CIUS based on sociodemographic characteristics, grade point average, and other addictive behaviors. Data were obtained from the ESTUDES, a representative survey of addictive behaviors of Spanish adolescents. The sample consisted of a total of 34,308 adolescents aged between 14 and 18 years old (M_age = 15.70; SD = 1.19; 51.7% females). Results indicated that the CIUS fit a unidimensional structure, exhibited measurement invariance with respect to sex and age, and demonstrated excellent reliability (ω = .94). Past-month tobacco, alcohol, and cannabis use, as well as past-year gambling and gaming, were related to CIUS scores. A standardized screening instrument that provides valid, reliable information on young people’s use of the Internet in Spain is a critical requirement for successful early detection and intervention in this population.

The study of narcissism has been hindered by conceptual, theoretical, and measurement in-consistencies. In this article, we report two studies that tested a novel unified conceptualization and theoretical approach to narcissism using the Unified Narcissism Scale–Revised. Study 1 revised the recently developed Unified Narcissism Scale to construct a preliminary 40-item measure in a sample of 395 American participants (M_age = 41). We confirmed the five-factor first-order model, the two-factor second-order model, and the one-factor third-order model. Study 2 considered the cross-cultural performance of the revised scale in the Chinese language in China (N = 326, M_age = 25.5 years) and in the English language in Sri Lanka (N = 354 M_age = 28.7 years) and constructed a final 35-item measure. In conducting these studies, we have demonstrated the cross-cultural importance of entitlement and self-esteem to the conceptualization of narcissism and suggest that the negative relationship between narcissism and agreeableness may be culture-specific to Western samples (as evidenced by the absence of this relationship in non-Western samples). In this article, we have constructed a measure of narcissism that has refined our understanding of the construct and created a tool to capture this understanding.

We evaluated how the number of response options affects the psychometric properties of the Big Five Inventory-2 (BFI-2). Using two large samples collected from a market research company (Ns = 893 and 1,213), we tested how different response options of the BFI-2 influenced scale score distributions, internal consistency estimates, convergent validity correlations, and criterion validity correlations. Results suggest that score distributions were impacted by the number of response options such that ceiling and floor effects were more common when using two or three response options than when using more options. Estimates of Cronbach’s alpha were generally lower with fewer scale points as compared with more scale points, but these effects disappeared when ordinal alpha was used. There were no systematic effects of response options on convergent validity and criterion validity correlations. Given these results, there seems to be few psychometric reasons for deciding whether to administer personality items with five, six, or seven scale points.

The “Dark” Triad (DT) refers to three personality constructs with ties to socially aversive behaviors: psychopathy, narcissism, and Machiavellianism. These constructs are commonly assessed via omnibus self-report inventories such as the Short Dark Triad (SD3) or the Dirty Dozen. Alternatively, researchers wishing to measure “dark” traits can compile stand-alone measures of each construct. Recently, the Five Factor Model Antagonistic Triad Measure (FFM ATM) was developed, which measures the DT from the perspective of the widely used Five Factor Model of personality. Initial validation studies indicated that the FFM ATM addresses common concerns with other omnibus inventories (e.g., allows for multifaceted examination of DT constructs). The current study tested the FFM ATM in relation to these other methods of measuring the DT (i.e., omnibus inventories and combinations of single-construct measures). Across three tests of validity (i.e., nomological network analysis, intraclass correlations, and incremental validity analyses), the FFM ATM showed favorable results and outperformed other measures of the DT.

The development of efficient and reliable online assessments has become increasingly important in the digital era. We developed a 10-min online word reading assessment of global English based on the existing paper-and-pencil version of our English silent word reading test. The test includes two parts, namely, random word recognition and contextual word reading. A total of 889 participants (437 children and 392 adults; 62.7% female) took part in the study. They were from various regions including mainland China, Hong Kong SAR, Macau SAR, Poland, the United States, and the Philippines. Reliability and validity analyses on various demographics samples (by age and country/region of origin) demonstrated that the WordSword Test is highly reliable and valid (e.g., the correlation of this test with other English reading measures were above .80). Education level was positively correlated with test performance, while the correlations between age and test performance were not consistent. Ninety-seven children participants also took the paper-and-pencil version of the WordSword Test. The correlation between performances on the online and paper-and-pencil versions of the test was .879, one year apart. With more children and adults taking the WordSword Test, we ultimately hope to establish norms by area, grade level, and age.

Anecdotal evidence has suggested that rater-based measures (e.g., parent report) may have strong across-trait/within-individual covariance that detracts from trait-specific measurement precision; rater measurement-related bias may help explain poor correlation within Autism Spectrum Disorder (ASD) samples between rater-based and performance-based measures of the same trait. We used a multi-trait, multi-method approach to examine method-associated bias within an ASD sample (n = 83). We examined performance/rater-instrument pairs for attention, inhibition, working memory, motor coordination, and core ASD features. Rater-based scores showed an overall greater methodology bias (57% of variance in score explained by method), while performance-based scores showed a weaker methodology bias (22%). The degree of inter-individual variance explained by method alone substantiates an anecdotal concern associated with the use of rater measures in ASD.

Commensurate measures of alcohol-related consequences across countries and cultures are critical for addressing the global burden of hazardous alcohol use. The Rutgers Alcohol Problem Index (RAPI), developed and validated in the United States, is a popular measure of alcohol problems. This study examined measurement invariance of the RAPI across samples of U.S. and Swedish high school seniors. Latent mean differences in alcohol problems across countries and differences in associations between alcohol problems with alcohol use and protective behavioral strategies (PBS) were also examined. The RAPI was scalar invariant. Swedish students reported fewer problems than U.S. students (latent mean difference = −0.19, p = .047). In both samples, the RAPI was positively correlated with alcohol use frequency and quantity (ps < .001), and negatively correlated with PBS use (ps < .05). Overall, the RAPI demonstrated measurement invariance, and we found evidence for its validity across samples of U.S. and Swedish high school seniors.

No tools quantify the experience, psychological, and practical impact of receiving a diagnosis from a non-deficit perspective. Autism is increasingly late diagnosed in adulthood. The Impact of Diagnosis Scale (IODS) was initially developed for borderline personality disorder. We aimed to develop a revised version suitable for autistic adults and potentially other diagnostic groups. Following a trial of a preliminary revision, the researchers and autistic research advisors co-produced an expanded pool of 46 items, scored on 7-point Likert-type scale, within 6 hypothesized domains. Scale reduction processes were applied to data from 125 formally diagnosed autistic adults. Following iterative rounds of factor analysis using maximum likelihood estimation with Promax rotation, 22 items were retained across 4 domains to comprise the IODS-R. The IODS-R adds new understanding to the experience of receiving an autism diagnosis in adulthood. It may be useful for evaluating diagnostic services and other diagnostic groups.

The Eating Disorder Examination–Questionnaire (EDE-Q) is a widely used self-report measure of eating pathology. Despite widespread use, investigations of its factor structure have proved inconclusive and rarely supported the “original” interpretation. The current study evaluates several proposed factor solutions of the EDE-Q using latent variable analysis in a sample of adult women with anorexia nervosa (AN). A total of 804 patients from a specialist treatment center in the United States participated in the study. Confirmatory factor analysis was conducted on 22 EDE-Q items assessing attitudinal features of eating pathology. Findings suggested that three full-item versions (none of which was the “original” interpretation) fit the data adequately, with a brief, seven-item version showing excellent fit. The study is one of the first to examine this within a sample of women with AN and provides an empirical foundation for how best to use the EDE-Q among clinical and research participants with AN. Findings suggest that the “original” factor structure lacks structural validity in women with AN. Its use should generally be discouraged, and future work on screening and treatment outcomes might consider the EDE-Q7.

This study evaluated the performance characteristics, construct validity, and reliability of two computerized, self-administered verbal and visual recognition memory tests based on the Remember-Know paradigm. Around 250 healthy control participants and 440 patients referred for neuropsychological assessment used an iPad to complete the Words and Faces recognition memory tests before or after concurrent neuropsychological testing. Performance accuracy was high but without ceiling effects. Education, but not age, was related to overall performance for both samples while the influence of gender and race differed across samples. In the clinical sample, overall performance was worse in those patients demonstrating memory impairment on clinical assessment. Words and Faces subtests demonstrated the strongest correlations with neuropsychological measures of verbal and nonverbal memory, respectively. Both showed moderate correlations with processing speed while Faces was also correlated with visuospatial skills. The memory tests showed good test–retest reliability over two testing sessions. These findings demonstrate acceptable psychometric properties in clinical and community samples and suggest that this computerized format is feasible for memory assessment in clinical contexts.

Culture influences responses to psychological measurements in ways unrelated to target constructs, thus biasing test scores and potentially contributing to under- and over-diagnosis of mental health problems in populations for which measures have not yet been normed. We conducted a systematic review of publications addressing response style among Latinx population groups in North and South America. In a final corpus of 24 studies, Latinx/Latin American populations were generally found to exhibit higher levels of extreme response style (n = 17), acquiescent response style (n = 10), and socially desirable responding (n = 5). The few publications (n = 3) that investigated midpoint responding reported no differences. Seven publications (29%) attempted to adjust scores to mitigate response style bias, using both scale design and statistical techniques. Findings suggest that researchers and clinicians should directly assess culturally patterned response style as a construct, rather than inferring style indirectly using other measures. For clinicians, knowledge of response style represents another facet of case conceptualization.