Cross-Validation of Multiple WAIS-IV Digit Span Embedded Performance Validity Indices Among a Large Sample of Adult Attention Deficit/Hyperactivity Disorder Clinical Referrals

Abstract

This study investigated the utility of four WAIS-IV Digit Span (DS) indices (traditional Reliable Digit Span [RDS], RDS-Working Memory [RDS-WM], RDS-Revised [RDS-R], and DS Age-Corrected Scaled Score [ACSS]) as embedded performance validity tests (PVTs) among a sample of 342 consecutive adults referred for neuropsychological evaluation of ADHD. All DS indices had acceptable classification accuracy (areas under the curve: .73–.76) for detecting invalid performance with optimal cut-scores of RDS ≤7 (35% sensitivity/93% specificity), RDS-WM ≤7 (56% sensitivity/86% specificity), RDS-R ≤12 (48% sensitivity/85% specificity), and ACSS ≤7 (46% sensitivity/87% specificity). Although all indices were able to detect invalid performance, DS indices incorporating the more complex working memory trials of the task yielded the best accuracy for identification of invalid test performance among adults referred for ADHD evaluation.

Keywords

neuropsychology assessment embedded measures tests ADHD

Introduction

Attention-Deficit/Hyperactivity Disorder (ADHD) is a prevalent psychiatric disorder that commonly results in referral for neuropsychological evaluation (Faraone & Biederman, 2005; Kessler et al., 2006; Schroeder, Martin, & Walling, 2019). Despite being a frequent referral question for neuropsychological evaluations, accurate assessment of ADHD is often complicated by concerns of invalid test performance, especially given the potential external incentives associated with a diagnosis, such as access to psychostimulant medication and academic/standardized testing accommodations (e.g., Teter et al., 2005). ADHD is also highly heterogeneous in terms of associated cognitive sequelae (e.g., Pievsky & McGrath, 2018), thereby leading to variability in neuropsychological performance and difficulty establishing standards for adequate test engagement and the presence of self-reported symptoms, particularly in adults (Barkley et al., 2011; Faraone et al., 2004; Suhr et al., 2008). Prior research reported high bases rate of performance invalidity (i.e., 20–50%; Martin & Schroeder, 2020; Suhr et al., 2008; Sullivan et al., 2007) and exaggerated symptom reporting (e.g., 21%; Leib et al., 2021) among adults referred for ADHD evaluation. These findings underscore the importance of performance validity testing in this population (Hirsch & Christiansen, 2018; Marshall et al., 2010; Resch et al., 2021), mirroring more general practice standards calling for routine administration of performance validity tests (PVTs) during all neuropsychological evaluations (Sweet et al., 2021).

The use of both freestanding and embedded PVTs not only ensures that test results are an accurate representation of a patient’s true cognitive abilities, but also increases confidence in conclusions drawn from such evaluations in clinical and research settings. Embedded PVTs may have particular utility given their ability to assess validity at numerous time points throughout an evaluation without inflating testing time (Boone, 2009). Among embedded PVTs, Reliable Digit Span (RDS; Greiffenstein et al., 1994) is one of the most heavily researched, commonly used, and well-validated embedded PVTs (Boone, 2007; Martin et al., 2015). RDS was originally developed using the Wechsler Adult Intelligence Scale-Third Edition (WAIS-III; Wechsler, 1997) Digit Span (DS) subtest, which only included Forward and Backward trials. RDS was later revised for the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV; Wechsler, 2008) and included the newly added Sequencing trial (RDS-R). Although RDS, and, to a lesser extent, RDS-R and DS age-corrected scaled score (ACSS) have been extensively cross-validated in a multitude of clinical populations (e.g., Schroeder et al., 2012; Webber & Soble, 2018), there is a relative paucity of research examining DS PVTs specifically among adult ADHD populations. This is surprising considering that DS is frequently administered as a test of attention/working memory or as part of the full WAIS-IV IQ testing during ADHD evaluations. As such, this study aimed to examine the utility of DS indices as embedded PVTs among a large sample of adults referred for ADHD evaluation as well as to compare classification accuracy and corresponding sensitivity/specificity of these indices, determine optimal cut-scores, and assess which index most accurately detects performance invalidity among adult ADHD referrals.

Method

Participants

This cross-sectional study examined data from a consecutive academic medical center sample of 348 adults referred for neuropsychological evaluation of possible ADHD by their treating physician or psychiatrist and who consented to including their data for research. Six patients were missing a criterion PVT and thereby excluded, resulting in a final sample of 342. The sample had a mean age of 27.8 (SD = 6.67), education of 15.7 years (SD = 2.06), and Test of Premorbid Functioning (TOPF; Pearson, 2009) estimated Full-Scale IQ of 105.88 (SD = 7.75). Sex distribution was 38% male (N = 130) and 62% female (N = 212). The sample had the following ethnoracial composition: Non-Hispanic White (N = 159; 46%), Hispanic (N = 84; 25%), Black (N = 48; 14%), Asian (N = 32; 9%), and other race/ethnicity (N = 19; 6%). All patients identified English as their primary language and completed evaluation procedures in English. Sixty-eight percent (N = 231) were college students. Eighty-five percent were classified into the valid group (N = 290) and 15% in the invalid group (N = 52) based on performance on seven, independent criterion PVTs (see Criterion PVT section below). Among the 290 in the valid group, 235 met formal diagnostic criteria for ADHD (see ADHD Diagnostic Workup section below).

Measures

ADHD Diagnostic Workup

All patients received a detailed and multimodal assessment to determine if they met Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition (DSM-5; APA, 2013) criteria for ADHD. This workup included: 1) a thorough chart/record review (including previous testing and academic records, if available); 2) a semi-structured clinical interview that assessed DSM-5 criteria and concomitant mental disorders, as well psychosocial and developmental history; 3) the Clinical Assessment of Attention Deficit-Adult; (CAT-A; Bracken & Boatwright, 2005), an inventory of childhood and adulthood ADHD symptoms with embedded symptom validity scales (Leib et al., 2021; White, Ovsiew, et al., 2020); 4) a uniform, comprehensive neuropsychological test battery; and 5) the Minnesota Multiphasic Personality Inventory-Second Edition-Restructured Form (Ben-Porath & Tellegen, 2008).

Criterion PVTs

All patients were administered a battery of seven freestanding and embedded PVTs that previously have been cross-validated in ADHD samples or ADHD-predominant mixed clinical samples, including the Dot Counting Test (Abramson et al., 2021; Boone et al., 2002); Rey 15-Item Test Recall/Recognition (Ashendorf et al., 2021), Rey Auditory Verbal Learning Test Effort Score (Pliskin et al., 2021), Brief Visuospatial Memory Test-Revised Recognition Discrimination (Bailey et al., 2018; Resch et al., 2020), Stroop Test Word Reading T-Score (White, Korinek, et al., 2020); Trail Making Test - Part A (White, Korinek, et al., 2020), and Verbal Fluency (F/A/S; White, Korinek, et al., 2020). Consistent with current empirical findings and neuropsychology practice standards, those with ≤1 criterion PVT failures were classified into the valid group and those with ≥2 failures were assigned to the invalid group (e.g., Critchfield et al., 2019; Jennette et al., 2021; Larrabee, 2014; Rhoads et al., 2021; Schroeder, Martin, Heinrichs, & Baade, 2019; Soble et al., 2020).

Digit Span PVTs

As part of their comprehensive neuropsychological evaluation, all patients were administered the WAIS-IV DS subtest. Four previously identified DS PVTs were examined in this study: (1) traditional RDS (Forward/Backward), (2) RDS-WM (Backward/Sequencing), (3) RDS-R (Forward/Backward/Sequencing), and (4) DS ACSS.

Data Analytic Plan

Spearman correlations evaluated associations among the DS indices and criterion PVTs in the valid group. Descriptive analyses were calculated for each DS PVT, and independent samples t-tests examined significant performance differences between the validity groups. Receiver operating characteristic (ROC) curve analyses determined the classification accuracy for each DS PVT, and optimal cut-scores that maximized sensitivity/specificity were selected for each DS PVT with a statistically significant area under the curve (AUC). Classification accuracy (AUCs) was interpreted as low (.50–.69), acceptable (.70–.79), excellent (.80–.89), or outstanding (≥.90; Hosmer et al., 2013). Optimal cut-scores were defined in this study as those that maximized sensitivity for invalid detection while maintaining minimally accepted specificity levels (i.e., >84% but preferable closer to >90%; Boone, 2013; Larrabee, 2003). All analyses were first conducted for the overall sample (i.e., all adult ADHD referrals), and then replicated among the subsample of patients who met DSM-5 diagnostic criteria for ADHD to investigate whether the four embedded DS PVTs functioned similarly in a more homogenous ADHD sample.

Results

DS PVTs demonstrated large, significant correlations with each other, but nonsignificant to small correlations with the criterion PVTs, suggesting the DS PVTs were largely independent from the criterion PVTs used to establish validity groups (see Table 1). Performance on the DS PVTs by validity group is presented in Table 2. In the overall sample, the valid group performed significantly better (i.e., higher scores) across all four DS PVTs relative to the invalid group, with large effects. Consistently, comparison of valid and invalid group performance within the ADHD subsample resulted in nearly identical findings as those observed in the overall sample. Given the sample diversity, DS differences based on racial group (i.e., White vs. non-White) were examined and revealed statistically significant differences (ps < .01) with White patients obtaining higher scores across RDS (M = 10.65; SD = 2.05), RDS-WM (M = 9.68; SD = 1.75), RDS-R (M = 16.01; SD = 2.58), and ACSS (M = 11.01; SD = 2.55) relative to non-White patients across RDS (M = 9.86; SD = 2.18), RDS-WM (M = 9.08; SD = 2.06), RDS-R (M = 14.86; SD = 2.91), and ACSS (M = 9.80; SD = 2.83), but effect sizes were minimal (ηp²s = .02–.04), indicating that racial group membership accounted for little variance in performance across these four DS indices.

Table 1.

Intercorrelations between Performance Validity Tests among the Valid Group.

	DCT	RFIT	FAS	RAVLT ES	BVMT-R RD	TMT-A	Stroop	RDS	RDS-WM	RDS-R	ACSS
DCT	–	−.21^**	−.03	−.13^**	−.04	−.13^*	−.29^**	−.17^**	−.23^**	−.21^**	−.27^**
RFIT	–	–	−.06	.11	.05	−.04	−.06	.14^*	.25^**	.19^**	.21^**
FAS	–	–	–	−.01	.05	.13^**	.16^**	.17^**	.14^*	.18^**	.18^**
RAVLT ES	–	–	–	–	.06	.06	.04	.01	.11	.07	.10
BVMT-R RD	–	–	–	–	–	.02	−.03	.03	.11	.09	.08
TMT-A	–	–	–	–	–	–	.20^**	−.05	.03	.01	.01
Stroop	–	–	–	–	–	–	–	.15^*	.13^*	.19^**	.17^**
RDS	–	–	–	–	–	–	–	–	.77^**	.92^**	.86^**
RDS-WM	–	–	–	–	–	–	–	–	–	.92^**	.84^**
RDS-R	–	–	–	–	–	–	–	–	–	–	.92^**
ACSS	–	–	–	–	–	–	–	–	–	–	–

Note. n = 290. DCT: Dot Counting Test; RFIT: Rey 15-Item Test/Recognition; FAS: Verbal Fluency Test; RAVLT ES: Rey Auditory Verbal Learning Test Effort Score; BVMT-R RD: Brief Visuospatial Memory Test-Revised Recognition Discrimination; TMT-A: Trail Making Test-Part A; RDS: Reliable Digit Span Forward and Backward; RDS-WM: RDS Backward and Sequencing; RDS-R: RDS Forward, Backward, and Sequencing; ACSS: Digit Span Age-Corrected Scaled Score.

*p < .05.

**p < .01.

Table 2.

Performance on Digit Span Embedded Validity Indictors by Validity Group.

			t	d
Overall Sample/Invalid	Valid (n = 290) M (SD)	Invalid (n = 52) M (SD)
RDS	10.23 (2.15)	8.46 (1.89)	−5.54***	0.87
RDS-WM	9.36 (1.94)	7.56 (1.64)	−6.32***	1.00
RDS-R	15.40 (2.82)	12.67 (2.46)	−6.55***	1.03
ACSS	10.37 (2.77)	7.67 (2.25)	−6.64***	1.07
ADHD Subsample/Invalid	Valid (n = 235)M (SD)	Invalid (n = 52)M (SD)
RDS	10.25 (2.05)	8.46 (1.89)	−5.77***	0.91
RDS-WM	9.31 (1.88)	7.56 (1.64)	−6.23***	0.99
RDS-R	15.38 (2.72)	12.67 (2.46)	−6.61***	1.05
ACSS	10.30 (2.72)	7.67 (2.25)	−6.50***	1.05

Note. RDS: Reliable Digit Span Forward and Backward; RDS-WM: RDS Backward and Sequencing; RDS-R: RDS Forward, Backward, and Sequencing; ACSS: Digit Span Age-Corrected Scaled Score.

***p < .001.

Among the overall sample, all four DS PVTs significantly differentiated valid from invalid performance with acceptable classification accuracies (i.e., AUCs = .73–.76; see Table 3). With respect to optimal cut-scores, RDS ≤7 had 35% sensitivity/93% specificity, RDS-WM ≤7 had 56% sensitivity/86% specificity, RDS-R ≤12 had 48% sensitivity/85% specificity, ACSS ≤7 had 46% sensitivity/87% specificity.

Table 3.

Accuracy of the Digit Span Embedded Validity Indicators for the Overall Sample.

Cutoff	SN (%)	SP (%)	30% Base Rate		20% Base Rate		10% Base Rate
Cutoff	SN (%)	SP (%)	PPV	NPV	PPV	NPV	PPV	NPV
RDS
AUC = .73***
≤5	.06	.99	0.72	0.71	0.60	0.81	0.40	0.90
≤6	.14	.98	0.75	0.73	0.64	0.82	0.44	0.91
≤7	.35	.93	0.68	0.77	0.56	0.85	0.36	0.93
≤8	.48	.79	0.49	0.78	0.36	0.86	0.20	0.93
≤9	.75	.60	0.45	0.85	0.32	0.91	0.17	0.96
RDS-WM
AUC = .76***
≤5	.10	.99	0.81	0.72	0.71	0.81	0.53	0.91
≤6	.25	.90	0.52	0.74	0.38	0.83	0.22	0.92
≤7	.56	.86	0.63	0.82	0.50	0.89	0.31	0.95
≤8	.73	.63	0.46	0.84	0.33	0.90	0.18	0.95
≤9	.89	.44	0.41	0.90	0.28	0.94	0.15	0.97
RDS-R
AUC = .77***
≤9	.08	.99	0.77	0.72	0.67	0.81	0.47	0.91
≤10	.17	.99	0.88	0.74	0.81	0.83	0.65	0.91
≤11	.33	.93	0.67	0.76	0.54	0.85	0.34	0.93
≤12	.48	.85	0.58	0.79	0.44	0.87	0.26	0.94
≤13	.67	.75	0.53	0.84	0.40	0.90	0.23	0.95
≤14	.79	.59	0.45	0.87	0.33	0.92	0.18	0.96
ACSS
AUC = .78***
≤5	.19	.98	0.80	0.74	0.70	0.83	0.51	0.92
≤6	.27	.95	0.70	0.75	0.57	0.84	0.38	0.92
≤7	.46	.87	0.60	0.79	0.47	0.87	0.28	0.94
≤8	.67	.74	0.52	0.84	0.39	0.90	0.22	0.95
≤9	.79	.58	0.45	0.87	0.32	0.92	0.17	0.96

Note. N = 342. RDS: Reliable Digit Span Forward and Backward; RDS-WM: RDS Backward and Sequencing; RDS-R: RDS Forward, Backward, and Sequencing; ACSS: Digit Span Age-Corrected Scaled Score; AUC: area under the curve; SN: sensitivity; SP: specificity; PPV: positive predictive value; NPV: negative predictive value. Bolded scores denote optimal cut-scores for detecting invalid performance.

***p < .001.

Comparison of the valid and invalid groups among the ADHD subsample (Table 4) yielded identical results as the overall sample. In particular, classification accuracies remained acceptable (i.e., AUCs = .74–.78), and optimal cut-scores mirrored those in the overall sample (RDS ≤7; RDS-WM ≤7; RDS-R ≤12; ACSS ≤7), with nearly exact sensitivity/specificity values.

Table 4.

Accuracy of the Digit Span Embedded Validity Indicators for the Subsample Diagnosed with Attention-Deficit/Hyperactivity Disorder.

Cutoff	SN (%)	SP (%)	30% Base Rate		20% Base Rate		10% Base Rate
Cutoff	SN (%)	SP (%)	PPV	NPV	PPV	NPV	PPV	NPV
RDS
AUC = .74***
≤5	.06	1.00	1.00	0.71	1.00	0.81	1.00	0.91
≤6	.14	.98	0.75	0.73	0.64	0.82	0.44	0.91
≤7	.35	.95	0.75	0.77	0.64	0.85	0.44	0.93
≤8	.48	.80	0.51	0.78	0.38	0.86	0.21	0.93
≤9	.75	.61	0.45	0.85	0.32	0.91	0.18	0.96
RDS-WM
AUC = .76***
≤5	.10	1.00	1.00	0.72	1.00	0.82	1.00	0.91
≤6	.25	.95	0.68	0.75	0.56	0.84	0.36	0.92
≤7	.56	.85	0.62	0.82	0.48	0.89	0.29	0.95
≤8	.73	.63	0.46	0.84	0.33	0.90	0.18	0.95
≤9	.89	.44	0.41	0.90	0.28	0.94	0.15	0.97
RDS-R
AUC = .77***
≤9	.08	.99	0.77	0.72	0.67	0.81	0.47	0.91
≤10	.17	.99	0.88	0.74	0.81	0.83	0.65	0.91
≤11	.33	.94	0.70	0.77	0.58	0.85	0.38	0.93
≤12	.48	.85	0.58	0.79	0.44	0.87	0.26	0.94
≤13	.67	.75	0.53	0.84	0.40	0.90	0.23	0.95
≤14	.79	.61	0.46	0.87	0.34	0.92	0.18	0.96
ACSS
AUC = .78***
≤5	.19	.98	0.80	0.74	0.70	0.83	0.51	0.92
≤6	.27	.94	0.66	0.75	0.53	0.84	0.33	0.92
≤7	.46	.87	0.60	0.79	0.47	0.87	0.28	0.94
≤8	.67	.75	0.53	0.84	0.40	0.90	0.23	0.95
≤9	.79	.58	0.45	0.87	0.32	0.92	0.17	0.96

Note. N = 235. RDS: Reliable Digit Span Forward and Backward; RDS-WM: RDS Backward and Sequencing; RDS-R: RDS Forward, Backward, and Sequencing; ACSS: Digit Span Age-Corrected Scaled Score; AUC: area under the curve; SN: sensitivity; SP: specificity; PPV: positive predictive value; NPV: negative predictive value. Bolded scores denote optimal cut-scores for detecting invalid performance.

***p < .001.

Discussion

This study cross-validated four previously identified PVTs derived from the WAIS-IV DS subtest in a large sample of adults referred for neuropsychological evaluation of ADHD. Among the total sample, results revealed that all four DS PVTs (i.e., RDS, RDS-WM, RDS-R, DS ACSS) reliably differentiated valid from invalid performance and demonstrated acceptable classification accuracies (see Table 3). Among the subsample of patients who met DSM-5 diagnostic criteria for ADHD, all four DS PVTs significantly distinguished valid from invalid performance and produced nearly identical classification accuracy statistics to the overall sample. Therefore, these results indicate that these cut-scores can be used in the evaluation of suspected ADHD. Importantly, given the high correlations between these four DS scores, they cannot be used as four independent PVTs as this may inflate the false positive rate (Boone, 2013). Rather, only one DS score (and preferably the one with the strongest psychometrics) should be selected as the PVT. Interestingly, although all four DS PVTs were able to detect performance invalidity, the traditional RDS PVT evidenced the least robust psychometric properties when compared to RDS-WM and RDS-R, even at a higher cut-score of ≤7, suggesting that the latter embedded DS PVTs may be more advantageous in detecting suboptimal performance when examining a patient with suspected ADHD. Finally, although the DS ACSS showed adequate detection of performance invalidity, it may ultimately be less useful in actual clinical practice, as the optimal cut-score represents a score that falls in the low average range (i.e., ≤7ss), which may lead to difficulties in distinguishing suboptimal engagement from a genuine weakness on this measure (see Ovsiew et al., 2020). Overall, the results from the current study are broadly consistent with research regarding the utility of RDS indices in the detection of invalid performance, as findings from previous studies resulted in similar classification accuracy across RDS indices (Babikian et al., 2006). However, previous findings were mixed regarding the sensitivity/specificity of individual RDS measures, indicating further research is needed to establish consistent guidelines regarding the use of individual RDS indices as PVTs (Webber & Soble, 2018; Webber et al., 2020; Young et al., 2012).

Methodological strengths of this current study included the utilization of a demographically diverse sample and the use of a standardized and multimodal diagnostic assessment protocol to facilitate the accurate diagnosis of ADHD. Additionally, the study analyses were replicated with the subsample of patients who met DSM-5 diagnostic criteria for ADHD to establish that the four DS PVTs functioned comparably regardless of whether patients met ADHD diagnostic criteria. Finally, a standardized battery of diverse and well-validated criterion PVTs were used to detect suboptimal neuropsychological performance, which maximizes the accuracy of our validity classifications and optimizes internal validity (Bilder et al., 2014). Despite these strengths, results should be interpreted in the context of a few limitations. Many of the criterion PVTs used to ascertain patients’ validity status were embedded measures, which are often less psychometrically robust than freestanding PVTs (e.g., Bain et al., 2021; Ovsiew et al., 2020; Pliskin et al., 2021). However, the seven independent criterion PVTs administered as a part of this neuropsychological battery evidenced adequate psychometric properties in other cross-validation studies. Considering research indicating that aggregating validity indices improves their capacity to detect invalidity (e.g., Larrabee, 2008), a high rate of false positive or false negative classification errors is unlikely in the current investigation. Further, corroborating informant reports of ADHD symptoms were largely unavailable; however, this is not uncommon in assessment of adult ADHD. Finally, although the sample’s mean estimated premorbid FSIQ (i.e., 105.88) closely approximated the population mean, the mean education attainment (i.e., 15.7 years) of the sample was relatively high. Thus, study findings may not generalize to individuals with ADHD and lower educational attainment such that replication among an ADHD sample with greater educational diversity is warranted.

In sum, results of this study emphasize the clinical utility of using RDS as an embedded PVT in neuropsychological evaluations for ADHD. Findings further suggest that RDS-WM and RDS-R, although less commonly used than RDS and DS ACSS, may better detect invalid test performance due to their high degree of sensitivity (56% and 48%, respectively) and specificity (86% and 85%, respectively). Because the WAIS-IV is already commonly used in ADHD evaluations (Woods et al., 2002), RDS indices (particularly RDS-WM and RDS-R) can easily be incorporated as additional PVTs as a part of a comprehensive neuropsychological assessment.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Hanaan Bing-Canar

Zachary J. Resch

Jason R. Soble

References

Abramson

D. A.

White

D. J.

Rhoads

Carter

D. A.

Hansen

N. D.

Resch

Z. J.

Jennette

K. J.

Ovsiew

G. P.

Soble

J. R.

(2021). Counting Test among an adult ADHD clinical sample and analyzing the effect of ADHD subtype and comorbid psychopathology. Assessment. Advance online publication.

American Psychiatric Association (2013). Diagnostic and statistical manual of mental disordersDSM-5) (5th ed.). American Psychiatric Publishing.

Ashendorf

Clark

E. L.

Humphreys

C. T.

(2021). The Rey 15-Item Memory Test in US veterans. Journal of Clinical and Experimental Neuropsychology, 43(3), 324-331. https://doi.org/10.1080/13803395.2021.1932761.

Babikian

Boone

K. B.

Arnold

(2006). Sensitivity and specificity of various digit span scores in the detection of suspect effort. The Clinical Neuropsychologist, 20(1), 145-159. https://doi.org/10.1080/13854040590947362.

Bailey

K. C.

Soble

J. R.

Bain

K. M.

Fullen

(2018). Embedded performance validity tests in the Hopkins Verbal Learning Test-Revised and the Brief Visuospatial Memory Test-Revised: A replication study. Archives of Clinical Neuropsychology, 33(7), 895-900. https://doi.org/10.1093/arclin/acx111.

Bain

K. M.

Soble

J. R.

Webber

T. A.

Messerly

J. M.

Bailey

K. C.

Kirton

J. W.

McCoy

K. J. M.

(2021). Cross-validation of three Advanced Clinical Solutions performance validity tests: Examining combinations of measures to maximize classification of invalid performance. Applied Neuropsychology: Adult, 28(1), 24-34. https://doi.org/10.1080/23279095.2019.1585352.

Barkley

R. A.

Knouse

L. E.

Murphy

K. R.

(2011). Correspondence and disparity in the self- and other ratings of current and childhood ADHD symptoms and impairment in adults with ADHD. Psychological Assessment, 23(2), 437-446. https://doi.org/10.1037/a0022172.

Ben-Porath

Y. S.,

Tellegen

(2008). MMPI-2-RF: Manual for administration, scoring, and interpretation. University of Minnesota Press.

Bilder

R. M.

Sugar

C. A.

Hellemann

G. S.

(2014). Cumulative false positive rates given multiple performance validity tests: Commentary on Davis and Millis (2014) and Larrabee (2014). The Clinical Neuropsychologist, 28(8), 1212-1223. https://doi.org/10.1080/13854046.2014.969774.

10.

Boone

K. B.

(2007). Assessment of feigned cognitive impairment: A neuropsychological perspective. Guilford Press.

11.

Boone

K. B.

(2009). The need for continuous and comprehensive sampling of effort/response bias during neuropsychological examinations. The Clinical Neuropsychologist, 23(4), 729-741. https://doi.org/10.1080/13854040802427803.

12.

Boone

K. B.

(2013). Clinical practice of forensic neuropsychology. Guilford Press.

13.

Boone

K. B.

Herzberg

(2002). The Dot counting test manual. Western Psychological Services.

14.

Bracken

B. A.,

Boatwright

B. S.

(2005). Clinical assessment of attention deficit – child and adult: Examiner’s manual. Psychological Assessment Resources.

15.

Critchfield

Soble

J. R.

Marceaux

J. C.

Bain

K. M.

Chase Bailey

Webber

T. A.

Alex Alverson

Messerly

Andrés González

O’Rourke

J. J. F.

(2019). Cognitive impairment does not cause performance validity failure: Analyzing performance patterns among unimpaired, impaired, and noncredible participants across six tests. The Clinical Neuropsychologist, 33(6), 1083-1101. https://doi.org/10.1080/13854046.2018.1508615.

16.

Faraone

SV,

Biederman

(2005). What is the prevalence of adult ADHD? Results of a population screen of 966 adults. Journal of Attention Disorders, 9(2), 384-391. https://doi.org/10.1177/1087054705281478.

17.

Faraone

Spencer

Aleardi

Pagano

Biederman

(2004). Meta-analysis of the efficacy of methylphenidate for treating adult attention-deficit/hyperactivity disorder. Journal of Clinical Psychopharmacology, 24(1), 24-29. https://doi.org/10.1097/01.jcp.0000108984.11879.95.

18.

Greiffenstein

M. F.

Baker

W. J.

Gola

(1994). Validation of malingered amnesia measures with a large clinical sample. Psychological Assessment, 6(3), 218-224. https://doi.org/10.1037/1040-3590.6.3.218.

19.

Hirsch

O.,

Christiansen

(2018). Faking ADHD? Symptom validity testing and its relation to self-reported, observer-reported symptoms, and neuropsychological measures of attention in adults with ADHD. Journal of Attention Disorders, 22(3), 269-280. https://doi.org/10.1177/1087054715596577.

20.

Hosmer

D. W.

Lemeshow

Sturdivant

R. X.

(2013). Applied logistic regression(3rd ed.). John Wiley, & Sons.

21.

Jennette

K. J.

Williams

C. P.

Resch

Z. J.

Ovsiew

G. P.

Durkin

N. M.

O’Rourke

J. J. F.

Marceaux

J. C.

Critchfield

E. C.

Soble

J. R.

(2021). Assessment of differential neurocognitive performance based on the number of performance validity tests failures: A cross-validation study across multiple mixed clinical samples. The Clinical Neuropsychologist. Advance online publication. https://doi.org/10.1080/13854046.2021.1900398.

22.

Kessler

R. C.

Adler

Barkley

Biederman

Conners

C. K.

Demler

Faraone

S. V.

Greenhill

L. L.

Howes

M. J.

Secnik

Spencer

Ustun

T. B.

Walters

E. E.

Zaslavsky

A. M.

(2006). The prevalence and correlates of adult ADHD in the United States: results from the National Comorbidity Survey Replication. The American Journal of Psychiatry, 163(4), 716-723. https://doi.org/10.1176/ajp.2006.163.4.716.

23.

Larrabee

G. J.

(2003). Detection of malingering using atypical performance patterns on standard neuropsychological tests. The Clinical Neuropsychologist, 17(3), 410-425. https://doi.org/10.1076/clin.17.3.410.18089.

24.

Larrabee

G. J.

(2008). Aggregation across multiple indicators improves the detection of malingering: Relationship to likelihood ratios. The Clinical Neuropsychologist, 22(4), 666-679. https://doi.org/10.1080/13854040701494987.

25.

Larrabee

G. J.

(2014). False-positive rates associated with the use of multiple performance and symptom validity tests. Archives of Clinical Neuropsychology, 29(4), 364-373. https://doi.org/10.1093/arclin/acu019.

26.

Leib

S. I.

Schieszler-Ockrassa White

C. D. J.

Gallagher

V. T.

Carter

D. A.

Basurto

K. S.

Ovsiew

G. P.

Resch

Z. J.

Jennette

K. J.

Soble

J. R.

(2021). Concordance between the Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF) and Clinical Assessment of Attention Deficit-Adult (CAT-A) over-reporting validity scales for detecting invalid ADHD symptom reporting. Applied neuropsychology: Adult. Advance online publication. https://doi.org/10.1080/23279095.2021.1894150.

27.

Marshall

Schroeder

O’Brien

Fischer

Ries

Blesi

Barker

(2010). Effectiveness of symptom validity measures in identifying cognitive and behavioral symptom exaggeration in adult attention deficit hyperactivity disorder. The Clinical Neuropsychologist, 24(7), 1204-1237. https://doi.org/10.1080/13854046.2010.514290.

28.

Martin

P. K.,

Schroeder

R. W.

(2020). Base rates of invalid test performance across clinical non-forensic contexts and settings. Archives of Clinical Neuropsychology, 35(6), 717-725. https://doi.org/10.1093/arclin/acaa017.

29.

Martin

P. K.

Schroeder

R. W.

Odland

A. P.

(2015). Neuropsychologists’ validity testing beliefs and practices: A survey of North American professionals. The Clinical Neuropsychologist, 29(6), 741-776. https://doi.org/10.1080/13854046.2015.1087597.

30.

Ovsiew

G. P.

Resch

Z. J.

Nayar

Williams

C. P.

Soble

J. R.

(2020). Not so fast! Limitations of processing speed and working memory indices as embedded performance validity tests in a mixed neuropsychiatric sample. Journal of Clinical and Experimental Neuropsychology, 42(5), 473-484. https://doi.org/10.1080/13803395.2020.1758635.

31.

Pearson

(2009). Advanced clinical solutions for WAIS-IV and WMS-IV: Administration and scoring manual. The Psychological Corporation.

32.

Pievsky

M. A.,

McGrath

R. E.

(2018). The neurocognitive profile of attention-deficit/hyperactivity disorder: A review of meta-analyses. Archives of Clinical Neuropsychology, 33(2), 143-157. https://doi.org/10.1093/arclin/acx055.

33.

Pliskin

DeDios Stern

Resch

Saladino

Ovsiew

Carter

Soble

(2021). Comparing the psychometric properties of eight embedded performance validity tests in the Rey Auditory Verbal Learning Test, Wechsler Memory Scale Logical Memory, and Brief Visuospatial Memory Test–Revised recognition trials for detecting invalid test performance. Assessment, 28(8), 1871-1881. https://doi.org/10.1177/1073191120929093.

34.

Resch

Z. J.

Pham

A. T.

Abramson

D. A.

White

D. J.

DeDios-Stern

Ovsiew

G. P.

Castillo

L. R.

Soble

J. R.

(2020). Examining independent and combined accuracy of embedded performance validity tests in the California Verbal Learning Test-II and Brief Visuospatial Memory Test-Revised for detecting invalid performance. Applied neuropsychology: Adult. Advance online publication. https://doi.org/10.1080/23279095.2020.1742718.

35.

Resch

Z. J.

Webber

T. A.

Bernstein

M. T.

Rhoads

Ovsiew

G. P.

Soble

J. R.

(2021). Victoria symptom validity test. A systematic review and cross-validation study. Neuropsychology Review, 31(2), 331-348.

36.

Rhoads

Neale

A. C.

Resch

Z. J.

Cohen

C. D.

Keezer

R. D.

Cerny

B. M.

Jennette

K. J.

Ovsiew

G. P.

Soble

J. R.

(2021). Psychometric implications of failure on one performance validity test: A cross-validation study to inform criterion group definition. Journal of Clinical and Experimental Neuropsychology, 43(5), 437-448. https://doi.org/10.1080/13803395.2021.1945540.

37.

Schroeder

R. W.

Martin

P. K.

Heinrichs

R. J.

Baade

L. E.

(2019). Research methods in performance validity testing studies: Criterion grouping approach impacts study outcomes. The Clinical Neuropsychologist, 33(3), 466-477. https://doi.org/10.1080/13854046.2018.1484517.

38.

Schroeder

R. W.

Martin

P. K.

Walling

(2019). Neuropsychological Evaluations in Adults. American Family Physician, 99(2), 101-108.

39.

Schroeder

R. W.

Twumasi-Ankrah

Baade

L. E.

Marshall

P. S.

(2012). Reliable digit span: A systematic review and cross-validation study. Assessment, 19(1), 21-30. https://doi.org/10.1177/1073191111428764.

40.

Soble

J. R.

Alverson

W. A.

Phillips

J. I.

Critchfield

E. A.

Fullen

O’Rourke

J. J. F.

Messerly

Highsmith

J. M.

Bailey

K. C.

Webber

T. A.

Marceaux

J. C.

(2020). Strength in numbers or quality over quantity? Examining the importance of criterion measure selection to define validity groups in performance validity test (PVT) research. Psychological Injury and Law, 13(1), 44-56. https://doi.org/10.1007/s12207-019-09370-w.

41.

Suhr

Hammers

Dobbins-Buckland

Zimak

Hughes

(2008). The relationship of malingering test failure to self-reported symptoms and neuropsychological findings in adults referred for ADHD evaluation. Archives of Clinical Neuropsychology, 23(5), 521-530. https://doi.org/10.1016/j.acn.2008.05.003.

42.

Sullivan

B. K.

May

Galbally

(2007). Symptom exaggeration by college adults in attention-deficit hyperactivity disorder and learning disorder assessments. Applied Neuropsychology, 14, 189-207.

43.

Sweet

J. J.

Heilbronner

R. L.

Morgan

J. E.

Larrabee

G. J.

Rohling

M. L.

Boone

K. B.

Kirkwood

M. W.

Schroeder

R. W.,

Suhr

J. A.

Conference Participants (2021). American Academy of Clinical Neuropsychology (AACN) 2021 consensus statement on validity assessment: Update of the 2009 AACN consensus conference statement on neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 35(6), 1053-1106. https://doi.org/10.1080/13854046.2021.1896036.

44.

Teter

C. J.

McCabe

S. E.

Cranford

J. A.

Boyd

C. J.

Guthrie

S. K.

(2005). Prevalence and motive for illicit use of prescription stimulants in an undergraduate student simple. Journal of American College Health, 53(6), 253-262. https://doi.org/10.3200/JACH.53.6.253-262.

45.

Webber

T. A.

Critchfield

E. A.

Soble

J. R.

(2020). Convergent, discriminant, and concurrent validity of non-memory-based performance validity tests. Assessment, 27(7), 1399-1415. https://doi.org/10.1177/1073191118804874.

46.

Webber

T. A.,

Soble

J. R.

(2018). Utility of various WAIS-IV Digit Span indices for identifying noncredible performance validity among cognitively impaired and unimpaired examinees. The Clinical Neuropsychologist, 32(4), 657-670. https://doi.org/10.1080/13854046.2017.1415374.

47.

Wechsler

(1997). WAIS-III administration and scoring manual. Psychological Corporation.

48.

Wechsler

(2008). Wechsler adult intelligence scale (4th ed.). Pearson Assessment.

49.

White

D. J.

Korinek

Bernstein

M. T.

Ovsiew

G. P.

Resch

Z. J.

Soble

J. R.

(2020). Cross-validation of non-memory-based embedded performance validity tests for detecting invalid performance among patients with and without neurocognitive impairment. Journal of Clinical and Experimental Neuropsychology, 42(5), 459-472. https://doi.org/10.1080/13803395.2020.1758634.

50.

White

D. J.

Ovsiew

G. P.

Rhoads

Resch

Z. J.

Lee

A. J.

Soble

J. R.

(2020). The divergent roles of symptom and performance validity in the assessment of ADHD. Journal of Attention Disorders. Advance online publication. https://doi.org/10.1177/1087054720964575.

51.

Woods

S. P.

Lovejoy

D. W.

Ball

J. D.

(2002). Neuropsychological characteristics of adults with ADHD: A comprehensive review of initial studies. The Clinical Neuropsychologist, 16(1), 12-34. https://doi.org/10.1076/clin.16.1.12.8336.

52.

Young

J. C.

Sawyer

R. J.

Roper

B. L.

Baughman

B. C.

(2012). Expansion and re-examination of Digit Span effort indices on the WAIS-IV. The Clinical Neuropsychologist, 26(1), 147-159. https://doi.org/10.1080/13854046.2011.647083.