Reliabilities of WMS–IV Discrepancy Score Comparisons for Subtests,Indexes,and Selected WAIS

Abstract

This investigation provides internal consistency reliabilities for the Wechsler Memory Scale–Fourth Edition (WMS–IV) subtest and index discrepancy scores using the standardization samples of the Adult and Older Adult batteries. Subtest reliabilities ranged from .00 to .93 for Adults and .25 to .94 for Older Adults. Three of 91 Adult coefficients and three of 24 Older Adult coefficients reached the recommended level (≥ .90). Index reliabilities ranged from .46 to .94 for 100 Adult Battery comparisons and .19 to 93 for 36 Older Adult Battery comparisons. Each battery had 10 coefficients ≥ .90. Ability–memory discrepancies were reported using WAIS–IV VCI, PRI, WMI, and GAI as intelligence standards. Ability–memory comparisons for the Adult Battery ranged from .82 to .93 with 12 of 20 comparisons ≥ .90; Older Adult Battery reliabilities ranged from .85 to .94 with six of 16 comparisons ≥ .90. In terms of discrepancy score reliabilities, the WMS–IV represents a marked improvement over the WMS–III.

Keywords

Wechsler Memory Scale–Fourth Edition discrepancy scores reliability memory Wechsler Adult Intelligence Scale–Fourth Edition

The Wechsler Adult Intelligence Scale–Fourth Edition (WAIS–IV; Wechsler, 2008a) and Wechsler Memory Scale–Fourth Edition (WMS–IV; Wechsler, 2009a) are the latest versions of the Wechsler intelligence and memory scales, respectively. The WAIS–IV is composed of 10 core subtests that yield the Verbal Comprehension (VCI), Perceptual Reasoning (PRI), Working Memory (WMI), and Processing Speed (PSI) indexes. The 10 core subtests are combined to yield a Full Scale IQ (FSIQ) while the six primary subtests of the VCI (Information + Similarities + Vocabulary) and PRI (Block Design + Visual Puzzles + Matrix Reasoning) comprise the General Ability Index (GAI). The WMS–IV Adult Battery contains 10 core subtests yielding the Auditory Memory (AMI), Visual Memory (VMI), Visual Working Memory (VWMI), Immediate Memory (IMI), and Delayed Memory (DMI) indexes. The Older Adult Battery contains seven core subtests that produce four indexes (i.e., AMI, VMI, IMI, and DMI). For both scales, the mean and SD on the subtests are 10 and 3, respectively. All index scores have a mean of 100 and SD of 15.

To enhance the clinical utility of the WMS–IV, 1250 individuals in the standardization sample were also administered the WAIS–IV. This co-norming design allows for standardized ability–memory comparisons to determine if memory and intellectual functioning differ significantly. Ability–memory comparisons primarily utilize the GAI as the measure of general cognitive functioning. However, comparisons between memory and cognitive capacity may also be made using the VCI, PRI, and/or WMI as the ability standard (Wechsler, 2009b). The WAIS–IV Processing Speed Index and subtests were not included in any analyses because they utilize test–retest correlations rather than internal consistency coefficients as estimates of reliability. Comparisons across memory domains are also encouraged (e.g., AMI vs. VMI), and the WMS–IV Technical and Interpretive Manual (Wechsler, 2009b) provides numerous tables to determine significant differences between memory measures as well as among the four WAIS–IV indexes and the WMS–IV indexes.

When conducting subtest, index, and ability–memory discrepancy analyses, it is essential to know whether the discrepancy scores of interest meet a minimum standard of reliability. If the goal of profile analysis is hypotheses testing, an acceptable level is typically ≥ .80 (Sattler, 2008), whereas for clinical decision making, a minimum standard is ≥ .90 (e.g., Guilford & Fruchter, 1978; Ponterotto & Ruckdeschel, 2007). However, practitioners who regularly administer the Wechsler intelligence and memory scales do not always have the luxury of cherry-picking the reliability of the various subtest, index, and discrepancy scores they interpret. Nevertheless, the WMS–IV test developers recommend that practitioners use care when interpreting test scores with reliabilities < .80, while extreme caution is needed when interpreting coefficients ≤ .75 (Drozdick et al., 2011).

Discrepancy score reliability data were not provided by the publishers of the WMS–IV and do not appear in any contemporary or earlier Wechsler intelligence or memory test manual. Yet, since the appearance of the Wechsler Bellevue Intelligence Scale (WB I; Wechsler, 1939) and the Wechsler Memory Scale, Form I (Wechsler, 1945), the interpretation of discrepancies between subtests, IQs (Rapaport et al., 1968), and ability–memory composites (Prigatano, 1978) has been a core focus of test interpretation because of their supposed diagnostic significance. The first publication (McNemar, 1957) to report discrepancy score reliabilities for a Wechsler scale used only the subtest scores of the 25–34 age range in the standardization sample of the Wechsler Adult Intelligence Scale (WAIS; Wechsler, 1955). Of 55 discrepancy score reliabilities, the range was .25–.84 (mdn = .60) with only 10 (18%) ≥ .70.

The Wechsler Adult Intelligence Scale–Revised (WAIS–R; Wechsler, 1981) subtest discrepancy score reliabilities have been reported for 229 psychiatric patients (Piedmont et al., 1989), 290 individuals with intellectual disability (Atkinson, 1991a), and 1880 participants from the WAIS–R standardization sample (Atkinson, 1991b). Results of these investigations were similar in that the vast majority of discrepancy scores were well below the desired minimal standard of ≥ .80. For example, for the average of the WAIS–R standardization sample, only the Vocabulary–Block Design discrepancy score reliability (r = .82) was sufficiently large to justify hypothesis formulation (Atkinson, 1991a). Fifty-one (92.7%) of the 55 possible subtest comparisons produced reliabilities ≤ .74, suggesting that it would be unwise to interpret these scores as reflecting psychological or neurological pathology.

Discrepancy score reliabilities for the Wechsler Adult Intelligence Scale–Third Edition (WAIS–III; Wechsler, 1997) have been reported for the standardization sample (Charter, 2001) and 100 men in a treatment program for substance abuse disorders (Brown & Ryan, 2004). For the normative sample, discrepancy score reliabilities were computed across the 13 age levels and the average of all ages for the following comparisons: Verbal IQ–Performance IQ, VCI–Perceptual Organization Index (POI), VCI–WMI, and POI–WMI. Analysis of subtest score differences used only the average correlations for all age groups. For the IQs and indexes, the range of reliabilities was .70–.88 (mdn = .83) with 47 (83.9%) of the coefficients ≥ .80. Subtest discrepancy score reliabilities ranged from .44 to .85 (mdn = .70) with 8 (12.1%) coefficients ≥ .80. Charter concluded that many of the IQ and index score discrepancies appear adequate for clinical application, whereas less confidence can be placed in subtest difference scores since many have reliabilities too low to justify interpretation. Discrepancy score reliabilities among substance abusers on the composite (range = .79 to .87; mdn = .82) and subtest scores (range = .34 to .85; mdn = .71) were highly similar to those of the standardization sample, indicating that interpretation of discrepancy scores for the WAIS–III is as defensible in a clinical sample as it is among presumably healthy individuals (Brown & Ryan, 2004).

Discrepancy score reliability information is also available for the Wechsler Abbreviated Scale of Intelligence (WASI; The Psychological Corporation, 1999), Wechsler Intelligence Scale for Children–Fourth Edition (WISC–IV; Wechsler, 2003), and WAIS–IV (Wechsler, 2008a). Across the 23 age groups in the WASI standardization sample, VIQ–PIQ discrepancy score reliabilities ranged from .78 to .91 (mdn = .85), while the subtest reliabilities ranged from .59 to .85 (mdn = .81). All but one of the 25 VIQ–PIQ discrepancies had coefficients ≥ .80, whereas 75% of the 12 subtest reliabilities were ≥ .80 (Ryan & Brown, 2005). Results for the WISC–IV (Glass et al., 2009) were derived using the average split-half reliabilities for the subtests and indexes. Composite score comparisons were computed separately for the 11 age ranges and the average of all age ranges in the standardization sample. Discrepancy score reliabilities for the subtests ranged from .50 to .82 (mdn = .74) and for the indexes the reliability ranges were .78–.84 (mdn = .82) on VCI–PRI, .79–.88 (mdn = .84) on VCI–WMI, and .80–.86 (mdn = .83) on PRI–WMI. All but three of the 36 (91.7%) index discrepancy score reliabilities were ≥ .80, whereas only five (7.6%) of the 66 subtest discrepancy scores achieved this degree of magnitude. Regarding the WAIS–IV (Glass et al., 2010), index discrepancy score reliabilities were computed separately for the 13 age ranges and the average of all age ranges in the standardization sample. Reliabilities were derived for 12 subtests using the average of the normative group. The range of discrepancy coefficients for the index comparisons were VCI–PRI = .85–.91 (mdn = .88), VCI–WMI = .83–.89 (mdn = .87), and PRI–WMI = .80–.88 (mdn = .85). Five of the index discrepancy reliabilities were ≥ .90. For the 66 subtest comparisons, the discrepancy score reliabilities ranged from .55 to .88 (mdn = .78) with 23 (34.8%) coefficients ≥ .80.

Only one article to date has examined discrepancy score reliabilities for a Wechsler memory test. Charter (2002) determined reliabilities for the Wechsler Memory Scale–Third Edition (WMS–III; Wechsler, 1997) using the 1250 participants in the standardization sample. Reliability coefficients for the eight primary index comparisons were Auditory Immediate (AI)–Visual Immediate (VI) range of .66–.86 (mdn = .80), AI–Auditory Delayed (AD) range of .00–.32, VI–Visual Delayed (VD) of .00–.29 (mdn = .00), AD–Auditory Recognition Delayed (ARD) of .25–.64 (mdn = .43), AD–VD of .58–.77 (mdn = .72), Immediate Memory (IM)–General Memory (GM) of .00–.43 (mdn = .11), IM–Working Memory (WM) of .73–.86 (mdn = .79), and GM–WM of .69–.87 (mdn = .78). Of the 104 (excluding average correlations) reliabilities, 19 (18.3%) were ≥ .80. Subtest discrepancy score reliabilities based on the average of all age groups ranged from .00 to .82 (mdn = .72), with only six (11%) of the 55 subtest coefficients ≥ .80. Ability–memory comparisons involving the eight primary indexes and the WAIS–III VIQ, PIQ, and FSIQ were also reported. Discrepancy score reliabilities for the VIQ ranged from .73 to .88 (mdn = .85), PIQ ranged from .71 to .86 (mdn = .81), and FSIQ ranged from .73 to .89 (mdn = .84). It is obvious that in many instances the WMS–III discrepancy score reliabilities for the primary indexes and subtest score comparisons are too low for practical application.

Because discrepancy score reliabilities of the WMS–IV have not been published, the present investigation provides this information for subtest scores, indexes, and ability–memory comparisons. This is an important topic that should assist clinicians in determining which discrepancy scores are potentially useful for developing hypotheses and clinical decision making. The availability of discrepancy score reliabilities should aid interpretation because it will allow practitioners to calculate confidence intervals for the difference scores being interpreted (Charter & Feldt, 2009). The importance of the present topic is confirmed by a recent survey of practicing clinical neuropsychologists in the United States and Canada. This survey reported that the WMS–IV is ranked as the second most frequently used neuropsychological instrument in general and as the most frequently administered tool for the assessment of memory functioning (Rabin et al., 2016).

Method

Participants

The standardization samples for adults age 16–69 years (n = 900) and older adults age 69–90 years (n = 500) served as participants when calculations were based exclusively on WMS–IV subtest and index scores. The WMS–IV norms are stratified according to the 2005 US Census on the variables of age, sex, race/ethnicity, education, and geographic region. All participants reported English as the primary language and were screened for medical and/or psychiatric problems. Individuals with a history of disease or injury that might dampen test performance (e.g., intellectual disability, traumatic brain injury, long-term substance abuse, learning disability, and Parkinson’s disease) were excluded as were persons who appeared to put forth low effort on at least one effort measure. The WAIS–IV standardization sample (n = 2200) provided internal consistency coefficients for the VCI, PRI, WMI, and GAI. The WAIS–IV norms, like those for the WMS–IV, were stratified according the 2005 US Census on the same demographic variables. Extensive exclusionary criteria similar to those used with the WMS–IV were applied to the WAIS–IV standardization sample. For all ability–memory comparisons, the co-normed WAIS–IV/WMS–IV sample (n = 1250) was utilized. Participants in this group were administered the WAIS–IV during standardization of the WMS–IV.

Procedure

Discrepancy score reliabilities were derived by the formula of Thorndike and Hagen (1969). This formula is applicable to two tests with equal SDs, for which internal consistency reliability coefficients are obtainable

r = {[(r_{1} + r_{2}) / 2] - r_{12}} / (1 - r_{12})

where r is the discrepancy score reliability, r_1, and r₂ are the split-half internal consistency reliability coefficients for tests 1 and 2, and r₁₂ is the correlation between these test scores.

To utilize the formula, internal consistency reliabilities and intercorrelational data for the composite and subtest scores were obtained from the WMS–IV Technical and Interpretive Manual (Wechsler, 2009b). Technical Manual Table 3.1 (p. 46) provides reliability coefficients for the five indexes across nine age ranges (i.e., 16–17, 18–19, 20–24, 25–29, 30–34, 35–44, 45–54, 55–64, and 65–69 years) and the average of all age ranges in the Adult standardization sample. Also in the Technical Manual Table are the reliabilities for five indexes across five age ranges (i.e., 65–69, 70–74, 75–79, 80–84, and 85–90 years) and the average of all age ranges in the Older Adult standardization group. Average reliabilities of 14 subtests in the 16–69 age range and eight subtests in the 69–90 age range were similarly obtained from Technical Manual Table 3.1. The intercorrelations of the WMS–IV subtests and indexes were taken from Technical Manual Table 4.1 (p. 57) for the total Adult standardization sample while Technical Manual Table 4.2 (p. 58) reports intercorrelations of index and subtest scores from the Older Adult normative group. Technical Manual Tables A.1 through A.14 (pp. 184–197) provided the necessary subtest and index score intercorrelations separately for each of the 14 age groups. To calculate the ability–memory discrepancy score reliabilities, the intercorrelations of the selected WAIS–IV composites (i.e., VCI, PRI, WMI, and GAI) with WMS–IV index scores were obtained from Technical Manual Table 4.12 (p. 75) for both the Adult and Older Adult batteries. The WAIS–IV Technical and Interpretive Manual (Wechsler, 2008b) provided the average reliabilities of the VCI, PRI, and WMI (Technical Manual Table 4.1, p. 42), while the internal consistency of the GAI was taken from Lichtenberger and Kaufman (2013, Table 5.1, p 163).

All calculated discrepancy score reliabilities in the present study pertain only to the Simple-Difference method (Wechsler, 2009b). Additional discrepancy score reliabilities could not be calculated because internal consistency coefficients and intercorrelations for the Predicted-Difference and Contrast Scores methods are not available. Likewise, discrepancy score reliability coefficients for the WMS–IV process subtest scores were not derived in the present analyses because internal consistency data are not accessible for these tasks.

Results

Discrepancy score reliabilities for the 14 subtests of the Adult Battery are presented in Table 1 and range from .00 to .93 (mdn =.77). Of the 91 coefficients, 35 (38.5%) were ≥ .80 and three (3.3%) were ≥ .90. Table 2 provides the discrepancy score reliabilities of the seven Older Adult Battery subtests, ranging from .25 to .94 (mdn = .79). Twelve (50%) were ≥ .80 while three (12.5%) are ≥ .90. The 10 discrepancy score reliabilities for the five primary indexes of the Adult Battery for nine age groups and the average of all groups are presented in Table 3. The reliability coefficients ranged from .46 to .94 (mdn = .76), with 49 coefficients ≥ .80 and 10 ≥ .90. Also displayed in Table 3 are the six discrepancy score reliabilities for the four primary indexes of the Older Adult Battery across five age groups and the average of all groups. The coefficients ranged from .19 to .93 (mdn = .57), with 15 (41.6%) of the 36 coefficients ≥ .80 and 10 (27.8%) ≥ .90. Table 4 shows the ability–memory discrepancy score reliabilities for the Simple-Difference method. For the Adult Battery, the coefficients range from .82 to .93 (mdn = .88), with 8 (40%) ≥ .80 and 12 (60%) ≥ .90. The Older Adult Battery ability–memory comparisons ranged from to .85 to .94 (mdn = .89), with 10 (62.5%) ≥ .80 and 6 (37.5%) ≥ .90.

Table 1.

WMS–IV Subtest Discrepancy Score Reliabilities for the Adult Battery.

Subtest	1	2	3	4	5	6	7	8	9	10	11	12	13
LM I
LM II	.27
VPA I	.79	.82
VPA II	.72	.74	.34
DE I	.77	.80	.84	.77
DE I Content	.73	.76	.80	.74	.12
DE I Spatial	.73	.77	.79	.73	.19	.56
DE II	.78	.80	.84	.77	.42	.44	.46
DE II Content	.74	.76	.81	.74	.42	.15	.60	.00
DE II Spatial	.72	.75	.80	.74	.61	.60	.47	.15	.58
VR I	.81	.84	.90	.83	.89	.75	.75	.82	.77	.77
VR II	.85	.87	.93	.86	.85	.80	.79	.85	.81	.79	.87
SA	.81	.85	.89	.82	.79	.75	.74	.81	.75	.76	.85	.91
SSP	.76	.80	.85	.77	.73	.69	.70	.77	.72	.73	.81	.87	.82

Note. DE = Designs; LM = Logical Memory; SA = Spatial Addition; SSP = Symbol Span; VPA = Verbal Paired Associates; VR = Visual Reproduction; WMS–IV = Wechsler Memory Scale–Fourth Edition.

Table 2.

WMS–IV Subtest Discrepancy Score Reliabilities for the Older Adult Battery.

Subtest	1	2	3	4	5	6	7
LM I
LM II	.25
VPA I	.80	.78
VPA II	.64	.58	.53
VR I	.84	.85	.92	.87	.75
VR II	.88	.87	.94	.90	.77	.84
SSP	.75	.77	.87	.79	.63	.76	.83

Note. LM = Logical Memory; VPA = Verbal Paired Associates; VR = Visual Reproduction; SSP = Symbol Span; WMS–IV = Wechsler Memory Scale–Fourth Edition.

Table 3.

Discrepancy Score Reliabilities for WMS–IV Index Scores in the Adult and Older Adult Batteries.

Age group	AMI–VMI	AMI–VWMI	AMI–IMI	AMI–DMI	VMI–VWMI	VMI–IMI
Adult battery
16 to 17	.92	.84	.69	.70	.80	.72
18 to 19	.94	.87	.76	.77	.85	.80
20 to 24	.86	.89	.50	.64	.84	.85
25 to 29	.90	.91	.65	.65	.85	.69
30 to 34	.92	.89	.71	.61	.87	.75
35 to 44	.93	.89	.73	.72	.84	.76
45 to 54	.89	.88	.68	.61	.86	.58
55 to 64	.93	.90	.74	.72	.83	.69
65 to 69	.92	.87	.69	.65	.82	.73
Average	.91	.88	.69	.68	.85	.73
Older adult battery
65 to 69	.93		.45	.35		.85
70 to 74	.91		.39	.25		.85
75 to 79	.93		.54	.36		.69
80 to 84	.92		.45	.40		.82
85 to 90	.90		.44	.19		.83
Average	.93		.50	.35		.86

Age Group	VMI–DMI	VWMI–IMI	VWMI–DMI	IMI–DMI
Adult battery
16 to 17	.72	.76	.74	.56
18 to 19	.89	.82	.87	.61
20 to 24	.64	.85	.86	.54
25 to 29	.69	.86	.86	.50
30 to 34	.71	.83	.84	.46
35 to 44	.74	.83	.84	.50
45 to 54	.69	.78	.85	.60
55 to 64	.76	.80	.85	.70
65 to 69	.72	.76	.80	.56
Average	.71	.82	.84	.58
Older adult battery
65 to 69	.83			.46
70 to 74	.71			.42
75 to 79	.82			.56
80 to 84	.80			.54
85 to 90	.74			.57
Average	.80			.54

Note. AMI = Auditory Memory Index; VMI = Visual Memory Index; VWMI = Visual Working Memory Index; IMI = Immediate Memory Index; DMI = Delayed Memory Index; WMS–IV = Wechsler Memory Scale–Fourth Edition.

Table 4.

WAIS–IV and WMS–IV Index Discrepancy Score Reliabilities for the Adult and Older Adult Batteries.

Composite	WMS–IV composite
Composite	AMI	VMI	VWMI	IMI	DMI
Adult battery
GAI	.91	.92	.85	.88	.89
VCI	.90	.93	.88	.90	.90
PRI	.91	.88	.82	.87	.88
WMI	.89	.91	.83	.87	.88
Older adult battery
GAI	.91	.93	–	.88	.87
VCI	.90	.94	–	.89	.88
PRI	.91	.89	–	.87	.85
WMI	.89	.91	–	.87	.85

Note. AMI = Auditory Memory Index; DMI = Delayed Memory Index; GAI = General Ability Index; IMI = Immediate Memory Index; VCI = Verbal Comprehension Index; PRI = Perceptual Reasoning Index; WMI = Working Memory Index; VMI = Visual Memory Index; VWMI =Visual Working Memory Index; WMS–IV = Wechsler Memory Scale–Fourth Edition.

Discussion

When conducting profile analyses with the WMS–IV, certain standards of discrepancy score reliability are necessary. For clinical decision making, internal consistency of at least ≥ .90 (e.g., Guilford & Fruchter, 1978; Ponterotto & Ruckdeschel, 2007) is reasonable, whereas if hypothesis development is the objective, a lesser standard (i.e., ≥ .80) is acceptable (Sattler, 2008). Reliance on discrepancy scores with reliabilities of ≤ .74 has been questioned since their clinical utility is uncertain (Drozdick et al., 2011). Therefore, serious thought is recommended before practitioners base their hypotheses or clinical decisions about individual examinees on discrepancy scores that lack a high degree of internal consistency (Charter, 2002).

There is substantial variation among the subtest discrepancy score reliability coefficients in the WMS–IV Adult Battery, with only three of 91 (3.3%) ≥ .90 and 35 of 91(38.5%) ≥ .80. The WMS–IV Technical and Interpretive Manual (Wechsler, 2009b) points out that when interpreting the Scale at the subtest level, clinicians should avoid overinterpretation of subtest differences. The reasons given for this guidance are (a) the presence of considerable normal variation in the cognitive functioning of normal individuals and (b) the fact that low scores and relatively large discrepancy scores occur frequently in the general population (Carrasco et al., 2015). The present study suggests a third reason for caution when interpreting subtest score differences, namely, that many discrepancy scores are themselves notoriously unreliable. In the present study, 56 of 91 (61.5%) subtest difference scores had reliabilities < .80. Nevertheless, Table F.4 (p. 156) in the WMS–IV Administration and Scoring Manual (Wechsler, 2009a) provides the critical values necessary for determining statistically significant differences for all possible pairs of primary subtest scores. Proper use of this resource suggests a need for practitioners to have specific interpretive hypotheses in mind prior to consulting WMS–IV Administration and Scoring Manual Table F.4 in order to reduce the probability of erroneous conclusions. For the Older Adult Battery, the number of discrepancy scores with acceptably high reliabilities is somewhat better since 12.5% (3/24) are ≥ .90 and 50% (12/24) are ≥ .80. Nevertheless, it would still be inappropriate to use Administration and Scoring Manual Table F.4 in a fishing expedition in search of statistically significant subtest differences. In general, the subtest discrepancy score reliabilities of the WMS–IV are improved over those of the WMS–III (Charter, 2002). For example, on the combined Adult and Older Batteries, 33% (38/115) of the coefficients were ≤ .74, whereas on the WMS–III, 65.4% (36/55) of the coefficients were at or below this level. Conversely, 40.9% (47/115) of WMS–IV coefficients were ≥ .80, but only 10.9% (6/56) of the WMS–III reliabilities were at or above this magnitude.

In the Adult Battery, discrepancy scores for the AMI–VMI, AMI–VWMI, and VMI–VWMI comparisons have reliabilities ≥ .80 across the nine age groups in the standardization sample. Most of these index discrepancy scores are interpretable and are likely to be useful for decision making and hypothesis development. For the most part, the difference score reliabilities for the VWMI–IMI and VWMI–DMI are ≥ .80 and suitable for generating hypotheses worthy of further investigation. The remaining index comparisons yielded scores that, with a few exceptions, are too low for serious interpretation (i.e., IMI–DMI, AMI–IMI, VMI–DMI, and AMI–DMI). Strict rules for interpreting WMS–IV discrepancy scores reliabilities do not exist, but a practical recommendation for clinical practice is to calculate reliabilities for each examinee comparable to his/her age peers. For example, the reliability of the VMI–DMI difference score is .89 for an examinee 18–19 years old, whereas the reliability is .64 if the examinee is 21 years of age. In the former instance, the discrepancy score may be used for hypothesis development and possible interpretation, whereas in the latter case, the coefficient is probably too low for either. For the Older Adult Battery, only the AMI–VMI discrepancy scores were ≥ .90 across the five age ranges and large enough to justify clinical decision making. The VMI–IMI and VMI–DMI comparisons were less impressive but nevertheless in many instances are high enough to allow for hypothesis generation. The discrepancy score reliabilities for the AMI–IMI, AMI–DMI, and IMI–DMI comparisons are so low that they should probably not be used clinically. It is typically a good idea to evaluate discrepancy score reliabilities according to the examinee’s age group, regardless of which WMS–IV battery is being utilized. Relative to the WMS–III (Charter, 2002), the index discrepancy score reliabilities on the WMS–IV have improved, as the former produced 18.7% (21/112) with coefficients ≥ .80 while the latter reported 47% (64/136) of this magnitude or better. A reduction in the number of poor reliabilities is also observed as 64.3% (72/112) of WMS–III coefficients were ≤ .74. Only 46.3% (63/136) of the WMS–IV reliabilities were at comparable levels.

For the Adult Battery, the ability–memory discrepancy score reliabilities were satisfactory, with all coefficients ≥ .82 and 40% ≥ .90. Reliabilities for the Older Adult Battery were similarly strong, with all values ≥ .85 and 37.5% (6/16) ≥ .90. Both batteries produced ability–memory discrepancy scores with sufficient reliability to justify their clinical application in terms of hypothesis development (36 instances) and decision making (14 instances). When predicting memory performance, the GAI and VCI yielded comparable reliabilities. Moreover, similarly reliable difference scores emerged regardless of which cognitive ability measures were used to predict the AMI and VMI scores. With respect to the WMS–III, ability–memory comparisons have been reported (Charter, 2002) for the eight memory indexes and the three WAIS–III IQs (i.e., Verbal, Performance, and FS). Discrepancy score reliabilities were highly similar regardless of which ability measure was used to predict memory performance. The range of WMS–III discrepancy score reliabilities across the 24 ability–memory comparisons was .71–.89 (mdn = .84), which is lower than that for the WMS–IV. On the latter Scale, the range of reliabilities across 36 ability–memory differences was .82–.94 (mdn = .89), with 38.9% (14/36) of the discrepancy score reliabilities ≥ .90. Unlike the WMS–III, all of the ability–memory comparisons provided by the WMS–IV allow for hypothesis formulation (22/36; 61.1%) or both hypothesis development and clinical decision making (38.9%).

The findings of the present study will allow practitioners to determine whether individual discrepancy scores have sufficient reliability for clinical decision making. The logical next step in the process involves computing confidence intervals (CIs) for each score and then comparing the results to an established discrepancy score standard. The recommended formulas for determining the lower and upper limits are taken from Charter (1999) and Charter and Feldt (2009)

Lower = (X_{1} - X_{2}) - Z [SD {(2 - r_{1} - r_{2})}^{1 / 2}]

Upper = (X_{1} - X_{2}) + Z [SD {(2 - r_{1} - r_{2})}^{1 / 2}]

where the obtained scores for Test 1 and Test 2 are represented by X₁ and X_2, respectively. The Z is the z-score of 1.96 for the two-tailed 95% CI, and the SD is the standard deviation for both tests (for subtests, SD = 3; for composites, SD = 15). The correlations r₁ and r₂ are the internal consistency reliabilities for the two tests. The CI represents the full range of credible hypotheses that are consistent with the actual difference and are based on a previously selected true difference score criterion that must be met prior to engaging in clinical decision making. If the CI includes zero, then this is considered a nonsignificant/unreliable result due to measurement error or chance. If the lower bound of the CI is above 0, the selected difference score is statistically significant, and possible explanations include (a) the previously selected difference score is between the upper and lower bound of the CI and therefore the true difference may be greater than, less than, or equal to the selected difference score, and (b) the upper bound of the CI is less than the clinician-selected difference score. If the lower bound of the CI is greater than the previously selected difference score, clinical interpretation is not advised. Consider an example in which an experienced assessment practitioner selects a 15-point discrepancy between the GAI and DMI as the minimum value required before clinical action is justified, and the obtained scores from the examination are 105 for GAI and 92 for DMI (discrepancy score reliability = .87). The obtained difference score of 13 points is statistically significant as determined by Table B.10 in the Technical and Interpretative Manual (Wechsler, 2009b). The internal consistencies are .97 and .94, respectively, for the GAI and DMI, and the Z = 1.96 for the two-tailed confidence limits at the p = .05 level of significance

Lower = (105 - 92) - 1.96 [15 {(2 - .97 - .94)}^{1 / 2}] = 4.18

Upper = (105 - 92) + 1.96 [15 {(2 - .97 - .94)}^{1 / 2}] = 17.50

The interval does not include zero and is therefore significant. However, the 13-point discrepancy is within the interval (i.e., 4.18–17.50) but is also below the selected 15-point difference needed to justify a clinical decision. Although clinical information may influence the final interpretation of the discrepancy, from a statistical perspective, additional evidence would be needed to defend clinical action as the true score difference between indexes may be as low as 4 points.

Consider a case involving the comparison of two indexes from the WMS–IV with a discrepancy score reliability of .54 (based on the average internal consistency for the Older Adult Battery). The assessment practitioner is interested in detecting possible problems with the retention of information over time. Because of the modest discrepancy score reliability and the specific hypothesis under investigation, an IMI > DMI discrepancy that occurs in < 5% of the standardization sample is selected as the minimum value required to justify clinical action. According to Table F.4 in the Administration and Scoring Manual (Wechsler, 2009a), this required difference is 14 points, which is both rare and statistically significant (Table F.3 in the Administration and Scoring Manual). The WMS–IV examination yields an IMI of 100 and a DMI of 80, the internal consistencies are .95 and .92 for the IMI and DMI, and the Z = 1.96 for the two-tailed confidence limits at the p = .05 level of significance

Lower = (100 - 80) - 1.96 [15 {(2 - .95 - .92)}^{1 / 2}] = 9.42

Upper = (105 - 92) + 1.96 [15 {(2 - .97 - .94)}^{1 / 2}] = 30.58

This is a statistically significant result because the interval is entirely above zero, including the minimum required 14-point discrepancy score, and the obtained score of 20 points exceeds the preselected required difference. In this situation, the examiner may decide that action should be taken; however, caution is warranted as the true difference may be as low as nine points. When the lower bound of the CI is above the previously chosen discrepancy score, the clinician can confidently recommend action that is statistically supported.

The present investigation demonstrated that the WMS–IV, in terms of discrepancy score reliabilities, provides a marked improvement over the WMS–III. However, the picture is incomplete because it was based exclusively on the WMS–IV Technical and Interpretive Manual (Wechsler, 2009b). Therefore, discrepancy score reliabilities were assessed only for the Simple-Difference method of comparing subtest scaled scores and selected WAIS–IV and WMS–IV indexes. Discrepancy score reliabilities for the Process and Contrast scaled scores as well as the Predicted-Difference method for evaluating ability–memory comparisons and the numerous additional scores were not examined. Because reliability coefficients for the Process, Contrast, and Predictive-Difference scores were not provided in the WMS–IV Manual, discrepancy score reliabilities could not be determined. Perhaps the publisher will correct these omissions in future revisions of the memory scale. It is also the case that the present study did not address discrepancy score reliabilities for the supplementary scores provided by the Advanced Clinical Solutions for the WAIS–IV and WMS–IV computer program (Wechsler, 2009c).

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Samuel T. Gontkovsky

References

Atkinson

(1991a). On WAIS-R difference scores in the standardization sample. Psychological Assessment, 3, 292-294.

Atkinson

(1991b). Mental retardation and WAIS-R difference scores. Journal of Mental Deficiency Research, 36, 537-542.

Brown

K. I.

Ryan

J. J.

(2004). Reliabilities of the WAIS-III for discrepancy scores: Generalization to a clinical sample. Psychological Reports, 95, 914-916.

Carrasco

R. M.

Grups

Evans

Simco

Mittenberg

(2015). Apparently abnormal Wechsler Memory Scale Index score patterans in the normal population. Applied Neuropsychology Adult, 22, 1-6. doi:10.1080/23279095.2013.816702

Charter

R. A.

(1999). Testig for true score differences using the confidence interval method. Psychological Reports, 85, 808.

Charter

R. A.

(2001). Discrepancy scores of reliabilities of the WAIS-III. Psychological Reports, 89, 453-456.

Charter

R. A.

(2002). Reliability of the WMS-III discrepancy comparisons. Perceptual and Motor Skills, 94, 387-390.

Charter

R. A.

Feldt

L. S.

(2009). A comprehensive approach to the interpretation of difference scores. Applied Neuropsychology, 16, 23-30. doi:10.1080/09084280802644110

Drozdick

L. W.

Holdnack

J. A.

Hilsabeck

R. C.

Hobokin

(2011). Essentials of WMS–IV assessment. Wiley.

10.

Glass

L. A.

Ryan

J. J.

Charter

R. A.

(2010). Discrepancy score reliabilities in the WAIS–IV standardization sample. Journal of Psychoeducational Assessment, 28, 201-208. doi:10.117/0734282909346710

11.

Glass

L. A.

Ryan

J. J.

Charter

R. A.

Bartels

J. M.

(2009). Discrepancy score reliabilities in the WISC–IV standardization sample. Journal of Psychoeducational Assessment, 27, 138-144. doi:10.1177/0734282908325158.

12.

Guilford

J. P.

Fruchter

(1978). Fundamental statistics in psychology and education (6th ed.). McGraw-Hill.

13.

Lichtenberger

E. O.

Kaufman

A. S.

(2013). Essentials of WAIS–IV assessment (2nd ed.). Wiley..

14.

McNemar

(1957). On WAIS difference scores. Journal of Consulting Psychology, 21, 239-240..

15.

Piedmont

R. L.

Sokolove

R. L.

Fleming

M. Z.

(1989). On WAIS-R difference scores in a psychiatric sample. Psychological Assessment A Journal of Consulting and Clinical Psychology: A Journal of Consulting and Clinical Psychology, 1, 155-159.

16.

Ponterotto

J. G.

Ruckdeschel

D. E.

(2007). An overview of coefficient alpha and a reliability matrix for estimating adequacy of internal consistency coefficients with psychological research measures. Perceptual and Motor Skills, 105.997-1014.

17.

Prigatano

G. P.

(1978). Wechsler memory scale: A selective review of the literature. Journal of Clinical Psychology, 34, 816-832.

18.

Rabin

L. A.

Paolillo

Barr

W. B.

(2016). Stability in test usage practices of clinical neuropsychologists in the United States and Canada over a 10-year period: Follow-up survey of INS and NAN members. Archives of Clinical Neuropsychology, 31, 206-230. doi:10.1093/arclin/acw007

19.

Rapaport

Gill

M. M.

Schafer

(1968). Diagnostic psychological testing (rev ed.). R. R. Holt (Ed.). New York, NY: International University Press.

20.

Ryan

J. J.

Brown

K. I.

(2005). Enhancing the clinical utility of the WASI: Reliabilities of discrepancy scores and supplemental tables for profile analysis. Journal of Psychoeducational Assessment, 23, 140-145.

21.

Sattler

J. M.

(2008). Assessment of children: Cognitive foundations (5th ed.). San Antonio, TX:

22.

The Psychological Corporation (1999). WASI: Wechsler abbreviated scale of intelligence. San Antonio: Sattler. Pearson.

23.

Thorndike

R. L.

Hagen

(1969). Measurement and evaluation in psychology and education (3rd ed.). Wiley.

24.

Wechsler

(1939). The measurement of adult intelligence. Williams & Wilkins.

25.

Wechsler

. (1945). A standardized memory scale for clinical use. Journal of Psychology: Interdisciplinary and Applied, 19, 87-95. doi:10.1080/0023980.1945.9917223

26.

Wechsler

(1955). Manual for the Wechsler Adult Intelligence Scale (WAIS). The Psychological Corporation.

27.

Wechsler

(1981). Manual for the Wechsler Adult Intelligence Scale–Revised (WAIS–R). The Psychological Corporation.

28.

Wechsler

(1997). WMS–III administration and scoring manual. ThePsychological Corporation.

29.

Wechsler

(2003). WISC–IV technical and interpretive manual. The Psychological Corporation.

30.

Wechsler

(2008a). WAIS–IV administration and scoring manual. San Antonio, TX: Pearson.

31.

Wechsler

(2008b). WAIS-IV technical and interpretive manual. San Antonio, TX: Pearson.

32.

Wechsler

(2009a). WMS-IV administration and scoring manual. San Antonio, TX: Pearson.

33.

Wechsler

(2009b). WMS-IV technical and interpretive manual. San Antonio, TX: Pearson.

34.

Wechsler

(2009c). Advanced clinical Solutions for the WAIS–IV and WMS–IV. San Antonio, TX: Pearson.

Reliabilities of WMS–IV Discrepancy Score Comparisons for Subtests,Indexes,and Selected WAIS–IV Composites

Abstract

Keywords

Method

Participants

Procedure

Results

Discussion

Footnotes

Declaration of Conflicting Interests

Funding

ORCID iD

References