Abstract
The reliability and validity of three short forms of the Dutch version of the Wechsler Memory Scale–Fourth Edition (WMS-IV-NL) were evaluated in a mixed clinical sample of 235 patients. The short forms were based on the WMS-IV Flexible Approach, that is, a 3-subtest combination (Older Adult Battery for Adults) and two 2-subtest combinations (Logical Memory and Visual Reproduction and Logical Memory and Designs), which can be used to estimate the Immediate, Delayed, Auditory and Visual Memory Indices. All short forms showed good reliability coefficients. As expected, for adults (16-69 years old) the 3-subtest short form was consistently more accurate (predictive accuracy ranged from 73% to 100%) than both 2-subtest short forms (range = 61%-80%). Furthermore, for older adults (65-90 years old), the predictive accuracy of the 2-subtest short form ranged from 75% to 100%. These results suggest that caution is warranted when using the WMS-IV-NL Flexible Approach short forms to estimate all four indices.
Keywords
Assessment of memory functioning is a core component of neuropsychological evaluations. A wide variety of memory tests and batteries exists (see Lezak, Howieson, Bigler, & Tranel, 2012, for a comprehensive overview), but the Wechsler Memory Scale (WMS) is one of the most widely used memory batteries internationally (Rabin, Barr, & Burton, 2005). The latest edition of the WMS, the Wechsler Memory Scale–Fourth edition (WMS-IV), consists of an Adult Battery for participants aged 16 to 69 years and an Older Adult Battery for participants aged 65 to 90 years. Moreover, three subtests were altered from subtests from previous versions of the WMS (i.e., three primary subtests: Logical Memory [LM], Verbal Paired Associates [VPA], and Visual Reproduction [VR]) and four subtests that were newly developed (i.e., one optional subtest: Brief Cognitive Status Exam and three primary subtests: Designs [DE], Spatial Addition [SA], and Symbol Span [SSP]). The six primary subtests can be used to assess indices for Immediate Memory (IMI), Delayed Memory (DMI), Auditory Memory (AMI), Visual Memory (VMI), and Visual Working Memory (VWMI). Hence, the complete WMS-IV is an extensive test battery which provides detailed information of different aspects of memory functioning (Wechsler, 2009).
However, the relatively long administration time of the WMS-IV and its predecessors have been criticized. According to the test’s manual, the administration time takes approximately 90 minutes at the 75th percentile in healthy adults, but can take up to over 2 hours in patients with cognitive impairment (Axelrod, 2001; Groth-Marnat, 2009; Miller, Axelrod, Rapport, et al., 2012; Wechsler, 2009). Although the full WMS-IV, with its long administration time, ensures that the core memory functions are assessed in an adequate and reliable manner, there is still a demand for short forms in clinical practice. Administration of the full WMS-IV in clinical practice is sometimes not feasible due to several reasons, such as patient’s fatigue, frustration and agitation when the patient is tested for several hours consecutively, or external constraints on the amount of time for a neuropsychological assessment of a broad spectrum of cognitive domains including memory.
In response to requests for short forms, the test developers designed the WMS-IV Flexible Approach to make short forms available and “expand the usability and utility of the WMS-IV” (Wechsler, 2010, p. 1). The WMS-IV Flexible Approach uses the primary subtests to generate three shorter memory assessments which still maintain four index scores: AMI, VMI, IMI, and DMI. First, a 3-subtest short form including the subtests LM, VPA, and VR (i.e., the Older Adult Battery for Adults [OAA]). Second, a 2-subtest short form including the subtests Logical Memory and Visual Reproduction (LMVR). Third, a 2-subtest short form including the subtests Logical Memory and Designs (LMDE). With regard to the length of time required for administration of these short forms, the WMS-IV manual states that the time required is approximately 60 minutes for the OAA, 30 minutes for the LMVR, and 45 minutes for the LMDE at the 90th percentile in patients with cognitive impairment (Hendriks, Bouman, Kessels, & Aldenkamp, 2014; Wechsler, 2009). Notably, because the full WMS-IV Older Adult Battery (65-90) does not contain all available WMS-IV subtests, only the LMVR short form was evaluated in older adults of 65 to 90 years old. See Table 1 for an overview of subtests comprising the actual WMS-IV-NL and the three short forms. Besides these short forms which comprised the WMS-IV primary subtests, in the United States, there is also a possibility to generate two alternate memory assessments by using two supplementary subtests (Logos and Names). However, these are not included in the current study (see the WMS-IV Flexible Approach Manual by Wechsler, 2010, for a detailed description on these two alternate forms). The purpose of the current study was to examine the accuracy and clinical utility of the three above mentioned short forms using the Dutch version of the WMS-IV (WMS-IV-NL: Hendriks et al., 2014; Wechsler, 2009).
Subtest Composites of the Full WMS-IV-NL (Adult Battery and Older Adult Battery) and the Three WMS-IV-NL Flexible Approach Short Forms (OAA, LMVR, and LMDE).
Note. WMS-IV-NL = Wechsler Memory Scale–Fourth edition; LMVR = Logical Memory and Visual Reproduction; LMDE = Logical Memory and Designs; IMI = Immediate Memory Index; DMI = Delayed Memory Index; AMI = Auditory Memory Index; VMI = Visual Memory Index; VWMI = Visual Working Memory Index.
Symbol Span is included in the Older Adult Battery, not in the 3-subtest short form (Older Adult Battery for Adults [OAA]).
Previous independent research has developed and examined the use of various shortened versions of the Wechsler Memory Scale–Revised (WMS-R; Wechsler, 1987; e.g., Axelrod, Woodard, Putnam, & Adams, 1996; Cañizares et al., 2000; Hoffman, Tremont, Scott, Adams, & Mittenberg, 1997; Woodard & Axelrod, 1995) and Wechsler Memory Scale–Third edition (WMS-III; Wechsler, 1997b; e.g., Axelrod, Ryan, & Woodard, 2001; Axelrod & Woodard, 2000). Overall, these studies resulted in high predictive accuracies for short forms that include three or more subtests. The percentage of individuals falling within 6 points (2 standard errors of measurement [SEMs]) of the actual WMS scores or falling within the same Wechsler classification ranged from 85% to 100%, with most studies revealing percentages of ≥90%. Moreover, these previous findings suggest that the predictive accuracies of short forms decrease with a reduced number of subtests. Accordingly, the predictive accuracies of short forms that include only two subtests varied widely across different studies. For the 2-subtest short forms, the percentage of individuals falling within 6 points of the actual WMS scores or falling within the same Wechsler classification ranged from 55% to 93%.
As the WMS-IV has changed considerably with respect to its predecessors, the findings on short forms of previous WMS versions cannot be applied to the WMS-IV. So far, however, only two studies have examined whether the IMI and the DMI could be reliably predicted using a parsimonious selection of WMS-IV subtests (Miller, Axelrod, Rapport, et al., 2012; Miller, Axelrod, & Schutte, 2012). Again, 3-subtest short forms were found more accurate than 2-subtest short forms. Miller, Axelrod, Rapport, et al. (2012) showed that the 3-subtest short form (LM, VPA, and VR) resulted in 97% and 96% of the sample falling within 8 points (2 SEMs) of actual IMI and DMI. In contrast, the 2-subtest short form (LM and VR) resulted in 76% and 78%, respectively. In addition, Miller, Axelrod, and Schutte (2012) revealed similar results with demographically corrected WMS-IV scores which can be calculated using the WMS-IV Advanced Clinical Solutions (see Pearson, 2009). The 3-subtest short form resulted in 95% and 98% of the sample falling within 6 points of actual IMI and DMI; the 2-subtest short form (LM and VR) resulted in 74% and 79%, respectively; and another 2-subtest short form (VPA and VR) resulted in 82% for the estimation of both IMI and DMI.
However, previous study designs have not studied whether the AMI and the VMI could be predicted using short forms. Furthermore, the subtest DE was not taken into account in these previous studies. In the present study, we examined the validity of the three WMS-IV-NL Flexible Approach short forms for estimating four WMS-IV-NL indices (i.e., IMI, DMI, AMI, and VMI) in a mixed clinical sample.
Method
Participants
The sample consisted of 235 patients in whom the WMS-IV-NL was administered as part of an extensive neuropsychological evaluation in several centers in the Netherlands and Belgium 1 . One hundred eighty-two patients completed the WMS-IV-NL Adult Battery (16-69 years old) and 53 patients completed the WMS-IV-NL Older Adult Battery (65-90 years old). The diagnoses of the participants who were assessed with the Adult Battery included 8 (4.4%) patients with mild neurocognitive impairment due to alcohol abuse; 12 (6.6%) with Korsakoff’s syndrome; 50 (27.5%) with acquired brain injury including 20 (11.0%) with traumatic brain injury (TBI) and 18 (9.9%) with a stroke (cerebral vascular accident); 99 (54.4%) with epilepsy including 67 (36.8%) with temporal lobe epilepsy and 19 (10.4%) with extratemporal lobe epilepsy; 1 (0.5%) with mild cognitive impairment; 2 (1.1%) with different types of dementia (early stage) and a sample of 10 (5.5%) patients with mixed psychiatric disorders. The diagnoses of the elderly group included 4 (7.5%) with mild neurocognitive impairment due to alcohol abuse; 2 (3.8%) with Korsakoff’s syndrome; 5 (9.4%) with acquired brain injury including 1 (1.9%) with TBI and 3 (5.7%) with a cerebral vascular accident; 6 (11.3%) with epilepsy including 3 (5.7%) with temporal lobe epilepsy; 16 (30.2%) with mild cognitive impairment; 18 (34.0%) with different types of dementia (early stage) and 2 (3.8%) with psychiatric disorders. The study included an additional number of 18 patients who were excluded because they did not complete the full-length WMS-IV-NL (n = 17), were unable to speak/understand the Dutch language or had a hearing or visual impairment which made normal test administration impossible (n = 1; visual impairment). The participant characteristics are shown in Table 2.
Patient Characteristics.
Note. NART IQ = National Adult Reading Test Intelligent Quotient. Education level was classified according to the Central Office for Statistics of the Netherlands (Central Office for Statistics of the Netherlands (CBS, 2011), which is based on the International Standard Classification of Education (United Nations Educational, Scientific and Cultural Organization Institute for Statistics, 2011). The NART IQ was not available from all patients. The sample size of the Adult Battery, n = 173 and Older Adult Battery, n = 37.
The WMS-IV-NL standardization study was approved by the Institutional Review Board of the Faculty of Social Sciences of Radboud University Nijmegen. Patient data were collected as part of the routine clinical assessment of each participating center.
Neuropsychological Tests
The WMS-IV-NL was administered and scored according to the manual procedures (Hendriks et al., 2014; Wechsler, 2009). This memory batteryconsists of an Adult Battery for participants aged 16 to 69 years and an Older Adult Battery for participants aged 65 to 90 years. The full WMS-IV-NL contains one optional subtest, the Brief Cognitive Status Exam and six primary subtests of which four subtests have immediate and delayed recall conditions: LM I and II, VPA I and II, DE I and II, VR I and II, SA, and SSP. These six subtests are considered primary subtests and are used to derive five index scores: AMI, VMI, VWMI, IMI, and DMI. In the Adult Battery, all primary subtests are included making it possible to compute all index scores. The Older Adult Battery consists of a selection of four primary subtests (LM I and II, VPA I and II, VR I and II, and SSP) making it possible to compute four index scores (AMI, VMI, IMI, and DMI).
In addition, for the present study, three short forms were derived according to the WMS-IV Flexible Approach manual procedures (Hendriks et al., 2014; Wechsler, 2010). One of these short forms is a 3-subtest combination including the subtests LM, VPA, and VR. This short form is labeled as “OAA” and is only available for participants aged 16 to 69 years. The other two short forms are 2-subtest combinations. One of these includes the subtests LM and VR and is labeled “LMVR.” The LMVR is available for participants aged 16 to 90 years. The other 2-subtest combination includes the subtests LM and DE and is labeled as “LMDE.” The LMDE is available for participants aged 16 to 69 years. All these versions make it possible to compute four index scores, namely AMI, VMI, IMI, and DMI. See Table 1 for an overview of subtests comprising the full WMS-IV-NL and each short form.
The applied authorized Dutch version of the WMS-IV is equivalent to the original American version and includes the same index scores and subtest scores with a similar factor structure (Bouman, Hendriks, Kerkmeer, Kessels, & Aldenkamp, 2015). The nonverbal visual stimuli are identical to those in the U.S. WMS-IV. Instructions, auditory stimuli, and scoring criteria were translated and adapted to the Dutch language (see Hendriks et al., 2014, for a detailed description of the development of the WMS-IV-NL).
Statistical Analyses
For both the WMS-IV full form as the WMS-IV Flexible Approach short forms, we used the scaled subtest scores (M = 10, SD = 3) and standard index scores (M = 100, SD = 15). We derived these scores from the test’s manuals (Hendriks et al., 2014; Wechsler, 2009; Wechsler, 2010).
Mean comparisons of the actual WMS-IV-NL index scores (Adult and Older Adult Battery) and the estimates from each of the short forms were analyzed with paired samples t tests. To protect for Type I and II errors when performing multiple comparisons, significance was a priori set at p < .01. Moreover, the magnitudes of the differences between the short and full forms were assessed using Cohen’s (1988) effect size. Pearson correlation coefficients were calculated to assess the degree of agreement between the actual index scores form the WMS-IV-NL and the index scores from the three short forms (OAA, LMVR, and LMDE). Because Pearson correlation coefficients of short and full form versions are spuriously high due to shared error variance, these correlations were corrected (r’) for redundant error of variance using a formula described by Girard and Christensen (2008). This formula is a modification of the correction originally reported by Levy (1967), which takes into account the reliability of the short form and the standard deviations of both the short and the long forms. The modified formula results in lower and upper bounds of the corrected correlation and adjusts the weight of the standard deviations by taking the number of subtests contributing to the short and full forms into account. In this study, the Pearson correlations and the corrected correlations were calculated. To examine the reliability, we used Cronbach’s alpha to estimate the internal consistency reliability of each subtest based on the items of our own study sample. Then, the internal consistency reliability coefficients for index scores were calculated using the formula recommended by Nunnally and Bernstein (1994).
Next, difference scores between the actual and short-form index scores were calculated by subtracting the actual index score from each of the short-form index scores. Frequency analyses of the index scores of the WMS-IV-NL short forms were conducted to reveal the percentage of estimated index scores that fell within 1 or 2 SEM (i.e., 4 or 8 points) or within 15 points (i.e., 1 standard deviation). As reported in the technical and interpretive manual for the WMS-IV-NL, the average SEM for the adult battery (i.e., 16-69 years of age) is 4.39 points for the IMI, 4.35 points for the DMI, 3.84 points for the VMI, and 3.54 points for the AMI, while the average SEM for the Older Adult Battery (i.e., 65-90 years of age) is 3.91 points for the IMI, 4.65 points for the DMI, 3.85 points for the VMI, and 3.61 points for the AMI (Hendriks et al., 2014). These results are in agreement with the average SEMs reported in the U.S. WMS-IV technical manual (Wechsler, 2009). Therefore, the SEM could be set to 4 points for all index scores, which is also useful for the comparison with previous studies by Miller, Axelrod, Rapport, et al. (2012) and Miller, Axelrod, and Schutte (2012).
Following a procedure used by Donders and Axelrod (2002), which was based on the procedure from Nunnally (1978), we used the following a priori criteria to determine whether a short form is acceptable for clinical application. First, the reliability estimates should be ≥.90. Second, the corrected correlations (lower bound) should be ≥.82. Third, >80% of each estimated WMS-IV-NL index score should fall within 2 SEM of the actual WMS-IV-NL index score. Additionally, effect sizes (Cohen’s d) of .20, .50, and .80 were considered small, medium, and large, respectively.
Results
Descriptive statistics, mean comparisons, effect sizes, internal consistency coefficients, and correlations between short-form and actual index scores are presented in Table 3. Of the three short forms evaluated for the WMS-IV-NL Adult Battery, the OAA short index scores did not differ significantly from the actual index scores (i.e., IMI OAA, DMI OAA, and VMI VR did not differ significantly from the corresponding full index scores). In addition, the LMVR index scores did not differ significantly from the actual index scores (i.e., IMI LMVR, DMI LMVR, AMI LM, and VMI VR). Because the significance level was a prior set at p < .01, a trend was seen for a difference between the actual IMI and the IMI LMVR, t(181) = −2.11, p < .05. Moreover, the effect sizes of this difference was negligible and, therefore, clinically insignificant (i.e., Cohen’s d = .04). Furthermore, all LMDE index scores were significantly higher than the actual index scores (IMI LMDE, DMI LMDE, AMI LM, and VMI DE), but effect sizes of these differences were small to negligible (i.e., Cohen’s d = −.20 to −.09).
Descriptive Statistics, Mean Comparisons, Effect Sizes, and Correlations Between WMS-IV-NL Actual and Short-Form Index Scores.
Note. WMS-IV-NL = Wechsler Memory Scale–Fourth edition; LMVR = Logical Memory and Visual Reproduction; LMDE = Logical Memory and Designs; IMI = Immediate Memory Index; DMI = Delayed Memory Index; AMI = Auditory Memory Index; VMI = Visual Memory Index. t value: comparison of summary scores with WMS-IV-NL scores; d value: Cohen’s (1988) effect sizes. Reliability/internal consistency alpha: based on the formula recommended by Nunnally and Bernstein (1994); Pearson r: correlation short with full form; r’: correlation based on the correction for redundancy by Girard and Christensen (2008); r’ min and r’ max: lower and upper bound correction factor, respectively.
For the WMS-IV-NL Older Adult Battery, only one short form, LMVR, was evaluated. Three of the four short index scores did not differ significantly from the actual index scores (i.e., IMI LMVR, DMI LMVR, and AMI LM). Only the VMI VR short index score was significantly higher than the actual index score, but the effect size was negligible and was considered clinically insignificant (Cohen’s d = −.04).
All of the short forms met the a priori specified criteria for reliability (i.e., ≥.90) and correlations (i.e., corrected correlations ≥ .82; see Table 3). That is, internal consistency reliability analysis revealed Cronbach’s alpha coefficients of >.91 for all short-form index scores. Moreover, all short-form index scores correlated highly with the actual index scores (r > .89, p < .001 for Pearson’s correlations and r > .81 for the lower bound of the corrected correlations). Of the three short forms evaluated for the WMS-IV-NL Adult Battery, the highest correlations were consistently found for the 3-subtest short form (i.e., OAA).
The frequency analyses are provided in Table 4. None of the short forms met the a priori specified criteria that >80% of each estimated WMS-IV-NL index score should fall within 2 SEM of the actual WMS-IV-NL index score. The mean difference between the short-form and actual WMS-IV-NL index scores ranged from −0.93 to 3.56 in the Adult Battery, and from −0.06 to 1.66 in the Older Adult Battery. For the Adult Battery, three short forms were examined. Not unexpectedly, the short form that yielded the highest classification accuracy was the 3-subtest short form (i.e., OAA). The percentage of estimates that fell within 2 SEM was 92.3% for IMI OAA, 95.1% for DMI OAA, and 72.5% for VMI OAA (see the fourth column of Table 4). The 2-subtest short forms (i.e., LMVR and LMDE) yielded lower classification accuracy levels than the OAA. For IMI, both LMVR and LMDE yielded comparable classification rates. For DMI and VMI, LMVR yielded somewhat better classification rates than LMDE, which indicates that LMVR could be chosen in favour of LMDE. Furthermore, as reported in the last two columns of Table 4, all LMDE index scores were slightly overestimated in comparison with the actual index scores. This is in line with the higher means observed for this short form in comparison with the actual index scores (see Table 3). For the OAA short form, only the IMI OAA index was slightly overestimated. For the LMVR short form, the results were mixed. IMI LMVR, DMI LMVR, and AMI LMVR were slightly overestimated, whereas VMI VR scores were slightly underestimated in comparison with the actual index scores.
Mean Difference and Percent Agreement Between WMS-IV-NL Actual and Short-Form Index Scores.
Note. WMS-IV-NL = Wechsler Memory Scale–Fourth edition; LMVR = Logical Memory and Visual Reproduction; LMDE = Logical Memory and Designs; IMI = Immediate Memory Index; DMI = Delayed Memory Index; AMI = Auditory Memory Index; VMI = Visual Memory Index; VWMI = Visual Working Memory Index. The mean difference is calculated by subtracting the actual index score from each of the short-form index scores. Positive differences indicate that short-form index scores were higher, whereas negative differences indicate that full-form index scores were higher.
For the Older Adult Battery, the percentage of estimates that fell within 2 SEM was 92.5% for IMI LMVR, 92.5% for DMI LMVR, 75.5% for AMI LMVR, and 100% for VMI LMVR (see the fourth column of Table 4). Furthermore, all short-form index scores were slightly overestimated in comparison with the actual index scores (see the last two columns of Table 4).
Discussion
The predictive accuracies of three WMS-IV-NL Flexible Approach short forms were evaluated in a mixed clinical sample. Our results reveal high correlations between all short-form and actual index scores for the WMS-IV-NL Adult and Older Adult Batteries (i.e., corrected correlation coefficients were comparable in all three short forms and ranged between .81 and .95). There was also a high degree of internal consistency for all short-form index scores (i.e., Cronbach’s alphas ranged between .91 and .97).
Furthermore, none of the short forms met the a priori criteria for the predictive accuracy (i.e., >80% of each short-form index score should fall within 2 SEM of the actual index score). These results may lead to concern about the usability of these short forms in clinical practice. In fact, the OAA yields accurate estimations of the IMI, DMI, and AMI (predictive accuracy ≥92%), but the estimation of the VMI fell below our specified criterion (predictive accuracy of 73%). The 2-subtest short forms (LMVR and LMDE) were overall less accurate (predictive accuracy ranged from 61% to 80%). Clearly, inclusion of three subtests results in a more accurate and reliable estimation of the actual index scores than the 2-subtest versions. These finding are in agreement with the results reported in the studies by Miller, Axelrod, Rapport, et al. (2012) and Miller, Axelrod, and Schutte (2012) on the prediction of WMS-IV IMI and DMI. They reported accuracy rates over 90% for the 3-subtest short form, and accuracy rates over 77% for 2-subtest short forms.
Our findings extend the results of Miller, Axelrod, Rapport, et al. (2012) and Miller, Axelrod, and Schutte (2012), in that we are the first to (a) examine the estimation of the AMI and VMI and (b) include the new visuospatial memory subtest, DE, in our short-form evaluation. The AMI index score can be reliably estimated with the OAA, because this index score is equal for the full WMS-IV-NL and the OAA short form: Both forms include two auditory subtests (LM and VPA). However, our results reveal that caution is needed when interpreting the estimated AMI from the LMVR and LMDE short forms, as well as the estimated VMI from all short forms. That is, the predictive accuracy rates fell below our a priori criteria of what is considered acceptable in clinical practice.
A comparison of both 2-subtest short forms revealed that LMVR is likely to have the most accurate estimations, but results only in a marginally better prediction than LMDE. For LMVR, 69% to 80% of the predicted WMS-IV-NL index scores fell within 2 SEM. For LMDE, 61% to 75% of the predicted WMS-IV-NL index scores fell within 2 SEM.
Because the full Older Adult Battery does not contain all available WMS-IV subtests (i.e., LM, VPA, VR, and SSP), only the LMVR short form can be used with the WMS-IV-NL Flexible Approach (Hendriks et al., 2014; Wechsler, 2010). Only the VMI VR short index score was significantly higher than the actual index score. However, the effect size was clinically insignificant, and both the short VMI VR and actual VMI index scores are composed of the subtests VR I and VR II. The difference found is attributable to different scaled scores. That is, for the Older Adult Battery, only results of participants aged 65 to 90 years are used for the establishment of the norm scores, whereas for the LMVR short form, all participants aged 16 to 90 years are used for the establishment of the norm scores. Moreover, the LMVR yields accurate estimations of the IMI, DMI, and VMI (predictive accuracy ≥ 93%), but the estimation of the AMI fell below our specified criterion (predictive accuracy of 76%).
It is also relevant to relate the current findings to previously reported results. First, using Wechsler’s (2009) nomenclature, the mean performance of our mixed clinical adult sample was in the “low average” range (between 83.0 [DMI LMVR] and 89.9 [VMI DE]), and the mean performance of our mixed clinical older adult sample was in the “borderline” range (between 73.3 [VMI] and 80.6 [AMI LM]). These findings are comparable to the average performance of three clinical groups reported in the U.S. Flexible Approach manual (Wechsler, 2010), that is, patients with TBI and patients with left or right temporal lobectomy. Moreover, the reliability coefficients derived from our mixed clinical sample (ranging from .91 to .97) are in agreement with the results described both in the U.S. and Dutch WMS-IV manuals, in which the estimates ranged from .87 to .97 (Wechsler, 2010) and 84 to .92 (Hendriks et al., 2014), respectively.
While our results provide support for the use of the 3-subtest short form when estimating the IMI, DMI, and AMI, caution is warranted when using the 2-subtest short forms and the OAA VMI. It is important to note that, obviously, the administration of the complete WMS-IV-NL is encouraged (Kaufman & Kaufman, 2001) as the full WMS-IV-NL provides a complete evaluation of the core memory functions. However, in clinical practice, brief memory instruments or short forms are often warranted due to practical or clinical constraints. Rather than using multiple brief memory instruments such as the California Verbal Learning Test (Delis, Kramer, Kaplan, & Ober, 2000) and the Location Learning Test–Revised (Bucks, Willison, Byrne, & Kessels, 2011), it is an advantage that the WMS-IV short forms assess a continuum of memory functions with normative data that come from one high quality normative sample. Furthermore, in this mixed clinical group, the OAA short form yielded the most accurate estimations of the WMS-IV-NL index scores. The 2-subtest short forms may be more suitable when administration time needs to be significantly shortened, for instance, as part of a first neuropsychological screening. The LMVR short form is more likely to have the most accurate estimations in comparison with the LMDE short form, especially in patients with low-range memory abilities. Nonetheless, in clinical practice, there may be circumstances in which the LMDE short form is more appropriate. For example, when a patient’s motor skills are impaired, the subtest VR cannot be administered. As a result, the OAA and LMVR short forms may not be eligible. Additionally, the selection of a particular short form may also depend on the cognitive constructs that need to be examined. When the clinician has a priori hypotheses about the patient’s visuospatial memory, the LMDE short form might be preferred. Importantly, it should be stressed that the results of short forms must be interpreted with caution, and clinicians should take the accuracy rates into account.
Some limitations have to be addressed. In the design of the current study, the full WMS-IV-NL has been administered in all participants. Consequently, the short forms were derived from subtests that were embedded in the full WMS-IV-NL. Patients’ fatigue or procedural learning may have had an impact on subtests administered later in the battery, and thus, on the performance of the short forms (Kaufman & Kaufman, 2001; Thompson, 1987). However, the shortened forms should not be used as stand-alone tests, but as part of a more extensive neuropsychological evaluation, in which other tests given earlier in the test battery might also affect performance on tests administered later in the battery.
The WMS-IV Flexible Approach is limited to the aforementioned short forms, which can be used to calculate four index scores (AMI, VMI, IMI, and DMI; Hendriks et al., 2014; Wechsler, 2010). Obviously, it would be possible to evaluate more WMS-IV short forms with other subtest combinations. For example, a 2-subtest combination of VPA and VR could be estimated, as Miller, Axelrod, Rapport, et al. (2012) already showed that VPA is a good choice for clinicians when they utilize prediction equations. Moreover, it would be interesting to evaluate short forms for the estimation of the working memory index. In particular, it would be interesting to examine a combination of one of the visual working memory subtests of the WMS-IV and one of the verbal working memory subtests of the WAIS-IV which are not part of the WMS-IV. A further limitation is that our clinical sample consisted of a mixed-etiology group with a variety of neuropsychological and psychological diagnoses. Future studies should focus on the applicability and predictive accuracy rates of WMS-IV short forms in specific clinical samples.
Footnotes
Acknowledgements
The authors would like to express their gratitude toward Mia de Bijl, Coby van Drie, Jos Egger, Luciano Fasotti, Sofie Geurts, Judith Grit, Marijke Miatton, Jeanine Noeverman, Jorrit Postma, Annemarie Schulenburg, Ludo Verdyck, Kiki Verhoeven, Arie Wester, Ellen Wingbermühle, Henriëtte van der Zee for their assistance in the data collection, and thank Pearson Assessment and Information (Amsterdam, Netherlands) for authorizing and funding the development of the WMS-IV-NL.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by Pearson Assessment B.V., Amsterdam, Netherlands; Academic Centre for Epileptology, Kempenhaeghe, Heeze, Netherlands; and Donders Institute for Brain, Cognition and Behaviour of Radboud University Nijmegen.