Reproducing the Wechsler Intelligence Scale for Children–Fifth Edition

Effects Coding

The WISC-V publishers used the effects-coding method to set the scale of their latent variable. Little, Slegers, and Card (2006) developed this method for scaling a latent variable to be analogous to effects coding in ANOVA. For a single latent variable, it requires that the set of loadings have an average value of one, or, equivalently, the loadings sum to the number of unique indicator variables. ¹ This constraint is shown in Equation 1.

∑ i = 1 p r λ i r = p ,

where r indexes one specific latent variable and p is the number of indicator variables for the rth latent variable. Using this constraint scales the latent variance to be the average of the indicator variables’ variances. Little et al. argued that this method provides an optimal balance across a latent variable’s possible indicators to establish its scale.

Modified Effects-Coding Method Used for WISC-V Models

Using effects coding should only affect the scale of the latent variable; it should not affect model fit. Model fit measures—including df—using effects coding should be identical to those from more traditional latent variable scaling methods (Little, Card, Slegers, & Ledford, 2007). Thus, Canivez and Watkins’ (2016) inability to replicate the results in the WISC-V technical manual by consistently getting more df indicates that the WISC-V publishers modified the Little et al.’s constraints (i.e., Equation 1).

The effects-coding derivation the WISC-V publishers used comes from making all the latent variables in the model subject to the same constraint. This is shown in Equation 2.

∑ i = 1 p ′ λ i = p ′ ,

where p′ is the number of indicator variables for all latent variables in the entire model.

To date, the WISC-V publishers have not stated that they used effects coding in any of their released documentation for the instrument. Through an exploration of a variety of latent variable scaling methods, however, I was able to replicate the results from the WISC-V technical manual using the constraint shown in Equation 2. To prove that the constraints shown in Equation 2 are those used in the WISC-V technical manual (Wechsler, 2014b), in the appendix, I provide R syntax (R Development Core Team, 2015) to fit the WISC-V CFA model preferred by the publisher (Model 5e). I use the lavaan package (Rosseel, 2012) and fit the model using the all-ages summary statistics provided in the WISC-V technical manual (Wechsler, 2014b). ² In Table 1, I compare my results with those provided in the technical manual. The χ² and model fit values are not the exact same because I used summary statistics and maximum likelihood estimation while the WISC-V publishers used individual test scores and weighted least squares estimation. Nonetheless, the values are very close and, more importantly, the df are the exact same.

OPEN IN VIEWER

Table 1.

Results From Fitting Model 5e From the WISC-V Technical Manual and the Modified Effects-Coding Scaling.

	Technical Manual a	Modified Effects Coding
Estimator	Weighted least squares	Maximum likelihood
Data used	Individual test scores	Covariances
χ2	353.00	362.58
Degrees of freedom	92	92
Comparative fit index	.98	.98
Tucker–Lewis non-normed fit index	.98	.98
Root mean squared error of approximation	.04	.04
Akaike information criterion	441	451 b
Bayesian information criterion	692	701 c

Note. WISC-V = Wechsler Intelligence Scale for Children–Fifth Edition.

a

Results are from the All Ages group for Model 5e (Wechsler, 2014b, p. 82).

b

Calculated via χ² + 2 × number of free parameters to match the technical manual’s calculation.

c

Calculated via χ² + ln(n) × number of free parameters to match the technical manual’s calculation.

Cautions on Using Modified Effects Coding

Despite its use by the WISC-V publishers, the modified effects-coding scaling should be employed with caution because it could produce multiple problems. First, it provides a parameterization of the latent variables that is not equivalent to more traditional scaling methods, which is contrary to Little et al.’s (2007) intention. Second, it changes the interpretation of the latent variance’s scale to be the average of all variables’ variances, not just the indicators for a specific latent variable. As the WISC-V publishers’ preferred model involves a higher order latent variable, it means that all the variable’s metrics are in some manifest-latent hybrid scale that does not have a simple interpretation. Third, while it will not necessarily influence the χ² values, the df difference will cause a change in values of fit measures that use df (or number of estimated parameters) in their calculation. The magnitude of this change depends on the particular situation, as it depends on the particular model used, the sample size, and how the df are used for a particular fit measure. Until the modified effects-coding method is better understood, it should likely only be used when replicating the CFA models from the WISC-V technical manual.

Clinically, the sequelae of using the modified effects-coding scaling are only speculative at this point as there has not been any research on it. At one extreme, it could be that the scaling method used (and the subsequent different df) is just minutia and only of import to psychometrically inclined clinicians and scholars. At the other extreme, because the modified effects-coding scaling method affects the model’s df, it could lead to selection of a non-optimal model, which could then lead to an inaccurate interpretation of whatever the test scores are measuring. As an example, for a given model the modified effects coding will produce fewer df than traditional scaling techniques. This means that for a given χ² value, the p values will be smaller when using modified effects coding. Consequently, using the modified effects coding could lead to rejecting latent variable models that actually fit the data well. In any case, future research can now examine these issues as independent clinicians and scholars should now able to replicate the WISC-V validation CFA models.