δ Scores are Exportable Across Cultural and Linguistic Boundaries

INTRODUCTION

We have constructed a latent measure of dementia severity, “δ”, and validated it in several datasets, including well characterized subjects participating in the Texas Alzheimer’s Research and Care Consortium (TARCC) study [1] and a convenience sample of Japanese persons [2]. The latent variable δ represents the “cognitive correlates of functional status”. It is uniquely related to dementia severity as measured by the Clinical Dementia Rating scale Sum of Boxes (CDR-SOB) [3] and accurately distinguishes cases with Alzheimer’s disease (AD) and mild cognitive impairment (MCI) from each other, and from controls [1, 5]. Although we have validated δ in datasets that are highly selected for AD cases, δ scores are likely to represent the severity of dementia itself, regardless of etiology. A δ homolog has been independently validated in the National Alzheimer’s Coordinating Center “Uniform Dataset” among cases with frontotemporal, Lewy body, and vascular dementias, in addition to AD [6].

δ can be constructed from a minimal cognitive assessment. We have constructed δ homologs from as little as an executive clock-drawing task (CLOX) [7] and either the Mini-Mental State Examination (MMSE) [8] or the Executive Interview (EXIT25) [9]. In both cases, the resulting composite variable achieves areas under the receiver operating curve (AUC /ROC) >0.95 for the diagnosis of “Alzheimer’s disease” versus controls [2, 5].

Given that δ composites (i.e., “d-scores”) retain their diagnostic accuracy when constructed from bedside measures, valid dementia case finding might be freed from the clinic if d-scores derived in a well characterized reference cohort can be generalized to other populations. This would allow the determination of δ in the field, either prospectively, or post hoc, and in datasets that are currently limited by their lack of comprehensive psychometrics and/or expert consensus clinical diagnoses.

Matsuoka et al. [10] recently validated a Japanese language translation of the CLOX in a convenience sample of elderly Japanese persons with normal cognition (NC), MCI, and dementia (n = 176). Their sample offers an opportunity to test the generalizability of a brief δ homolog. We hypothesize that a δ composite constructed in a subset of the TARCC sample will generalize both to the remainder of the TARCC participants, and to Matsuoka’s sample. Moreover, we hope to take advantage of TARCC’s relatively extensive psychometric battery by constructing the composite from factor loadings that have been adjusted for formalcognitive performance measures. Finally, we plan to limit the composite to cognitive measures only, after dropping δ’s target indicator, i.e., Instrumental Activities of Daily Living (IADL). The retention of the diagnostic accuracy of such a “restricted” δ composite would extend δ’s utility to the future prediction of IADL and other clinical outcomes. It would also facilitate the assessment of δ in a wide range of clinical settings, by removing the requirement to obtain IADL information from a qualified informant.

METHODS

RESULTS

Table 1 presents mean demographic variables for both TARCC subgroups and the Japanese sample. There were no significant differences between the TARCC subgroups on any measure. However, the Japanese sample differed significantly from the TARCC samples with regard to age, education, gender, and CDR global scores. This is to be expected when comparing two convenience samples.

Table 2 presents cross-group differences in the observed dT2J indicator variables. Once again, there were no significant differences between the TARCC subgroups on any measure. However, the Japanese sample differed significantly from the TARCC samples with regard to observed CLOX1, CLOX2, and MMSE scores. This is again to be expected.

Table 3 presents cross-group differences in observed T2J composite indicator variables, within diagnostic subsets. In Controls, CLOX2 and MMSE scores differed significantly. In MCI, only MMSE scores differed significantly. In AD, CLOX2 and MMSE scores differed significantly. However, these significant cross group differences were clinically negligible (i.e., <3 MMSE and <2 CLOX points).

Figure 1 presents the δ model parameterized for the TARCC test sample (n = 998). All observed indicators are adjusted for age, education, ethnicity, and gender. The model has excellent fit [RMSEA = 0.012; CFI = 0.999; ChiSQ = 36.31 (32), p = 0.027]. A Baysean model produced nearly identical parameter estimates. dT2J exhibited acceptable factor determinancy by Grice’s method (i.e., TISMC = 0.86; MC = 0.72).

Table 4 presents each composite’s correlations with the CDR^SOB and /or the CDR^Global in three samples: TARCC validation, TARCC test, and the Japanese sample. All correlations were strong, ranging from T2J (Japanese) x CDR^Global: r =−0.79, p <  0.001, to d(test) x CDR^SOB: r =−0.94, p <  0.001. In general, the correlations were slightly less strong for CDR^Global versus CDR^SOB, and for the Japanese sample versus TARCC.

Table 5 presents each composite variable’s AUCs for the discriminations between 1) AD versus NC, 2) AD versus MCI, and 3) NC versus MCI. Each composite’s AUC was best for the relatively easy discrimination between AD cases and controls (AUC = 0.95, 95% CI = 0.91–0.98). However, they also performed well at more difficult discriminations, including AD versus MCI (AUC = 0.81, 95% CI = 0.72–0.90), and MCI versus NC (0.84, 95% CI = 0.77–0.91). In the TARCC, T2J’s performance was slightly less robust than d’s on every discrimination. However, T2J’s discriminations were statistically indiscriminable from d’s. Similarly, T2J’s performance in the Japanese sample was statistically indiscriminable from d’s performance in the TARCC validation and test samples on every discrimination.

DISCUSSION

We have achieved three important objectives by this analysis. First, we have demonstrated that a δ composite, constructed in one cohort, can be generalized to another well-characterized convenience sample without loss of its psychometric properties. That this can be accomplished across cultural and linguistic barriers and despite significant differences in demographics and observed cognitive performance is a testament to a latent construct’s relative freedom from psychometric measurement error, as has been recently reviewed [31]. Nevertheless, the TARCC and our Japanese cohort are both convenience samples. The generalization of δ to a population-based cohort might be more problematical because of a priori differences in the prevalence of dementia, and could limit unrestricted application of δ homologs.

Second, we have succeeded in extracting a composite from a “restricted” set of indicators, without sacrificing diagnostic accuracy. This achieves two goals. First, it allows us to use the robust battery of our validation cohort to improve the psychometric properties of the final composite, T2J. Second, the restricted composite allows us to estimate δ scores from cognitive performance data alone, without reference to δ’s target indicator (in this case IADL items). This is a remarkable advance. First, it frees us from the necessity of obtaining information from a reliable informant (in the case of IADLs) or expert clinicians (in the case of CDR scores). Dementia diagnosis by latent variables then becomes much more feasible for cases without informants, in remote clinical settings, or in the post hoc analysis of datasets that lack CDR or even IADL ratings. Second, this should allow us to use cognitive performance data to estimate the target indicator in cases where only cognitive performance data are known.

This is important because δ-like models are modular and can be directed to any target indicator. We havepreviously constructed one such δ “ortholog” representing “the cognitive correlates of depressive symptoms”, and associated it with the default mode network by voxel based morphometry [32]. By a similar method, we have identified serum insulin-like growth factor binding protein-2 as a biomarker of age-specific cognitive change [33]. Other potential ortholog targets might be driving ability, decision-making capacity, financial, and/or medication management. Now it becomes feasible to estimate these outcomes from δ’s cognitive performance indicators alone. As we have demonstrated here, these need not be very sophisticated measures. They might be simple bedside measures that are easily acquired, at the bedside, via telephone, or the Internet. Moreover, they might represent only a subset of the indicators used to construct the original latent variable [2].

It is also important to note that both δ and its alternative orthologs could be constructed from the same cognitive assessment, and simultaneously estimated in individuals after a single clinical interview. Moreover, given the high AUCs we have achieved for some discriminations, those outcomes might be confidently estimated and interpreted without expert review or adjudication.

δ scores have had difficulty discriminating MCI from controls. We originally thought this weakness might be due to a lack of variance in δ’s IADL indicator(s) among MCI cases. However, MCI samples often demonstrate impairments in IADL [34], and we later found δ-scores can accurately discriminate controls from MCI in non-Hispanic whites [35]. Therefore, we speculated that TARCC clinicians might be experiencing difficulties with the diagnosis of MCI in Hispanic cases. If so, then T2J should perform better in distinguishing controls from MCI in the Japanese sample, where Japanese clinicians are examining ethnic Japanese patients in their native language. However, although the performance of T2J in Japanese persons was slightly more robust than in TARCC, it did not perform statistically better than in TARCC (Table 5). This then suggests that MCI may itself be a heterogeneous condition, composed of subgroups with widely varying δ scores and hence widely varying risks of near term conversion to dementia.

δ ranks subjects along a continuous dimension of dementia severity. Thus, longitudinal change in δ is a very strong determinant of future CDR scores independently of baseline δ [6, 35]. In contrast, g’ is unrelated to functional status, and therefore to dementia, by definition. Thus, it ranks cases along an orthogonal dimension of cognitive performance that is irrelevant to future dementia status. As a consequence, g’ and Δg’ make trivial contributions future CDR scores independently of baseline δ and Δδ [36].

Since MCI is defined by cognitive impairment in the absence of dementia, it should be comprised of at least three subgroups: cases with poor g’, cases with intermediate d-scores, and cases with both. These are broadly recognizable as “amnestic”, “non-amnestic”, and “multi-domain” MCI. Only the latter two would be at near-term risk of conversion, and g’s contribution to the last category would be superfluous. Thus, an adjustment for g’ may improve upon the ability of δ to make this important discrimination.

We have not demonstrated δ’s factor invariance across language or culture. That would have been impractical here because d, from which T2J’s factor weights were derived, was indicated by cognitive measures not available in our Japanese sample. It may not even be feasible to achieve factor invariance across widely disparate populations or cohorts, because δ’s factor loadings are determined in part by each indicator’s measurement error, which may vary across samples. Our aim instead has been pragmatic. Regardless of whether T2J can be found to demonstrate measure invariance, it achieves a statistically indiscriminable classification of Japanese persons.

This is partly because, as latent variables, δ homologs will be relatively immune to the unique measurement biases of their indicators. However, any latent construct’s “reification” as a composite score potentially introduces new measurement error. In very constrained conditions, this can lead to unacceptable discrepancies in a composite’s psychometric properties [20]. Regardless, the methods that are most commonly employed to generate factor composites lead to identical outcomes [30].

As in this study, other δ homologs we have tested all exhibit acceptable determinancy. Moreover, although the reification of δ as a restricted T2J composite may have introduced some measurement error, this 1) explicitly does not include the measurement biases that characterize its psychometric indicators (e.g., cultural or linguistic psychometric biases) and 2) empirically does not affect the psychometric performance of T2J relative to a homolog constructed and validated within Japanese persons, despite statistical differences in these two samples’ demographics and observedcognitive performance (and those measures’ administration in three different languages).

In summary, latent variables can precisely quantify dementia status, in contrast to categorical diagnoses such as “MCI”. The resulting composites rank order subjects along a continuous dimension of dementia severity. We have demonstrated that a latent variable’s target indicator can be omitted from its composite, indeed the majority of its indicators can be omitted, without a drop in diagnostic accuracy. The resulting T2J dementia phenotype is generalizable across cultural and linguistic boundaries. If this can be demonstrated in other δ homologs, δ composites may hold great promise for dementia case-finding.