Abstract
Background:
The Quick Dementia Rating System (QDRS) is a brief and rapid tool that can be administered by an informant without the need for a trained assessor.
Objective:
Our objective was to examine the validity, reliability, and cost-effectiveness of the informant QDRS in a Singapore memory clinic sample.
Methods:
We assessed a total of 177 older adults, among whom, 32 had no cognitive impairment (NCI), 61 had mild cognitive impairment (MCI), and 84 had dementia. Elderly underwent 1) the informant QDRS, 2) the Clinical Dementia Rating (CDR) as the gold standard diagnosis, 3) the Mini-Mental State Examination (MMSE), and 4) the Ascertain Dementia 8 (AD8) as comparisons to the QDRS. The extent to which the QDRS may reduce the recruitment cost (time) of clinical trials was also calculated.
Results:
The QDRS had excellent internal consistency (Cronbach alpha = 0.939). It correlated highly with the CDR-global (R = 0.897), CDR Sum-of-Boxes (R = 0.915), MMSE (R = –0.848), and the AD8 (R = 0.747), showing good concurrent validity. With an optimal cut-off of 1.5 for MCI (sensitivity 85.2%, specificity 96.3%) and 6 for dementia (sensitivity 90.1%, specificity 89.2%), the QDRS achieved a higher overall accuracy of 85.0%, as compared to MMSE (71.2%) and AD8 (73.4%). A simulated clinical trial recruitment scenario demonstrated that pre-screening with the QDRS followed by a confirmatory CDR would reduce the time needed to identify NCI subjects by 23.3% and MCI subjects by 75.3%.
Conclusion:
The QDRS is a reliable cognitive impairment screening tool which is suitable for informant-administration, especially for identification of MCI.
Keywords
INTRODUCTION
Mild cognitive impairment (MCI) has been conceptualized as an intermediate phase between normal cognitive aging and overt dementia [1]. The weighted prevalence of MCI is 1.2% in adults aged 60 years and above from a Singapore population study [2]. Studies suggested that approximately 60% to 65% of individuals with MCI advancing to developing dementia [3–5]. If MCI can be identified in the early stage, it may allow early intervention and treatment.
Currently, there are several performance-based and informant-based tools for the detection of MCI [6]. However, there remain several challenges. Firstly, performance-based tools are affected by education. The Mini-Mental State Examination (MMSE) and Montreal Cognitive Assessment (MoCA) are generally useful for detecting cognitive impairment, but they may wrongly categorize patients with low education leading to false positives [7, 8]. Secondly, dementia is a progressive disease, which needs to detect cognitive and behavioral changes over time, and current tools have not been design to capture longitudinal changes [9]. Thirdly, most brief screening tools have limited ability to achieve dementia staging [10]. To address these challenges, the Clinical Dementia Rating (CDR) has been used to determine the presence or absence of dementia and to stage its severity, taking into consideration of both individual’s cognitive and functional status. Hence the CDR has been applied in clinic and research settings as gold standard for clinical evaluation [11, 12]. However, the CDR has a long administration time (>1 h) and requires extensive training for the administration of the scale. As a result, despite its high accuracy, it cannot be implemented as a screening tool for MCI. Hence, a brief, concise, yet accurate tool is essential for the early detection of MCI in the memory clinic setting.
The Quick Dementia Rating System (QDRS) may be a suitable screening tool to overcome the above limitations. The QDRS is a 10-item questionnaire which takes 5 min to complete by an informant without the need for a trained clinician or rater [9]. Due to its advantages of inexpensive, efficient, and remote collection, the QDRS can be popularized in the screening process. In addition, previous study have confirmed that the QDRS has achieved comparable results to the CDR [13], and that QDRS score was significantly related to biomarkers of AD (amyloid deposition, hippocampal volume, and apolipoprotein [APOE] ɛ4 status) [14]. Thus it has been shown to be a feasible surrogate endpoint of the primary outcome measure in a dementia treatment trial [15]. Hence, the QDRS may replace time-consuming screening methods for early detection of MCI. To this end, the QDRS has been validated in several countries apart from the USA, such as Russia, Iran, and Korean [10, 17], there is an urgent need for further validation of the QDRS in a sample of Singapore older adults.
Hence, we studied whether the QDRS can serve as a valid screening tool for MCI patients in a multi-ethnic Asian memory clinic cohort. This study aims to 1) perform an external validation and compare the performance between QDRS and CDR in Singaporean older adults, 2) compare the discriminant validity of the QDRS with other commonly used screening tools, and 3) evaluate whether pre-selecting only participants positive on the QDRS can reduce the overall recruitment time of healthy controls or MCI subjects for clinical trials.
METHODS
Study population and research procedures
Participants from an ongoing memory clinic study were approached to complete the QDRS between July 2020 to August 2021. The inclusion criteria were: aged 50 years or older, sufficient language skills for neuropsychological assessment, and informed consent. Participants with major psychiatric illness or substance abuse disorder were excluded as were those without informants. Demographic information such as age, gender, ethnicity, and years of formal education were collected from all the participants. We recruited 246 subjects who met the inclusion criteria. Nine subjects who did not have a reliable informant and another 60 subjects who did not consent or complete the informant QDRS were excluded. Therefore, a total of 177 subjects were included in the final analysis.
This study ethics approval was obtained from the National Healthcare Group Domain-Specific Review Board and conducted in accordance with the Declaration of Helsinki. Written informed consent was obtained in the preferred language of the participants.
Administration of the QDRS
Reliable informant with frequent interactions with the study subjects with at least 10 h a week were asked to self-administer and complete the QDRS during their study visit. The QDRS assessment covers the following 10 questions: 1) memory and recall, 2) orientation, 3) decision-making and problem-solving abilities, 4) activities outside the home, 5) function at home and hobbies, 6) toileting and personal hygiene, 7) behavior and personality changes, 8) language and communication abilities, 9) mood, and 10) attention and concentration. Each question on the scale is scored on five choices assigned 0, 0.5, 1, 2, and 3 points, respectively. The QDRS total score is the sum of all the 10 questions, with scores ranging from 0 to 30 [9]. A higher score indicates worse cognitive and functional impairment. Above questions are divided into two subdomains: cognitive (questions 1, 2, 3, 8) and behavioral (questions 4, 5, 6, 7, 9, 10).
Disease staging and cognitive assessments
Trained raters who administered the CDR to the participants and their caregivers were blinded to their QDRS scores, clinical diagnoses, and other assessments. The QDRS score was also not considered for the scoring and staging of the CDR. The CDR rates cognitive and daily functioning using six categories: Memory, Orientation, Judgment and Problem-solving, Community Affairs, Home and Hobbies, and Personal Care. A global CDR score of 0 indicates No Cognitive Impairment (NCI), 0.5 indicates Mild Cognitive Impairment (MCI), and CDR 1, 2, or 3 indicates mild, moderate, and severe dementia, respectively. The CDR Sum of Boxes (CDR-SoB) was calculated by adding up the individual CDR categories giving a score from 0–18, with higher scores signifying more severe stages [11, 12]. The MMSE and the Ascertain Dementia 8 (AD8) were also separately conducted for all participants or informants as brief global cognitive assessments [18].
Calculating the time cost saving of QDRS utilization
We estimated the resources and time saved if the QDRS is used a selection tool to pre-screen subjects for prodromal or pre-symptomatic AD clinical trials. We devised two hypothetical clinical trials in which 100 participants with CDR = 0 (preclinical AD trial), and 100 participants with CDR = 0.5 (early prodromal AD trial) were to be recruited respectively. Two scenarios were demonstrated: 1) only the CDR was used to recruit eligible subjects, 2) only the participants who screened positive on QDRS, which was used as a pre-screening tool, underwent CDR for the selection of eligible subjects. A prevalence of 75% of CDR = 0 and 15% of CDR = 0.5 among the Singapore older people was assumed based on previous studies [11, 19]. This cost-saving calculation method was previously utilized in another study to screen for significant brain amyloid burden [20].
Statistical analyses
Descriptive statistics were used to present demographic characteristics of patients, QDRS, CDR, CDR Sum-of-Boxes (CDR-SoB), AD8, and MMSE. One-way analysis of variance (ANOVA) with Tukey-Kramer post hoc tests were used for continuous data and chi-squared analyses were used for categorical data. Concurrent (criterion) validity for cognition was assessed by comparing the mean performance of the CDR, CDR-SoB, AD8, and MMSE. Internal consistency was assessed with the Cronbach alpha reliability coefficient, with coefficients greater than 0.7 indicating good measures of internal consistency. Receiver operator characteristic (ROC) curve analysis was used to determine the discriminative ability between groups using the CDR as the gold standard, with results reported as area under the curve (AUC) with 95% confidence intervals (CIs). Analyses were conducted using SPSS v26.0.
Using the identified cut-off, we assessed the accuracy of the QDRS to detect CDR = 0.5, using the following formula [21]:
TP, True Positive; TN, True Negative; FP, False Positive; FN, False Negative
A logistic regression was performed to ascertain the effects of age, gender, year of education, race, and QDRS total score on the likelihood that subjects have CDR = 0.5 in subjects with no dementia (CDR <1).
We also compared the accuracy of the QDRS with other screening tools such as the MMSE and the AD8, both of which have been previously validated with established cut-offs in Singapore [18, 22–24].
The cost and time saving that could be obtained if the QDRS was used as a case selection tool to increase the proportion of CDR = 0 or CDR = 0.5 participants in clinical trials were estimated.
RESULTS
Sample characteristics
Sample characteristics and QDRS scores by CDR global stages are presented in Table 1. The mean age of the whole sample was 76.3±8.3 years (range 52–91) with a mean education of 6.4±4.9 years (range 0–20) and a mean QDRS, MMSE, and AD8 of 7.4±7.2 (range 0–27), 19.5±7.8 (range 0–30), and 4.5±3.0 (range 0–8), respectively. Among 177 participants, 32 (18.1%) were diagnosed as CDR = 0, 61 (34.5%) as CDR = 0.5, and 84 (47.5%) as CDR ≥1. There were significant differences between CDR diagnostic groups in MMSE, QDRS and its subdomains, and AD8 (p < 0.001). In addition, there were no significant differences between the excluded and included groups in terms of age, years of education, race, and gender.
Sample characteristics
MMSE, Mini-Mental State Examination; QDRS-Inf, Informant Quick Dementia Rating System; AD8, Ascertain Dementia 8.
Reliability and scale score features of the QDRS
The internal consistency was excellent, with Cronbach alpha coefficients of 0.90 (95% CI: 0.87–0.92) and 0.94 (95% CI: 0.93–0.95) for the two subdomains (cognitive and behavioral) and the total QDRS scores respectively. Correlations among the two subdomains and the QDRS total were high (Table 2).
Internal consistency, score features, and interscale correlation of the QDRS
Concurrent validity of QDRS
Excellent concurrent validity was found, as demonstrated by the high correlation between QDRS the CDR-global (R = 0.897, p < 0.001), CDR-SoB (0.915, p < 0.001), MMSE (–0.848, p < 0.001), and AD8 (0.747, p < 0.001). See Supplementary Table 1 for strength of associations between QDRS and other screening measures.
Discriminant validity of the QDRS
Subjects with CDR 0 ranged from 0 to 5.5 on the QDRStotal, M(SD) = 0.5(1.1), while those with CDR 0.5 ranged between 0–10.5 on the QDRStotal, M(SD) = 3.4(2.5), and those with CDR ≥1 ranged between 3.0–27.0 on the QDRStotal, M(SD) = 13.0(6.5). The ROC curves of the QDRStotal score are shown in Fig. 1.
Receiver operating characteristics curves of the QDRS. A) CDR 0 versus 0.5; B) CDR 0 and 0.5 versus ≥1.0.
Table 3 summarizes all discriminant indices of QDRS by CDR stages. A cut-off score of 1.5 displayed good sensitivity (85.2%) and specificity (93.8%), with excellent AUC of 0.88 (95% CI: 0.81–0.96) in discriminating CDR = 0 controls from CDR = 0.5 subjects (p < 0.001). Using this cut-off, 52 out of 61 subjects with CDR = 0.5 were correctly identified, whereas of the 31 subjects with CDR = 0, 2 were incorrectly included (i.e., 54 positives with 52 TP and 2 FP). An optimal cut-off score of 6 had the best AUC of 0.98 (95% CI: 0.96–0.99) in discriminating CDR = 0 and CDR = 0.5 from CDR ≥1 subjects (p < 0.001). By applying this cut-off, 79 out of 84 subjects with CDR ≥1 were correctly identified, and 10 out of 61 subjects (CDR = 0.5) were incorrectly included (i.e., 89 positives with 79 TP and 10 FP).
Discrimination ability of the QDRS by CDR stages
Accuracy of the QDRS in identifying CDR = 0.5
The optimal QDRS cut-offs were: 0 to 1 for CDR = 0, 1.5 to 5.5 for CDR = 0.5, and 6 and above for CDR ≥1. Supplementary Figure 1 shows the distribution of subjects on each QDRS score range with their CDR groupings. Using the cut-off range of 1.5 to 5.5, the QDRS had excellent discriminant validity in identifying CDR = 0.5 (sensitivity: 68.9%, specificity: 92.4%, PPV: 85.7%, NPV: 85.2%, accuracy: 85.0%). 42 out of 61 subjects with CDR = 0.5 were correctly identified, but 2 with CDR = 0 and 5 with CDR ≥1 subjects were incorrectly included.
Comparison of QDRS with other screening tools
Table 4 summarizes all the obtained and established cut-off scores for detecting CDR = 0.5 and CDR ≥1 for the QDRS, MMSE, and AD8. The AD8 and MMSE had excellent accuracy in identifying people with dementia, but both had limited accuracy in identifying MCI subjects, with only 18 (30%) and 21 (34%) out of 61 subjects correctly identified respectively. In contrast, the QDRS correctly identified more MCI subjects: 42 (69%) out of 61 and provided a better overall accuracy of 85.0% compared to the AD8 at 71.2% and the MMSE at 68.5%.
Comparison of QDRS with AD8 and MMSE in detecting CDR = 0.5 and CDR ≥1
Time cost saving using QDRS as pre-screening tool
Clinical trial for preclinical AD, i.e., CDR = 0 subjects
By estimating a CDR = 0 prevalence of 75% in the community population, a total of 100/0.75 = 133 participants (100 positives, 33 negatives) on average should be recruited for CDR screening to identify 100 CDR = 0 subjects. This CDR-only screening scenario would lead to a total recruitment administration time of 133*45 = 5985 min (almost 100 h).
The sensitivity and specificity obtained using the cut-off<1.5 on the QDRS was 85.2% and 93.8% respectively in identifying CDR = 0, thus increasing the number of subjects required for QDRS to 133/0.852 = 156 (117 positive, 39 negative). Among whom 100 TP would screen pass the QDRS and be confirmed with CDR alongside 35×(1–0.938) = 2 FP subjects, that would be rejected by the CDR. The total recruitment administration time for this two-step screening scenario with the QDRS would be 102×45 = 4590 min (76.5 h), resulting in a modest 23.3% reduction in the CDR administration time (Fig. 2A).
Cost-effective analysis using QDRS as pre-screening tool. A) Clinical trial for preclinical AD; B) Clinical trial for prodromal AD. P, positive; N, negative; TP, true positive; FP, false negative.
Clinical trial for prodromal AD, i.e., CDR = 0.5 subjects
Estimating a CDR = 0.5 prevalence of 15% in the community population, a total of 100/0.15 = 667 participants (100 positives, 567 negatives) on average should be recruited for CDR screening to identify 100 CDR = 0.5 subjects. This CDR-only screening scenario would lead to a total administration time of 667*45 = 30015 min (approximately 500 h).
The QDRS cut-off range of 1.5 to 5.5 obtained a sensitivity of 94.0% and specificity of 89.2% respectively, thus increasing the number of subjects required for QDRS to 667/0.94 = 710 (107 positive, 604 negative). Using this cut-off, an estimated 604×(1–0.892) = 65 FP subjects would be included. The QDRS pre-screening would lead to a total of 100 + 65 = 165 participants being screened for subsequent administration of CDR for confirmation. The total time cost for this 2-step approach with the QDRS would be 165×45 = 7425 min (approximately 124 h), corresponding to a 75.3% reduction in the CDR administration time (Fig. 2B).
DISCUSSION
In this Asian memory clinic-based study, we found that the QDRS had a good discriminant validity in identifying MCI, and its results were highly correlated with the CDR. Our study showed that the QDRS displayed an excellent internal consistency and was highly correlated with CDR-global (0.897, p < 0.001), and CDR-SoB (0.915, p < 0.001). The sensitivity and specificity of the QDRS were high in discriminating CDR = 0 from CDR = 0.5, as well as CDR = 0.5 from CDR = 1 (all above 0.8). Moreover, the overall accuracy of QDRS was higher than that of both AD8 and MMSE in detecting CDR = 0.5 and CDR ≥1.
In this study, we suggested the cut-off of QDRS for discriminating MCI from NCI was 1.5 (sensitivity = 85.2%, specificity = 93.8%, AUC = 0.88), and 6 (sensitivity = 94.0%, specificity = 89.2%, AUC = 0.98) for discriminating MCI from dementia, which was consistent with previous study in USA [9]. Compared to other Asian countries, the cut-off for CDR 0.5 group in our study was higher than Korean (1.0) and lower than Iran (3.0). In CDR 1 group, the Singapore cut-off was higher than both Korean (5.0) and Iran (5.5) [10, 16]. The discrepancy is possibly due to the different demographic and risk factors in each study.
We found the range for Normal (0 to 1) and MCI (1.5 to 5.5) is rather narrow, but the range was less “narrow” as compared to other commonly used tool such as the MoCA. The MoCA has a maximum score of 30, using the locally validated cut-off, the range for Normal is 30 to 27, and MCI is 26 to 24, with an average range of 2-3 points. Although both the QDRS and MoCA has the same maximum score of 30, and a similar score range for non-dementia subjects (0 to 5.5 versus 30 to 24), but the QDRS has been reported better in capturing cognitive or functional changes as it measures changes in the increment of 0.5 instead of 1.0 [9].
We found the QDRS to be a time-cost effective screening tool for identifying MCI. The CDR has been widely used as a dementia staging instrument in many studies in Asian populations [11], but its resource-intensive nature is an obvious limitation. In the memory clinical setting where the prevalence of MCI and dementia is relatively high, using the QDRS will significantly reduce the number of subjects needed to go through the CDR screening for diagnostic confirmation. Our study showed that the QDRS achieved 85.0% accuracy in the diagnosis of MCI (CDR = 0.5), making it an effective screening tool in clinical practice and in clinical trials. We estimated that by using the QDRS as a pre-screening tool, the time required to administer CDR was reduced by 23.3% in screening CDR = 0, and 75.3% in screening CDR = 0.5 subjects. The reduction in administration time demonstrated the effectiveness of the QDRS as an effective mass screening tool without placing any additional burden on the recruitment personnel.
The QDRS is feasible to be self-administered by the informant and does not require any trained administering personnel. During the COVID-19 pandemic, using the QDRS informant version via the phone or internet to caregivers can minimize face-to-face contact and allow for sustainable follow-up. We acknowledge that the reliability and validity of an informant administered instrument heavily depends on the informant making the rating. The reliability of the informant version may be declined, if the informants with depression or anxiety due to patient care. In addition, caregivers with lower educational level tend to have a poorer understanding of dementia in Singapore [26], and misconceptions may also lead to inaccuracy of the QDRS. Patient-based QDRS has been found to be highly correlated with the CDR [22, 25], hence patients may rate themselves on the QDRS in the absence of a reliable informant but need further validation in an Asian population. Furthermore, self-administration by patients with more severe cognitive impairment may not be appropriate, who may not be able to make an accurate assessment of their own cognitive abilities. By contrast, a reliable informant may capture their progressive cognitive changes [18].
One of the strengths of this study is the validation of the informant-based QDRS in an academic research setting where the prevalence of MCI and dementia are relatively higher. Under such setting, we can recruit patients with a high level of cooperation and adherence to follow-up, and can administrate CDR as the gold standard, allowing for more accurate and clear cut-off. The current study extends the original work of the QDRS to suggest the feasibility of the QDRS as a screening measure in Singaporean and other Asian populations. Another strength is the comparison of QDRS with other screening tools. Several limitations require acknowledgement. First, the present study was conducted in Singapore, thus the generalizability of the current finding to other countries requires confirmation. Second, our study population was recruited from hospital-based memory clinics. Future studies can aim to validate the QDRS in settings where MCI and dementia prevalence is lower, such as the community, or in telehealth settings where screening could be conducted via a website without face-to-face interaction. As the QDRS is conveniently self-administered, we are hoping to develop a digital form of the questionnaire for it to be disseminated to informants remotely.
Conclusion
The QDRS is a reliable and valid screening tool among older adults in Singapore in a clinical setting. Using the appropriate cut-off, the QDRS can accurately identify people at risk of dementia as well as those with dementia.
Footnotes
ACKNOWLEDGMENTS
The authors thank all participants for their involvement. They also thank the research team from the Memory Ageing and Cognition Centre (MACC) in National University of Singapore for data collection and management.
This work is supported by the Singapore National Medical Research Council (grants number NMRC/CG/NUHS/2010 and NMRC/CG/013/2013).
