Abstract
Background:
Early detection of cognitive decline allows timely intervention to delay progression of dementia. However, current cognitive evaluation tools often include items delivered via verbal forms of instruction, which can cause poor performance in patients with hearing loss.
Objective:
To develop and validate a cognitive screening battery, the Reading Cognitive Test Kyoto (ReaCT Kyoto), comprising test items given through non-verbal instruction.
Methods:
A cross-sectional and multi-center study was conducted in the three medical institutes. ReaCT Kyoto was designed to evaluate domains of “registration,” “repetition,” “delayed recall,” “visuospatial recognition,” “orientation in time and place,” and “executive function.” The Japanese version of the Mini-Mental State Examination Test (MMSE-J) and ReaCT Kyoto were applied by experienced psychotherapists. Concurrent validity was evaluated between the ReaCT Kyoto Test and MMSE-J and between the ReaCT Kyoto Test and physician-diagnosed dementia.
Results:
ReaCT Kyoto was validated in a sample of 115 participants. The mean age of subjects was 81.0±6.4 years, and the sample comprised 53.0% females. The area under the receiver operating curves was 0.95 for detecting physician-diagnosed dementia. When classifying patients in accordance with presence or absence of hearing loss, the AUCs were 0.93 and 0.97 for those with and without hearing loss, respectively. With a cut-off score of < 29 points for suspected dementia, ReaCT Kyoto correctly classified 90.4% of the subjects as belonging to the group with or without physician-diagnosed dementia.
Conclusion:
ReaCT Kyoto provides an appropriate solution for detection of cognitive impairment in persons with or without hearing loss.
INTRODUCTION
Cognitive decline or dementia, and acquired hearing impairment due to age or other pathology have become increasingly common as aging society is rapidly progressing, especially among the developed countries. As a result of the demographic shift in the population, the prevalence of dementia among individuals aged 60 and older is estimated at 5–7% in most world regions and among those aged 71 and older at 13.9% in the United States [1, 2]. Age-related hearing loss, or presbycusis, affects approximately 30% of adults older than age 65 and nearly 50% of those aged 75 and older [3], and these figures can be expected to increase in the next decade. Hearing loss is associated with multiple negative outcomes, not only communication problems, but also isolation, depression, and possibly cognitive decline. The association between hearing impairment and dementia was proposed by Granick et al. in 1976 [4]. Kay et al. also documented that moderate or severe hearing loss was more common in old people with dementia living at home in Newcastle [5]. Recently, two publications regarding the association between hearing impairment and dementia indicate that age-related hearing loss is independently associated with worse cognitive function and incidental dementia [6], and that even mild levels of hearing loss in midlife and later life increases the long-term risk of cognitive decline and dementia [7]. Several possible mechanisms have been proposed to explain the association between hearing impairment and dementia. One hypothesis is that hearing impairment increases cognitive load and affects cortical processing, diverting cognitive resources to auditory processing at the expense of other processes [8, 9]. Conversely, hearing loss might cause cognitive dysfunction indirectly through associated social isolation or depression. Another possibility is that dementing illness themselves cause hearing impairment through pathological effects on neurological pathways [10, 11]. However, the evidence remains controversial across studies as to why and to what extent hearing impairment is correlated with cognitive decline and dementia. Limitations of previous studies include unclear definitions of hearing impairment and cognitive function assessment tools that contain verbal communication-related test items. Cognitive screening tests including the Mini-Mental State Examination (MMSE) frequently rely on items being correctly heard or seen. There remains the possibility that the moderate reduction in cognitive performance measured by verbally instructed task items is an artifact of hearing impairment. Hearing loss is often unrecognized, and therefore not considered in the differential diagnosis of poor performance in cognitive tests. Moreover, conditions as complex as dementia and hearing impairment share symptoms such as social isolation, depression, and forgetfulness about what patients’ family or caregivers said a little while before. Although whether early intervention in hearing loss could delay the progression of cognitive dysfunction is another important clinical question, a better understanding of the etiology behind the association between hearing impairment and cognitive decline should contribute to the development of interventions that preserve or improve cognitive function in patients with hearing loss in any case.
In the present multicenter clinical study, we developed and validated a cognitive screening battery, the Reading Cognitive Test Kyoto (ReaCT Kyoto), which comprises test items given through non-verbal instruction for people with hearing loss as well as those without hearing loss. This tool was also designed to distinguish hearing loss from cognitive impairment. Furthermore, ReaCT Kyoto should detect hidden cognitive decline in patients with hearing loss in addition to hidden hearing loss in patients with dementia by comparison with cognitive performance measured by the MMSE.
METHODS
Development of the ReaCT Kyoto Test
The ReaCT Kyoto development committee consisted of two neurologists, two clinical psychotherapists, four otolaryngologists, and two epidemiologists. Based on a review of previous studies, all the members of the committee discussed and generated items that might be useful to evaluate the degree of the cognitive decline in two rounds of panel conferences. Specifically, ReaCT Kyoto was designed to evaluate the domains of “registration,” “repetition,” “delayed recall,” “visuospatial recognition,” “orientation in time and place,” and “executive function (verbal fluency).”
In particular, for the evaluation of “registration,” “repetition,” and “delayed recall,” the development committee pre-selected 312 independent words from 59,233 categorematic words included in the Modern Magazine Two-Million Linguistic Survey Vocabulary List provided by the National Institute of Japanese Language. The members then anonymously voted for 20 eligible words from the 312 words, and 44 words that received two or more votes were chosen. Seven sets of five words were determined from the 44 words to check the face validity in two rounds of web-based modified Delphi methods and one face-to-face meeting. In both rounds of the modified Delphi methods, the level of agreement between the members of the committee was evaluated using a 9-point Likert scale, from 1 (not appropriate at all) to 9 (very appropriate). A positive or negative consensus was defined as 75% or more of the respondents scoring 7 and above or 3 or below, respectively.
Study participants
A cross-sectional, multi-center study was conducted to evaluate the reliability and validity of the ReaCT Kyoto scale in the following facilities: Kyoto University Hospital, Shiga General Hospital, and Kyoto Teishin Hospital. Participation was limited to those who regularly visited the department of otolaryngology due to hearing loss or the department of neurology due to cognitive decline, and aged older than 65 years. Exclusion criteria were persons with history of aphasia, drug or alcohol abuse, or with endocrine and/or metabolic disorders that might affect cognitive function. Patients with vascular dementia were also excluded because of potentially having aphasia. All participants were administered both the Japanese version of the Mini-Mental State Examination Test (MMSE-J) and ReaCT Kyoto in a quiet, separated room.
Cognitive evaluation and diagnosis of dementia
The Japanese version of the MMSE-J was performed by an experienced clinical psychotherapist according to the manufacturer’s instructions (Nihon Bunka Kagakusha, Tokyo, Japan, and PAR, Inc., Lutz, FL, USA). Diagnosis of dementia was made by two expert neurologists specializing in cognitive-behavioral neurology based on the 2011 guidelines in the National Institute on Aging-Alzheimer’s Disease Association Criteria and a detailed clinical assessment. The subjects were classified as non-cognitive impairment, mild cognitive impairment, or dementia.
Hearing assessment
Hearing examination, including pure-tone air- and bone-conduction audiometry, was conducted in a sound-proof room by an audiologist using a Rion AA-H1 audiometer (Rion Co., Ltd., Tokyo, Japan). Hearing thresholds at octave frequencies from 0.125 to 8 kHz were measured. Hearing impairment was defined as the pure-tone average of air-conduction hearing threshold greater than 40 decibels hearing level (dB HL) at four frequencies (0.5, 1, 2, and 4 kHz) in the better ear, according to the WHO definition [12, 13]. No subjects demonstrated profound hearing loss, as verbal communication would not have been practical; however, the MMSE-J was administered by a clinical psychotherapist using a bullhorn in some cases.
Statistical analyses for psychometric evaluation
Patient characteristics were described using descriptive statistics. For each item, we quantified ceiling and floor effects using the proportions with which the extreme response options were chosen. The data were then analyzed to identify the underlying components using exploratory factor analysis based on an iterated principal factor method. The number of components was determined by an eigenvalue greater than 1. Cronbach’s alpha was calculated to evaluate the reliability. To assess criterion-related validity, we then categorized participants into four groups based on total scores if the unidemensionality of the scale was ascertained based on the result of factor analysis, and calculated the proportions diagnosed as having dementia by physicians. Concurrent validity was evaluated with the Spearman’s correlation coefficients between the ReaCT Kyoto Test scores and total scores of MMSE-J. Furthermore, to evaluate the discrimination ability to detect physician-diagnosed dementia, the areas under the receiver operating curves and sensitivity and specificity in each score were calculated. The Hosmer-Lemeshow test was employed to ascertain model performance, and a calibration slope was drawn to evaluate the calibration ability. We also assessed the discrepancy of the test performance when classifying the subjects according to presence or absence of hearing loss. A p-value less than 0.05 was considered statistically significant for all analyses. The statistical software used was STATA version 15.1 (College Station, TX, USA).
Ethical issues
The study was performed in accordance with the ethical standards in the Declaration of Helsinki of 1975 and its later amendments or comparable ethical standards. The protocol of the present study was approved by the Institutional Review Boards and Ethics Committees of Kyoto University Hospital and Kyoto University Graduate School of Medicine (R0950). Written informed consent was obtained from all the participants as well as accompanying partners or caregivers of the persons with dementia and/or hearing loss.
RESULTS
Development of the ReaCT Kyoto Test
Table 1 shows the initial version of ReaCT Kyoto Test drafted by the ReaCT Kyoto development committee. In accordance with the discussions of the committee, the initial version of ReaCT Kyoto scale comprised eight items: registration (one item), repetition (one item), delayed recall (one item), visuospatial recognition (one item), orientation in time (one item), orientation in place (one item), and executive function (verbal fluency) (two items). The visuospatial function domain assessed the ability to draw pictorial figures consisting of two rectangles. The verbal fluency domain, which comprised two items, assessed the ability to present as many names of vegetables and words starting with ka- as the subjects could within a minute. With regard to the word sets to evaluate “registration,” “repetition,” and “delayed recall,” we finalized one main combination (shoe/photograph/telephone/bank/sea) and four spare combinations of five words based on two rounds of web-based modified Delphi methods and an additional face-to-face meeting.
All the instructions of the test were given by written Japanese with legible font size for elderly people (a regular gothic type font and 80 points in A4 size paper, in a bright room). Before starting cognitive evaluation using ReaCT Kyoto, reading comprehension was evaluated via the general instruction of ReaCT Kyoto and inquiries of name, date of birth, and age of the patients, all of which were given by written Japanese. The patients who were not able to answer their name, date of birth, or age were excluded from the present study. Each item of the ReaCT Kyoto Test was scored on a range of 0–5 points. The score for each item of “registration,” “repetition,” and “delayed recall” was defined as the number of correct answers from a set of five presented words. With regard to the item of “visuospatial recognition,” the score was defined as 0/2.5/5 points when they were able to copy none/one/two of the presented pictorial figures by hand. “Orientation in time and place” was evaluated using two items: “orientation in time” and “orientation in place.” The score of “orientation in time” was defined as the number of correct answers for the year, month, date, season, and day of the week when the patient was asked, and the score of “orientation in place” was defined as the number of correct answers for the region, prefecture, city, name of the facility, and floor where the patient was asked. With regard to the two items of “executive function” (verbal fluency), the number of words presented by the patient was classified into six categories of scores: 0 words as 0 points, 1–3 words as 1 point, 4–6 words as 2 points, 7–9 words as 3 points, 10–12 words as 4 points, and more than 12 words as 5 points.
Item response frequencies
The characteristics of the 115 participants are shown in Table 2. The mean age of the subjects was 81.0 years (standard deviation (SD):±6.4 years), women (53.0%) slightly outnumbered men, and years of education ranged between 3 and 24 (mean±SD: 13.0±3.4 years). All the participants were literate. Analysis of correlation coefficient indicated that there is some degree of positive correlation between the scores of ReaCT Kyoto and years of education (r = 0.38). 42.6% of the subjects were diagnosed as having dementia. The participants consist of 49 with normal cognitive function, 15 with MCI, 38 with AD type dementia, 7 with AD with concomitant small vessel disease, 3 with dementia with Lewy bodies, 2 with primary age-related tauopathy, and 1 with progressive supranuclear palsy. The proportion of participants with hearing loss was 61.7%. We described the characteristics classified in accordance with presence or absence of hearing loss in Table 2, suggesting that those with hearing loss were more likely to be male and less likely to be diagnosed as dementia.
Mean completion time for both MMSE-J and ReaCT Kyoto Test was approximately 30 min. For each item of ReaCT Kyoto Test, Table 3 shows the distribution of item responses, including the percentage of respondents who chose the “best” and “worst” response options to ascertain ceiling and flooring effects.
Construct validity
Polychoric correlations among the items are shown in Table 4. The correlations ranged from 0.33 to 0.86. Results of the exploratory factor analysis showed that ReaCT Kyoto Test had an uni-dimensional structure on the basis of the pattern of eigenvalues, decreasing remarkably with the second and later factors (Table 5).
We therefore calculated the total scores of all eight items, which had a range of 0–40 points. The mean total score was 27.6 (SD:±8.1), and the minimum and maximum scores of ReaCT Kyoto Test in the present study were 6 and 40, respectively.
Reliability
Cronbach’s alpha for all eight items was 0.863.
Concurrent validity and criterion-based validity
As shown in Fig. 1, a strong correlation was observed between the scores of the MMSE-J and ReaCT Kyoto Test (r = 0.90). With regard to the criterion-based validity, there were clear dose-response relationships between the proportion having definitive diagnosis of dementia and the quartiles of the total score of the ReaCT Kyoto Test (Fig. 2).
Predictive validity
The AUC was 0.95 (95% CI: 0.91–0.99) for detecting physician-diagnosed dementia. The calibration performance under the Hosmer-Lemeshow test was p = 0.43. The ROC curve and calibration slope are shown in Figs. 3 and 4, respectively. In addition, Table 6 shows the test performance of the ReaCT Kyoto Test, including sensitivity, specificity, likelihood ratio, and the proportion of those correctly diagnosed at each cutoff point. With a cut-off score of < 29 points as having suspected dementia, ReaCT Kyoto correctly classified 90.4% of the subjects as belonging to the group with or without physician-diagnosed dementia.
When classifying patients in accordance with presence or absence of hearing loss, the AUCs were 0.93 (95% CI: 0.87–0.99) and 0.97 (95% CI: 0.93–1.00) for those with and without hearing loss, respectively.
DISCUSSION
This study developed and validated a novel cognitive test battery independent of the subject’s hearing condition. The results of validation tests and reliability tests showed that the scale could be used to evaluate cognitive functions in older participants regardless of their hearing loss. The diagnostic performance of ReaCT Kyoto was found to be satisfactory in sensitivity, specificity, and abilities of discrimination and calibration. Ceiling and flooring effects of most items were acceptable. The results of exploratory factor analysis showed that the scale comprised one factor, and thus we calculated total scores of the ReaCT Kyoto Test for analysis. For criterion-based validation testing, a clear dose-response relationship was observed between the proportion having physician-diagnosed dementia as the external criterion and patients categorized in accordance with the quartiles of total scores of ReaCT Kyoto Test. The results were also satisfactory regarding concurrent validity, since a strong correlation between the MMSE-J and ReaCT Kyoto Test was observed. ReaCT Kyoto also has strength in terms of independence from the examiner’s skills and therefore feasibility in clinical settings, since an examiner applying the ReaCT Kyoto simply indicates the written instructions and determines scores according to subjects’ replies.
Since the need for tests adapted to persons with visual and/or auditory sensory impairments has been recognized [14], several previous studies developed or adapted cognitive screening and assessment tools for persons with dementia and acquired hearing impairment. Frequently adapted tests are the MMSE and Montreal Cognitive Assessment (MoCA) [15]. Uhlmann et al. tested 71 Alzheimer’s disease patients with varying degrees of hearing impairment using a written modified version of the MMSE [16]. De Silva et al. also presented data comparing the standard and written versions of MMSE for 82 subjects over the age of 65 with a range of cognitive and hearing impairment. Contrary to their hypotheses, both studies found no clinically significant difference between the standard and written MMSE in the hearing-impaired group. Moreover, a version of the MoCA to be administered to the severely hearing impaired (HI-MoCA) was developed to assess its performance in adults over the age of 60. HI-MoCA was developed through the conversion of verbal instructions into visual ones, and was found to be equivalent to the standard MoCA for people with or without hearing impairment [17]. Although attempts have been made to adapt cognitive tests for persons with acquired hearing loss by deleting or creating written versions for hearing-dependent items, their validity in accurately detecting dementia among those with acquired hearing impairment is still not fully established. No study has reported the validity of such tests in relation to the detection of dementia in people with hearing impairment. In addition to cognitive tests for those with hearing impairment, some tests for early detection of cognitive impairment have been also developed adopting non-verbal tasks. Kandiah et al. reported the development and validation of a novel visual-based cognitive evaluation tool that can be administered to a multilingual population without translation or adaptation [18]. The test was validated in a sample of 206 subjects, revealing high sensitivity and specificity comparable to those of the MoCA administered in the same cohort. Another attempt to overcome differences of languages, educational levels, or cultures has been reported by Carnero-Pardo et al. [19]. In comparison with the MMSE, the authors emphasized the advantages of a visual-based phototest and memory impairment screen from the viewpoint of cost-benefit effectiveness, while the memory impairment screen cannot be used for illiterates and the specificity of MMSE was relatively low due to the low educational level of the samples. The Eurotest, a screening tool for cognitive impairment based on the knowledge and handling of money, was reported by the same research group from Spain [20]. The advantage of the Eurotest is that it can be used with no modification in any country that uses the euro as currency, and it can be also easily adapted to any other currency. A limitation of the developed test for illiterates or multi-lingual population is that these tests could only suggest the existence of cognitive decline, and thus clinical and neuropsychological tests are also needed to diagnose dementia at the same time. In addition, assessments of language or executive function are not fully satisfactory for evaluation of the relevant neuropsychological functions related to dementia or Alzheimer’s disease.
As mentioned above, the association between hearing loss and cognitive decline in older adults has been recently advocated in several epidemiological studies [3, 6] and systematic reviews [7, 21], whereas the cause-result link of how hearing loss increases the risk of developing dementia is controversial and still under debate. One point that should be taken into account when analyzing previous studies reporting hearing loss and cognitive decline is the impact of sequential approvals of acetylcholinesterase (AChE) inhibitors, such as donepezil (Aricept ™, Eisai and Pfizer), on the symptomatic treatment of mild to moderate Alzheimer’s disease in the period between 1990–2000 [22]. Although there is no way to cure Alzheimer’s disease, AChE inhibitors might improve symptoms and even delay disease progression in the early and middle stages of the disease. Various guidelines for the treatment of Alzheimer’s disease recommended the use of AChE inhibitors after placement on the market around 2000 [23–25], and as the treatment of Alzheimer’s disease and quality of life of patients with Alzheimer’s disease dramatically changed once AChE inhibitors were approved, there is a possibility that treatment using AChE inhibitors might also impact age-related hearing loss or use of hearing aids in aged persons. These factors are thought to be one of the reasons for the equivocal evidence regarding the benefit of early intervention for hearing loss in middle or older age people in the prevention or delay of progression of cognitive impairment.
A variety of approaches have recently been introduced to detect the link between cognitive decline, especially in Alzheimer’s disease, and auditory disorders. Gates et al. administered three central auditory tests including dichotic sentence identification, dichotic digits, and synthetic sentence identification to persons with Alzheimer’s disease and found that central auditory dysfunction is a precursor to the onset of Alzheimer’s disease [26]. Iliadou et al. also reported the results of evaluating auditory perception in a group of older adults diagnosed with mild cognitive impairment. Auditory perception was evaluated by assessing temporal resolution in the Gaps-In-Noise test, and suggested that central auditory processing evaluation in the elderly population should be a promising indicator to achieve earlier diagnosis of dementia [27]. Swords et al. reviewed the literature concerning auditory dysfunction in Alzheimer’s disease and concluded that a dichotic listening task showed high diagnostic and prognostic significance in persons with Alzheimer’s disease; therefore, audiological approaches are a low-cost way to identify an early window of Alzheimer’s disease pathology [28]. These data indicate that cognitive decline in persons with Alzheimer’s disease is involved in auditory dysfunction, not only peripherally due to age-related degeneration of inner ear structure but also centrally due to alterations in the processing and perception of auditory signals. In addition, these findings strongly support the fact that evaluation for both cognitive and hearing impairment is recommended to identify and diagnose earlier older persons with Alzheimer’s disease and age-related hearing loss.
Several limitations of ReaCT Kyoto should be considered in potential future studies for improvement and validation. First, in agreement with other screening tests, ReaCT Kyoto should not be used in isolation for the diagnosis of dementia. Neuropsychological examinations are clinically needed to confirm the existence of dementia. While there is a good correlation between MMSE-J score and ReaCT Kyoto score, three patients showed discrepancy. The patients are all in their late 80s, diagnosed as having dementia due to Alzheimer’s disease, and with a poorer performance in terms of executive function (verbal fluency). However, visual acuity or reading comprehension are basically preserved in all the three patients. The exact reason regarding the discrepancy still appears to be unclear. Second, since ReaCT Kyoto uses written instructions throughout the test, persons with visual impairment or visual cognition decline are not intended as targets for ReaCT Kyoto. Third, in agreement with other cognitive assessment batteries, the educational levels of subjects might affect the results of performance on ReaCT Kyoto, as ReaCT Kyoto uses written instructions, although virtually no one is practically illiterate in modern Japanese society. Fourth, the item of visuospatial recognition showed high ceiling effects. In a future study, revision may be needed to accurately evaluate visuospatial recognition over a wider range. Last, it was a Japanese multicenter study, and only Asian patients participated. Further studies are required to ascertain the generalizability in other countries.
In summary, while there is an urgent need for early detection of cognitive impairment or dementia, substantial difficulties are often found in distinguishing cognitive decline from other sensory disabilities in daily clinical practice. Although the relatively small sample size in the present study may appear to be a limitation, ReaCT Kyoto provides an appropriate solution for detection of cognitive impairment in persons with or without hearing loss. ReaCT Kyoto could be potentially used for future cross-sectional or longitudinal studies to examine the clinical question of whether wearing hearing aids improve or delay the onset or progression of cognitive impairment.
Footnotes
ACKNOWLEDGMENTS
The authors thank Ms. Atsuko Kokuryu, Ms. Noriko Uchida, Ms. Ai Ishida, Ms. Kanako Kondo, Ms. Toshiko Fujikawa, and Ms. Miyoko Oka at Kyoto University Hospital, Ms. Tomoko Okina and Mr. Norio Suzuki at Shiga General Hospital, and Dr. Hideyuki Yoshioka and Ms. Seiko Kawae at Kyoto Teishin Hospital, for conducting the MMSE-J and ReaCT Kyoto. This research was supported by the Japan Agency for Medical Research and Development under Grant No. 18dk0207031.
