Abstract
Although much of the research on service use by older adults with dementia relies on proxy reports by informal caregivers, little research assesses the accuracy of these reports, and that which does exist, does not focus on home care services. This brief report compares proxy reports by family caregivers to those with dementia with provincial Ministry of Health records collected for payment and monitoring. The four home care services examined include home nursing care, adult day care, home support, and respite care. Data come from a province-wide study of caregivers in British Columbia, Canada. Caregiver reports are largely consistent with Ministry records, ranging from 81.0% agreement for home support to 96.6% for respite care. Spouses living with the care recipient (the vast majority of the sample) are the most accurate. Others, whether living with the care recipient or not, have only a 50-50 chance of being correct.
Much of social science research in gerontology depends on self-reports by older adults themselves and, especially when the older adult has dementia, by their caregiver. However, research on the accuracy of self-reports of service use is relatively scant; research on caregiver proxy reports is fewer still and on caregiver proxy reports of service use by those with dementia is even less available. Bhandari and Wagner’s (2005) review of 42 studies on the accuracy of self-reported utilization data noted “variable” accuracy and did not include studies on proxy reports by caregivers to those with dementia. Callahan and colleagues’ (2015) recent comparison of caregivers to those with dementia focused on clinical trials data. Furthermore, existing research focuses on hospital use, nursing home admissions, and clinical visits, not on home care service utilization. This article compares proxy reports by family (primary) caregivers of home care service utilization (specifically home nursing care, adult day care [ADC], home support, and respite care) by the care recipient with dementia with provincial Ministry of Health (MoH) records collected for payment and monitoring. Accuracy is important; without it, one cannot understand the validity of the data and how to interpret it.
Method
Data come from a program of research known as the Alzheimer Drug Therapy Initiative (ADTI), British Columbia, Canada, a coverage with evidence development endeavor consisting of four studies: Cost and Utilization, Clinical Epidemiology, Seniors Medication Study, and the Caregiver Appraisal Study (CAS). The final report from the complete program is available at http://www2.gov.bc.ca/assets/gov/health/health-drug-coverage/pharmacare/adtiresearchstudiesuvic.pdf. The data reported here come from the CAS, consisting of a nonrepresentative province-wide sample of caregivers to those with dementia in British Columbia, Canada. A description of this sample appears in several publications (see Chappell, 2016; Chappell, Dujela & Smith, 2014a,b).
Briefly, 906 caregivers participated. They were English speaking and provided three or more hours/week of care for a person with physician-diagnosed dementia and on cholinesterase inhibitors (ChEIs) for 6 months or longer. Recruitment occurred primarily from phone calls made by PharmaCare personnel who informed patients that their ChEIs were covered by the provincial insurance program, and of the research, asking whether they would like to be connected by phone, contacted by the researchers, or mailed information about the research (68% of eligible participants came from this source; 23% from physician referrals). Caregivers were interviewed in person (M = 97.26 min) between September 15, 2008 and October 14, 2012. Due to missing data for three care recipients, the final sample size for the analyses reported here was 903. Caregivers were asked about their own and their care recipients’ health, social support, and sociodemographic information as well as their own perceptions of the effectiveness of ChEIs.
Of interest here is the question from the interview: “Does [the care recipient with dementia] use any of the following formal services? home nursing care, ADC, foot care, handi-dart, geriatrician/other specialist, emergency room (ER) visits, home support, respite care, other.” Of these, home nursing care, ADC, home support, and respite care were also available in MoH records through the Home and Community Care (HCC) database. ER visits are recorded by the MoH only when there is admission to hospital so were not suitable for comparison. Service use in the database (yes/no) was determined from the date of interview to 6 months prior. Duplicates were excluded; multiple services of the same type within this period were counted as one use.
Previous experience indicated that neither caregiver nor older adults were good at distinguishing private from publicly funded services, so no attempt was made to do so in asking the question. Home nursing and ADC are based on need; there is no charge. Home support and respite (referring only to temporary facility stays) are income (not asset) tested with possible copayment usually not exceeding CAD 35/dayfor the former and less than CAD10/day for the latter.
In assessing consistency between caregiver reports and MoH records, four categories were created: a positive match where both said the care recipient used the service, a negative match where both said the service was not used, false positive where the caregiver said the care recipient did use the service but the records indicated it was not used, and a false negative where the caregiver said the care recipient did not use the service but records indicated they did. False negatives and false positives were differentiated because false positives could be accounted for through private purchase (discussed further below).
Accuracy was measured in terms of percentage of correct responses by caregivers for each service, and the sum of correct positive and correct negative responses. We also examined both types of errors across the four services. Caregiver responses were coded as −1 for a false negative response, 0 for a correct match, and +1 for a false positive response, and summed for three services (we excluded respite care because there were few mismatches). Those receiving a negative total score (−1, −2, or −3) were grouped as False Negatives (n = 70). Those receiving scores of 0 constituted Correct Matches (all responses correct; n = 624). Those scoring 1, 2, or 3 were grouped as False Positives (n = 195). The 12 caregivers (1.9%) missing a response on at least one service or making two errors, one false positive and one false negative that canceled each other out in the computation were excluded from analyses. One-way between-subjects ANOVAs, including post hoc analyses using the Bonferroni adjustment for Type 1 error, were used when comparing the three groups on quantitative variables. Chi-square tests of independence were used for categorical variables. The variables chosen were based on past research (Callahan et al., 2015) or intuitive relevance (caregiver: age, years caring, hours/week caring, burden, relation, and lives with care recipient; care recipient: age, age at diagnosis, gender, number of chronic conditions, and number of services.
Results
Caregivers tended to be women (68.8%), spouses (67.7%), and had a number of their own health problems (sleep problems, 50.7%; arthritis, 48.9%; high blood pressure, 43.0%; back problems, 41.8%). On average, they had 4.4 chronic conditions (range: 0-13), had been caregiving for 3.1 years (range: 0-20), and were 68.7 years of age (range: 20-93). Their median education was greater than high school but less than a university degree, and median monthly income was just below CAD 3,500.
Care recipients were half female (50.9%), primarily lived in the community (80.7%) rather than in a facility, suffered from Alzheimer’s disease (58.5%) rather than another type of dementia, currently had mild dementia (52.8%) rather than moderate (38.6%) or severe (8.5%). They had a number of health problems: ear trouble, including, but not restricted to hearing loss, 51.3%; high blood pressure, 49.3%; arthritis, 49.1%; and sleep problems, 46.3%. They had an average of 4.8 chronic conditions (range: 0-13), were 79.8 years old (range: 50-100), and were 76.7 years old when diagnosed with dementia (range: 50-99). Their median level of education was less than high school and income was about CAD 2,700 per month.
As shown in Table 1, Correct Matches are high, ranging from a low of 81.0% for home support to a high of 96.6% for respite care. For all services, False Positives outnumber False Negatives. The former range from 2.7% for respite care to 16.0% for home support. The latter range from 0.7% for respite care to 3.0% for home support. Most caregivers (69.1%) correctly identified service use for all four services; few (1.1%) identified none (see Table 2).
Matches: Percentage of Correct and Incorrect Responses per Service.
Matches: Total Number of Matches.
Bivariate analyses revealed no relationship between group membership (False Negatives, Correct Matches, False Positives) and several of the sociodemographic variables—caregiver: sex, education, poverty, rural/urban residence as measured by the Urban Index (Allan & Cloutier-Fisher, 2006), number in household, depression and anxiety measured separately by the Hospital Anxiety and Depression Scale (Mykletun, Stordal, & Dahl, 2001), self-esteem as measured by the Rosenberg Self-Esteem Scale (Rosenberg, 1989), self-rated health, burden as measured by the short Zarit Burden Scale (Bédard et al., 2001), number of chronic conditions, perceived mental and perceived physical health, and perceived closeness of the relationship with the care recipient after memory loss but prior to beginning the ChEI medication; and with care recipient: education and dementia diagnosis (see references for details on the measures).
However, other factors were significant. As shown in Table 3, all relationships, whether with caregiver or care recipient characteristics, showed the Correct group differing from the False Positives and the False Negatives but the two error categories not differing from one another. There was one exception, False Positives received fewer formal services than False Negatives. Otherwise, the Correct group were more likely to have been older, caregiving a shorter amount of time, providing fewer hours/week of care, and living with the care recipient. They were also more likely to be spouses (elaborated below), and to be caring for someone who was younger, male, diagnosed with dementia at a younger age, and had fewer chronic conditions.
Group Differences.
The importance of examining group differences is dramatically revealed in Figure 1 which examines relation (spouse, adult child, other) of the caregiver to the care recipient. Among caregivers not living with the care recipient, approximately 50% fall in the Correct group, whether they be spouses, adult children, or others. Among those living with the care recipient, however, spouses are much more likely to be correct (81.6%), with adult children the least likely to be correct (53.9%), other caregivers improve significantly (from 50.0% to 71.4%), but this category is small.
Group membership by caregiver relation and living arrangements.
Conclusion
Although caregiver proxy reports of care recipient service utilization tend to be consistent with Ministry records, caregivers who are most accurate are older, have been providing care for fewer years, provide fewer hours of care per week, live with the care recipient, and are spouses. Spouses living with the care recipient in particular are relatively accurate in reporting the services examined here. Others, however, have approximately a 50-50 chance of being correct. Such findings suggest that, where possible, service records need to be checked to verify service use, especially when the caregiver is an adult child, whether living with the parent or not. Expect caregivers, especially those who are not spouses, to overestimate the number of services utilized. For all four services examined, especially notable for home support and home nursing, overreporting was more common than underreporting. Other studies have tended to find underreporting more common than overreporting (Bhandari & Wagner, 2005; Callahan et al., 2015; Ritter et al., 2001; Schmitz, Russell, & Cutrona, 2002), but they have not examined home care services, nor typically, care to those with dementia.
Although Callahan and colleagues (2015) argue that “Given the large differences in health care utilization between persons with dementia compared with those without dementia . . . self-reported utilization provides valid estimates of the overall cost of dementia” (p. 12), the findings reported here suggest otherwise, at least for home care services. More research on proxy caregiver reports of the use of home care by those with dementia needs to be conducted. The findings reported here are based on an ad hoc analysis of data collected for a different purpose; future studies should include a time frame in the interview question to make the comparison with the administrative database “cleaner.”
The data reported here are not representative. Nevertheless, analytic generalization is still possible. The findings suggest, other than for spouses living with their care recipient, caution should be used when interpreting self-reports of home care service use by proxy caregivers. The government records refer to home care services covered by the provincial government. Services paid for directly by the care recipient or a third party are not included. Given these are home care services that could include other forms of payment, this likely accounts for some of the discrepancy, especially for home support services. However, the high percentage of Correct Matches (81.0%-96.6% for the individual services) suggests it would not be large. Furthermore, this issue would only effect the category False Positive for home support services (accounting for only 3% of responses), not Correct or False Negative, further minimizing this problem.
Footnotes
Authors’ Note
Ethics approval was received from all relevant health authority and university review bodies. Anyone interested in accessing the primary data should contact the first author. Access to provincial data is not available without permission from the province.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The data reported here were collected with a research grant from the Ministry of Health, province of British Columbia, Canada. Any opinions expressed are those of the authors and not of the government.