Remote Health Monitoring for Older Adults and Those with Heart Failure: Adherence and System Usability

Abstract

Background: Remote health monitoring technology has been suggested as part of an early intervention and prevention care model. Older adults with a chronic health condition have been shown to benefit from remote monitoring but often have challenges with complex technology. The current study reports on the usability of and adherence with an integrated, real-time monitoring system over an extended period of time by older adults with and without a chronic health condition. Materials and Methods: Older adults 55 years of age and over with and without heart failure participated in a study in which a telehealth system was used for 6 months each. The system consisted of a wireless wristwatch-based monitoring device that continuously collected temperature and motion data. Other health information was collected daily using a weight scale, blood pressure cuff, and tablet that participants used for health surveys. Data were automatically analyzed and summarized by the system and presented to study nurses. Results: Forty-one older adults participated. Seventy-one percent of surveys, 75% of blood pressure readings, and 81% of daily weight measurements were taken. Participants wore the watch monitor 77% of the overall 24/7 time requested. The weight scale had the highest usability rating in both groups. The groups did not otherwise differ on device usage. Conclusions: The findings indicate that a health monitoring system designed for older adults can and will be used for an extended period of time and may help older adults with chronic conditions reside longer in their own homes in partnership with the healthcare system.

Introduction

“Automated hovering” has been suggested as part of an early intervention and prevention healthcare model in which patients are “watched over” not just at an occasional doctor's visit, but also daily as outpatients via mobile devices and real-time wireless data transmission and analysis technologies.¹ Older adults with a chronic health condition have been shown to benefit from automated hovering.² Proactively detecting symptom changes earlier with healthcare provider intervention is intuitively better than reactively treating patients who may wait too long and become seriously ill before seeking treatment. Recent meta-analyses support this proactive model of care, showing improvement in quality of life and reduced healthcare costs when key symptom tracking for chronic conditions was used with early intervention.^2,3 The relationship between telemonitoring and improved outcomes, however, has not been consistent across studies.^3,4

Even though electronic monitoring devices can help patients manage symptoms, technology can be difficult for older adults with chronic illness to adopt.^5,6 Prior descriptive research⁷ and a meta-analysis⁸ found that older adults were concerned about data privacy but were willing to use remote monitoring and share data with healthcare professionals if that could help identify emerging health problems. Therefore, older adults may use monitoring technology if these criteria are met.

Usability is usually defined as the extent to which people can quickly learn to reliably and safely operate a device. Devices such as blood pressure (BP) cuffs are only useful if participants use them, whereas other sensors, such as wearable monitors, require less user effort but must be put on. The variety of components that may require attention can be overwhelming, possibly further causing a reluctance to use technology; discomfort and/or disabilities associated with a chronic condition may compound the difficulty. These issues highlight the need for usability to be empirically measured. The importance of verifying usability in order to help older adults adopt technology is reflected in recent work addressing perceptions regarding the ease of monitoring system use for persons with a chronic condition.^9,10

Likewise, how well components function over time—reliability—is also critical for devices expected to operate continuously and detect changes in real time that may require attention, such as a fall or decreased activity level. Recent work suggests that devices that are used regularly, such as a scale or BP cuff, appear to perform reasonably well with minimal data loss.^11,12 The current study uses a monitoring system with both components expected to operate continuously and ones to be used daily. This offers a unique opportunity to examine (1) how well a continuously operating wireless home health monitoring system—designed to be low cost and usable by chronically ill seniors—performs in a home environment and (2) how well older adults adhere to using such a system with potential to reduce costs while improving delivery of care.

The impact of chronic illness on society is enormous. In 2010, 86% of all U.S. healthcare spending was for chronic medical conditions.¹³ Heart failure (HF) is a chronic condition associated with increased mortality, risk of rehospitalization within 30 days of discharge for an HF-related admission, higher healthcare costs, and decreased quality of life.¹⁴ Because HF may respond well to early treatment and intervention and because worsening HF symptoms often result in hospital admission, it may be cost-effective to monitor HF patients remotely. The current study included monitoring of persons with HF to investigate whether the adherence rates and usability ratings would differ for those with a chronic condition. If actual use data collected over a 6-month period show that the level of adherence is equal for older adults with and without HF, this would be an important finding, suggesting that the system is usable by both older adults with a chronic condition and those coping with the physical and cognitive changes associated with normal aging.

This study sought to establish whether older adults with and without the chronic condition of HF can and will use over an extended period of time a remote health monitoring system designed for ease of use. A secondary goal was to explore whether a monitoring system using a cellular wireless connection could reliably collect health data in real time over an extended period. The results reported in this article are from a broader trial that included HF participants randomized to usual care with nurse visits or to usual care with nurse visits and the telehealth system. This report focuses on describing the results pertaining to system reliability and adherence for those receiving the telehealth system.

Portions of the study results have been presented and published in abstract form as conference proceedings.^15,16

Materials and Methods

Participants

Persons with and without HF were recruited through local hospitals and advertisements for this Institutional Review Board–approved study. Interested persons were prescreened to verify the following criteria: living in independent housing with reliable electricity and phone service, not participating in another clinical trial, fluent in English, have adequate hearing and vision, ambulatory (with assistive devices if needed), and not suffering from dementia (as determined by two or fewer errors on the Short Portable Mental Status Questionnaire¹⁷ and a recall of seven or more items on the Wechsler Memory Scale-III¹⁸). An additional exclusion criterion included unavailability for 21 or more sequential days during the 6-month study. Those with HF had their diagnosis verified either by providers (if recruited through hospitals) or by a registered nurse who interviewed study candidates and reviewed their health histories and medication regimens during a clinical interview (if recruited through advertisements). HF participants were further assessed for their New York Heart Association functional class. Enrolled participants were compensated $15 a week for using the system and $25 for each monthly in-home nurse visit.

Study Equipment

The monitoring system consisted of Food and Drug Administration–cleared devices (Fig. 1). The central feature was the watch device that collected skin temperature and motion data using built-in tri-axial accelerometers (AFrame Digital, Inc., Reston, VA). Other components included a one-button BP cuff (model UA-767PBT) and a digital weight scale (model UC-321PBT) from A&D Medical (San Jose, CA). Wi-Fi hotspot devices from two U.S. service providers (AT&T [Dallas, TX] and Verizon [New York, NY]) allowed the system to access the Internet wirelessly through the cellular antenna network to obviate the need for a home Internet connection. A Samsung (San Jose) Galaxy™10-inch touchscreen tablet was used for administering daily health surveys. For ease of use, the majority of the tablet's features were disabled, and it was set up to automatically connect to the Internet through the Wi-Fi hotspot and only display the health surveys, a welcome screen with graphical health and wellness status, and system information to participants (Fig 2 ). Therefore, the only participant burden from the tablet was remembering to charge the device periodically and use the power button to wake it from sleep mode.

Fig. 1.

Overview of the telehealth home monitoring system and its component parts. API, application programming interface; DSL, digital subscriber line; PHI, protected health information.

Fig. 2.

Participant screen on the tablet device.

Measures and Procedures

To assess perceptions of system usability and to measure the amount of prior experience with technology (a potential factor in usability), participants were asked to complete two measures. Device usability was measured with a modified (range, 8–40) System Usability Scale (SUS),¹⁹ whereas the Technology Experience Questionnaire (TEQ)²⁰ was used to measure a participant's familiarity with technology. The TEQ asked participants to rate their level of interaction over the last year with various electronic devices such as cell phones and digital home appliances. The TEQ contained 42 items and used a Likert scale to measure how frequently the participant used each type of technology on a scale from 1 (“Not sure what it is”) to 5 (“Used frequently”); thus, the range of scores was from 42 (little experience) to 210 (frequent experience).

Each participant read and signed the informed consent after having the chance to interact with the study equipment and ask questions. Participants were provided contact information for the study nurse who would follow up with them to schedule their first in-home nursing visit. In most cases, the wireless connectivity meant that the installation and training usually took less than 1 h. Participants were given the TEQ and SUS questions and asked to complete and return to the nurse on their first visit. This was done to ensure that participants had some experience with the system before rating usability. The SUS was administered again when the system was removed 6 months later.

A registered nurse visited each participant's home monthly, during which she would ask if there have been any medical issues, doctor's visits, or hospital admissions. She would also complete a short physical examination and, if necessary, suggest that the individual seek further medical attention.

Between visits, the nurses kept abreast of monitoring data trends and alerts—analyzed and presented by the monitoring system—that could potentially indicate changes in symptoms, contacting participants as necessary.

Analytic Strategy

Statistical analysis was completed using PASW (SPSS) Statistics for Windows version 18.0 (SPSS Inc., Chicago, IL). The sample size for each group to detect a difference in proportion at the 0.8 (medium effect) level with an alpha of 0.05 was determined to be 25. To allow for attrition, the size for each group was increased to 30 for a goal of 60 participants.

To measure adherence, the percentages of data collected were calculated by comparing the number of minutes, or days, of that particular type of data divided by the number of minutes, or days, that participants were involved in the study (although not on vacation or in the hospital). In cases in which a participant used the BP cuff or scale more than once on a given day, only the first reading was counted. Skin temperature was sent to the system's server once per minute and was used to determine if the watch was being worn. A reported skin temperature below 76°F generally indicates a watch is not being worn; this criterion was used to identify nonadherence with watch monitor wearing. This mean temperature comparison value was determined based upon prior comparisons with ambient temperatures and extensive recording of readings over days, weeks, and months of usage (24/7 basis) from multiple subjects in prior studies and in operational use who had consented to continuous wearing of the watch monitor.

Based on feasibility trial results,¹¹ participants were estimated to wear the wrist monitor at least 75% of the time and complete at least 75% of requested physiological readings during their 6 months of participation. The 75% figure was arbitrary but within range of the 80% medication adherence level generally considered an acceptable target rate.^12,21 The estimated system usability was 80% or higher and motivated by the work of Giger et al.,⁹ who reported usability ratings of over 80% (6 out of 7) during the course of 3 months when using devices similar to those used in the current study.

Network reliability for a continuous monitoring system was determined using the number of “pings” (messages saying “I am here”) that reached the remote server. The router sent a “ping” to the server every 3 min, so each ping suggests uptime, whereas missing network pings indicate that the network was down. Ideally, the wireless connection would remain active and transmit data from the wearable monitor 24 h/day for 6 months.

Results

Sample Description

Recruitment fell short of the goal of 60 participants, and the post hoc achieved power to detect change at the 0.05 level with 41 participants is 0.70. Demographic information for the 41 study participants is presented in Table 1, and the result of public advertising is that many of the participants were likely in an early stage of HF as very few were short of breath at low levels of exertion. Chi-squared tests were conducted for the categorical variables of gender, racial group, marital status, education, income, and presence of edema between the HF and non-HF groups. Groups only differed by the prevalence of edema [χ² (1, n = 41) = 6.13, p = 0.006]. Independent-samples t tests were conducted on the variables of age, cognition (Short Portable Memory Status Questionnaire, Wechsler Memory Scale-III), technology experience (TEQ), body mass index, shortness of breath, and pain. No group differences were found, and these variables were excluded from subsequent analyses.

Table 1.

Demographic, Health, and Cognition Variables by Group

	HEART FAILURE	NON–HEART FAILURE	TOTAL
Number of subjects	21	20	41
Mean (SD) age (years)	71.3 (11.7)	72.2 (4.3)	71.8 (8.8)
Female (%)	57.1	50.0	53.7
White (%)	61.9	85.0	73.2
African American (%)	38.1	15.0	26.8
Education (n)
Less than high school	1	1	2
High school graduate	5	2	7
Vocational school	1	2	3
Some college/associate's degree	6	5	11
Bachelor's degree	3	2	5
Master's degree	3	6	9
Doctoral degree	2	2	4
Marital status (n)
Single	2	0	2
Married	12	13	25
Widowed	6	5	11
Divorced	1	1	2
Separated	0	1	1
Income yearly (n)
≤$15,000	3	1	4
$15,001–49,999	10	12	22
$50,000–99,999	5	4	9
≥100,000	2	3	5
Did not answer	1	0	1
Cognition [mean (SD)]
SPMSQ-Errors	0.65 (0.67)	0.60 (0.68)	0.63 (0.67)
WMS-Recall	10.63 (4.00)	10.05 (4.51)	10.33 (4.23)
Health indicators
BMI (kg/m²) [mean (SD)]	29.33 (5.08)	27.50 (5.21)	28.37 (5.17)
Pain (0–5) self-report [mean (SD)]	1.38 (0.92)	1.45 (1.15)	1.41 (1.02)
Pain (1–4) nurse OBS [mean (SD)]	1.43 (0.93)	1.50 (1.00)	1.46 (0.95)
Edema (nurse OBS) (n)
Yes	9	3	12
No	12	17	29
SOB (nurse OBS) (n)
No SOB	11	17	28
SOB with heavy exertion	7	2	9
SOB with moderate exertion	2	0	2
SOB at rest	1	1	2
Technology experience [mean (SD)]^a	118.80 (31.64)	130.00 (25.38)	124.40 (28.87)

On a scale of 47 (no experience) to 210 (Frequent experience).

BMI, body mass index; OBS, observation; SD, standard deviation; SOB, shortness of breath; SPMSQ, Short Portable Mental Status Questionnaire; WMS, Wechsler Memory Scale.

Adherence

The watch, scale, BP cuff, and daily survey adherence results are summarized in Table 2 (excluding times when participants were away for vacation or in the hospital or when the system had lost Internet connection). Overall 71% of requested daily surveys were taken. Overall adherence rates for BP and weight were 75% and 82%, respectively, and participants wore the watch monitor 77% of the overall time requested (24/7, for 6 months each).

Table 2.

Summary of Participant Adherence and Network Performance by Group

	HEART FAILURE	NON–HEART FAILURE	TOTAL
Number of days of data	3,552	3,331	6,883
Time network was up (%)	94	92	93
Watch worn 24/7 (%)	73	81	77
Daily blood pressure taken (%)	72	78	75
Daily weight taken (%)	77	86	82
Daily survey taken (%)	64	78	71

A multivariate analysis of variance test was conducted to test for group differences on adherence between the HF and non-HF groups on the four adherence measures. Presence of edema was included as a covariate in the model. There were statistically significant differences on the combined dependent measures for edema (F _{4, 35} = 3.03, p = 0.030, Wilks' lambda = 0.74, η² = 0.257) and HF (F _{4, 35} = 3.06, p = 0.029, Wilks' lambda = 0.74, η² = 0.259). The follow-up tests for each dependent variable showed that edema did not significantly affect any of the dependent variables for adherence. Using a Bonferroni adjusted alpha level of 0.02, the only differences between the HF and non-HF groups that approached significance were percentage of time the watch was worn (F _{1, 38} = 4.38, p = 0.043, η² = 0.103) and daily surveys taken (F _{1, 38} = 3.57, p = 0.066, η² = 0.086), with the HF group trending lower on both measures (Table 3).

Table 3.

Summary of System Usability Scores by Group

	HEART FAILURE		NON–HEART FAILURE
TIME	MEAN	SD	MEAN	SD
Watch device
1	25.47	4.86	26.59	4.12
2	29.82	3.96	27.75	7.01
Blood pressure cuff
1	24.94	5.31	27.56	4.93
2	29.18	4.67	26.88	5.50
Weight scale
1	27.00	5.28	28.69	5.00
2	29.82	3.96	27.75	7.01
Tablet device
1	25.47	4.99	27.62	5.14
2	28.65	4.89	25.50	5.83

SD, standard deviation.

Usability

The usability scale, given as a paper measure, had some data loss. The target goal on the SUS was 80% (a value of 25.6); the descriptive statistics are shown in Table 3. A repeated-measures analysis of variance was conducted to test differences in usability over time for all devices and for differences between HF and non-HF groups with presence of edema included as a covariate. The three-way interactions were not significant. The interaction effect for usability ratings over time (F _{1, 31} = 4.91, p = 0.034, Wilks' lambda = 0.863, η² = 0.137) and HF was significant, with the HF group showing higher ratings. The interaction between usability ratings and device over time (F _{3, 29} = 4.68, p = 0.009, Wilks' lambda = 0.67, η² = 0.326) was also significant (Fig. 3).

Fig. 3.

Usability ratings by group, time, and device. BP, blood pressure; HF, heart failure; SUS, System Usability Scale; Wt.scale, weight scale. Bars = standard error.

Follow-up tests were conducted on the usability ratings for each of the devices. Using a Bonferroni corrected alpha level of 0.02, the weight scale had significantly higher ratings over the other devices (Table 4 and Fig. 3).

Table 4.

Comparison of System Usability Scores by Device

				95% CONFIDENCE INTERVAL
DEVICE 1, DEVICE 2	MEAN DIFFERENCE (DEVICE 1 AND 2)	STANDARD ERROR	P VALUE	LOWER BOUND	UPPER BOUND
Watch
BP cuff	0.24	0.37	0.534	−0.53	0.99
Weight scale	−0.97	0.25	<0.001^a	−1.48	−0.47
Tablet	0.53	0.49	0.291	−0.48	1.54
BP cuff
Watch	−0.24	0.37	0.534	−0.99	0.53
Weight scale	−1.21	0.35	0.002^a	−1.92	−0.49
Tablet	0.30	0.61	0.634	−0.95	1.54
Weight scale
Watch	0.97	0.25	<0.001^a	0.47	1.48
BP cuff	1.21	0.35	0.002^a	0.49	1.92
Tablet	1.50	0.48	0.004^a	0.52	2.48
Tablet
Watch	−0.53	0.49	0.291	−1.54	0.48
BP cuff	−0.29	0.61	0.634	−1.54	0.95
Weight scale	−1.50	0.48	0.004^a	−2.48	−0.52

p < 0.02 indicates a significant difference.

BP, blood pressure.

Reliability

The wireless cell-based network connectivity was up 93% of the time (Table 2), and a t test (t ₃₉ = 0.70, p = 0.489) showed that network reliability was equivalent across groups. There was a higher rate of network downtime, as well as data loss, due to issues with Wi-Fi hotspot stability (using a new fourth generation card); these issues were corrected through upgrades, such as internal data storage with feed forward capabilities, leading to improved connectivity and data collection without changing what the participants experienced while using the system.

Discussion

This study has provided insight into the attitudes and experience of older adults regarding remote health monitoring and provides data on actual usage over an extended period of time by those individuals. Participants used the scale and BP cuff daily 75% and 82% of the time, respectively. Adherence rates for wearing the watch 24/7 over 6 months was 77% and for taking daily surveys was 71%. The difference in survey completion rate is most likely due to survey length, which was longer for HF patients, who had additional, HF-specific questions. The adherence for the watch device is promising as it was a prototype model, causing some initial concern that participants may perceive it as an obvious medical assistive device. Regarding system usability, an 80% (25.6) score or higher was achieved for many of the devices as well as the overall system. Consistent with prior work,¹⁰ usability ratings generally improved over time, with a statistically significant finding for the weight scale. HF participants' usability ratings for all devices increased over time, whereas those from non-HF participants were stable. The device that was rated as the easiest to use was the most frequently used device over the course of 6 months, suggesting that attitudinal and cognitive barriers that older adults sometimes have towards technology can be successfully addressed in an effort to increase use and thereby patient engagement/adherence.²²

Prior to the study, individuals with HF were thought to be more inclined to use the system given their clear medical reason to do so. Table 2, however, shows that adherence rates trended higher for non-HF participants despite higher usability ratings from HF participants. These results are consistent with those of Katz et al.,²³ who found that knowledge of symptom management may not be enough of a motivator for adherence. For example, participants had the highest adherence with the weight scale but were less likely to wear the watch monitor, which was the only device capable of capturing real-time information and only had to be put on in order to work. One possibility is that because HF participants have been widely instructed by their providers to monitor weights for increases that can indicate fluid build-up, they were accustomed to regular weighings. Results from non-HF participants, however, show the same trend; it is possible that high adherence with the scale is a habit related to Americans' interests with weight and health. Future research may find that key factors for adherence could be medical trends or emphasis that comes from a healthcare provider.

Analysis placed overall system reliability at 93%. The most common reason for data loss was the Wi-Fi hotspot cellular connectivity device. In many cases, the connection did not reset after a brief power outage or after automatic firmware updates that required the device to be manually restarted. Efforts to walk older adults through the process remotely were often unsuccessful as the power button was small and flush-mounted on the edge of the device, making it difficult for older adults to find. It was discovered that a higher-cost Wi-Fi hotspot device had fewer problems. Engineers from a major carrier were consulted about the issues with the less expensive devices, with no cause for the issues found. The cellular network was reliable and did not fail during the study. These findings suggest that the wireless connection device was the weak link. The temporary nature of the study suggested that the Wi-Fi hotspot would be the most cost-effective and lowest support option available. It now appears that careful evaluation and selection of all available options for wired or wireless access to the Internet should be undertaken in consultation with equipment and service providers when assessing access to meet future research or operational medical monitoring requirements.

The presence of usual care, no-contact, control groups may have improved the study design because nurse visits and participant payments could affect overall adherence and usability. Also, because the majority of participants were recruited from advertisements, the diagnosis of HF relied on the clinical interview instead of an ejection fraction. Edema and shortness of breath were overall mild among the participants; it is possible that usability ratings would be different with moderate or severe symptoms.

In summary, this study shows that chronically ill older adults can use home monitoring systems designed for their use for an extended period of time. The results inform the design community of the potential for and pitfalls of the use of wireless technology that does not require a home Internet connection and highlights the need for usability testing as the device perceived as the easiest to use was indeed used the most. Encouraging results show that a well-designed system can be installed quickly and that older adults with support and training can show relatively high and stable adherence rates while coping with a chronic health condition. Overall, the majority of components performed well during extended use. These findings show the potential such systems have to provide support to older adults in their own homes in partnership with healthcare professionals monitoring the system's data analysis and trends that require attention. Although the contribution that remote monitoring systems have on reducing healthcare costs and outcomes remains a matter of debate, the foundation for any potential solution exists in replication across studies and in consistent implementation of reliable and usable remote monitoring systems; that goal appears to be close.

Footnotes

Acknowledgments

The authors thank Joshua Russell and Ronald Andringa who worked to coordinate and implement the program protocols and provided technical support for study participants. The authors also thank Drs. Ken Brummel-Smith and Alice Pomidor for their advice and assistance with the study protocol and for serving as the project's medical directors, as well as Bruce Wilson for his assistance and support with the home monitoring system. Finally, the authors thank Dr. Roxanne Hauber for her invaluable help with recruitment and training of the study nurses. This research was supported by a grant from the National Institute on Aging, “Non-intrusive Automated Portable Collection Systems for Aging Surveys II,” SBIR Phase II grant 2R44AG02196-02.

Disclosure Statement

At the time of the study, A.P., C.T.S., and C.C. had a financial interest in AFrame Digital, Inc., the company that developed the wrist-based, real-time monitoring system. J.E., N.C., W.R.B., L.S.-F., M.M., and C.B.E. declare no competing financial interests exist.

References

Asch

, Muller

, Volpp

. Automated hovering in health care—Watching over the 5000 hours. N Engl J Med, 2012; 36:1–3.

Omboni

, Gazzola

, Carabelli

, Parati

. Clinical usefulness and cost effectiveness of home blood pressure telemonitoring: Meta-analysis of randomized controlled studies. J Hypertens, 2013; 31:455–468.

Chaudhry

, Mattera

, Curtis

, Spertus

, Herrin

, Lin

. Telemonitoring in patients with heart failure. N Engl J Med, 2010; 363:2301–2309.

Koehler

, Winkler

, Schieber

, Sechtem

, Stangl

, Bohm

. Impact of remote telemedical management on mortality and hospitalizations in ambulatory patients with chronic heart failure: The telemedical interventional monitoring in heart failure study. Circulation, 2011; 123:1873–1880.

Corotto

, McCarey

, Adams

, Khanzie

, Whellan

. Heart failure patient adherence: Epidemiology, cause, and treatment. Heart Fail Clin, 2013; 9:49–58.

Kerby

, Asche

, Maciosek

, O'Connor

, Sperl-Hillen

, Margolis

. Adherence to blood pressure telemonitoring in a cluster-randomized clinical trial. J Clin Hypertens, 2012; 14:668–674.

Claes

, Devriendt

, Turnoy

, Milisen

. Attitudes and perceptions of adults 60 years and older towards in-home monitoring of the activities of daily living with contactless sensors: An explorative study. Int J Nurs Stud, 2015; 52:134–148.

Peek

STM

, Wouters

EJM

, van Hoof

, Luijkx

, Boeije

, Vrijhoef

HJM

. Factors influencing acceptance of technology for aging in place: A systematic review. Int J Med Inform, 2014; 83:235–248.

Giger

, Pope

, Vogt

, Gutierrez

, Newland

, Lemke

, Lawler

. Remote patient monitoring acceptance trends among older adults residing in a frontier state. Comput Hum Behav, 2015; 44:174–182.

10.

Hawley-Hague

, Boulton

, Hall

, Todd

. Older adults' perceptions of technologies aimed at falls prevention, detection or monitoring: A systematic review. Int J Med Inform, 2014; 83:416–426.

11.

Charness

, Fox

, Papadopoulos

, Crump

. Metrics for assessing the reliability of a telemedicine remote monitoring system. Telemed J E Health, 2013; 19:487–492.

12.

Goldstein

, Gathright

, Dolansky

, Gunstad

, Sterns

, Redle

, Josephson

, Hughes

. Randomized controlled feasibility trial of two telemedicine medication reminder systems for older adults with heart failure. J Telemed Telecare, 2014; 20:293–299.

13.

Gerteis

, Izrael

, Deitz

, LeRoy

, Ricciardi

, Miller

, Basu

. Multiple chronic conditions chartbook. AHRQ Publications Number Q14-0038. Rockville, MD: Agency for Healthcare Research and Quality, 2014.

14.

Roger

. The epidemiology of heart failure. Circ Res, 2013; 113:646–659.

15.

Evans

, Papadopoulos

, Charness

, Tsien Silvers

, Boot

, Schlachta-Fairchild

. Patient adherence and usability of a telehealth system for real-time automated hovering in an older adult sample [abstract]. Telemed J E Health, 2014; 20(5):A135–A136.

16.

Evans

, Tsein Silvers

, Papadopoulos

, Charness

, Boot

, Schlachta-Fairchild

, et al. Reliability, usability, and patient adherence with a real-time automated hovering telehealth system in older adults with chronic health conditions [abstract]. Telemed J E Health, 2015; 21(5):A126–A127.

17.

Pfeiffer

. A short portable mental status questionnaire for the assessment of organic brain deficit in elderly patients. J Am Geriatr Soc, 1975; 23:433–441.

18.

Wechsler

. Wechsler Adult Intelligence Scale III, 5th ed. San Antonio, TX: The Psychological Corporation, 1997.

19.

Brooke

. SUS—A “quick and dirty” usability scale. In: Jordan

, ed. Usability evaluation in industry. London: Taylor & Francis, 1996:189–194.

20.

Czaja

, Charness

, Dijkstra

, Fisk

, Rogers

, Sharit

. Computer and technology experience questionnaire. CREATE Technical Report CREATE-2006-03. 2006. Available at http://create-center.gatech.edu/publications_db/report%203%20ver1.3.pdf (last accessed May 18, 2015 ).

21.

Osterberg

, Blaschke

. Adherence to medication. N Engl J Med, 2005; 353:487–497.

22.

Charness

, Boot

. Aging and information technology use: Potential and barriers. Curr Dir Psychol Sci, 2009; 18:253–258.

23.

Katz

, Ashmore

, Barboa

, Trueblood

, McLaughlin

, Mathews

. Knowledge of disease and dietary compliance in patients with end-stage renal disease. Psychol Rep, 1998; 82:331–336.