Effects of Electronic Health Information Technology Implementation on Nursing Home Resident Outcomes

Abstract

Objective: To examine the effects of electronic health information technology (HIT) on nursing home residents. Methods: The study evaluated the impact of implementing a comprehensive HIT system on resident clinical, functional, and quality of care outcome indicators as well as measures of resident awareness of and satisfaction with the technology. The study used a prospective, quasi-experimental design, directly assessing 761 nursing home residents in 10 urban and suburban nursing homes in the greater New York City area. Results: No statistically significant impact of the introduction of HIT on residents was found on any outcomes, with the exception of a significant negative effect on behavioral symptoms. Residents’ subjective assessment of the HIT intervention were generally positive. Discussion: The absence of effects on most indicators is encouraging for the future development of HIT in nursing homes. The single negative finding suggests that further investigation is needed on possible impact on resident behavior.

Keywords

long-term care technology quality of care

Introduction and Background

The past decade has seen dramatically increased interest in and promotion of electronic health information technology (HIT). Such technology is viewed as a solution to persistent problems in the quality of care across health settings, increasing efficiency while offering significant potential for cost savings (Committee on Quality of Health Care in America, Institute of Medicine, 2001; Hillestad et al., 2005; Poon et al., 2006). Growing numbers of health care providers have implemented at least some components of HIT. The evidence to support the high hopes for HIT has been uneven, however, with studies showing mixed results across settings and outcome measures (Garg et al., 2005; Harrison, Koppel, & Bar-Lev, 2007; Linder, Ma, & Bates, Middleton, & Stafford, 2007; Sidorov, 2006; Walker et al., 2008).

Long-term care facilities have lagged behind other providers in comprehensive adoption of HIT (Brandeis, Hogan, Murphy, & Murray, 2007), and existing technology may be underutilized (Liu & Castle, 2008). Furthermore, there is a dearth of empirical research regarding the impact of HIT in nursing homes (Brandeis et al., 2007; Subramanian et al., 2007). Our review uncovered no studies that assessed the impact of HIT on resident outcomes. Despite the lack of scientific evidence, representatives of the long-term care industry and HIT vendors continue to assert the many benefits of HIT in long-term care (Lourde, 2009), and widespread adoption is likely to be inevitable over time. Given the mixed evidence in other settings, as well as the highly vulnerable nature of nursing home residents, assessing the impact of HIT on resident outcomes should be a major priority for researchers.

It is in particular of critical importance to determine whether unintended negative consequences for nursing home residents result from HIT introduction. Harrison et al. (2007) found “disturbingly mixed reports” on HIT outcomes in health care settings, with research showing unanticipated negative consequences of implementation, some of which resulted in actual harm (p. 542). Several of the negative consequences documented by Harrison and colleagues could potentially affect nursing home residents, including changing or disrupting oral communication among clinicians or with patients; causing cognitive overload for providers by emphasizing “overcomplete” information entry; and the inflexibility of electronic records, causing lost detail about resident conditions. Most problematic for the long-term care context, given the importance of person-centered care (Tellis-Nayak, 2007), is evidence of changes in the provider–patient relationship, as professionals become more occupied with the computer and less oriented toward the patient (Ludwick & Doucette, 2009). Thus, it is possible that HIT could lead to less personalized and more routinized care, as well as less direct observation and interaction with residents, and in turn to negative clinical outcomes such as increased falls, diminished function, and dissatisfaction with care.

Because of the possibility of unanticipated negative clinical outcomes as well as the rapid expansion of this technology in nursing homes, it is necessary to begin efforts to evaluate the impact of HIT implementation. The purpose of the present study was to examine outcomes among nursing home residents, using a prospective, quasi-experimental design that derived information from multiple sources. To determine the impact of HIT on measurable resident outcomes, we used two approaches. First, we examined subjective resident responses to the introduction of HIT in the nursing home, including the degree to which residents were aware of the change and the perceived impact on their care. Second, we compared intervention and comparison groups on strategically selected clinical and quality of life outcome variables, measured at two time-points approximately 9 months apart. The study represents to our knowledge the first evaluation of HIT in long-term care using direct research assessments of residents.

Method

Setting and Facility Selection

In 2006, the New York State Legislature approved funding to support a demonstration project of the adoption and implementation of HIT in a group of nursing homes in the New York City metropolitan area. Because this demonstration project resulted from a collective bargaining agreement between the employee union and a consortium of nursing home operators, the potential facilities were restricted to the 120 unionized, for-profit facilities participating in the bargaining agreement. A “Quality Care Oversight Committee” (QCOC) was formed with provider and union representation to oversee the implementation and ongoing management of the project. Twenty facilities were selected to receive HIT based on several criteria, including ensuring diversity of location, readiness of the facility to adopt the technology, and commitment to paying for the technology after the demonstration concluded. The first facility began implementation in spring 2007, and implementation in all facilities was complete by early 2008.

The investigators conducted an entirely independent evaluation of the impact of HIT on residents. Five of the 20 facilities receiving HIT were selected as intervention sites because resources available to the study did not allow for the very intensive resident assessments to take place in all 20 facilities. In addition to the intervention facilities, five comparison facilities were purposively selected from the remaining 100 facilities of the original pool of 120 so as to maximize comparability with the intervention facilities. Comparability was based on the following criteria: size, location in the same area (e.g., borough), number of licensed nursing staff hours per resident day, and number of survey deficiencies. In addition to similarity in location and for-profit status, comparison and intervention facilities were generally well matched, with similar bed sizes (M = 246.17 for the comparison and M = 238.00 for the intervention group), number of deficiencies (M = 5.50 for the comparison and M = 4.20 for the intervention group), and nursing hours (M = 71.00 minutes for the comparison and M = 63.75 minutes for the intervention group).

Resident Selection and Recruitment

Figure 1 shows the design and participant flow during the study. A total of 761 residents were assessed at Time 1, prior to the implementation of HIT in the intervention facilities. At Time 2, 482 residents were assessed. The primary reasons that residents were not assessed at Time 2 were death of the resident or progression of dementia to the stage that the resident could not participate. A number of residents were discharged from the facilities, and their ultimate status (including whether or not still alive) is unknown. Very few residents refused the interview at either time point. The primary analysis was conducted using an intent-to-treat approach, with inclusion of all residents with Time 1 data. There were no significant differences in demographic characteristics such as gender and education between completers and noncompleters. However, as would be expected, completers were somewhat younger than noncompleters (78.4 years vs. 81.1 years; p = .001). At baseline, residents were blind to their own treatment assignment (and to the actual existence of a treatment and evaluation). Baseline data were completed approximately 1 month prior to implementation of the intervention in each facility and 9 months later (and at similar points for the comparison facilities).

Figure 1.

Study enrollment

Intervention

In the HIT intervention facilities, a formal “kickoff” and training period preceded the introduction of HIT. The HIT intervention was comprehensive, converting most facility records to electronic records. The system allowed for scheduling and mobile capture of assessments, interventions, and treatments, as well as online entry of progress notes by discipline. It further allowed for real-time reporting of sentinel events, quality indicators, and quality measures. Secure, wireless, and “ruggedized” PDAs for use by CNAs and nurses allowed access to resident records at any location on the job and enabled staff to enter orders or chart information at bedside or anywhere in the facility. Desktop personal computers were configured on every nursing station and within every department for clinical management. The system also included computerized physician order entry, allowing physicians to securely approve orders and access medical records remotely. Conversion of the facility from paper to electronic records took on average 3 months, and all systems were fully operational during the evaluation period.

Outcome Variables

This evaluation focuses on the impact of HIT introduction on resident outcomes. It is also likely that HIT implementation affects process quality; that is, the quality of the care performed by staff. Changes in care processes could in turn affect resident outcomes. Thus, the ideal model for a study such as this implies a comprehensive analysis, with examination of both direct and indirect effects. However, the resources available for this study did not permit both an evaluation of resident outcomes involving in-depth personal assessments and an evaluation of changes in care processes. We determined that assessing outcome quality was the most pressing task given the current state of knowledge. Because of the vulnerability of nursing home residents, it is possible that any major change in resident care could have negative impacts on quality, and examining resident outcomes in this initial study appears the most critical priority. Therefore, only an analysis of direct effects is undertaken here.

To assess the potential impact of the introduction of HIT in nursing homes, we examined a number of prespecified resident outcomes. It is important to note that guidance from prior research as to the selection of outcome variables was not available, given that longitudinal assessment of HIT implementation in nursing homes has not been conducted previously. Furthermore, the research on acute and ambulatory care settings has focused on a narrow range of process outcomes, such as medication errors, infection due to medical care, and adherence to care guidelines for specific disease conditions (Kazley & Orzcan, 2008). Although such outcomes are appropriate in those settings where contact with patients is short term, the nursing home is a long-term residential environment in which more global quality of care and satisfaction outcomes are particularly salient (Kane, 2003). For the purposes of this study, we included two categories of outcomes to assess the impact of HIT on residents.

Resident Satisfaction

A goal of the study was to determine effects on residents’ subjective assessments of quality of life, including whether residents reported any positive or negative aspects of the introduction of HIT in their facility. Given that studies cited earlier have found potential perceived disruption in the patient–provider relationship in acute care settings, we wished to examine whether any such effects occurred in the nursing home setting. In so doing, we respond to Kane’s (2003) call for inclusion of the “resident voice” in examining factors that may affect quality of life in nursing homes. At the end of the second wave of data collection, residents in the treatment group were asked an initial question regarding whether they were aware of the transition to HIT in the nursing home. They were shown a photograph of the handheld device used in the facility, and asked: “Have you noticed the nurses and CNAs using any handheld computerized devices, like the one pictured here?”

In addition to the direct inquiry about reactions to the technology, we also examined treatment and control differences in two measures of subjective satisfaction with nursing home care. It is possible that the introduction of the new technology could affect residents’ assessments of the care provided to themselves as well as their overall level of satisfaction with care in the unit on which they reside. Two established scales were employed that measure these dimensions separately.

Satisfaction with own care

The interview included a scale about the resident’s satisfaction with his or her own care. Sample questions include “Do you feel comfortable with the nursing assistants who take care of you?”; “Are you satisfied with the care you get?”; “In the past week, how worried have you been about getting help when you need it?” The Cronbach’s alpha was .78 at baseline and .81 at follow-up for this sample.

Satisfaction with unit environment

The interview also contained a measure of the resident’s overall satisfaction with the immediate living environment. The Satisfaction with Unit Environment scale examined residents’ assessments of problems with roommates, next-door neighbors, and other unit mates. These questions sought direct information about the impact of living in close proximity to individuals who may be cognitively impaired or behaviorally disturbed. Questions dealt with such issues as lack of privacy, interference with sleep, mood of other residents, and noise and interruptions by other residents (Teresi, Holmes, & Monaco, 1993). We included this scale because, in addition to satisfaction with the care one receives, it is possible that perceived noise and nuisance in the unit as a whole might be adversely affected by the new technology. Specifically, the mechanism for changes in this scale could be decreases in staff time or attentiveness due to the HIT, which might affect not only the resident’s satisfaction with his or her own care but also more generally with the unit environment. The Cronbach’s alpha was .82 at baseline and .86 at follow-up for this sample.

Quality Outcome Measures

We selected several outcome measures that have been widely used to assess changes in quality of care. We focused on generally accepted quality outcomes that could be reliably measured and that could plausibly be expected to change over a 9-month study period. In addition, we sought measures that have been used in previous research on the impact of staffing (both availability and behaviors) on quality outcomes, given that changes in staff care processes may be affected by HIT introduction (although not specifically assessed in the present study). We ruled out some commonly used indicators (such as pressure ulcers or hospitalizations) due to the 9-month time frame, given that incidence rates would be low (Wiener, 2003).

Four outcomes have been frequently used in the United States and internationally in studies of processes of care (including the availability and quality of nursing care): ADL function, falls, resident mood, and behavioral symptoms (Arling, Kane, Mueller, Bershadsky, & Degenholtz, 2007; Du Moulin, van Haastregt, & Hamers, 2010; Grabowski, Aschbrenner, Rome, & Bartels, 2010; Li, Cai, Mukamel, & Glance, 2010; Nakrem, Guttormsen Vinsnes, Harkless, Paulsen, & Seim, 2009; Rubenstein, Powers, & MacLean, 2001). In addition, mortality is regularly used as a quality indicator (Castle, 2008). These five outcomes can be reliably measured and indicate clinically undesirable outcomes that could be associated with HIT introduction.

Assessment of Outcomes

To assess the impact of HIT implementation, it is important to note that data routinely collected as part of the Minimum Data Set (MDS) were not used in this study because they were supplied by staff, the target of the intervention. A primary component of the HIT intervention itself was systematically changing the way that MDS data are collected; that is, a shift from pen-and-paper recording to real-time electronic recording. Thus, the intervention itself could create large, systematic changes in MDS reporting between Time 1 and Time 2 that could not be predicted or accounted for in the study.

For these reasons, the data collection process involved directly assessing resident outcomes primarily through direct interview and/or observation. Most outcomes were assessed using the INCARE (Institutional Comprehensive Assessment and Referral Evaluation), a version of the CARE originally developed for use with community residents (Golden, Teresi, & Gurland, 1984; Gurland & Wilder, 1984), which is a multilevel–multisource instrument that allows at least some assessment to be completed across all levels of residents. In addition, an extended interview was conducted with residents capable of taking part in it. Information was also obtained independently from rater observation and chart review. The measurement of specific outcome variables was as follows.

Falls

There were data on number of falls for each month of the study. We constructed variables for average number of falls per month for the 3 months before baseline and for the period over which the intervention took place. For those residents who died or left the study for other reasons, the date of departure was used to determine the number of months over which the average was computed.

Mortality

There were data on mortality for all residents over the study period.

Functional Status

Functional status was measured with the Performance Activities of Daily Living (PADL) scale, a 27-item scale that measures an individual’s inability to perform activities of daily living independently (Kuriansky & Gurland, 1976). Inability to perform various upper and lower body movement tasks associated with eating, dressing, and grooming, such as putting on a sweater, buttoning and unbuttoning a sweater, guiding a spoon to the mouth, and combing hair was assessed. Performance times were recorded, and items are rated as to whether the task was performed with and without cueing, or could not be performed at all. The Cronbach’s alpha measuring internal consistency was .90 at baseline and .90 at follow-up for this sample.

Behavioral Symptoms

Rater observation was employed, in which a trained research assistant collects a rating of behavior obtained through observations. Each individual was observed for 5 minutes, on a total of four occasions at each time point, using a 14-item observational measure of affect and a 37-item measure of behavior. The observations were collected at different times for each resident: at the interview (which could vary throughout the day), and in morning, midday, and late afternoon. Observations were made in a wide variety of locations, including in resident rooms, hallways, day room, and dining area. Similarly, observations often included situations where the resident was interacting with others, receiving care, or in activities.

Frequency of behavioral states were coded as follows: “occurs not at all”; “occurs with very little frequency (once or twice during the observation period)”; “occurs with some frequency (several times)”; “occurs with moderate frequency (many times, but not continuous)”; “occurs with great frequency (almost continuously).” In the Behavior Observation Checklist measure, typical items include “disruptive of others,” “repetitive questioning,” “wandering,” “argumentative,” “asking for help,” “noisy,” “uncooperative,” and “picks/pulls clothing.” Interrater reliability, estimated using the intraclass correlation coefficient, ranged from .80 to .95 across samples.

Resident Mood

Residents were assessed using the Feeling Tone Questionnaire (FTQ; Toner & Teresi, 1990; Toner, Teresi, Gurland, & Tirumalasetti, 1999). The psychometric properties of this measure have been examined in large-scale studies of depression in residents of nursing homes (Teresi, Abrams, & Holmes, 2000; Teresi, Abrams, Holmes, Ramirez, & Eimicke, 2001). The FTQ contains 16 questions asked directly of the resident. Typical items are “Are you feeling well?”; “Are you feeling happy today?”; “Do you feel lonely?”; “Do you have a good appetite?”; “Do you sleep well?” Each item is coded “yes,” “no,” or “equivocal (sometimes, it depends).” Each verbal response made by the resident is also rated by the interviewer for affect, using a 5-point continuum from 1 (laughs, praises, enthusiastic, emphatically positive) to 5 (extreme negative—cries, groans, curses, is negative). As has been commonly done in previous studies, we used the total scale that combines the responses and affect ratings. The Cronbach’s alpha for the FTQ for this sample was .79 at baseline and .81 at follow-up. In addition to the FTQ, the Comprehensive Assessment and Referral Evaluation Depression scale (Golden et al., 1984) was administered to residents capable of responding as part of the assessment.

We also collected data to be used as covariates in statistical models, including the resident’s gender, age, and cognitive status. Cognitive status was measured by the Comprehensive Assessment and Referral Evaluation (CARE) Diagnostic Scale developed by Gurland and colleagues (Gurland et al., 1977). The Cronbach’s alpha was .79 at baseline and .80 at follow-up for this sample.

Statistical Models and Methods

The primary purpose of the analysis was the evaluation of the HIT intervention by examining treatment differences between intervention and comparison groups for the outcomes described previously. The outcome variables were measured at baseline prior to the intervention and follow-up 9 months later. The intervention took place over the 9-month interval between assessments. A 2 × 2 repeated measures design (Treatment × Time) forms the core of the statistical models for evaluation of the intervention.

Analyses were carried out using general linear mixed models, with treatment and time included as levels of fixed classification factors and individuals included as levels of a random factor. Facilities were included in the model as levels of an additional random classification factor, taking into account variance associated with facilities and that residents are grouped by facilities. This allows inferences to the population of facilities from which we sampled although the number of facilities in the study is small and estimates of variances are not precise. The model also included the Treatment × Time interaction and sex, age, and cognitive status of the resident. The cognitive variable was treated as a time-varying covariate. The key test in the examination of intervention effects is the test of the interaction of the factors for treatment and time. Three covariates were included in the models: age, gender, and cognition.

The falls variable (average number of falls per month) was analyzed in the same repeated-measures model as described for the INCARE variables (the repeated measures classification factor is the 3 months prior to baseline vs. the 9 months following). The mortality data (whether the resident died or did not) were analyzed in a logistic-linear model with binomial error with the same model factors as for the preceding models except omitting time and the Treatment × Time interaction, which are not relevant to this model because death is based on status at 9 months after baseline.

Tables 2 and 3 show separate analyses for seven outcome variables (mortality, N = 761, is not shown in a table), with each outcome variable presented in two sections, the first with three rows and the second with six rows (excluding headings). The first section shows least squares means for the treatment-by-time interaction, contrasts on these LS means, and in parentheses probabilities for the tests of these contrasts. The lower-right cell for each variable gives the probability for the test of the Treatment × Time interaction, which is the test of treatment effect. This section is the focus of interest in the evaluation. The second section shows the lines of the analysis for the fixed effects, with estimates, standard errors, and ps. Each effect and the least squares means are adjusted for all other variables in the model.

Results

Participants

The treatment group included 428 residents and the comparison group 333 residents (sample sizes in the tables vary due to missing data). A description of both groups is provided in Table 1, including demographic and outcome variables. Despite the lack of random assignment, the groups were substantially equivalent on all baseline characteristics, except for race/ethnicity and the FTQ score (see Table 1).

Table 1.

Characteristics of Residents in the Study Arms at Time 1

	Control		Treatment
	N	%	N	%
Age M (SD)	333	79.6 (11.4)	428	79.2 (10.3)
Gender
Male	106	31.83	156	36.45
Race/ethnicity
Black	76	22.82	158	36.92***
Hispanic	28	8.41	65	15.19
White	209	62.76	196	45.79
Education (years) M (SD)	167	11.0 (3.5)	300	10.6 (3.8)
FTQ Total M (SD)	324	58.7 (6.4)	424	57.3 (7.4)**
Average number of falls M (SD)	333	0.09 (0.26)	428	0.06 (0.16)
Total satisfaction with long-term care living M (SD)	207	3.0 (3.5)	245	3.5 (3.9)
Satisfaction with care M (SD)	202	2.4 (2.5)	241	2.1 (2.3)
CARE Diagnostic Scale M (SD)	319	5.9 (3.0)	416	5.9 (3.0)
PADL Total Scale M (SD)	234	1.4 (3.5)	318	2.0 (4.0)
Observed Behavior Scale M (SD)	333	4.7 (2.8)	428	4.9 (2.5)

p < .05. **p < .01. ***p < .001.

Resident Satisfaction

As noted earlier, at Time 2 residents were shown a photograph of the handheld device used by staff and asked if they were aware of its use. A total of 124 residents (51%) answered affirmatively and were asked a series of questions about their opinion on the effects of the technology. The fact that only half of residents interviewed were even aware of a change in care is worth noting, suggesting that for many residents the impact of the technology was so minimal as to be unnoticed.

The response from residents who were aware of the change to HIT was generally positive. Nearly three quarters (70.8%) of these respondents agreed that “the handheld device helps staff to better manage my care.” A similar percentage (72.8%) reported that they are “pleased that staff use the handheld devices to better track and manage my care.” Over two thirds of residents (69.3%) reported that staff using the handhelds did not interfere with the time they spent with him or her. Finally, residents were asked to rate changes in their care since the introduction of HIT in the facility. The majority (62.2%) felt that the care had stayed the same, and 30.6% believed that it improved; only 7.1% felt that it had declined. Thus, from the resident perspective, use of the computerized technology does not appear overall to have led to resident dissatisfaction or poor communication.

In addition to directly asking about impact on care, we compared treatment–control differences between the pretest and posttest on the two scales measuring resident satisfaction. As shown in Table 2, no significant differences were found on either of these measures, again supporting the finding that the HIT implementation did not negatively affect residents’ subjective assessments of care.

Table 2.

Analysis of Satisfaction Variables

Outcome^a
	Satisfaction with own care scale (N = 436)			Satisfaction with long-term care scale (N = 445)
	Baseline (Time 1) least squares mean	Follow-up (Time 2) least squares mean	Time 2–Time 1 (probability)	Baseline (Time 1) least squares mean	Follow-up (Time 2) least squares mean	Time 2–Time 1 (probability)
Control	2.41	2.54	0.12 (.56)	3.06	2.39	−0.67 (.05)
Treatment	2.19	2.36	0.18 (.34)	3.41	3.37	−0.05 (.88)
Treatment-control (probability)	−0.23 (0.43)	−0.17 (0.59)	−0.06 (0.84)	0.35 (0.52)	0.97 (0.10)	−0.62 (0.18)
	Estimate	SE	p	Estimate	SE	p
Treatment	.172	.321	.593	−.972	.591	.100
Time	−.178	.186	.338	.048	.306	.876
Treatment × Time	.056	.280	.840	.621	.459	.177
Age	.001	.009	.883	−.045	.015	.002
Sex	.336	.212	.114	−.120	.328	.543
Cognitive	−.002	.036	.955	−.112	.057	.049

Note. The table shows separate analyses for 2 dependent variables. Analysis was by general linear mixed models. The analysis for each dependent variable is given in 2 sections, the first with 3 rows and the second with 6 rows (excluding headings). The first section shows least squares means for the treatment-by-time interaction, contrasts on these LS means, and in parentheses probabilities for the tests of these contrasts. The lower-right cell in each block gives the probability for the test of the Treatment × Time interaction, which is the test of treatment effect. This section is the focus of interest in the evaluation. The second section shows the lines of the analysis for the fixed effects, with estimates, standard errors, and ps. Each effect and the least squares means are adjusted for all other variables in the model.

The outcome variables are coded so that a higher score indicates a poorer outcome—that is, less satisfaction on the satisfaction scales.

Effect of HIT Intervention on Resident Outcomes

For all other outcome variables except one, there were no statistically significant effects of the HIT intervention (Table 3). That is, the changes over time between before and after the introduction of HIT did not differ between the treatment and control facilities. There was also no treatment effect for mortality (p = .94), with estimated control and treatment means of 0.081 and 0.078, respectively (not shown in table). A negative treatment effect was found on one outcome: the measure of observed behavior. Residents in the treatment facilities experienced an increase in observed disruptive behaviors, whereas a reduction over time in the control facilities was observed.

Table 3.

Analysis of Primary Clinical Outcome Variables

Outcome^a
	Average Number of Falls (N = 718)			Performance Activities of Daily Living (N = 544)			Feeling Tone Questionnaire (N = 717)			Observed Behavior Scale (N = 718)
	Baseline (Time 1) least squares mean	Follow-up (Time 2) least squares mean	Time 2– Time 1 (probability)	Baseline (Time 1) least squares mean	Follow-up (Time 2) least squares mean	Time 2– Time 1 (prob.)	Baseline (Time 1) least squares mean	Follow-up (Time 2) least squares mean	Time 2– Time 1 (probability)	Baseline (Time 1) least squares mean	Follow-up (Time 2) least squares mean	Time 2– Time 1 (probability)
Control	0.09	0.08	0.00 (.80)	1.39	1.50	0.11 (.76)	58.46	57.55	−0.90 (.09)	4.57	4.04	−0.52 (.01)
Treatment	0.06	0.06	0.00 (.98)	1.94	1.62	−0.32 (.33)	57.31	57.55	0.24 (.61)	4.92	5.13	0.21 (.19)
Treatment-control (probability)	−0.02 (0.29)	−0.02 (0.36)	0.00 (0.83)	0.55 (0.24)	0.12 (0.82)	0.43 (0.38)	−1.14 (0.29)	−0.01 (1.0)	−1.14 (0.10)	0.35 (0.51)	1.09 (0.05)	−0.73 (0.003)
	Estimate	SE	p	Estimate	SE	p	Estimate	SE	p	Estimate	SE	p
Treatment	.020	.022	.364	−.115	.515	.823	.006	1.143	.996	−1.086	.546	.047
Time	−.0003	.011	.978	.322	.330	.328	−.236	.457	.605	−.214	.162	.186
Treatment × Time	.003	.016	.834	−.437	.502	.384	1.137	.701	.105	.735	.248	.003
Age	.0005	.0006	.453	.010	.013	.440	−.041	.0233	.079	−.002	.007	.773
Sex	−.013	.013	.323	−.223	.279	.425	−.031	.501	.951	.410	.158	.009
Cognitive	.004	.002	.088	.298	.047	<.0001	.166	.078	.033	−.124	.025	<.0001

Note. The table shows separate analyses for 5 dependent variables. Analysis was by general linear mixed models. The analysis for each dependent variable is given in 2 sections, the first with 3 rows and the second with 6 rows (excluding headings). The first section shows least squares means for the treatment-by-time interaction, contrasts on these LS means, and in parentheses probabilities for the tests of these contrasts. The lower-right cell in each block gives the probability for the test of the Treatment × Time interaction, which is the test of treatment effect. This section is the focus of interest in the evaluation. The second section shows the lines of the analysis for the fixed effects, with estimates, standard errors, and ps. Each effect and the least squares means are adjusted for all other variables in the model.

All outcome variables are coded so that a higher score indicates a poorer outcome—greater impairment for: average number of falls, performance activities of daily living, feeling tone questionnaire, and observed behavior score.

Discussion

This study conducted personal assessments of nursing home residents in five treatment and five comparison facilities, using a range of measures with demonstrated reliability and validity. Time 1 assessments were conducted shortly before HIT was introduced, and Time 2 assessments were conducted approximately 9 months later. Changes that occurred in the treatment facilities were compared to those in the control facilities. With one exception, there was no statistically significant impact of the introduction of HIT on residents for a number of clinical and quality of life outcomes. For one variable, a statistically significant negative effect was found, with comparison facilities showing improvement on a measure of behavioral disturbances, whereas the treatment facilities showed no change. Without detailed observational data on the effect of HIT on staff behaviors, it is not possible to determine the mechanism for this finding. Further research is needed to determine whether there is a relationship between HIT and behavioral issues and what mechanisms underlying such a relationship might exist.

It is also important to note that this study found no measurable improvement in resident condition as a result of the HIT intervention. Therefore, claims that HIT in nursing homes will have direct benefits for residents should be tempered by the findings of this research. Consistent with a number of studies of HIT in acute and ambulatory care, there appears to be no demonstrable positive effect of the technology on residents. If this finding is supported by future research, studies of cost and efficiency will be of key importance to make the case for HIT in nursing homes.

The findings regarding resident response to the technology are encouraging. Residents who were sufficiently competent to answer direct questions about the technology generally expressed positive sentiments about it and were not disturbed or upset about its use. Moreover, we found no differences between the treatment and control groups over time in overall levels of satisfaction with the care they were receiving. However, it is worth considering that a minority of individuals in the treatment arm did express some dissatisfaction with the use of the handheld device (23.1% said the device interferes with time the staff spends with them). Future studies should explore in greater detail resident perceptions of HIT introduction.

The study has a number of limitations that point to directions for future research. First, in this project resource limitations made it impossible examine the impact on processes of providing care (which studies of acute and ambulatory care have emphasized). Moving from handwritten charting to electronic health records could lead, for example, to changes in coordination of care and thus affect common problems of nursing home residents such as falls or behavioral disturbances (Brandeis et al., 2007). Care transitions might be better managed with electronic access to information using HIT (Resnick, Manard, Stone, & Alwan, 2009). Furthermore, electronic reminder prompts may improve the responsiveness of provider behavior in nursing homes (Field et al., 2009; Linder et al., 2007). In contrast, HIT intervention might negatively affect time spent on paperwork and documentation; several studies from non-long-term care settings have found mixed results, with some studies actually showing increased documentation time (Overhage, Perkins, Tierney, & McDonald, 2001; Poissant, Pereira, Tamblyn, & Kawasumi, 2005; Tierney, Miller, Overhage, & McDonald, 1993). Studies that examine both the impact on care processes and in turn the effects of any changes on resident outcomes are greatly needed.

Second, randomized assignment of facilities was not possible due to the circumstances of the demonstration project; however, the two study arms were equivalent on almost all baseline measures. Future studies should assess the impact of HIT in nursing homes using randomized, controlled designs. Second, the number of facilities was small, with five facilities in each condition. It would be ideal to include a larger number of facilities in future studies. Third, the study was limited to the New York City region and included exclusively for-profit and unionized facilities. Although there are no grounds for expecting that the effect of HIT introduction on resident outcomes would systematically differ in other types of nursing homes, it remains a possibility that future research should explore.

Finally, the study followed residents only over a 9-month period following HIT implementation; it is possible that effects might have become more evident over a longer period of time. It is important to note, however, that the most negative effects would be expected over the relatively short term because of the disruption in facility activities resulting from implementation and adjustment to the technology. We therefore would not expect more negative outcomes over time although positive outcomes might be more likely to be measured later. Moreover, given the high attrition rates observed in nursing home populations of frail, very old individuals, longer follow-up results in analytic challenges.

Despite these limitations, this study represents the first attempt to assess directly the impact on residents of the introduction of HIT in nursing homes. It used well-established assessment methods and obtained a relatively large sample of respondents. The results therefore have significant implications even as we await more representative and controlled studies in the future. Most important, based on the findings of this study, concerns about negative outcomes for residents do not appear to be a major barrier to implementation of HIT. The absence of any effects on key indicators such as resident mortality, ADL function, falls, and subjective measures such as satisfaction with care, allow reasonable confidence that the intervention does not unintentionally harm residents. The findings from the resident perspective were also generally positive, and use of the computerized technology does not appear to have led to widespread resident dissatisfaction or poor communication.

The only reservation to this overall assessment based on outcomes we examined lies in the area of observed behavior problems and the reports of some individual residents regarding possible adverse outcomes on care delivered. It is encouraging that only a single outcome showed a negative treatment effect. Furthermore, as noted earlier, it is impossible to determine the mechanism of this effect. We therefore recommend that future investigations carefully measure impact on resident behavior problems and that direct care staff be alert for any such effects.

Footnotes

Acknowledgements

The authors acknowledge the helpful advice and assistance of Mary Jane Koren, David Lipsky, Ariel Avgar, Scott White, and members of the Quality Care Oversight Committee: Martin Scheinman, Jay Sackman, and William Pascocello. They also thank Leslie Schultz for assistance with project coordination and data management.

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article:

We are grateful for primary support for the project from the Quality Care Oversight Committee in New York State. Additional support was provided by the Commonwealth Fund, grant 2R01AG014299-06 from the National Institute on Aging, and an Edward R. Roybal Center grant from the National Institute on Aging (1 P50 AG11711-01). Mark Lachs acknowledges support from a mentoring award in patient-oriented research from the National Institute on Aging (K24 AG022399).

References

Arling

Kane

R. L.

Mueller

Bershadsky

Degenholtz

H. B.

(2007). Nursing effort and quality of care for nursing home residents. The Gerontologist, 47, 672-682.

Brandeis

Hogan

Murphy

Murray

(2007). Electronic health record implementation in community nursing homes. Journal of the American Medical Directors Association, 8, 31-34.

Castle

N. G.

(2008). Nursing home caregiver staffing levels and quality of care: A literature review. Journal of Applied Gerontology, 27, 375-405.

Committee on Quality of Health Care in America, Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press.

Du Moulin

M. F. M. T.

Van Haastregt

J. C. M.

Hamers

J. P. H.

(2010). Monitoring quality of care in nursing homes and making information available for the general public: State of the art. Patient Education and Counseling, 78, 288-296.

Field

T. S.

Rochon

Lee

Gavendo

Baril

J. L.

Gurwitz

J. H.

(2009). Computerized clinical decision support during medication ordering for long-term care residents with renal insufficiency. Journal of the American Medical Informatics Association, 16, 480-485.

Garg

A. X.

Adhikari

N. K. J.

McDonald

Rosas-Arellano

M. P.

Devereaux

P. J.

Beyene

Haynes

R. B.

(2005). Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: A systematic review. Journal of the American Medical Association, 293, 1223-1238.

Golden

R. R.

Teresi

J. A.

Gurland

B. J.

(1984). Development of indicator scales for the Comprehensive Assessment and Referral Evaluation (CARE) interview schedule. Journal of Gerontology, 39, 138-146.

Grabowski

D. C.

Aschbrenner

K. A.

Rome

V. F.

Bartels

S. J.

(2010). Quality of mental health care for nursing home residents: A literature review. Medical Care Research and Review, 67, 627-656. doi:10.1177/107755871036253810.1177/1077558710362538

10.

Gurland

Kuriansky

Sharpe

Simon

Stiller

Birkett

(1977). The Comprehensive Assessment and Referral Evaluation (CARE)—rationale, development and reliability. International Journal of Aging & Human Development, 8, 9-42.

11.

Gurland

B. J.

Wilder

D. E.

(1984). The CARE interview revisited: Development of an efficient, systematic, clinical assessment. Journal of Gerontology, 39, 129-137.

12.

Harrison

M. I.

Koppel

Bar-Lev

(2007). Unintended consequences of information technologies in health care–An interactive sociotechnical analysis. Journal of the American Medical Informatics Association, 14, 542-549.

13.

Hillestad

Bigelow

Bower

Girosi

Meili

Scoville

Taylor

(2005). Can electronic medical records transform healthcare? Potential health benefits, savings and costs. Health Affairs, 24, 1103-1117.

14.

Kane

R. A.

(2003). Definition, measurement, and correlates of quality of life in nursing homes: Toward a reasonable practice, research, and policy agenda. The Gerontologist, 43(Suppl. 2), 28-36.

15.

Kazley

A. S.

Ozcan

Y. A.

(2008). Do hospitals with electronic medical records (EMRs) provide higher quality of care? An examination of three clinical conditions. Medical Care Research and Review, 65, 496-513.

16.

Kuriansky

Gurland

(1976). The performance test of activities of daily living. International Journal of Aging and Human Development, 7, 343-352.

17.

Cai

Mukamel

D. B.

Glance

L. G.

(2010). The volume-outcome relationship in nursing home care: An examination of functional decline among long-term care residents. Medical Care, 48, 52-57.

18.

Linder

Bates

D. W.

Middleton

Stafford

R. S.

(2007). Electronic healthcare use and quality of ambulatory care in the United States. Archives of Internal Medicine, 167, 1400-1405.

19.

Liu

Castle

N. G.

(2008). Health information technology in nursing homes. Journal of Applied Gerontology, 28, 38-58.

20.

Lourde

(2009). Long-term care health information technology: Providers are increasingly looking to improve quality and claim earned reimbursement with HIT. Provider, 35, 21-32.

21.

Ludwick

D. A.

Doucette

(2009). Adopting electronic medical records in primary care: Lessons learned from health information systems implementation experience in seven countries. International Journal of Medical Informatics, 78, 22-31.

22.

Nakrem

Guttormsen Vinsnes

Harkless

G. E.

Paulsen

Seim

(2009) Nursing sensitive quality indicators for nursing home care: International review of literature, policy and practice. International Journal of Nursing Studies, 46, 848-885.

23.

Overhage

J. M.

Perkins

Tierney

W. M.

McDonald

C. J.

(2001). Controlled trial of direct physician order entry: Effects on physicians’ time utilization in ambulatory primary care internal medicine practices. Journal of the American Medical Informatics Association, 8, 361-371.

24.

Poissant

Pereira

Tamblyn

Kawasumi

(2005). The impact of electronic health records on time efficiency of physicians and nurses: A systematic review. Journal of the American Medical Informatics Association, 12, 505-516.

25.

Poon

E. G.

Cina

J. L.

Churchill

Patel

Featherstone

Rothschild

J. M.

Gandhi

T. K.

(2006). Medication dispensing errors and potential adverse drug events before and after implementing bar code technology in the pharmacy. Annals of Internal Medicine, 145, 426-434.

26.

Resnick

H. E.

Manard

B. B.

Stone

R. I.

Alwan

(2009). Use of electronic information systems in nursing homes: United States, 2004. Journal of the American Medical Informatics Association, 16, 179-186.

27.

Rubenstein

L. Z.

Powers

C. M.

MacLean

C. H.

(2001). Quality indicators for the management and prevention of falls and mobility problems in vulnerable elders. Annals of Internal Medicine, 135, 686-693.

28.

Sidorov

(2006). It ain’t necessarily so: The electronic health record and the unlikely prospect of reducing health care costs. Health Affairs, 25, 1079-1085.

29.

Subramanian

Hoover

Gilman

Field

T. S.

Mutter

Gurwitz

J. H.

(2007). Computerized physician order entry with clinical decision support in long-term care facilities: Costs and benefits to stakeholders. Journal of the American Geriatrics Society, 55, 1451-1457.

30.

Tellis-Nayak

(2007). A person-centered workplace: the foundation for person-centered caregiving in long-term care. Journal of the American Medical Directors Association, 8, 46-54.

31.

Teresi

Abrams

Holmes

(2000). Measurement of depression and depression recognition among individuals with cognitive impairment. In: Albert

Logsdon

(Eds.), Assessing quality of life in Alzheimer’s disease (pp.127-151). New York, NY: Springer.

32.

Teresi

J. A.

Abrams

Holmes

Ramirez

Eimicke

(2001). Prevalence of depression and depression recognition in nursing homes. Social Psychiatry and Psychiatric Epidemiology, 36, 613-620.

33.

Teresi

J. A.

Holmes

Monaco

(1993). An evaluation of the effects of commingling cognitively and noncognitively impaired individuals in long-term care facilities. The Gerontologist, 33, 350-358.

34.

Tierney

W. M.

Miller

M. E.

Overhage

J. M.

McDonald

C. J.

(1993). Physician inpatient order writing on microcomputer workstations: Effects on resource utilization. Journal of the American Medical Association, 269, 379-383.

35.

Toner

Teresi

J. A.

(1990). Development of a direct patient assessment screening instrument for the detection of affective disorder in cases of dementia [Abstract]. The Gerontologist, 30(Suppl.), 185.

36.

Toner

J. A.

Teresi

J. A.

Gurland

B. J.

Tirumalasetti

(1999). The Feeling-Tone Questionnaire: Reliability and validity of a direct patient assessment screening instrument for the detection of depressive symptoms in cases of dementia. Journal of Clinical Geropsychology, 5, 63-78.

37.

Walker

J. M.

Carayon

Leveson

Paulus

R. A.

Tooker

Chin

Stewart

W. F.

(2008). EHR Safety: The way forward to safe and effective systems. Journal of the American Medical Informatics Association, 15, 272-277.

38.

Wiener

J. M.

(2003). Assessment of strategies for improving quality of care in nursing homes. The Gerontologist, 43(Suppl. 2), 19-27.