Outcomes of health information technology utilization in nursing homes: Do implementation processes matter?

Abstract

We examined several outcomes of health information technology utilization in nursing homes and how the processes used to implement health information technology affected these outcomes. We hypothesized that one type of health information technology, electronic medical records, will improve efficiency and quality-related outcomes, and that the use of effective implementation processes and change leadership strategies will improve these outcomes. We tested these hypotheses by creating an original survey based on the case study literature, which we sent to the top executives of nursing homes in seven US states. The administrators reported that electronic medical record adoption led to moderately positive efficiency and quality outcomes, but its adoption was unrelated to objective quality indicators obtained from regulatory agencies. Improved electronic medical record implementation processes, however, were positively related to administrator-reported efficiency and quality outcomes and to decreased deficiency citations at the next regulatory visit to the nursing home. Change leadership processes did not matter as much as technological implementation processes.

Keywords

electronic medical records health information technology long-term care nursing homes quality of care

Introduction

The US federal government has successfully used financial incentives for increasing health information technology (health IT) implementation in some industries, and some researchers have suggested broadening their use to include nursing homes.¹ Such policy is based on the premise that health IT substantially impacts quality and cost-related outcomes in nursing homes, which given the mixed and meager outcomes in the research literature^2–6 may be an optimistic assumption. In this study, we examine this assumption by systematically building an electronic medical record (EMR) outcomes scale based on the quotations of hundreds of clinicians from four multi-case studies^7–9 (Alexander et al., 2007). We therefore examine the extent to which the most important proximal outcomes of EMR implementation, as reported by case study interviewees, are experienced at a larger cross section of nursing homes. Perhaps more importantly, however, our approach allows us to answer the growing cry from researchers to consider how implementation strategies affect the efficacy of health IT.^7,10–12 We use the same studies to build a scale of effective implementation processes. We argue that ineffective implementation processes and poor organizational change leadership in nursing homes are limiting the potential of one type of health IT, EMRs, to positively impact quality and efficiency outcomes in some nursing homes, leading to the uneven and often meager results seen in the case study literature. We suggest that with heightened attention paid to implementation processes, health IT outcomes will improve.

Literature review and hypotheses development

EMR implementation outcomes

First, EMRs may improve the quality and efficiency of care in nursing homes for several reasons. EMRs allow for increased accuracy, completeness, and legibility of and access to medical information. Multiple staff can view a file simultaneously and files are not misplaced, and with instantaneous and convenient access to each resident’s file, documentation accuracy and completeness should be improved. Indeed, these outcomes were reported by the clinicians interviewed by Rantz et al.,⁵ Kramer et al.,⁸ Cherry et al.,¹³ Degenholtz et al.,¹⁴ Meehan,¹⁵ Munyisia et al.,¹⁶ and Zhang et al.¹⁷ Accurate and timely documentation allows different clinicians to understand the changes in resident condition and in medications given in real time, leading to improvements in the quality of care and reductions in medical errors. Medical error reductions were found by Qian et al.,⁴ and Rantz et al.⁵ found that activities of daily living, range of motion, and high-risk pressure sores of residents improved. A second way EMRs can improve the quality of care is by prompting clinicians to follow evidence-based medical recommendations based on conditions in the resident’s chart, and charts may be programmed with reminders regarding the residents’ care plans. These prompts have led to slight improvements in clinician action¹⁸ and to increased clinician compliance.⁴ These mechanisms capitalize on the use of better data for clinicians to make better care-related decisions, which should ultimately improve the efficiency and quality of care (Kruse et al., 2017).19 In addition, the data in aggregate form can help managerial staff supply data to regulators, apply for grants and thus become better funded, and make better managerial decisions.^17,20

A third way that EMRs can improve the efficiency and quality of care is through staff oversight and productivity improvements.^7–9 With EMRs, it is more observable which cares were completed on each resident each day as electronic charts are searchable while paper charts are not. With more oversight, employees may do more in order to appear better to their supervisors, and supervisors may better understand what employees do and may make better scheduling, delegating, and coaching decisions. In addition, EMRs increase the efficiency of documentation,¹⁷ for example, by pre-filling common fields with default values that only rarely need to be changed, which may allow nursing staff to spend more time on direct care.²¹

The impact of EMRs on the documentation time is not clear in the literature. While Mei et al.²¹ found that documentation time fell for a simple fall-reporting software and Bezboruah et al.⁷ found that documentation time rose in some nursing homes that implemented EMRs at least initially, it is not clear whether EMRs will generate staff time-savings. Qian et al.⁴ compared the EMR administration records and paper-based records and found no significant differences in the nursing time spent on various activities such as medication administration, documentation, and communication. Munyisia et al.²² used a longitudinal case study design to evaluate whether nursing time with patients improved after employees had time to adjust to EMRs, but even 23 months after implementation, no difference in direct care time was found.²² Klinger and White²³ and Cherry et al.¹³ found that some employees spent more time with residents, while others spent less after EMR implementation. If increasing nursing time with residents is a goal of EMR adoption, it may be important to include training or incentives regarding it because it does not appear to happen automatically.

A fourth way that EMRs could improve the efficiency and quality of care is by increasing the speed of service provision.^7,8,13 Electronic alerts can be used to ensure timely responses to everything from resident call lights (used by residents to signal an urgent need for assistance) to medication administration to family member inquiries.

Hypothesis 1. The use of EMRs is positively associated with quality and efficiency outcomes in nursing homes.

Implementation processes

While in theory EMRs should positively impact efficiency and quality outcomes, many studies, especially larger studies, have found meager or mixed findings. Hitt and Tambe² used a more sophisticated methodological design that compared the same organization’s performance before and after EMR adoption and found only marginal improvements in employee productivity and efficiency. Pillemer et al.³ in a well-done quasi-experimental design, found EMRs had virtually no impact on resident health and quality of life. Pillemer et al.³ did find that EMRs were negatively associated with poor resident behaviors and that residents viewed the technology as generally positive; however, in a qualitative study, Cherry et al.¹³ found a range of resident and family overall reactions to EMRs, with residents lamenting their caregivers’ “playing” on computers. Klinger and White²³ and Bezboruah et al.⁷ reported that nursing staff generally did not believe they were more efficient or documented more accurately, and Cherry et al.¹³ reported nursing staff did not think EMRs improved the quality of care.

Large longitudinal studies of Medicare data and the Department of Veterans’ Administration found that the benefits of health IT use were unexpectedly small.^24,25 A few studies^4,7,26 found that EMRs sometimes even had adverse effects on the quality of care. With a strong theoretical rationale for improved outcomes, researchers have begun to consider why so many studies, especially the best studies, report uninspiring outcomes. Several have conjectured that poor implementation processes are hindering the effectiveness of health IT (Kellerman and Jones, 2013).^{7,11,27,28,29}

Several qualitative studies present evidence of implementation processes we believe to be suboptimal. One area of concern was in the involvement and treatment of employees prior to and during the EMR implementation process. Many case studies report staff resistances to using technology and learning problems, technical and connectivity concerns, documentation duplication, unnecessarily complex information management including wasting time clicking through screens, and difficulties in delivering care or the quantity or quality of time spent with residents.^{7,23,26,30,31} Yet, nursing homes with positive employee relations,^9,23 those that included end-users in systematic planning,⁷ and those that used employee empowerment practices² usually had better outcomes. When asked by researchers, nursing staff had innovative ideas for improving the software to improve efficiency and quality outcomes and desired meetings with the vendors for software customization.^13,15

In every study reviewed, staff reported not receiving enough training,^7,11,13,30 and implementation processes that emphasized training led to better outcomes.^9,32 Several staff members also expressed that off-site technological assistance was insufficient and used computers and screens unlike those the employees would use, and on-site training and assistance was more helpful.^7,11,30 After training, staff reported that workload initially increased due to learning curves and charting twice (electronically and in writing),^7,30 and employees who experienced incremental roll-outs reported better outcomes than those who were introduced to all new technical features simultaneously.^7,30

Based on these results, we conjecture that nursing homes that include the employees who will use the EMRs during the planning and customized design phase of the EMR project will not only face less resistance to EMRs but will also end up with EMR systems that save employees time, as unnecessary screens can be eliminated and default answers can be appropriately set. We also believe that additional training time, delivered on-site using the exact screens that employees will be using, will result in greater meaningful use of EMR systems and their electronic alerts enabling greater increases in the quality of care. In addition, including additional staff on busy shifts as employees adjust to EMRs, and incremental EMR roll-outs, will allow employees the time to form new habits that take advantage of the technological features and customizable prompts that they learned in training and lead to fewer counterproductive EMR work-arounds.

Another area of concern was in the area of change leadership. While reviewing hundreds of quotations in qualitative studies, we saw plenty of evidence of organizations either following or not following the advice given in the change leadership literature. Change leadership touts the importance of organizational characteristics such as learning, innovation, experimentation, and questioning while preparing the organizations for change. In this change process, there is constant seeking for new perspectives and encouragement of participation throughout the organization. Organizational leadership are more aware of trends, crises, and evolutions in their organizational environments, which prepare them to be resilient and adaptive to changes.³³ Research on nursing homes³⁴ suggests that the presence of change leadership and an organizational climate of innovation are likely to assist organizations preparing for change.

Studies of EMR implementation processes found that administrators faced challenges with obtaining adequate information about the costs and benefits of health IT, or they did not prioritize information gathering or learning about health IT systems.^7,11,35,36 Song et al.³⁶ found that pre-implementation business case analyses lacked the rigor associated with comparable capital acquisition projects, Bezboruah et al.⁷ found little systematic planning for EMR implementation, and Ko et al.¹¹ found limited involvement of nurses in EMR planning and little planning overall. Rationales given for EMR implementation were often remarks about what other institutions were doing, lacking the rigor of benchmarking studies, or a belief by the management team that EMR investment was inevitable.^7,36 A lack of clinical leadership and early communication with employees and other important stakeholders for buy-in were also common (Bezboruah et al., 2014).¹¹ Many nursing homes failed to plan how the EMRs would impact staff workflow and consequently implemented cumbersome systems that were not optimally utilized since employees found work-arounds.¹¹ Often, the administrators, who made decisions in the absence of clinical staff, did not understand the software or the needs of clinical staff and were too reliant on the biased views of vendors for information and advice.^11,37

We gathered information from the study by Szydlowski et al.²⁹ on nursing and suggest the following attributes would indicate a more ideal change leadership process: leadership of the EMR acquisition team (1) developed a clear vision, (2) clarified the necessity of EMR implementation, (3) proactively built a broad coalition to support implementation, (4) empowered employees to implement EMRs, (5) monitored and communicated progress of EMR implementation, and (6) gave individualized attention to employees struggling with EMR implementation. Even if management did not enact the best practice implementation processes that we uncovered, managers who were skilled at leading change initiatives should still improve the quality and efficiency of their nursing homes as they lead their organizations through technological change.

Hypothesis 2. Better EMR implementation processes are positively associated with quality and efficiency outcomes in nursing homes.

Hypothesis 3. Effective change leadership strategies are positively associated with quality and efficiency outcomes in nursing homes.

Method

Data

The purpose of this study was to (1) estimate the impact of EMRs on nursing home quality and efficiency outcomes (hereafter “outcomes”) and (2) determine whether good implementation processes and change leadership strategies improve nursing home outcomes (hereafter “processes” and “change leadership”). We began by designing an original survey using an inductive approach, by examining the detailed practitioner responses³⁸ recorded in several qualitative case studies (including Alexander, 2007).^7–9 We selected all the case studies available at the time that included detailed reporting of raw data, which we obtained from a government report and conference paper drafts available online. Following a process advised by Bowen and Gao,³⁹ two researchers and two graduate assistants independently reviewed all the case studies. We independently categorized quotations into themes and then met to discuss the results. Outcomes or processes quotations that were mentioned in only one case study were dropped. Quotation themes were turned into survey items. By group consensus, similar survey items were combined and succinct language for the items was developed. The draft survey was reviewed by three researchers and two nursing home administrators and pilot tested. Changes to the survey were made based on the feedback received.

Electronic mail addresses were obtained from state Departments of Health/Aging or nonprofit advocacy organizations for all nursing home administrators in Illinois, South Carolina, Kansas, Texas, Georgia, Minnesota, and Missouri. The survey was conducted online between July and October 2011. Three emails containing cover letters and a survey link were sent in 1-week increments to 2747 nursing home executives; 284 surveys were received (over three-fourths were the chief executive officer or administrator, with the remainder being the chief information officer or the director of nursing).

Survey measures

We grouped outcome items into three positive and one negative theme that emerged from the data when we conducted exploratory factor analysis. All items were measured on a 5-point Likert-type scale with the lead question, “To what extent has your EMR implementation resulted in. …” The first positive outcome was improved data access. It was measured with the following items: (1) improved census reporting, (2) improved completeness of resident information, (3) increased availability of resident information, (4) enhanced information sharing with other health care providers, and (5) higher quality and more complete reports. The second positive outcome was that the speed of service provision increased, which was measured with the following items: (1) faster delivery of services to new residents, (2) improved on-call response time, and (3) quicker response to family member inquiries. The third positive outcome was staff oversight and productivity improvements, which were measured with the following items: (1) improved staff productivity, (2) reduced administrative staff needed, and (3) improved managerial oversight of employees. The negative outcome was staff resistance to EMRs (Alexander, 2007).^7,8 It was measured by asking the level of resistance to EMR implementation expressed by each employee group and by asking about employee turnover.

Seven items were included in the processes scale. They were measured on a 5-point scale, and all questions began with “To what extent did the EMR implementation process include. . .” The items were (1) staff training, (2) early communication with key stakeholders, (3) use of additional staff in early adoption stages, (4) Incremental roll-out, (5) on-site technological assistance, (6) nursing managers in implementation decisions, and (7) nursing staff in implementation decisions.

“Change leadership” was measured by a six-item scale adapted from Harold et al.⁴⁰ These scale consists of the following items: (1) developed a clear vision, (2) made it clear why EMR implementation was necessary, (3) built broad coalition up-front to support implementation, (4) empowered employees to implement EMRs, (5) monitored and communicated progress of EMR implementation, and (6) gave individualized attention to those having trouble with EMR implementation. Although drawn from the management literature, these items were similar to the health IT leadership change themes identified by Szydlowski and Smith²⁹ in their study of nursing.

EMR adoption was measured by a single variable “Does your organization use electronic medical records?” Because nursing homes were in the early stage of health IT adoption and EMRs were often the first systems adopted, a binary variable satisfactorily captures EMR use.² The survey also asked for the date of EMR implementation to calculate the length of time the facility had the system. EMRs can have initial negative effects, since it takes several months for employees to adapt to them.³⁵

Regulator data measures

Regulator data were obtained from the Centers for Medicare and Medicaid Services (CMS), containing the size, ownership, chain status, occupancy rates, funding, quality ratings (5-point scales), and regulatory citations for all certified facilities. We use the facility’s overall rating, the quality indicator (containing nine items, such as the percentage of residents with pressure sores, falls, or other injuries), and the count of regulatory citations received as distal quality outcomes.

Statistical strategy

Scale validation

Since our survey items were not previously validated, we followed Lance and Vandenberg’s⁴¹ and Acock’s⁴² instructions to conduct confirmatory factor analysis (CFA) to generate scales. We used maximum likelihood estimation with imputation to adjust for missing data.

Response bias checks

We tested for survey response bias, which involves comparing survey responders with non-responders.⁴³ Responding nursing homes had on average 99 beds compared to 104 beds among non-responders (p > 0.10). In all, 46 percent of the responders were for-profit, 47 percent were nonprofit, and 5 percent were government-owned, compared to 71, 24, and 3 percent, respectively, of non-responders (p < 0.01). In total, 6.5 percent of the responders and non-responders were located within a hospital (p > 0.10). Fifty percent of responders, versus 58 percent of non-responders, were part of a regional or national chain (p < 0.01). In total, 3.5 percent of responders versus 5.2 percent of non-responders did not care for Medicaid residents (p > 0.10), and 3.5 percent of responders versus 6.0 percent of non-responders did not care for Medicare residents (p < 0.10).

Regressions

We conducted ordered probit regressions to test Hypotheses 1 and 2. All measures were adjusted for resident health conditions (case mix). We used the Heckman⁴⁴ sample selection correction method, a weighting technique, which can minimize survey response bias.⁴⁵

Results

Descriptive statistics

Table 1 reports descriptive statistics for our survey. As expected, the availability and completeness of health data increased a moderate to large amount at most nursing homes. Similarly, the speed at which resident concerns were addressed increased. The administrators reported that new residents received needed care, call lights (for urgent needs) were answered, and family needs were addressed more quickly after the EMRs were introduced. In addition, the productivity of nursing and administrative staff improved slightly. Most administrators detected a moderate to large increase in the level of oversight that they had over employees after EMR implementation, and they believed that employee productivity had increased moderately. However, over 50 percent of administrators responded that there had not been a reduction in staff due to productivity gains. Our results also indicated that staff turnover was usually not impacted by the introduction of the EMRs, and staff resistance to EMRs was relatively low in the administrator’s view.

Table 1.

Outcomes of EMR implementation as reported by nursing home administrators.

	Number of responses	Mean	Standard deviation	Minimum	Maximum
Improved data
Improved census reporting	109	2.991	1.213	1	5
Improved completeness of resident information	111	3.423	1.066	1	5
Increased availability of resident information	110	3.782	0.961	1	5
Enhanced information sharing with other health care providers	111	2.865	1.132	1	5
Higher quality and more complete reports	109	3.459	1.085	1	5
Improved speed of service provision
Faster delivery of services to new residents	110	2.764	1.116	1	5
Improved on-call response time	100	2.220	1.177	1	5
Quicker response to family member inquiries	110	2.964	1.188	1	5
Staff resistance to health IT
CNAs’ resistance to IT	101	2.307	0.821	1	5
Nurses’ resistance to IT	108	2.435	0.899	1	5
Nursing managers’ resistance to IT	108	2.102	0.937	1	5
Increased turnover	106	1.528	0.783	1	5
Oversight and productivity improvements
Improved staff productivity	110	2.873	1.101	1	5
Reduced administrative staff needed	110	1.573	0.784	1	4
Improved managerial oversight of employees	111	3.243	1.223	1	5

IT: information technology; CNA: certified nursing assistant.

Sample sizes vary because questions marked “unsure” are excluded. All variables measured on the following Likert-type scale: 1 (not at all), 2 (some), 3 (moderate), 4 (large), and 5 (extreme).

Table 2 reports descriptive statistics for our survey respondents who had implemented EMRs. Consider first the processes scale. The administrators reported that they provided a large amount of training and on-site technical help to employees and communicated with stakeholders to a moderate extent. Administrators were less likely to provide additional staff to assist employees during implementation, but most implemented the technology incrementally. Administrators included staff in implementation decision-making, especially supervisory staff.

Table 2.

Implementation processes reported by nursing home administrators.

Variable	Observations	Mean	Standard deviation	Minimum	Maximum
Process variables
Include staff training	109	3.917	0.873	1	5
Include early communication with key stakeholders	103	3.340	1.053	1	5
Use additional staff in early adoption stages	107	2.925	1.052	1	5
Incremental roll-out	107	3.477	1.093	1	5
Include on-site technical assistance	110	3.473	1.171	1	5
Inclusion in decision-making
Include nurse managers in implementation decisions	109	3.349	1.117	1	5
Include nursing staff in implementation decisions	110	3.009	1.128	1	6
Change leadership
Developed a clear vision	108	4.009	0.743	1	5
Made it clear why EMR implementation was necessary	109	4.174	0.768	1	5
Built a broad coalition up-front to support implementation	107	3.766	0.875	1	5
Empowered employees to implement EMRs	109	3.899	0.816	1	5
Monitored and communicated progress of EMR implementation	109	3.899	0.816	1	5
Gave individualized attention to those having trouble with EMR implementation	108	4.148	0.783	1	5

EMR: electronic medical record.

Sample sizes vary because questions marked “unsure” are excluded. All variables measured on the following Likert-type scale: 1 (not at all), 2 (some), 3 (moderate), 4 (large), and 5 (extreme).

Finally, we consider the change leadership. This six-item scale had internal consistency reliability (α) of 0.84 and a mean of 4.03 (95% CI = 3.9–4.2). Administrators reported following these effective change leadership practices to a large extent.

Tests of the factor structure of survey items

We modeled a CFA using four outcome variables and two implementation variables (implementation processes and participation in decision-making). It yielded insignificant fit, so we modified our results along theoretical and statistical lines.⁴² We dropped two items that had factor loadings below 0.40 (“increased turnover” and “reductions in administrative staff”).

The modified six-factor model fit our data well (χ²(df = 152) = 188.3, p < 0.05; comparative fit index (CFI) = 0.97; root mean square error of approximation (RMSEA) = 0.05, 90% CI = 0.02–0.07). These values are better than the fit cut-offs of CFI = 0.95 and RMSEA = 0.08.^42,46 We combined data and staff productivity outcomes into a single scale because the covariance between these two variables was over 0.90, suggesting they measured the same latent variable. The resulting five-factor model was a good fit for our data (χ²(df = 157) = 189.3, p < 0.05; CFI = 0.97; RMSEA = 0.04, 90% CI = 0.01–0.06). Our best model compares favorably to a CFA where all survey measures load upon one latent variable (χ²(df = 170) = 491.9, p < 0.01; CFI = 0.70; RMSEA = 0.13, 90% CI = 0.12–0.14) and to a three-factor model of positive outcomes, negative outcomes, and implementation processes (χ²(df = 165) = 230.8, p < .01; CFI = 0.94; RMSEA = 0.06, 90% CI = 0.04–0.08). Since our variables do not load on a single factor, we also pass the Harman single-factor test, suggesting little common method variance.⁴⁷

Means and zero-order correlations

Given our CFA results, we created three scales to measure the survey outcomes of EMR implementation. We combined the data and staff productivity and oversight items into a scale for efficiency, which had internal consistency reliability (α) of 0.90 and a mean of 3.23 (95% CI = 3.06–3.39). We created a second scale measuring the speed at which residents’ needs were met (α = 0.85), which had a mean of 2.66 (95% CI = 2.47–2.85). We created a third scale for employee resistance to EMRs (α = 0.74), which had a mean of 2.08 (95% CI = 1.96–2.20).

We created two scales for health IT implementation processes. We labeled the first scale implementation processes (α = 0.76), which had a mean of 3.43 (95% CI = 3.29–3.57). The second scale measured employee participation in implementation decision-making (α = 0.88), which had a mean of 3.18 (95% CI = 2.98–3.38).

Table 3 also reports correlations between our processes and outcomes variables. We found positive associations between employee productivity and implementation processes (r = 0.44, p < 0.01) and participation in decision-making (r = 0.43, p < 0.01). We also find positive associations between the speed of service provision and implementation processes (r = 0.34, p < 0.01) and participation in decision-making (r = 0.41, p < 0.01). Smaller associations were detected between effective change leadership and employee productivity (r = 0.34, p < 0.01) and speed of service provision (r = 0.22, p < 0.01).

Table 3.

Means, standard deviations, and correlations among variables.

Only adopters of EMRs	Mean	Standard deviation	1	2	3	4	5
1. Implementation processes	3.43	0.75
2. Participation in decision-making	3.18	1.06	0.52***
3. Employee productivity	3.23	0.88	0.44***	0.43***
4. Speed of service provision	2.66	1.02	0.34***	0.41***	0.78***
5. Resistance to change	2.08	0.66	0.25***	0.06***	0.08	0.10
6. Change leadership	4.03	0.63	0.44***	0.42***	0.22**	0.22**	−0.24**
All respondents and/or OSCAR data			7	8
7. Uses EMR	0.44	0.50
8. Citations	2.31	3.96	−0.03
9. Overall quality rating	2.84	1.28	0.06	−0.14***

EMR: electronic medical records; OSCAR: Online Survey, Certification, and Reporting.

*, **, and *** indicate statistical significance at the 10, 5, and 1 percent levels, respectively, in two-tailed tests.

Hypothesis testing: multivariate analysis

Table 4 has the regression results analyzing the impact of EMR utilization on regulatory data quality outcomes. Using EMRs was not related to the nursing home’s overall regulatory score, quality index score, or deficiency citations. Recognizing that EMR implementation might not have positive results initially since staff face a learning curve and may resist technologies, we also tested whether the length of time since EMR implementation was related to the quality of care. Again, we found no such relationship. Administrators reported positive outcomes from using EMRs, but EMR use was unrelated to distal objective measures, providing partial support for Hypothesis 1.

Table 4.

The impact of the use of electronic medical records, and the length of their use, on regulator quality data and staffing levels.

	Overall star rating	Quality star rating	Deficiency citations
Uses EMR	0.185[0.179]	0.090[0.163]	0.126[0.209]
Total years EMR in place	0.014[0.026]	0.023[0.024]	−0.046[0.035]
Nonprofit ownership (includes public)	0.284[0.214]	−0.368** [0.153]	−0.383[0.240]
Number of certified beds	−0.003** [0.001]	−0.000[0.001]	0.001[0.002]
Chain status	−0.034[0.145]	0.130[0.137]	−0.278[0.174]
Located within a hospital	−0.139[0.299]	−0.168[0.268]	0.518[0.346]
Number of Medicaid patients	0.247[0.371]	0.173[0.322]	−0.061[0.416]
Number of Medicare patients	1.072** [0.440]	0.833** [0.412]	−0.739[0.554]
Observations	222	219	222
Log-likelihood	−1092.5	−1073.4	−1103.4
χ²	17.86	17.09	9.29

Coefficients presented with standard errors in brackets. The Heckman sample selection corrections and ordered probit were utilized in estimation. Sample selection equation included ownership, chain status, location within a hospital, number of residents, percentage of occupied beds, Medicaid or Medicare payment, retirement community status, and state. Sample selection equation had 3725 observations.

**Statistical significance at the 5 percent level, in two-tailed tests.

Table 5’s regression results corroborated our correlational analysis; the implementation processes were positively related to our efficiency and speed of service outcomes, and unrelated to employee resistance to EMRs. While the overall and quality star rating regressions were insignificantly related to our implementation processes, we found that these processes decreased deficiency citations. Participation in decision-making and change leadership were not significantly related to any outcomes and were therefore excluded from Table 5.

Table 5.

The impact of EMR implementation processes on regulator quality data and staffing levels.

	Efficiency outcomes	Speed of service provision	Employee resistance to EMRs	Overall star rating	Quality measures star rating	Deficiency citations	RN staffing levels
Implementation processes	0.469*** [0.110]	0.447*** [0.129]	0.119[0.112]	−0.005[0.166]	0.014[0.131]	−0.323** [0.152]	−0.006[0.125]
Number of certified beds	0.004*** [0.001]	0.004*** [0.002]	−0.000[0.001]	−0.005** [0.002]	−0.001[0.003]	0.001[0.002]	−0.004[0.003]
Not for-profit (includes public)	−0.170[0.235]	−0.531** [0.264]	0.164[0.256]	0.678* [0.363]	−0.392[0.246]	−0.212[0.380]	0.023[0.464]
Chain status	0.079[0.165]	0.067[0.197]	−0.245* [0.141]	−0.080[0.241]	0.125[0.213]	−0.095[0.277]	0.033[0.188]
Located within a hospital	−0.148[0.260]	−0.179[0.197]	0.106[0.234]	−0.300[0.377]	0.010[0.303]	0.447[0.399]	0.264[0.353]
Number of Medicaid patients	0.401[0.351]	0.895[0.423]	0.338[0.300]	0.588[0.560]	−0.148[0.396]	−0.256[0.606]	0.335[0.516]
Number of Medicare patients	−0.596[0.826]	0.182[0.955]	−1.217[0.790]	−0.193[1.223]	7.122[160337]	−6.434[16080]	0.256[1.579]
Constant	2.319*** [0.693]	2.478*** [0.753]	0.6030.752	−2.416*** [0.288]	−2.381*** [0.289]	−2.420*** [0.287]	−2.395*** [0.281]
Observations	92	92	92	95	95	95	95
Log-likelihood	−465.48	−479.86	−450.23	−509.2	−510.9	−514.7	−505.7
χ²	37.33	30.31	12.35	11.18	4.975	6.299	11.94

EMR: electronic medical records.

Beta coefficients reported with standard errors in brackets. The Heckman sample selection corrections and ordered probit were utilized in estimation. Due to a larger number of values taken in efficiency and speed scales (over 15 rather than 5), standard Heckman models are used. Sample selection equation included ownership, chain status, location within a hospital, number of residents, percentage of occupied beds, Medicaid or Medicare payment, retirement community status, and state. Only survey respondents who implemented EMRs were included in estimation. In all, 3598 observations were used in the selection equations.

*, **, and *** indicate statistical significance at the 10, 5, and 1 percent levels, respectively, in two-tailed tests.

Discussion

Outcomes of EMR implementation

We synthesized the efficiency and quality outcomes of EMR use reported in several case studies and created a survey to analyze them in a broader cross section of nursing homes. Overall, we found similar efficiency and speed of service outcomes as those reported in qualitative studies, with the exceptions that staff resistance to EMRs and turnover were lower than expected. It is possible that administrators underestimate staff resistance to EMRs, as found by Bezboruah et al.,⁷ but Ko et al.¹¹ found staff reports of health IT resistance to be relatively low as well. Ko et al.¹¹ did suggest that perhaps the staff who participated in the employee focus groups were not random. We encourage future research to survey a broader cross section of nursing staff to see whether our finding is due to low levels of staff resistance to EMR or low recognition of such resistance by administrators.

Our finding that administrator reports of EMR outcomes were quite positive should be viewed in light of research that shows that managerial staff are more likely than non-managerial staff to report favorable outcomes of health IT^6,48 and our finding that EMR adoption did not improve regulator quality indices or deficiency citations. However, outcomes noticed by administrators may not result in objective quality outcomes because they impact different outcomes, such as residents’ quality of life rather than counts of bed sores or falls (components of the regulator quality index). Existing research is mixed on the extent to which health IT improves communication between staff and residents, and little is known about other factors related to residents’ quality of life, such as timeliness of the delivery of messages from family or pain medication. Klinger and White²³ and Hitt and Tambe² suggested that communication increases after health IT implementation only in nursing homes with positive employee relations. Pillemer et al.³ found no differences in resident-reported mood outcomes as a result of health IT. It may be that EMRs impact proximal outcomes, such as the speed of service delivery and the quality of clinical data, but these outcomes do not matter enough to affect distal quality measures such as regulator five-star ratings because something else matters more, or they could impact different dimensions of quality (like quality of life instead of health). It could also be that EMR’s impact on the proximal outcomes is overstated due to measurement artifacts such as common method bias. Future research will need to explore these relationships using sources other than the administrator for information.

Implementation processes

We also estimated the extent to which certain implementation practices and general change leadership practices led to positive distal (regulator) outcomes, as this has been cited as a reason for the lackluster performance of health IT (Kellerman and Jones, 2013).^7,27,28 While effective change leadership practices are important, the implementation processes we uncovered and the scale we developed in this article were more important predictors of the quality of care.

This article makes another contribution to the nursing home literature. The implementation processes scale that we created was drawn from the qualitative literature and was more strongly correlated with the efficiency and quality administrator-reported outcomes than the existing change leadership scale. In addition, it predicted deficiency citations, a distal measure of nursing home quality, while the change leadership scale did not.

Limitations

A limitation of our study was that our response rate was low, but it is similar to other surveys of health IT.²⁷ Since many administrators use email filters, it is plausible that many administrators never saw our survey, reducing the response rate but not in a way that would be expected to create bias. Although web-based surveys tend to have lower response rates than paper surveys,^49,50 their results tend to be similar.⁵¹ The most significant problem with a low response rate is non-response bias, which we checked for using the procedures recommended by Werner et al.⁴³ Due to the existence of a national regulator dataset, we were able to test for differences between the sample and the population on observable variables. It is important to note that not only do we compare the sample and population for differences in observables, we also statistically correct for these differences using the Heckman⁴⁴ correction, a two-stage regression model that first models the probability of becoming a survey participant and then adjusts the results of the final regression to mimic the population.

One problem with a low response rate is that although we statistically control for observable variables, we cannot know whether administrators who were more comfortable with technology and more knowledgeable about its implementation were more likely to respond to the survey. The statistical problem that would result would be restriction of range, as the potential respondents who did poorly on technology implementation would not respond. Restriction of range makes it more difficult to detect a true relationship that exists since the observations on one side of the distribution are omitted. If this is true, the actual importance of technological implementation procedures is greater than we have found, but the absolute value of the means of our statistics is inflated. This does not call into question the core finding of this article.

Another potential problem with survey research of our type is common method bias. We test for this with the Harman single-factor test, which suggested little common method variance.⁴⁷ Because our finding that nursing homes that used better implementation processes had higher quality of care outcomes was replicated when we used objective regulator data, and that those results could not be due to common method bias, we have more confidence in our results regarding the impact of implementation processes on quality outcomes in nursing homes than in our results about the direct relationship between EMR implementation and nursing home outcomes, which relied upon same-source data.

Conclusion

We analyzed the extent to which EMR implementation impacts quality and efficiency outcomes in nursing homes and examined the extent to which implementation processes impacted these outcomes. We linked an original survey of executives in seven states to objective regulator data. We used qualitative case studies and CFA to generate the scales used in the analysis and regressions controlling for some types of sample selection to test our hypotheses. We found that EMR use led to positive administrator-reported efficiency and quality outcomes, but was unrelated to regulatory quality outcomes. We did, however, find that the following implementation processes improved efficiency-related outcomes, increased the speed at which residents received care, and decreased regulatory citations in nursing homes: (1) early communication with key stakeholders, (2) using additional staff during early adoption stages, (3) high levels of staff training, (4) incremental EMR roll-outs, and (5) ensuring staff access to on-site technical assistance. Our literature review and analysis also suggest that change leadership practices that value innovation and employee empowerment can result in nursing homes being adaptive to implementing health IT and make the implementation process more streamlined. We encourage practitioners to utilize these practices and researchers to examine whether our findings generalize to other industries or technologies.

Footnotes

Acknowledgements

We are grateful to Jason Smith, David Coursey, Jason Lambert, and Elena Radeva. IRB approval was obtained from University of Texas at Arlington IRB Protocol: 2011-0532 “A comparative study of the implementation of health information technology,” 11 June 2011.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Darla J Hamann

References

Abramson

McGinnis

Moore

, et al. A statewide assessment of electronic health record adoption and health information exchange among nursing homes. Health Serv Res 2014; 49(1 pt. 2): 361–372.

Hitt

Tambe

. Health care information technology, work organization, and nursing home performance. Ind Labor Relat Rev 2016; 69(4): 834–859.

Pillemer

Meador

Teresi

, et al. Effects of electronic health information technology implementation on nursing home resident outcomes. J Aging Health 2012; 24(1): 92–112.

Qian

Hailey

. The impact of electronic medication administration records in a residential aged care home. Int J Med Inform 2015; 84(11): 966–973.

Rantz

Hicks

Petroski

, et al. Cost, staffing and quality impact of bedside electronic medical record (EMR) in nursing homes. J Am Med Dir Assoc 2010; 11(7): 485–493.

Weech-Maldonado

Davlyatov

Lord

. EHR implementation among nursing homes: is it associated with better financial performance? Innov Aging 2018; 2(Suppl_1): 599–600.

Bezboruah

Hamann

Smith

. Management attitudes and technology adoption in long-term care facilities. J Health Organ Manag 2014; 28(3): 344–365.

Kramer

Richard

Epstein

, et al. Understanding the costs and benefits of health information technology in nursing homes and home health agencies: case study findings. Denver, CO: University of Colorado, Denver; U. S. Department of Health and Human Services, 2009, http://aspe.hhs.gov/daltcp/reports/2009/HITcsf.pdf

Lipsky

Avgar

Lamare

. Organizational strategies for the adoption of electronic medical records: Toward an understanding of outcome variation in nursing homes. In: Proceedings of the 59th annual meeting of the labor and employment relations association, San Francisco, CA, 3–6 January 2009, pp. 73–85. Champaign, IL: Labor and Employment Relations Association.

10.

Buntin

Burke

Hoaglin

, et al. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff 2011; 30(3): 464–471.

11.

Wagner

Spetz

. Nursing home implementation of health information technology: Review of the literature finds inadequate investment in preparation, infrastructure, and training. Inquiry 2018; 55: 778902.

12.

Wang

Biedermann

. Adoption and utilization of electronic health record systems by long-term care facilities in Texas. Perspect Health Inf Manag 2012; 9: 1.

13.

Cherry

Ford

Peterson

. Experiences with electronic health records: Early adopters in long-term care facilities. Health Care Manage Rev 2011; 36: 265–274.

14.

Degenholtz

Resnick

Lin

, et al. Development of an applied framework for understanding health information technology in nursing homes. J Am Med Dir Assoc 2016; 17(5): 434–440.

15.

Meehan

. Electronic health records in long-term care: staff perspectives. J Appl Gerontol 2015; 36: 1175–1196.

16.

Munyisia

Hailey

. The changes in caregivers’ perceptions about the quality of information and benefits of nursing documentation associated with the introduction of an electronic documentation system in a nursing home. Int J Med Inform 2011; 80: 116–126.

17.

Zhang

Shen

. The benefits of introducing electronic health records in residential aged care facilities: a multiple case study. Int J Med Inform 2012; 81(10): 690–697.

18.

Judge

Field

DeFlorio

, et al. Prescribers’ responses to alerts during medication ordering in the long-term care setting. J Am Med Inform Assoc 2006; 13(4): 385–390.

19.

Kruse

Mileski

Vijaykumar

, et al. Impact of electronic health records on long-term care facilities: systematic review. JMIR Med Inform 2017; 5(3): e35.

20.

Jiang

Hailey

, et al. The impact of electronic health records on risk management of information systems in Australian residential aged care homes. J Med Syst 2016; 40: 204.

21.

Mei

Marquarda

Jacelon

, et al. Designing and evaluating an electronic patient falls reporting system: perspectives for the implementation of health information technology in long-term residential care facilities. Int J Med Inform 2013; 8(2): e294–e306.

22.

Munyisia

Hailey

. The effect of an electronic health record system on nursing staff time in a nursing home: a longitudinal study. Australas Med J 2014; 7(7): 285–293.

23.

Klinger

White

. Lessons from a health information technology demonstration in New York nursing homes. The Commonwealth Fund, 2010, https://www.commonwealthfund.org/publications/case-study/2010/apr/lessons-health-information-technology-demonstration-new-york (accessed 25 September 2013).

24.

Agha

. The effects of health information technology on the costs and quality of medical care. J Health Econ 2014; 34: 19–30.

25.

Spetz

Burgess

Phibbs

. The effect of health information technology implementation in Veterans Health Administration hospitals on patient outcomes. Healthc 2014; 2(1): 40–47.

26.

Zhang

Gong

, et al. Unintended adverse consequences of introducing electronic health records in residential aged care homes. Int J Med Inform 2013; 82(9): 772–788.

27.

Penoyer

Cortelyou-Ward

Noblin

, et al. Use of electronic health record documentation by healthcare workers in an acute care hospital system. J Healthc Manag 2014; 59(2): 130–144.

28.

Kellermann

Jones

. What it will take to achieve the as-yet-unfulfilled promises of health information technology. Health Aff 2013; 32(1): 63–68.

29.

Szydlowski

Smith

. Perspectives from nurse leaders and chief information officers on health information technology implementation. Hosp Top 2011; 87(1): 3–9.

30.

Alexander

Rantz

Flesner

, et al. Clinical information systems in nursing homes: an evaluation of initial implementation strategies. Comput Inform Nurs 2007; 25(4): 189–197.

31.

Brandeis

Hogan

Murphy

, et al. Electronic health record implementation in community nursing homes. J Am Med Dir Assoc 2007; 8: 31–34.

32.

Zarowitz

Resnick

Ouslander

. Quality clinical care in nursing facilities. J Am Med Dir Assoc 2018; 19(10): 833–839.

33.

Dumas

Beinecke

. Change leadership in the 21st century. J Organ Change Manag 2018; 31(4): 867–876.

34.

von Treuer

Karantzas

McCabe

, et al. Organizational factors associated with readiness for change in residential aged care settings. BMC Health Serv Res 2018; 18(1): 77.

35.

Chaudhry

Wang

, et al. Systematic review: Impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006; 144: E12–E22.

36.

Song

McAlearney

Robbins

, et al. Exploring the business case for ambulatory electronic health record system adoption. J Healthc Manag 2011; 56(3): 169–180.

37.

Bezboruah

Hamann

. Health IT adoption in nursing homes: the role of IT vendors. Int J Innov Tech Manag 2018; 15(1): 185001.

38.

Hinkin

. A brief tutorial on the development of measures for use in survey questionnaires. Organ Res Methods 1998; 1(1): 104–121.

39.

Bowen

Gao

. Structural equation modeling (pocket guides to social work research). New York: Oxford University Press, 2011.

40.

Harold

Fedor

Caldwell

, et al. The effects of transformational and change leadership on employees’ commitment to a change: a multilevel study. J Appl Psychol 2008; 93(2): 346–357.

41.

Lance

Vandenberg

. Confirmatory factor analysis. In: Drasgow

Schmitt

(eds) Measuring and analyzing behavior in organizations: advances in measurement & data analysis. San Francisco, CA: Jossey-Bass, 2002, pp. 221–256.

42.

Acock

. Discovering structural equation modeling using stata. College Station, TX: The Stata Press, 2013.

43.

Werner

Praxedes

Kim

. The reporting of nonresponse analyses in survey research. Organ Res Methods, (2007). 10, 287–295.

44.

Heckman

. Sample selection bias as a specification error. Econometrica 1979; 47: 153–161.

45.

Cuddeback

Wilson

Orme

, et al. Detecting and statistically correcting sample selection bias. J Soc Serv Res 2004; 30(3): 19–33.

46.

Kline

. Principles and practice of structural equation modeling. 2nd ed. New York: The Guilford Press, 2005.

47.

Podsakoff

Organ

. Self-reports in organizational research: problems and prospects. J Manag 1986; 12: 531–544.

48.

De Veer

AGE

Francke

. Attitudes of nursing staff towards electronic patient records: a questionnaire survey. Int J Nurs Stud 2010; 47: 846–854.

49.

Lin

Van Ryzin

. Web and mail surveys: an experimental comparison of methods for nonprofit research. Nonprof Volunt Sec Q 2012; 41(6): 1014–1028.

50.

Manfreda

Bosnjak

Berzelak

, et al. Web surveys versus other survey modes: a meta-analysis comparing response rates. Int J Market Res 2008; 50(1): 79–104.

51.

Teo

. Online and paper-based survey data: are they equivalent? Brit J Educ Technol 2013; 44(6): E196–E198.