Do Recipients of an Association-Sponsored Quality Award Program Experience Better Quality Outcomes Compared With Other Nursing Facilities Across the United States?

Abstract

Background: This study explores the relationship between recipients of an association-sponsored Quality Award Program and select measures of quality in U.S. nursing facilities, examined both cross-sectionally and over time. Method: Data used came from Quality Award Program recipients over 9 years (2003-2011) and a set of quality indicators from the Online Survey Certification and Reporting (OSCAR) database. These data were analyzed using a variety of multivariate regression techniques. Results: Cross-sectionally, the award recipients demonstrated higher performance on most of the quality indicators, compared with both the broader field of nursing facilities and also a more stringent comparison group of facilities. Comparing quality from the 4 years before the award was received to the 4 years after the award, the majority of quality indicators demonstrated improvement in those facilities receiving an award. Conclusion: These results indicate that, in general, the nursing facilities that participate in and receive the Silver or Gold American Health Care Association (AHCA) and National Center for Assisted Living (NCAL) National Quality Award perform at a higher level of quality for residents and sustain that heightened performance over a period of time.

Keywords

long-term care nursing facilities quality improvement quality award American Health Care Association National Center for Assisted Living Baldrige

Introduction

Health care has traditionally focused on what has become known as “little q” (quality) outcome measures (e.g., specific health department survey deficiencies), rather than “Big Q” organizational approaches (e.g., the Baldrige Criteria for Performance Excellence) to improving quality. This focus has produced some results but has had its own limitations as documented in numerous studies bemoaning the limited quality gains in health care (Institute of Medicine [IOM], 2001). Specifically, with respect to nursing facilities, “little q” initiatives have had center-stage (Castle & Ferguson, 2010). Quality has largely been externally driven by survey and compliance initiatives, as well as the remedies required by state survey agencies for deficient care. However, the Centers for Medicare & Medicaid Services (CMS) has attempted to raise the bar of quality by elevating transparency for consumers using “Big Q” approaches, such as publishing the results of the nationwide five-star ratings on their website, referred to as Nursing Home Compare (CMS, 2010).

Still, the belief exists that quality in the field has not been attended to or been driven by providers or their respective trade associations with “Big Q” initiatives (Castle & Ferguson, 2010). This is not necessarily true and is evidenced by the recent broad stakeholder collaborative, Advancing Excellence (www.nhqualitycampaign.org). This effort to improve quality has been a provider initiative that has produced a large amount of sharing of high-impact practices by long-term care organizations (such as consistent assignment practices and reduction of staff turnover; www.nhqualitycampaign.org). Another significant provider trade organization initiative that has been advanced over the past 20 years is the American Health Care Association and National Center for Assisted Living (AHCA/NCAL) National Quality Award Program, which is overseen by an independent board of overseers. The study presented here explores the relationship between the AHCA/NCAL National Quality Award Program recipients and select independent measures of quality of care collected by the government in U.S. nursing facilities. The analyses are first conducted cross-sectionally and, second, to determine whether any improvement and sustainability of performance in quality occurred; the analyses are also conducted over time.

Established in 1996, the AHCA/NCAL National Quality Award Program is based on the national Malcolm Baldrige Health Care Criteria for Performance Excellence. The Baldrige Program is a public–private partnership established by the Malcolm Baldrige National Quality Improvement Act of 1987. The Baldrige criteria contain eight categories: organizational profile; leadership; strategic planning; customer focus; measurement, analysis, and, knowledge management; workforce focus; operations focus; and results. These categories, and the performance standards within each, are meant to work in concert within a health care setting to establish higher organizational performance. The Baldrige criteria are updated every 2 years, and the Quality Award applications are updated accordingly to maintain relevance in the evolving business environment.

The Quality Award Program has three progressive application levels: Bronze, Silver, and Gold. The awards are progressive in that the Bronze award shows “Commitment to Quality,” the Silver award “Achievement in Quality,” and the Gold award “Excellence in Quality” (www.ahcancal.org/quality_improvement/quality_award/Pages/default.aspx). The Bronze level is based on the organizational profile of the Baldrige criteria, and applicants are assessed on their ability to describe the links between their mission, key customers, and strategic position. The Bronze award recognizes that a provider has developed a foundation to begin a journey of continuous quality improvement. Facilities receiving the Bronze award are also not using all of the Baldrige performance improvement tools and thus were not incorporated in this analysis. At the Silver level, applicants provide a thorough assessment of their systematic approaches to quality performance and deployment of these approaches. Applicants are also required to describe performance measures and sustainable organizational and process results that are linked to their key customer requirements, success factors, and challenges. At the Gold level, applicants respond to the full Health Care Criteria for Performance Excellence and must demonstrate that they are achieving high levels of performance in the areas of health care, customer satisfaction, finances, workforce, processes, and leadership. Gold- and Silver-level applicants are also required to show evidence of successful improvements resulting from their performance improvement system. Applications are reviewed by trained individual examiners at the Bronze level, a group of trained examiners at the Silver level, and a group of trained examiners and judges at the Gold level (www.ahcancal.org/quality_improvement/quality_award/Pages/default.aspx). At all three levels, applicants receive a feedback report that provides information on strengths and opportunities for improvement based on the information outlined in the application and serves as a resource for organizational quality improvement. The analyses presented here are based on Silver (n = 164) and Gold (n = 11) skilled nursing facility recipients from 2003 through 2011 (i.e., 175 recipients).

Literature Review and Hypotheses

The impact of external quality initiatives, such as those by the Joint Commission, has revealed some positive comparative results for those who participate in these types of organizational improvement programs (Wagner, McDonald, & Castle, 2012b) in the skilled nursing facility field There is also a general movement and acceptance of broader quality approaches in nursing facilities, encouraging internal organizational commitment evidenced by the recent Quality Assurance and Performance Improvement (QAPI) initiative advanced by the CMS (Smith, Castle, & Hyer, 2013).

No previous external quantitative studies have been conducted on the performance of the Silver and Gold Quality Award recipients of the Quality Award Program or the AHCA/NCAL National Quality Award criteria. However, the Malcolm Baldrige National Quality Award (MBNQA) is applicable to any industry. As such, there is research that the MBNQA is effective overall in other sectors (Link & Scott, 2011) and is an effective performance improvement tool (Evans & Jack, 2003; Ford & Evans, 2001).

Link and Scott (2011) examined the benefits received through the MBNQA application process. They used a web-based survey for all 273 MBNQA applicants from 2007 through 2010. A total of 45 firms and organizations responded. They identified cost savings of the applicants because of performance excellence and gains to consumers from greater satisfaction from higher quality products (Link & Scott, 2011). Ford and Evans (2001) reviewed the literature on organizational change, assessment, and learning and concluded that the self-assessment used in the Malcolm Baldrige Award Criteria for Performance Excellence is effective. Using a sample (n = 1,469) of directors from manufacturing firms, Evans and Jack (2003) validated many of the key linkages used in the Baldrige model (e.g., they discuss the link between employee satisfaction and performance and the link between work system improvement and productivity).

Analyses of the Baldrige process indicate that leadership drives the systems of an organization which can then create positive results. Research in 2001, by Meyer and Collier on the pilot health care criteria, provides validation of this theory. Their research demonstrates that improvements in leadership lead to improvements in the system categories of the Baldrige criteria (Foster & Chenoweth, 2011).

A 2011 study found that hospitals that have achieved the National Baldrige Award are more likely to receive the 100 Top Hospitals recognition than their peers (Meyer & Collier, 2001). The top 100 hospitals are selected based on a quantitative analysis that assesses performance on quality, efficiency, financial considerations, and consumer assessment of care. In addition, Baldrige hospitals (those that achieve the Award or those that receive a site visit) are more likely to display faster 5-year performance improvement. The study also showed that Baldrige hospitals outperformed peers on a risk-adjusted mortality index, risk-adjusted complications index, patient safety index, CMS core measure scores, severity-adjusted average length of stay, and adjusted operating profit margin (Meyer & Collier, 2001). Leading up to receiving the Baldrige Award, a number of health care organizations reported decreasing mortality rates, increased customer satisfaction, and decreasing employee turnover (Shook & Chenoweth, 2012).

Holding the above evidence in consideration, we hypothesize that those nursing facilities that achieve the Silver- and Gold-level awards from the AHCA/NCAL National Quality Award Program, having implemented the Baldrige criteria, are more likely to have superior performance outcomes than those that do not. Therefore, the first hypothesis (H1) is that Silver and Gold recipients will have better performance with respect to quality of care than other organizations. We assert that nursing facilities applying “Big Q” Baldrige principles will benefit from better organizational outcomes, including those reflective of quality of care to their residents.

A second hypothesis (H2) is that these same award recipients will also have better quality of care performance than a more stringent national control group defined as facilities that have not experienced seven or more deficiency citations (described below) in any year.

A third hypothesis (H3) is that these same Silver and Gold recipients sustain their quality of care performance over a period of time after receiving the award. This is a critical element for our study, based on the need to prove or disprove that these organizations can maintain the actual outcomes (i.e., quality) reported over an extended period of time.

Method

The analysis of the performance of AHCA/NCAL National Quality Award recipients was conducted using data from a 9-year time period. Over this period, the Quality Award Program received 9,856 applications and awarded Quality Awards to 4,157 organizations. The analyses presented here are based on Silver (n = 164) and Gold (n = 11) skilled nursing facility recipients from 2003 through 2011 (i.e., 175 recipients). The success rate in those years was 50% at the Bronze level, 13% at the Silver level, and 8% at the Gold level. The remaining 3,975 nursing facilities were Bronze-level recipients. The analyses consisted of three steps and were based on the above hypotheses.

The first analysis was a cross-sectional analysis using the award group of Silver and Gold recipients compared with all other skilled nursing facilities. The second analysis used a comparison group of facilities of higher quality (i.e., nursing facilities without seven or more deficiency citations at any level in a year), which eliminated the poorest performing facilities and made the groups in the study more similar to the Silver/Gold recipient facilities. The third analysis used this same higher quality comparison group and examined intervals before and after the award, which helped determine improvement and sustainability of performance with respect to quality of care over time.

Data

Primary Data

AHCA member status, with respect to Silver- and Gold-level award recipients, was identified from the 2011 AHCA member master list. Information spanned a 9-year period (i.e., 2003 through 2011) and included the date of the award. The date of the first award was used for each awardee in the analysis. A facility identification number was included with the AHCA member master list, such that the information could be matched with the Online Survey Certification and Reporting (OSCAR) data set, described below.

From the file of 182 Gold and Silver awards that was provided, five records were dropped for facilities for which there was no OSCAR record with the same provider number, and two records with missing provider numbers were dropped. The remaining 175 award records were used in analyses.

Secondary Data

OSCAR

The OSCAR data set contains data collected as part of annual (i.e., every 9-15 months) state/federal nursing facility inspections to ensure quality of care for residents. Other studies provide extensive detail regarding how and why these data are collected, and comprehensively list the data elements (Office of Inspector General [OIG], 2011). The current state/federal nursing facility inspection process (used as part of certification) is described in detail by the CMS (www.cms.hhs.gov).

In short, nursing facilities that accept residents with Medicare and/or Medicaid payments are included in these state/federal nursing facility inspections (often generically termed the survey process). This includes most (i.e., approximately 97% [roughly 15,000 facilities]) of the nursing facilities in the United States (OIG, 2011). The survey process includes the recording of many characteristics of the facility (e.g., profit status, number of beds) and aggregate characteristics of residents (e.g., number with dementia). Facility characteristics (e.g., number of beds, ownership, chain membership, occupancy, Medicaid occupancy, and three measures of staffing) and resident-aggregated characteristics (e.g., number of residents with limitations in activities of daily living [ADLs], psychiatric conditions, and dementia) in this analysis were used from the OSCAR data set. The data are commonly used in this way as a secondary source of facility/resident characteristics, and many of these characteristics are considered to be accurate and reliable (Hyer et al., 2011; Kash, Hawes, & Phillips, 2007; Zhang, Paek, & Wan, 2009).

When a nursing facility is determined not to meet minimum standards included in the state/federal nursing facility inspections, a deficiency citation is issued (CMS, 2008). These deficiency citations are also used for analyses in this study (and are described further below). In addition, the OSCAR data set indicates, for each citation issued, the scope and severity of harm. Scope generally refers to the number of residents affected and severity to the level of harm deficient practices pose to residents, with a letter value assigned (i.e., “A” to “L,” with letters J, K, and L representing immediate jeopardy to resident health or safety).

Area Resource File (ARF)

A limited number of variables were used from the ARF. This is a public data source with data available at the county level. Per capita income in the market and the number of elderly in the market came from the ARF (www.arfsys.com). These characteristics are often used as independent variables when examining nursing facility quality (e.g., Hyer et al., 2011; Kash et al., 2007).

Dependent Variables

Ten outcome characteristics representing quality of care were used in the analyses as dependent variables: (a) all deficiency citations, (b) quality of care deficiency citations, (c) J,K,L-level deficiency citations, (d) physical restraint use, (e) pressure ulcers, (f) catheter use, (g) antipsychotic medication use, (h) antidepressant medication use, (i) antianxiety medication use, and (j) use of hypnotic medications.

Approximately 180 deficiency citations exist. One of the analyses in this study uses a simple count of these deficiency citations. The deficiency citations are also grouped into 16 categories, which group like areas together. These categories were developed by the CMS and have considerable face validity although one limitation of using these categories is that they were not defined using empirical estimation (such as factor analysis). One category groups together 25 quality of care deficiency citations. A count of these specific quality of care citations was used in the analyses. In addition, a count of the highest scope and severity deficiency citations (i.e., J, K, and L) was used in analyses for this study.

Physical restraint use is an important quality indicator because restraint use is associated with an increased risk of morbidity and mortality in nursing facility residents (Castle & Engberg, 2005). Pressure ulcers affect both the comfort and the medical outcomes of nursing facility residents with impaired mobility (Brandeis, Ooi, Hossain, Morris, & Lipsitz, 1994). In general, high catheterization rates imply lower quality of resident care (Outslander & Kane, 1984). More details of indwelling catheter use as a quality measure can be found in the review by Schnelle and Smith (2001). Inappropriate use of medications is a key measure of nursing facility quality, and evidence exists that some nursing facilities inappropriately use medications—especially psychoactive medications (OIG, 2011; Perri, 2005). Psychoactive medications are generally classified into four categories: antianxiety, sedative/hypnotic, antipsychotic, and antidepressant. Each is included in the analyses. Thus, in general, for the outcomes examined, lower levels of medication use are regarded as beneficial.

All of these outcome variables were chosen based on their availability in the data and their common use for examining the quality of nursing facilities (Castle & Ferguson, 2010). In the analyses, measures representing the percent of residents with physical restraint use, pressure ulcers, catheter use, antipsychotic medication use, antidepressant medication use, antianxiety medication use, and use of hypnotic medications were used.

Independent Variables

The independent variable of interest was Quality Award Program recipient status. For all of the analyses, Gold and Silver award recipient nursing facilities were coded as 1, whereas other facilities were coded as 0.

Additional independent variables included in the analyses were derived from the prior research in this area that has examined nursing facility quality (e.g., Wagner, McDonald, & Castle, 2012a). Specifically, facility variables included in this research are number of beds, ownership, chain membership, occupancy rate, Medicaid occupancy, and three nurse staffing measures. Examples of prior research examining quality including these factors include Hyer et al. (2011), Hyer et al. (2013), and Wagner and Castle (2012). Resident variables included in this research are resident case-mix, residents with psychiatric conditions, and residents with dementia, as supported by prior research examining quality and including these factors (e.g., Hurtado, Berkman, Buxton, & Okechukwu, 2008; Hyer et al., 2011). Variables representing the operating environment included the Herfindahl index, elderly per square mile (in county), and per capita income (in county). These factors have also been included in prior research examining quality, including Donoghue (2010), Hyer et al. (2011), and Hyer et al. (2013).

The number of nursing facility beds was used as a measure of size. Two classes of facility ownership were used (for-profit and not-for-profit), and two classes of multifacility corporation membership were used (chain or non-chain). The overall occupancy rate is the percent of beds occupied by residents. The percent of residents paid for by Medicaid was used as a measure of Medicaid resident occupancy. Three different types of staffing measures were included in the analyses: the number (measured as full-time equivalent [FTE]) of registered nurses (RNs), licensed practical/vocational nurses (LPNs/LVNs), and nurse aides per resident.

An average ADL score previously used by others (e.g., Wagner & Castle, 2012) was used to represent resident case-mix. For each of three ADL questions (i.e., eating, toileting, and transferring) in the OSCAR data, a score from 0 to 2 was used: no assistance (0), moderate need for assistance (1), and high degree of need for assistance (2), respectively. We then calculated these scores, with higher scores indicating a greater average ADL impairment within the facility. The numbers of residents per facility with psychiatric conditions and dementia were used to calculate the percent of residents in each facility with these conditions.

The Herfindahl index (also known as Herfindahl–Hirschman index, or HHI) was used in the analyses as a measure of local nursing facility market competition. The county was the market area used in this analysis. This index is calculated by taking each nursing facility’s percentage share of beds in the county and dividing that by the squared market shares of all nursing facilities in the county (Department of Justice; www.justice.gov). The Herfindahl index ranges from close to 0 to 10,000. The number of elderly in a market represents a count of those aged 65 and older per 1,000 of the total population in each county, and per capita income in the market represents the average income for all residents in a county.

Analyses

Descriptive statistics (means and standard deviations) for the quality indicators and independent variables (means, standard deviations, and percent) are presented. We examined the level of collinearity among the independent variables, and multicollinearity, by using the variance inflation factor (VIF) test (Fox & Monette, 1992). The correlation between the variables was generally low.

A comparison group was identified that did not have seven or more deficiency citations (at any scope and severity level for any deficiency citation) in any year, which made the groups in the study more similar. Seven or more deficiency citations were used as this represents the average number of citations received. In essence, this raised the bar for the comparison group in the analyses, which was also a more appropriate standard to compare with the AHCA award recipients. The comparison group included both nursing facilities that have never applied for the AHCA/NCAL awards and nursing facilities that have received the Bronze award as well. Silver and Gold applicants are not eligible to receive the award if the average score for their three most recent surveys exceeds their state’s average deficiency score for the past 3 years or if they have received an immediate jeopardy or substandard quality of care citation in the past 3 years. Bronze applicants who have received an immediate jeopardy or substandard quality of care citation in the past 3 years are also not eligible to receive the award. Small facilities (<40 beds) and large facilities (>800 beds) were also excluded. These exclusions are common when examining quality of care, as both small and large facilities are considered unrepresentative of nursing facilities in general (Donoghue, 2010).

The regression analyses included logistic regression, binomial regression, and ordinary least squares (OLS) regression. That is, a logit was used for the binary outcomes of receiving any deficiency citation and deficiency citations for quality of care (adjusted odds ratios [AORs] are provided).

For many nursing facilities in the sample, the counts for the J,K,L citations were low (i.e., they are Poisson overdispersed). Negative binomial regression is a standard method commonly used to examine overdispersed Poisson data (Gardner, Mulvey, & Shaw, 1995). Thus, negative binomial regression (Gardner et al., 1995) was used in multivariate analyses for the outcome of number of J,K,L citations. The negative binomial coefficients are reported in incident rate ratio (IRR) form. An IRR is similar to an odds ratio; that is, estimates greater than one represent a positive association between the independent variable and the outcome. In our case, high values of the quality indicators are indicative of poor quality, and thus, coefficients less than one are representative of better quality.

OLS regression was used for the remaining outcomes of physical restraint use, pressure ulcers, catheter use, antipsychotic medication use, antidepressant medication use, antianxiety medication use, and use of hypnotic medications.

The potential for correlation of outcomes within markets exits with the data structure, which can bias the standard errors of the estimates. Thus, the Huber–White sandwich estimator, clustered by market (i.e., the county), was used for all the multivariate analyses (Zeger & Liang, 1992). Indicators for each state were also included in the analyses.

Several analyses were used to examine the association of Silver and Gold award recipients to quality. First, the cross-sectional association of Silver and Gold award recipients and quality is examined. One limitation of this approach is that better nursing facilities may become Silver and Gold recipients. Thus, to overcome this limitation in the second analysis, we examine time trends. Dummy variables were created for the following years: 4 or more years prior to award, 3 years prior to award, 2 years prior to award, 1 year prior to award, year of award, 1 year after award, 2 years after award, 3 years after award, and 4 years after award. That is, when a facility first received a Silver or Gold award from 2003 to 2011, this was coded as the year of the award, and then dependent and independent variables (listed above) from the OSCAR were used from each of the prior 4 years to the award and each of the 4 years after the award. This analysis gives some information on the impact of Silver- and Gold-level recipient status over time.

To parsimoniously show the findings, the coefficients on these dummies are plotted for each outcome and an indication of statistical significance is added to the figures (Figure 1). For these figures, the x-axis in the graphs indicates −4 years before award, −3 years before award, −2 years before award, −1 year before award, 0 is award year, and so on.

Figure 1.

Regression results examining quality indicators and award status over time.

Results

Table 1 shows the dependent and independent variables used in the analyses. For example, the awardees had an average of 2.77 deficiency citations, whereas the non-awardees (all other facilities) had an average of 4.14 deficiency citations.

Table 1.

Comparison of Award Status of Nursing Facilities With Dependent and Independent Variables Used in Analyses.

Variable categories	Award status
	No award; all other nursing facilities (n = 15,644)		No award; nursing facilities with seven or less deficiency citations (n = 7,372)		Gold or Silver award (n = 175)
	M	SD	M	SD	M	SD
Dependent variables
Count of any deficiency citation	6.14	5.19	4.14	5.19	2.77	3.50
Count of deficiency citation for quality of care	2.46	1.68	1.40	1.52	0.82	1.10
Count of any J,K,L deficiency citation	0.04	0.21	0.03	0.16	0.00	0.07
Percent of residents with physical restraint use	3.0	6.1	3.1	6.1	2.0	4.4
Percent of residents with pressure ulcers	7.2	5.4	6.6	5.3	6.5	4.1
Percent of residents with catheters	6.1	5.3	6.2	6.0	6.2	4.5
Percent of residents taking antipsychotic medications	23.3	11.8	25.5	15.2	22.0	11.1
Percent of residents taking antidepressant medications	48.3	14.1	49.3	14.5	52.8	11.3
Percent of residents taking antianxiety medications	23.4	11.4	23.2	11.7	21.4	10.2
Percent of residents taking hypnotic medications	8.0	7.2	8.8	8.5	6.0	6.3
Independent variables
Number of beds	103.15	51.12	108.11	63.96	117.92	59.61
For-profit (= 1)	0.76	0.43	0.69	0.46	0.79	0.41
Chain membership (= 1)	0.54	0.50	0.54	0.50	0.81	0.40
Occupancy rate percent	81.3	14.1	81.6	16.6	89.8	9.1
Percent of Medicaid residents in facility	63.4	21.2	60.7	24.5	59.3	17.2
FTE RNs per resident	0.06	0.05	0.07	0.09	0.08	0.05
FTE LPNs/LVNs per resident	0.13	0.07	0.14	0.08	0.11	0.06
FTE nurse aides per resident	0.37	0.14	0.37	0.15	0.34	0.13
Average ADL score	1.20	1.11	1.20	1.12	1.18	1.11
Percent of residents with a psychiatric condition	27.2	19.4	26.5	19.1	24.4	15.1
Percent of residents with dementia	47.5	18.2	47.2	19.6	47.5	17.2
Herfindahl index (in county)	2,221	2,514	2,221	2,514	1,890	2,271
Number of elderly per 1,000 of the total population (in county)	121	450	146	439	105	177
Average US$ per capita income (in county)	27,566	8,142	27,513	8,119	28,067	7,622

Note. FTE = full-time equivalent; RN = registered nurse; LPN = licensed practical nurse; LVN = licensed vocational nurse; ADL = activity of daily living.

Table 2 shows the regression results examining the quality indicators and award status. Only the parameter estimates for award status are provided; however, all of the regressions included all independent variables listed in Table 1. The first column shows the cross-sectional association of awardees compared with non-awardees (i.e., all other facilities). This shows that awardees demonstrated significantly higher (i.e., better) quality on most of the measures compared with non-awardees (except catheter usage). That is, of the 10 measures examined, nine were of significantly higher (i.e., better) quality for awardees compared with non-awardees. For example, Silver and Gold award recipients are less likely to receive any deficiency citation (AOR = 0.65, p ≤ .001) or deficiency citations for quality of care (AOR = 0.57, p ≤ .001). Thus, we find support for the first hypothesis that Silver and Gold award recipients will have better performance with respect to quality of care than other organizations.

Table 2.

Regression Results Examining Quality Indicators and Cross-Sectional Association With Silver and Gold Award Status.

Dependent variables	Cross-sectional association of Gold or Silver Award (n = 175) recipients compared with all other facilities (n = 15,644)	Cross-sectional association of Gold or Silver Award (n = 175) recipients compared with seven or less deficiency citations (n = 7,372)
Dependent variables	Adjusted OR (95% CI)	Adjusted OR (95% CI)
Any deficiency citation	0.65*** [0.52, 0.93]	0.75*** [0.53, 0.92]
Deficiency citations for quality of care	0.57*** [0.41, 0.90]	0.72*** [0.59, 0.91]
	Incident rate ratio (95% CI)	Incident rate ratio (95% CI)
Any J,K,L deficiency citation	0.25*** [0.20, 0.83]	0.88*** [0.71, 0.93]
	Coefficient (SE)	Coefficient (SE)
Physical restraint use	−0.08** (0.03)	−0.06** (0.02)
Pressure ulcers	−0.06* (0.03)	−0.06* (0.03)
Catheterized	0.02 (0.02)	0.03 (0.02)
Antipsychotic medications	−0.14** (0.06)	−0.11* (0.05)
Antidepressant medications	−0.29*** (0.09)	−0.30** (0.07)
Antianxiety medications	−0.07* (0.04)	−0.10 (0.05)
Hypnotic medications	−0.13** (0.05)	−0.11* (0.04)

Note. Independent variables included in all regression analyses (not shown above): number of beds, for-profit, chain member, average occupancy rate, Medicaid occupancy, FTE RNs per resident, FTE LPNs/LVNs per resident, FTE NAs per resident, case-mix (ADLs), psychiatric conditions, dementia, competition (Herfindahl index), number of elderly per 1,000 of the total population (in county), and average US$ per capita income (in county). OR = odds ratio; CI = confidence Interval; FTE = full-time equivalent; RN = registered nurse; LPN = licensed practical nurse; LVN = licensed vocational nurse; NA = nurse aide; ADL = activity of daily living.

Statistically significant at p ≤ .05 level or better. **Statistically significant at p ≤ .01 level or better. ***Statistically significant at p ≤ .001 level or better.

The second hypothesis was tested using a more stringent comparison group. The second column of Table 2 shows the cross-sectional association of awardees compared with non-awardees (i.e., facilities with seven or less deficiency citations). This shows that awardees demonstrate significantly higher (i.e., better) quality on most of the measures (except catheter usage and antianxiety medications) compared with non-awardees. That is, of the 10 measures examined, eight were of significantly higher (i.e., better) quality for awardees compared with non-awardees. For example, Silver and Gold award recipients are less likely to receive any deficiency citation (AOR = 0.75, p ≤ .001) or deficiency citations for quality of care (AOR = 0.72, p ≤ .001). Thus, we find support for the second hypothesis that the award recipients will also have better performance with respect to quality of care than a more stringent national control group that has not experienced seven or more deficiencies.

Time trends were examined to determine whether awardees continue to improve their quality indicators. This analysis gives some information on the impact of Silver and Gold award status over time. For the first panel of figures, for example, the y-axis represents the likelihood of the number of deficiency citations received in the year being less than the comparison group. For any deficiency citation, award recipients are better than the comparison facilities and remain better over time. Examining the other panels, we find, for the count of quality of care deficiency citations specifically, awardees had better quality of care than the comparison facilities and remained better over time. Going from left to right on the panel of figures, we find that for the J,K,L deficiency citations, awardees are better than the comparison facilities and also perform better over time on this measure. For the percent of residents physically restrained, awardees are better than the comparison facilities and remain better over time. For the percent pressure ulcers, awardees are better than the comparison facilities and remain better over time. For the percent catheterized, awardees were not better than the comparison facilities, but this is not statistically significant. For the percent of residents receiving antipsychotic medications, awardees were not better than comparison facilities (this is also not statistically significant). For the percent hypnotic medications, awardees are not better than comparison facilities (but this is not statistically significant). For the percent antidepressant medications, awardees are better than the comparison facilities and remain better over time. For the percent antianxiety medications, awardees are not better than comparison facilities (but this is not statistically significant). Thus, we find partial support for the third hypothesis that the Silver and Gold recipients sustain their performance with respect to quality of care over a period of time after receiving the award.

Discussion

Following H1 (Silver and Gold recipients will have better performance with respect to quality of care than other organizations), the analyses show that Silver and Gold recipients have better performance than other organizations with respect to the 10 quality indicators examined. Following H2 (award recipients will also have better quality of care performance than a more stringent national control group), the analyses show that Quality Award recipients also have better performance than a more stringent national control group, which has not experienced seven or more deficiencies, with respect to the 10 quality indicators examined. In addition, following H3, the Silver and Gold recipients seem to sustain their performance over a period of time after receiving the award. That is, with respect to six of the 10 quality indicators examined, awardees sustained their performance for higher quality of care over time.

In these analyses, we cannot determine with specificity how Silver and Gold award recipients achieve better performance. However, we speculate that inadequate assessments by staff at critical points in resident care (a known shortfall of nursing facilities; Wagner & Castle, 2012) may influence Silver and Gold award recipients achieving better performance. The Baldrige criteria (i.e., customer focus, measurement) emphasize the importance of conducting resident assessments. As such, this may be one reason why Silver and Gold award recipients achieve better performance.

In addition, we speculate that enhanced communication practices between staff (e.g., effective documentation practices) may be one reason Silver and Gold recipients achieve better performance. Again, Baldrige criteria (i.e., knowledge management, workforce focus) emphasize resident communication practices. This may be another mechanism by which Silver and Gold recipients achieve better performance.

Research has identified leadership as a critical element in the performance of nursing facilities. For example, Dana and Olson (2007) discussed and reviewed the importance and uniqueness of the significant role of leadership in long-term care. The influence of leadership on quality systems was explored by Olson, Dana, and Ojibway (2005) in a review of Silver and Gold applicants and structured interviews that resulted in seven common themes embodied by these high-performance organizations. In addition, Castle and associates have identified leadership as highly influential on the care nursing facility residents receive (Castle, Ferguson, & Hughes, 2009). The Baldrige criteria (i.e., leadership) emphasize this as a critical element; the leadership category focuses on how the leaders lead the organization and serve as leaders for the community. This may be yet another mechanism by which Silver and Gold recipients achieve better performance.

Some authors have noted that the quality of care in some nursing facilities remains low (Castle & Ferguson, 2010). Moreover, federal investigations (e.g., IOM, OIG) continue to document quality problems (OIG, 2011). Quality problems may not be emblematic of all nursing facilities, but a challenge some nursing facilities face is how to improve quality of care (Gaugler, 2016). Silver and Gold award recipients, using Baldrige, have potentially overcome this quality challenge. Thus, Baldrige principles (i.e., programs/initiatives) may be useful for nursing facilities.

Nevertheless, very few nursing facilities have successfully incorporated Baldrige principles (Smith et al., 2013). This may be an endemic quality issue for the industry, as nursing facilities lag behind acute care providers in that many have been unable to incorporate basic quality improvement procedures into their routine operations (Smith et al., 2013). One reason nursing facilities may be reticent to incorporate even basic quality improvement techniques is that the return on investment may appear to be unclear. Provider organizations invest substantial resources and time to receive levels of recognition for achieving quality as deemed by this (and other) award programs. Examining the benefits from these investments may be useful. One potential benefit that could promote investment in quality improvement techniques could come from survey and compliance recognition.

Government agencies tasked with survey and compliance (and themselves already stressed with resource issues) may consider taking the recognition of Quality Award recipients in mind when assigning and scheduling survey teams in the field. Currently, nursing facilities that are determined to have a greater number of quality problems, more serious problems than average, and a demonstrated pattern of quality problems are included in the Special Focus Facility (SFF) initiative (CMS, 2008). For nursing facilities, inclusion in the SFF program entails having two survey inspections per year (rather than the standard one survey) and the potential to be terminated from the Medicare and/or Medicaid programs. The reverse approach may also be used, where nursing facilities with demonstrated excellent performance could be surveyed at a longer time interval.

Initiatives such as pay for performance (P4P), also known as value-based purchasing (VBP), are gaining the attention of policy makers as a market-based option to promote quality improvements in nursing facilities (Kane, Arling, Mueller, Held, & Cooke, 2007). These may also promote the use of quality improvement techniques by nursing facilities. The benefits from this (and other) award programs may then be more credible.

The actions discussed above would all be positive incentives for the participating organizations to continue attaining higher quality and investing the necessary internal resources and time to advance that agenda. Considering the evolving public and private partnerships being advanced with Accountable Care Organizations (ACOs), preferred provider relationships, and health care reform, this movement would seem to be a nice adjuvant to both promoting quality of care and contracting arrangements. Contracts are highly dependent upon quality metrics. For example, five-star ratings as part of Nursing Home Compare are often used. Given that that Silver and Gold recipients of the AHCA/NCAL National Quality Award Program have better performance than other organizations, this may also be a viable and simple metric for contracting purposes.

Limitations and Future Research

There are some limitations to the current study. First, the AHCA/NCAL National Quality Award Program is sponsored by and limited to members of one trade organization, although we deliberately used an industry-wide comparison group to aid in mitigating this particular issue. Second, the use of cross-sectional data has the challenge of being a snapshot of performance at one point in time, and once again this research study attempted to mitigate this challenge with the use of longitudinal data used as part of the study.

As noted, Bronze-level recipients were viewed as only beginning the quality journey and were not incorporated in this analysis. Sensitivity analyses (not shown) verified this rationale. That is, these Bronze-level recipients were not significantly different from the nursing facilities with no awards, with respect to quality indicators such as the average number of deficiency citations.

The data had a few records with missing provider numbers (n = 14) and were dropped. This was unlikely to influence the findings, as the findings show that Silver and Gold recipients in general had better quality performance. These providers were included in the comparison group; thus, the bias would make the findings more conservative.

Several quality indicators are used in the analyses; however, these do not necessarily provide a comprehensive picture of nursing facility quality. Measures included as part of the public Nursing Home Compare analyses could be used in the future. Although similar to the OSCAR measures used here, these are primarily clinical measures of quality. Many other quality indicators exist, including quality of life (QoL) measures and measures of resident satisfaction. Reflecting the orthogonality of quality indicators, these may not necessarily follow the same patterns of findings as the clinical quality indicators examined.

As noted previously, the customer focus as part of Baldrige, in particular, may have several benefits for Silver and Gold award recipients. Resident satisfaction scores are likely to be enhanced. This could be examined in the future. Similarly, residents’ QoL may be enhanced. In turn, facilities may experience fewer QoL deficiency citations. This was not included as part of this analysis but could be incorporated into follow-up research.

Finally, the use of the OSCAR data has some described limitations and was replaced by a new data collection system, called Certification and Survey Provider Enhanced Reporting (CASPER), in 2012. Even with this recent development, the OSCAR data are still being widely used by the research community as a robust proxy measure for quality outcomes for the time period associated with this study.

Conclusion

According to the AHCA (2014), more than 1.3 million elderly and disabled individuals reside in nursing facilities. Nursing facilities that are working on improving quality of care and those utilizing the Baldrige framework can successfully attain their quality improvement goals to deliver excellent care to residents. We are in need of these and multiple other initiatives, such as the AHCA/NCAL National Quality Award Program, to improve the quality of nursing facilities throughout the country.

The overall results of this study provide some strong evidence that an internal association-sponsored quality award program, based on Baldrige concepts, can be used to signify an organization that is performing at a higher level of quality. In a largely reactive field that has limited financial and human resources to both provide and monitor services that take care of people, this research provides information to policy makers and the general public on additional indicators of quality which suggest that long-term care facilities are providing a better quality of care to vulnerable nursing facility residents.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Nicholas Castle, PhD, Health Policy and Administration, Penn State University, 1994, is a professor in the Department of Health Policy and Management, University of Pittsburgh. He currently has more than 200 publications in peer-reviewed journals. He serves on five journal editorial boards, including The Gerontologist, and is a fellow of the Gerontological Society of America.

Doug Olson, PhD, Health Services, Research, Policy and Administration, University of Minnesota, 2000, is a professor in the Department of Management and Marketing at the University of Wisconsin–Eau Claire. He also serves as the director of the Center for Health Administration and Aging Services Excellence at his university. He has published in variety of peer-reviewed and trade journals, presents regularly at both state and national meetings, serves on a variety of academic and national boards, and has many years of health care administrative experience prior to his academic career.

Urvi Shah, MPH, University of Michigan is the senior quality improvement manager for the American Health Care Association (AHCA). In her role at AHCA, she oversees the Bronze Quality Award, the first step in a progressive three-step criteria-based program that recognizes members for their improvements in quality. She also serves as the association’s liaison to the National Quality Forum and has recently managed the endorsement of the PointRight® Pro-30™ readmission measure. In the past 2 years, she has authored the annual AHCA Quality Report and has recently become the staff liaison for the workforce committee.

Kevin Hansen, PhD, Aging Studies, University of South Florida; JD, William Mitchell College of Law, is an assistant professor of health care administration at the University of Wisconsin–Eau Claire, and he helps lead the National Emerging Leadership Summit for Health Care Administrators each year. He is a member of the Gerontological Society of America and the American College of Health Care Administrators, and has served as a reviewer for the Journal of Applied Gerontology.

References

American Health Care Association. (2014). Characteristics, utilization and government funding: 2014 update. Washington, DC: American Health Care Association. Retrieved from http://www.ahcancal.org/research_data/trends_statistics/Documents/ST_rpt_STStats2011_20110906_FINAL_web.pdf

Brandeis

G. H.

Ooi

W. L.

Hossain

Morris

J. N.

Lipsitz

L. A.

(1994). A longitudinal study of risk factors associated with the formation of pressure ulcers in nursing homes. Journal of the American Geriatrics Society, 42, 388-393.

Castle

N. G.

Engberg

(2005). Staff turnover and quality of care in nursing homes. Medical Care, 43, 616-626.

Castle

N. G.

Ferguson

Hughes

(2009). Humanism in nursing homes: The impact of top management. Journal of Health and Human Services Administration, 31(4), 483-508.

Castle

N. G.

Ferguson

J. C.

(2010). What is nursing home quality and how is it measured? The Gerontologist, 50, 426-442.

Centers for Medicare & Medicaid Services. (2008). Action plan for (further improvement of) nursing home quality. Retrieved from http://www.cms.hhs.gov/certificationandcomplianc/downloads/sfflist.pdf

Centers for Medicare & Medicaid Services. (2010). Design for Nursing Home Compare five-star quality rating system: Technical users’ guide. Retrieved from https://www.cms.gov/CertificationandComplianc/Downloads/usersguide.pdf

Dana

Olson

(2007). Effective leadership in long term care (American College of Health Care Administrators Position Paper). Retrieved from http://www.achca.org/assets/docs/ACHCA_Leadership_Need_and_Opportunity_Paper_Dana-Olson.pdf

Donoghue

(2010). Nursing home staff turnover and retention: An analysis of national level data. Journal of Applied Gerontology, 29, 89-106.

10.

Evans

Jack

(2003). Validating key linkages in the Baldrige Performance Excellence model. Quality Management Journal, 10, 7-24.

11.

Ford

Evans

(2001). Baldrige assessment and organizational learning: The need for change management. Quality Management Journal, 8, 9-25.

12.

Foster

D. A.

Chenoweth

(2011). Comparison of Baldrige Award applicants and recipients with peer hospitals on a national balanced scorecard. Thomson Reuters. http://www.nist.gov/baldrige/upload/baldrige-hospital-research-paper.pdf

13.

Fox

Monette

(1992). Generalized collinearity diagnostics. Journal of the American Statistical Association, 87, 178-183.

14.

Gardner

Mulvey

Shaw

(1995). Regression analyses of counts and rates: Poisson, overdispersed Poisson, and negative binomial models. Psychological Bulletin, 118, 392-404.

15.

Gaugler

J. E.

(2016). Understanding quality of care in nursing homes and other residential settings. Journal of Applied Gerontology, 35, 263-266.

16.

Hurtado

D. A.

Berkman

L. F.

Buxton

O. M.

Okechukwu

C. A.

(2008). Schedule control and nursing home quality: Exploratory evidence of a psychosocial predictor of resident care. Journal of Applied Gerontology, 27, 375-405.

17.

Hyer

Thomas

K. S.

Branch

L. G.

Harman

J. S.

Johnson

C. E.

Weech-Maldonado

(2011). The influence of nurse staffing levels on quality of care in nursing homes. The Gerontologist, 51, 610-616.

18.

Hyer

Thomas

K. S.

Harman

Johnson

C. E.

Weech-Maldonado

(2013). Do Medicaid incentive payments boost quality? Florida’s direct care staffing adjustment program. Journal of Aging & Social Policy, 25, 65-82.

19.

Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academies Press.

20.

Kane

R. L.

Arling

Mueller

Held

Cooke

(2007). A quality-based payment strategy for nursing home care in Minnesota. The Gerontologist, 47, 108-115.

21.

Kash

B. A.

Hawes

Phillips

C. D.

(2007). Comparing staffing levels in the Online Survey Certification and Reporting (OSCAR) system with the Medicaid Cost Report data: Are differences systematic? The Gerontologist, 47, 480-489.

22.

Link

A. N.

Scott

J. T.

(2011, December). Economic evaluation of the Baldrige Performance Excellence Program. Retrieved from http://www.nist.gov/baldrige/publications/economic_evaluation_2011.cfm

23.

Meyer

S. M.

Collier

D. A.

(2001). An empirical test of the causal relationships in the Baldrige Health Care Pilot Criteria. Journal of Operations Management, 19, 403-426.

24.

Office of Inspector General. (2011). Medicare atypical antipsychotic drug claims for elderly nursing home residents. Washington, DC: Department of Health and Human Services.

25.

Olson

Dana

Ojibway

(2005, April). Mapping the road to quality results. Provider, pp. 69-71.

26.

Outslander

Kane

(1984). The cost of urinary incontinence in nursing homes. Medical Care, 22, 69-79.

27.

Perri

(2005). Adverse outcomes associated with inappropriate drug use in nursing homes. Annals of Pharmacotherapy, 39, 405-411.

28.

Schnelle

J. F.

Smith

R. L.

(2001). Quality indicators for the management of urinary incontinence in vulnerable community-dwelling elders. Annals of Internal Medicine, 135(8, Pt. 2), 752-758.

29.

Shook

Chenoweth

(2012, October). 100 Top Hospitals CEO insights: Adoption rates of select Baldrige award practices and processes. Truven Health Analytics. Retrieved from http://www.nist.gov/baldrige/upload/100-Top-Hosp-CEO-Insights-RB-final.pdf

30.

Smith

Castle

N. G.

Hyer

(2013). Implementation of quality assurance and performance improvement programs in nursing homes: A brief report. Journal of the American Medical Directors Association, 14, 60-61.

31.

Wagner

McDonald

Castle

N. G.

(2012a). Impact of voluntary accreditation on deficiency citations in U.S. nursing homes. The Gerontologist, 52, 561-570.

32.

Wagner

McDonald

Castle

N. G.

(2012b). Joint Commission accreditation and quality measures in U.S. nursing homes. Policy, Politics & Nursing Practice, 13, 8-16.

33.

Zeger

S. L.

Liang

K. Y.

(1992). An overview of methods for the analysis of longitudinal data. Statistics in Medicine, 11, 1825-1839.

34.

Zhang

N. J.

Paek

S. C.

Wan

T. T. H.

(2009). Reliability estimates of clinical measures between Minimum Data Set and Online Survey Certification and Reporting Data of US nursing homes. Medical Care, 47, 492-495.