Abstract
Consumers play a key role in the U.S. nursing home (NH) oversight through a federally established complaint process. However, past variation by state in complaint numbers and rates raised questions about the uniformity of the process. We examined state variation in numbers of complaints at intake and substantiated complaints, percentages of NHs with at least one complaint and one substantiated complaint, number of allegations per complaint, and complaint substantiation rates. We found state variation most prominently at the intake level, ranging from 0.4 to 30.4 complaints per NH. The investigation process appears to reduce this variation: however, variation remains among states in frequency and prevalence of substantiated complaints. Further work is needed to ensure federal standards concerning the handling of consumer complaints are applied equally across the states. This includes policies affecting how complaints are initially filed, in addition to how complaints are investigated.
Keywords
Introduction
Consumer complaints are a critical piece of the U.S. government’s oversight of nursing home (NH) quality. While NH quality assurance depends largely on mandated annual surveys, a key role exists for consumers through a federally established process enabling any concerned party to file a complaint and potentially initiate an investigation. The complaint process is the only way for consumers (e.g., residents, family members, ombudsmen, and others concerned about care) to alert the government to problems in an NH that may emerge between surveys. However, researchers and government oversight officials have found persistent state-by-state differences in NH complaint processes and rates (Hansen et al., 2019; Stevenson, 2006; U.S. Government Accountability Office [U.S. GAO], 1999, 2003, 2011). Variation among the states raises questions about the fairness of the complaint process, meaning the extent to which federal standards are applied equally across the states, and whether consumers in every state have an equal chance of having their concerns heard and investigated.
The objective of this article is to examine the extent to which NH complaints vary by state. Prior research examined this question (Hansen et al., 2019; Stevenson, 2006) but used data collected before publication of government oversight studies that recommended revisions to the NH complaint and complaint substantiation processes (U.S. GAO, 2011, 2015). This study uses recent data to examine and compare multiple measures of NH complaints by state: numbers of complaints and substantiated complaints, percentages of NHs with at least one complaint and one substantiated complaint, number of allegations per complaint, and complaint substantiation rates. This analysis provides the most comprehensive view yet of NH complaints across the nation, with an examination of variation by state in numbers and percentages of complaints and substantiated complaints in the context of recommendations for more uniform and consistent state interpretation of federal standards.
Background
Both annual survey and complaint investigation processes are guided by protocols set at the federal level but carried out at the state level (U.S. GAO, 2015). The complaint process begins with the intake and prioritization process, during which the survey agency records details about what was alleged to have occurred and categorizes each allegation. A single complaint may contain multiple allegations. Complaints can be filed by residents, families, or others with concerns about the care a resident is receiving. Long-term care ombudsmen are frequently involved, either by advising families on how to file complaints or filing them independently. Based on the level of harm alleged, the complaint is assigned a priority level that determines whether an on-site investigation is to be conducted and within what time frame, as outlined in the State Operations Manual (Centers for Medicare and Medicaid Services [CMS], 2016). If no actual harm is alleged in a complaint, it may be investigated initially offsite through an administrative review.
As part of the onsite-inspection process, an important question for the surveyors is whether the allegations in the complaint are substantiated, which means there are indications that the practices of the NH are likely inconsistent with regulatory standards. Only substantiated complaints are further assessed to determine if any federal regulations have been violated, and if so, what level of deficiency citation should be issued. Information on substantiated complaints that lead to federal deficiency citations is made public on the federal Nursing Home Compare website (Stevenson, 2005). No data are provided to the public on complaints that are not officially substantiated. While excluding invalid or nuisance complaints appears to be justifiable, as Stevenson (2005) noted it assumes that complaint intake and substantiation processes are uniform nationwide and of equal stringency.
Two decades ago, after identifying inconsistencies by state in complaint intake and investigation, the federal government bolstered complaint protocols. This included establishing requirements concerning the timely investigation of complaints alleging serious harm and strengthening federal oversight of state complaint investigation systems (Institute of Medicine, 2001; U.S. GAO, 1999). Nevertheless, oversight studies continued to find considerable variation in state processes and rates, including differences in the ease of filing a complaint, resources and tools available to train intake staff, how complaints were prioritized for investigation, and when a complaint should be substantiated (U.S. GAO, 2011, 2015). The U.S. GAO (2015) reported increases in numbers of complaints filed by consumers nationally and by state (though they did not analyze substantiation rates), and described changes to some state complaint processes that may have affected the numbers of complaints. It reported, for example, that complaints rose in California after the state took steps to ensure all complaints were entered into the federal tracking system and in Michigan after it provided more options for filing complaints, such as by email. Variation among states in complaints also may stem from differences in NH quality or differences in consumers’ motivation or inclination to complain.
Scholars have suggested that consumer complaints about NHs could be part of a new strategy to more effectively ensure NH quality of care (Stevenson, 2006, 2018). However, any such effort may be problematic if there is variation among states in NH complaint processes. While the 2015 GAO report highlights some of this variation, the GAO report utilized data from 2011 to 2014, a period that includes the introduction of the Minimum Data Set 3.0, Medicare postacute care reimbursement moving from RUG-III to RUG-IV, and other major CMS initiatives (e.g., National Partnership to Improve Dementia Care in Nursing Homes) that may have increased community awareness of NH quality. This study adds to the prior work on complaints using 2017 data; data that would reflect changes to the complaint process made after the 2015 GAO report. In addition, this study details and compares state variation in the results of two crucial steps, complaint intake and substantiation, using multiple measures of complaints. The results will provide baseline information needed for further research into the collection and use of NH complaint data to best reflect consumer concerns about NH quality, and thereby potentially improve NH quality.
Methods
Data and Sample Selection
The source of information on allegations and complaints is the ASPEN Complaints/Incidents Tracking System (ACTS). These data are based on information collected by state agencies as part of the federally required inspection process. CMS requires all states to track NH complaint investigations. For each allegation in a complaint, the state is required to record how each allegation was handled from the initial reporting of allegations at intake to closure, including key dates, prioritization level, overall findings, and proposed action. Most importantly, ACTS includes whether a complaint allegation was unsubstantiated or substantiated (CMS, 2016). The ACTS data used in this analysis are part of a larger data set that includes data from the Certification and Survey Provider Enhanced Reports (CASPER) and the Area Health Resource File (AHRF). CASPER contains data on facility characteristics (e.g., ownership status, occupancy rates), aggregate resident characteristics, and staffing levels. The AHRF includes socioeconomic and provider information concerning the county in which each NH is located.
Our sample is restricted to information about allegations and complaints filed between November 28, 2016, and November 27, 2017. We selected these dates due to our focus on the complaint process. On November 28, 2017, CMS implemented significant changes to the regulatory standards NHs must meet, and we wanted to ensure that we were examining only variation in how states administer the complaint process, not the response to the new federal policies. We further restricted the sample to free-standing NHs in the continental United States that did not have missing or erroneous data for describing the NH. 1 A total of 341 NHs were excluded from the sample: 163 because of staffing data and 178 because of they could not be linked with the AHRF. This resulted in an analytic sample of 14,194 free-standing NHs. We found no evidence that NHs dropped from the analysis were significantly different from those included in the analysis.
Allegation and Complaint Outcomes
Each complaint can consist of multiple allegations. All allegations that are part of the same complaint are identified by common intake identification numbers. The ACTS data identify whether each allegation was unsubstantiated or substantiated.
Given this structure, we calculated six variables for all NHs in the analytic sample: counts of the number of complaints per NH and per 100 residents, number of substantiated complaints per NH and per 100 residents, and binary indicators of whether a facility received at least one complaint and at least one substantiated complaint. The numbers of complaints and substantiated complaints identify the intensity of complaint (e.g., how many) a facility receives and whether those complaints rise to level of being substantiated. In contrast, the two binary indicator variables indicate prevalence of complaints by identifying whether the facility received any complaint and any substantiated complaint. As there can be multiple allegations in a complaint, we defined a substantiated complaint as being a complaint in which at least one allegation was substantiated (U.S. GAO, 2011).
The second set of outcome variables was restricted to facilities that received at least one complaint. The first of these variables was the number of allegations per complaint. This variable identified the number of specific issues associated with a single complaint at intake and may reflect variability in a state’s process for logging a complaint. The second variable in this set was the percentage of complaints that are substantiated. Similar to other variables, a complaint was considered substantiated if at least one allegation was substantiated. This variable measured the percent of complaints with at least one allegation that had merit in the view of the inspector.
Empirical Strategy
The empirical strategy was to identify state variation in each of our eight outcome variables. This was done by calculating state means for each of our allegation and complaint variables. We further calculated the means of the state means and identified the minimum and maximum value and standard deviation (SD) for each state mean. In addition, we calculated median values. Finally, we examined whether there was an association between allegations per complaint and percent of complaints substantiated.
Results
Number of Complaints and Substantiated Complaints
There were 14,194 NHs distributed among the U.S. states in our data, with each state receiving a mean of 5.2 complaints per NH and 6.5 complaints per 100 residents from November 28, 2016, to November 27, 2017 (see Table 1; see Online Appendix Table A for median values). This is an increase over the 3.9 complaints per NH found in 2014 (U.S. GAO, 2015). The numbers varied widely, from a mean of 0.4 complaints per NH and 0.4 per 100 residents in Alabama to a mean of 30.4 complaints per NH (see Figure 1) and 40.2 per 100 residents in Washington State. For complaints that were substantiated, there was still a considerable gap between the lowest, also Alabama, with a mean of 0.2 per NH and 0.2 per 100 residents, and the highest, California, with 7.5 per NH and 8.7 per 100 residents. However, substantiated complaints per NH and per 100 residents exhibited less than one third of the variation by state, as measured by the SD.
State Variation in Frequency and Prevalence of Nursing Home Complaints.
Note. The sample includes free-standing nursing homes from November 28, 2016 to November 27, 2017. A complaint is defined as having a unique intake identification number and can consist of multiple allegations. A complaint is considered substantiated if at least one allegation is indicated as substantiated in the CASPER data. CASPER = Certification and Survey Provider Enhanced Reports.

U.S. map showing state variation in the numbers of complaints filed per nursing home. Lighter shades represent lower mean numbers, with the means by state ranging from 0.4 to 30.4.
Percentage of NHs With At Least One Complaint and One Substantiated Complaint
In our examination of prevalence of complaints, we found 76.7% of NHs in each state received at least one complaint, ranging from 30% in Alabama to 98.7% in Texas. The state-by-state variation was slightly wider for the prevalence of substantiated complaints, ranging from 11.7% in Alabama to 91.3% in California (see Figure 2).

U.S. map showing state variation in the percentage of nursing homes with at least one substantiated complaint. Lighter shades represent lower percentages, with the percentages by state ranging from 11.7% to 91.3%.
Allegations and Substantiated Complaints in NHs With At Least One Complaint
As a single complaint can have multiple allegations, we assessed the mean number of allegations per complaint in NHs with at least one complaint. We found minimal variation in the number of allegations per complaint, with a mean of 2.3 and a range from 1.1 to 4.7. We further considered the likelihood of a complaint being substantiated among NHs receiving at least one complaint. In each state, surveyors substantiated an average of 34.3% of complaints, varying widely from 12.4% in Rhode Island to a high of 67.6% in Indiana (see Figure 3). We found no significant relationship between allegations per complaint and percentages of complaints substantiated.

U.S. map showing state variation in the percentage of total complaints in the state that were substantiated. Lighter shades represent lower percentages, with the percentages ranging from 12.4% to 67.6%.
Discussion
Consumer complaints are an important tool for the assessment of NH quality (Hansen et al., 2019; Stevenson, 2006). Complaints are the primary mechanism for residents, family, ombudsman, and others to express their concerns to regulators, and they have the potential to trigger investigations during the months between annual surveys. If complaints are an indicator of quality, one would expect that the distribution of complaints from state to state would be fairly consistent, particularly given that NH complaint protocols are established federally and apply equally to all states.
However, we found wide variation by state in the numbers of total complaints (those recorded at intake), from a low of 0.4 per NH in Alabama to a high of 30.4 per NH in Washington state. We also found variation in the number of complaints that were substantiated upon investigation. This variation was narrower than the variation for total complaints, but it was still evident, particularly between states with relatively low numbers of total complaints and those with relatively high total complaint numbers. Overall this suggests there are differences among the states in how they interpret and/or administer federal complaint protocols.
Prior reports have highlighted differences that would affect the numbers of complaints initially filed and recorded. For example, Michigan provided consumers with more complaint filing options and California increased efforts to enter all complaints into the federal tracking system (U.S. GAO, 2015). Prior reports also have found differences potentially affecting investigations, including how staff in different states determined whether to substantiate a complaint and differences in resources available to conduct investigations (U.S. GAO, 2003, 2011). Similarly, the Office of the Inspector General (2017) reported inadequate investigative staffing in states that failed to meet standards for investigating serious complaints within expected timeframes.
Numbers of complaints and substantiated complaints also may vary if NH quality varies from state to state. Stevenson (2006) found relationships between complaints and measures of NH quality, such as nurse staffing and deficiencies cited on annual surveys. Other factors also could lead to state differences, such as the availability of state ombudsmen whose federally defined role includes advocating for quality improvements in long-term care. However, it seems unlikely that quality and state ombudsmen vary to the extent that complaints vary in our data—comparing, for instance, Oregon, with three complaints per NH, and Washington, with more than 30, or South Carolina, with just over one complaint per NH, and neighboring North Carolina, with nearly seven.
In addition to complaint frequency, we also examined state variation in the prevalence of complaints and of substantiated complaints—that is, the likelihood of a state’s NHs having at least one complaint or at least one substantiated complaint. We found variation in both, but it was greater for the percentage of NHs with at least one substantiated complaint. This appeared to be related to total complaint numbers, particularly for states with few total complaints per NH. While a relatively high percentage of NHs in most states had at least one complaint, the chances of having any substantiated complaints appeared to decrease more for states with the lowest mean numbers of total complaints (nearly 30% of states had fewer than two complaints per NH).
The results of our analyses of complaint numbers and prevalence suggest the outcome of the complaints process is largely influenced by whether complaints are filed in the first place. The U.S. GAO (2011) highlighted differences in how states record and track complaints, noting in a later report (U.S. GAO, 2015) it had asked CMS to clarify protocols to states concerning complaint intake, and CMS had done so with some states on an as-needed basis. The GAO reports included the complaint frequencies for all states but reviewed the processes for only a limited number. The results of this study indicate state variation has persisted, suggesting there is a need for research to more thoroughly explore the relationship between numbers of complaints filed and state-level policies and processes that affect whether and how consumers file complaints and how complaints are recorded.
Further research also is needed into the factors concerning whether complaints are substantiated, given that states with higher numbers of complaints (six or more per NH) appear to have lower substantiation rates than those with lower numbers (less than two per NH). We do not know if complaints were not substantiated because they were deemed to lack merit or because of other factors, such as survey agency staffing deficits that hindered states’ responses to the complaints. Walshe and Harrington (2002) found a relationship between the funding of survey agencies and their ability to conduct annual regulatory surveys of NHs, with higher funding associated with more deficiency citations.
The step at which complaints are substantiated or not is critical because it determines what complaint information is made public on NH Compare. Research has suggested that public reporting of NH performance improves care as providers make changes to appeal to potential residents (Werner et al., 2010). However, evidence also suggests providers enhance their profiles partly by changing only care the government routinely measures (Werner et al., 2011). Consumer complaints represent qualitatively different information that could be presented as a separate measure to supplement the commonly used measures of quality. When complaints are recorded and investigated through a robust and consistent process they best serve their potential use for more effective quality assurance.
Considerable attention is needed concerning the factors influencing whether a complaint is initially filed and recorded, where variation was greater in our study. This study details improbable variations ranging from a total of 94 complaints in Alabama to 6,301 complaints in Washington, states with nearly the same number of NHs. An intake process that limits or hinders complaint filings could keep regulators from learning about quality lapses between annual surveys, resulting in negligent acts or errors going unaddressed. It is important to consider, however, that facilitating complaint filings also may be problematic if the process is not equitable and brings greater regulatory scrutiny upon one NH over another.
Given our findings, this study has several limitations. To our knowledge, no research has been conducted to directly test the validity of the ACTS data on which this analysis was based. However, the same individuals who conduct federally regulated annual survey visits to NHs are also responsible for investigating complaints, and the system requests the same data from the states rather than states generating their own reports. The extensive nature (e.g., intake, interviews, and on-site visits) of the complaint investigation process, as outlined in the State Operations Manual (CMS, 2016) substantiates the face validity of the information contained within the ACTS data. As a further limitation, we examined 1 year of data, but it is a recent year, and looking at a single year provides us with a clearer view, given the management and quality changes that can occur from year to year in NHs. Also, we looked only at numbers and percentages of complaints, not considering types of complaints. Other research has shown, however, that the majority of complaints fall into a few fairly broad categories: care or services, resident rights, and resident neglect and abuse (Hansen et al., 2019). Furthermore, we did not investigate the relationship of complaints to NH characteristics and indicators of quality. Future research should investigate the NH-level factors associated with numbers and prevalence of complaints as well as the relationship between complaints and deficiency citations and other quality measures.
Conclusion
Our results show that there is considerable variation between states in the frequency and prevalence of complaints, while the variance in frequency does appear to be reduced through the investigation process. Given the potential for complaints to spur investigation of potentially deficient NH practices, our results suggest more research is needed to understand state differences in multiple phases of the complaint process—knowledge of and the ease of filing a complaint, and the influence of long-term care ombudsmen and NH lawsuits, as well as investigation resources. Research also is needed concerning the extent to which complaints reflect NH quality or the complaint administration process. Results of these studies could lead to quality improvement through procedures to ensure the complaint process gives consumers a voice across the states, while being fair to providers.
Supplemental Material
Appendix_Table_A_1 – Supplemental material for U.S. State Variation in Frequency and Prevalence of Nursing Home Complaints
Supplemental material, Appendix_Table_A_1 for U.S. State Variation in Frequency and Prevalence of Nursing Home Complaints by Lindsay J. Peterson, John R. Bowblis, Dylan J. Jester and Kathryn Hyer in Journal of Applied Gerontology
Footnotes
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: J.R.B. provides consulting services to long-term care providers, including nursing homes. The other authors have no conflicts of interest to disclose.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by the Patrick and Catherine Weldon Donaghue Medical Research Foundation. University of South Florida Institutional Review Board Review Number: Pro00038157
Supplemental Material
Supplemental material for this article is available online.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
