Abstract
Policy makers, payers, and the general public are increasingly focused on health care quality improvement. Measuring quality requires robust data systems that collect data over time, can be integrated with other systems, and can be analyzed easily for trends. The goal of this project was to study effective tools and strategies in the design and use of clinical registries with the potential to facilitate quality improvement, value-based purchasing, and public reporting on the quality of care. The research team worked with an expert panel to define characteristics of effectiveness, and studied examples of effective registries in cancer, cardiovascular care, maternity, and joint replacement. The research team found that effective registries were successful in 1 or more of 6 key areas: data standardization, transparency, accuracy/completeness of data, participation by providers, financial sustainability, and/or providing feedback to providers. The findings from this work can assist registry designers, sponsors, and researchers in implementing strategies to increase the use of clinical registries to improve patient care and outcomes. (Population Health Management 2014;17:127–133).
Introduction
U
Clinical registries have the potential to fulfill the need for more widespread patient data to improve treatment, policy making, and public transparency.
1
The Agency for Healthcare Research and Quality recognizes a registry as “an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure and that serves predetermined scientific, clinical, or policy purpose(s).”
2
Information and communication technologies can improve the timeliness and lower the cost of data collection, analysis, and reporting, and provide the capacity to link data across sites of care for a more valid and comprehensive picture of health and health care. With increasing frequency, applications of registry data have migrated from within provider institutions out to health plans, purchasers, and the general public. Current applications of registry data include: • Quality improvement (QI) programs. Registries owned and operated by clinical specialty societies were created primarily to provide traditional peer review data and QI tools for their members. Participation in these registry programs has increased as payers and regulators have provided incentives for their use (see the next bullet point). • Value-based purchasing (VBP) strategies. VBP links employer and health plan contracting to information on provider cost and quality. At the core of VBP are data that inform purchasers and payers of provider performance, and that provide feedback to providers about their comparative performance versus peers.
3
Electronic provider registries now transmit required practice data directly to VBP managers to fulfill reporting requirements. This lowers reporting costs and increases data accuracy, completeness, and timeliness. • Public reports on the performance of health care providers. Public reporting on the quality of care is essential for informed patient decision making, public policy making, and a well-functioning health care marketplace. Public transparency also provides an additional impetus for providers to improve patient outcomes. Insurance claims have been the most frequent source of data for quality reporting. Drawbacks of using claims data are that they do not contain clinical data to adjust for differences among patient populations treated by different providers or to accurately measure patient outcomes. Registries have the potential to respond to this important need because they provide significantly better clinical detail than insurance claims.
Registries in different clinical areas have varied mechanisms for data collection, analysis, and use, especially for the purposes of QI. The goal of this project was to study effective tools and strategies in the design and use of clinical registries with the potential to facilitate QI, VBP, and public reporting on the quality of care. A preliminary, high-level environmental scan of clinical registries in the United States and internationally was conducted to identify examples of effective registry design and utilization from the perspective of QI, VBP, and public reporting. Based on results from that environmental scan, in-depth analyses of effective registries were conducted in 4 clinical areas (cancer, cardiovascular, maternity, and joint replacement).
Methods
Registries exist in a variety of clinical areas and differ widely in terms of types of users, funders, and technologic sophistication. 4 This variability across registries makes it unclear which registries are most useful for QI, what characteristics make them useful, and which registry elements are important under different circumstances. To understand the factors that enable registries to be used successfully for quality monitoring and improvement, an analytic framework called “positive deviance” was employed; this framework has been used elsewhere to identify and learn from programs and organizations that demonstrate consistently exceptional performance. 5 This approach was first used in public health to identify interventions designed around uncommon but beneficial health behaviors that some community members already practiced, and it has been used most often for behavior interventions at the level of the individual. 6 The positive deviance framework has since been applied to evaluation of health care organizations, 5 community health programs, 6 and health intervention planning. 7 In consultation with an expert panel, the research team defined validated, objective criteria for identifying registries that are most effective in key clinical areas and the factors that make them effective (ie, “positive deviants”). In-depth analyses were conducted of a sample of effective registries to learn from their experience.
Based on a literature review, a list of characteristics was compiled and grouped by domain, with the goal of learning more about these characteristics from effective registries. The research team began with a list of 5 primary domains that appeared to have characteristics that represented the strongest correlation with efficacy in the areas of QI and/or public reporting. The team subsequently convened a panel of 6 experts in the field of clinical registries, including registry users, developers, researchers, and policy makers, to elicit feedback on the domains and characteristics identified and to determine which should be the focus of the in-depth analysis. An updated list of domains and characteristics was circulated to the expert panel twice for refinement and final consensus. The final list contained 7 domains that guided in-depth analysis of effective registries (Table 1).
The research team conducted an inventory of current registries in 4 clinical areas of interest (cancer, cardiovascular, maternity, and joint replacement) through a literature review and Web search. The team focused on registries that collect individual patient information through a course of treatment rather than population-level data. The team worked with the project funder to select clinical areas based on the presence of prominent registry programs, the size of the clinical area in the health care industry, and the proportion of total health care spending associated with each area.
The research team used the final list of registry characteristics to identify a sample of effective registries in each of the 4 clinical areas. The focus was on registries that were clearly effective based on Web searches, literature reviews, and consultations with the expert panel. The research team conducted in-depth interviews with staff from 2 to 5 effective registries in each clinical area. Specific registries in each area are noted in Table 2. The goal was to learn from best practices of effective registries rather than include a representative sample of registries in the interviews. The team contacted each of the registries included in the study, and those organizations identified their key informants. The research team found that the domains they had identified were helpful for structuring the interviews; however, 6 key themes emerged after the interview data were analyzed. The results of this study are reported by theme rather than domain area.
Results
The research team found that exemplary registries were effective in one or more of 6 key areas: data standardization, transparency, accuracy/completeness of data, participation by providers, financial sustainability, and/or feedback to providers. Examples of successful practices in each of the key areas are noted in the following sections.
Data standardization
In order to be useful for comparison, registry data must be standardized across clinical practices; however, this is often a challenge, given the individual nature of many clinical charting systems. State cancer registries have been particularly effective in standardizing their data. Standard data collection techniques across state cancer registries are monitored through the North American Association of Central Cancer Registries, which serves as their accrediting and organizing body. In addition, the National Program of Cancer Registries (NPCR), administered by the Centers for Disease Control and Prevention (CDC), and the Surveillance, Epidemiology and End Results (SEER) Program, administered by the National Cancer Institute of the National Institutes of Health, provide funding to cancer registries and analyze data from across registries to produce reports on national trends for such topics as cancer incidence, type, and mortality, among others.
A benefit of the robust data collection process employed by the NPCR and SEER programs is the ability to aggregate data and draw conclusions about cancer incidence and treatment across the country. Standardization through federal regulation and designation of cancer as a reportable disease allows direct comparisons at the local, state, and national levels to inform policy decisions. Although this standardization might stifle local innovations in data collection and reporting, the level of data aggregation is invaluable in decision making for state and national programs and policies.
Other clinical areas have been creative in reaching standardization. For example, the American College of Cardiology's (ACC's) PINNACLE registry is an electronic medical record (EMR)-based ambulatory care registry focused on outpatient management of chronic conditions. Cardiologists enter data about their patients into their practice's EMR, and the data are then transmitted to the PINNACLE registry electronically. An important PINNACLE innovation is a “system integrator” that pulls EMR data to populate the registry automatically, regardless of which EMR the practice uses. The systems integrator is designed to “bolt on” to a practice's existing electronic record software and automatically prepare and send data to the registry. Although this system assists with data transference to the registry, it does not guarantee that all data are standardized. Practices receive error messages for data that do not “fit” into the registry system and must reconcile them. Although it requires some manual oversight, this technology is crucial to future dissemination of EMR-based registries. Such data standardization can allow for comparison of outcomes across providers and institutions.
In order to manage some of the challenges associated with registries pulling data from various providers, the principal players in cardiovascular (CV) registry development—the ACC, the Society of Thoracic Surgeons (STS), and the American Heart Association—have a number of collaborative registries in addition to individual registries. Such collaborations allow for standardization of data collection among different specialties of CV providers who treat similar conditions. These registries represent a continuing attempt to jointly develop clinical protocols that indicate how to achieve the best results for patients who currently have multiple treatment options available to them.
Joint replacement registries also are working on standardization. The recent launch of the American Joint Replacement Registry and the Food and Drug Administration's involvement in initiating the International Consortium of Orthopaedic Registries represent novel efforts to pool data resources at the national and global levels, thereby creating hitherto unprecedented capacity for data aggregation and opportunities to improve safety and reduce costs through better postmarket surveillance of implantable devices. These groups will address data standardization as a critical component of their overall strategies to merge data from multiple sources.
Transparency
Transparency of data is crucial to improving clinical quality and patient outcomes. CV registries are at the forefront of sharing data publicly, and do so far more often than registries in the other 3 clinical areas reviewed. Until recently, CV registry data were shared exclusively with physicians who subscribed to registry programs; however, some information is beginning to be shared with payers, and results calculated from some registry data are becoming available to the public.
For example, STS's Adult Cardiac Surgery Registry provides information for public reports that rank cardiac surgery groups. STS collaborates with Consumers Union to make selected registry data available in Consumer Reports, including an online selection tool on the magazine's Web site. The tool uses patient outcomes such as survival and complications (risk adjusted from registry data) plus process of care measures to produce ratings (1, 2, or 3 stars) for several hundred surgical groups around the country. In addition, STS mounts a similar report on its own Web site. The voluntary nature of these reports means some degree of selection bias (physician groups that perform badly can choose not to participate), but the reports do benefit patients who are seeking relevant care.
Accuracy/completeness of data
Data accuracy and completeness are key to ensuring data can be analyzed effectively. Cancer registries in the United States have robust methods to ensure accuracy and completeness of registry data, and data completeness has become a key metric for evaluating cancer registries by regulators. The Texas Cancer Registry, for example, performs state-specific automated data quality and completeness checks on all cases upon submission. Similarly, the Louisiana Tumor Registry conducts both visual and electronic audits. Another tactic involves employment of certified tumor registrars (CTRs), utilized by the New York State Cancer Registry for groups of a certain size, as required by state law. All 1500+ American College of Surgeons Commission on Cancer (CoC)-Certified Cancer Programs also are required (as of January 2012) to have a CTR performing data abstraction. Benchmarking of data quality is made possible through a mechanism administered by the CDC, which provides quarterly reports to allow registry administrators to review data quality and compare performance against peers.
State cancer registries also benefit from data exchange agreements with departments of public health in neighboring states that allow data transfer when patients travel out of state to receive cancer evaluation or treatment; for example, the Louisiana Tumor Registry maintains agreements with 19 neighboring states. Data use agreements such as those that the Texas Cancer Registry has with federal and state agencies also assist in sharing data with external organizations. Using the model of cancer registry data collection and exchange may help other clinical areas build more robust data systems.
Although this organized and regulated system of cancer registries results in more complete and systematic data collection, challenges (including timeliness, data access, and electronic reporting) remain with regard to using those data for QI and consumer choice. Historically, state cancer registries have served public health, clinical, and health services research needs. Individual hospital-based registries, accredited through the American College of Surgeons CoC, support each hospital's QI programs with the help of CoC-sponsored measurement and monitoring tools as well as benchmarks.
Both of the 2 effective maternal registries reviewed are linked to newborn data registries for ease of access and analysis of mother plus child information. The Newfoundland and Labrador Provincial Perinatal Program (NLPPP) operated as a biostatistics database until recently and has applied for “registry status” from its provincial Ministry of Health; this will allow it to collect patient identifiers and receive resources from the Ministry. Another Canadian program, Better Outcomes Registry and Network Ontario (BORN Ontario), already has “registry status” and collects patient identifiers that allow for better capture of the care continuum from preconception to postpartum. BORN Ontario has been effective in linking data from its maternity registry with other registries supported by the Ontario Ministry of Health in collaboration with The Institute for Clinical Evaluative Sciences Data Warehouse of the University of Ottawa. These data linkages allow for accurate and robust measurement and analysis of patient outcomes.
Advances are being made in joint replacement registries to collect similarly robust patient data. The HealthEast Joint Replacement Registry in Minnesota utilizes a client/server-based system that merges demographic information with clinical data, plus a Web-based implant catalog to obtain device details. All data collection for this registry is electronic, with the exception of a single data element: an administrative coordinator enters the reason for revision manually after a surgeon reviews the revision procedure. Once the data are entered into the system, they can be linked to administrative claims and EMR data. A rigorous audit process ensures data quality and accuracy, and the data entry burden for clinicians is practically nonexistent. As an international example, Sweden's registries have adopted a fully Web-based system and have extensive linking capacity to administrative claims databases as well as quality and pharmaceutical registries. With the move toward EMRs in all clinical settings, consideration should be given to the ability for EMR data to be linked to administrative data to ensure complete patient records when data are transferred to clinical registries.
Participation by providers
One of the key challenges to registry effectiveness is provider participation. Providers are often apprehensive about participating in clinical registries, lest their performance be compared to others or shared with peers. However, provider motivation to participate in registries has increased because of various factors.
In the United States, the legal mandate to report cancer cases encourages a high rate of participation among providers of cancer treatment. Similarly, clinical registries in countries with single-payer health care systems have robust data because providers are required to report patient information. This was found to be the case in the Canadian maternity registries reviewed.
Although there are currently no legal mandates for cardiologists to submit patient care information to CV registries, payers increasingly require registry participation as a condition of eligibility for bonus programs, preferred provider network status, and reimbursement. Cardiologist and cardiac surgeon participation in registries is relatively high, in part because of these payer incentives.
Financial sustainability
Financial sustainability of registries continues to be a challenge, particularly for registries not supported by public funds. The federal government funds state cancer registries across the United States, and many state cancer registries that make data available for researchers receive grant and/or partnership funds from academic institutions. The Newfoundland/Labrador and Ontario maternity registries reviewed receive funds from the Canadian government.
CV registries have found a variety of ways to maintain financial stability, including subsidies from pharmaceutical and device manufacturers, dues paid by participating providers, and revenues from the sale of registry data and from grants/contracts for studies derived from those data. Funding models are evolving, particularly with respect to industry support. PINNACLE, currently funded by the pharmaceutical industry and free to subscribing cardiology practices, has considered charging fees to participating physician groups based in part on the registries' Physician Quality Reporting System (PQRS) reporting feature. The ACC is considering charging an annual fee per participating physician that is a small percentage of the average participant's annual PQRS payments from Medicare. This is one way of transitioning from a registry that is initially free to participants to one that is financially self-sustaining. STS registries are funded through STS member dues, data sales, and research grants rather than industry funds.
Feedback to providers
Improving clinical quality requires that data be made available to providers in a timely manner. CV registries are particularly effective because the principal focus of CV registries is feedback to physicians (process and outcomes measures) for QI rather than population-based data on the incidence and prevalence of cardiovascular disease.
The PINNACLE registry can report PQRS (the Centers for Medicare and Medicaid Services [CMS] pay-for-reporting system that contains over 200 quality measures) measures directly to CMS. Most observers expect CMS eventually will make this a pay-for-performance program (ie, required for reimbursement rather than a voluntary bonus). The registry also can report provider feedback at the individual physician or group level and benchmark performance measures to national averages. 2,8 These are all aspects of “meaningful use” as defined by the Health Information Technology for Economic and Clinical Health (HITECH) Act, providing an important incentive for physician practices to consider EMR-based registries. (“Meaningful use” refers to EMR applications [mostly related to performance improvement] required in order for health care providers to be eligible for federal incentive subsidies). Although initiatives such as the federal HITECH Act (signed into law on February 17, 2009 as part of the American Recovery and Reinvestment Act) contained specific incentives to accelerate adoption of EMRs and provided incentives for the growth of office-based EMRs, multiple information technology platforms (especially in the outpatient setting) can be an impediment to wide diffusion of EMR-based registries.
Based on the same statistical model used to provide risk-adjusted feedback to registry subscribers, any physician (or patient) can use a publicly available online risk calculator on the STS Web site to assess a cardiac surgery patient's predicted risk of mortality and complications, and length of stay, among other factors. This tool aids in shared decision making when patients and their physicians consider alternative courses of treatment.
The NLPPP maternity registry has demonstrated effectiveness in the area of performance improvement, specifically in collaboration with the Society of Obstetricians and Gynaecologists of Canada risk-management initiative. The Society provides hospitals with data for analysis of scenarios that lead to adverse outcomes to support a Province-wide performance improvement initiative. BORN Ontario has shown effectiveness in the area of performance improvement through the development of an online dashboard. This interactive dashboard uses key performance indicators to give providers immediate feedback on repeat elective Cesarean section prior to 39 weeks and other conditions associated with increased risks.
Patient outcomes data are the gold standard for joint replacement registries, including identifying and following revision surgeries. Tracking surgical revision enables earlier detection of device failures and can add tremendous value to clinical decision making. All of the registries reviewed are able to track implant revisions. Internationally, Sweden measures quality via a balanced scorecard with established quality indicators, and the Swedish registries provide physicians with annual reports benchmarking their results (including revision rates) by region, by peer group, and by institution. In addition, a few domestic and international joint replacement registries enable physicians to track facility performance and/or patient outcomes in real time via a dynamic, Web-based reporting system.
Discussion
The growing focus on QI, VBP, and public reporting by policy makers, payers, and patients will require increased investment in clinical data collection, analysis, and reporting. The findings from the present study can assist existing and emerging clinical registries to develop robust systems to support such QI. The registries studied did not all contain each of the 6 elements for effectiveness; however, all of the registries studied do have mechanisms for data standardization. This speaks to the importance of having standard definitions for data entry and a mechanism for ensuring data quality. Registry developers and users should consider this when designing, updating, and implementing registries.
Key examples of effectiveness were found in each of the clinical areas studied. For example, cancer registries are effective at standardizing data collection, maternity registries link prenatal data with antenatal outcomes, and joint replacement registries link patients with outcomes. However, of the 4 clinical areas studied, cardiology has the most robust innovations from the perspective of QI and public reporting, spurred by changes in practice, competition between specialties, and payer incentives. American maternity and joint replacement registries suffer from the siloed nature of current private-sector efforts; one can see examples of foreign registries that have made great strides in these clinical areas, possibly because those countries have provincial or national health systems. The US system of state cancer registries, which benefits from national standards, mandated reporting, and public financing, offers robust data for research and surveillance purposes but exhibits the fewest innovations to enhance QI applications.
Clinical registries have the potential to support VBP as well. For example, several ACC registry programs now include payer reports. The registries' clinical data allow for more valid and comprehensive measures of provider performance than would be possible with the limited data elements available from insurance claims. As payment models shift from fee for service toward value-based payment models, 9 those paying for care as well as those providing care will see registries as an increasingly valuable asset.
The use of registry data for public reporting of provider performance is in its infancy and advancing slowly. In the case of proprietary registries (eg, owned by specialty societies), it might be difficult for public interest in greater transparency and informed consumer choice to outweigh a professional society's members' concerns when it comes to publishing reports cards about themselves. In spite of that, one registry owner (STS) partners with a national consumer publication to release limited information on provider performance. Alternatively, in the public sector, state cancer registries are apprehensive about using patient-level data to generate reports on provider outcomes for fear of compromising patient confidentiality. However, changes brought by the PPACA, as well public desire for increasing transparency in health care, may lead to further registry innovation across many clinical areas. The recent federal “fiscal cliff” legislation 10 contains language that would substitute participation in a “qualified clinical data registry” for certain quality measurement reporting requirements under Medicare. Such incentives should encourage registry programs to seek “qualified” designation and more physicians to participate in registry programs.
Conclusion
The research team found that effective registries were successful in 1 or more of 6 key areas: data standardization, transparency, accuracy/completeness of data, participation by providers, financial sustainability, and/or feedback to providers. Cardiology registries have made the greatest strides toward QI and transparency through the use of data integrators, establishing partnerships between professional societies in registry design and implementation, implementing creative mechanisms for registry financing, providing rapid feedback to providers, and using consumer-oriented products to share outcomes data. This qualitative study does not allow for measuring the strength of the relationship between registry characteristics and effectiveness. However, the findings show that although it is rare for registries to have all of the elements necessary for effectiveness, focusing on 1 or 2 key elements can help registries be better leveraged for improving health care performance. Registry owners should consider the tools they currently have in place and use the examples cited in this study as models for making changes to improve their effectiveness. Registry developers should consider the elements detailed in this study when designing new registries to ensure they are as effective as possible.
Footnotes
Author Disclosure Statement
Drs. Klaiman and Legnini, and Ms. Pracilio, Ms. Kimberly, and Ms. Cecil declared no conflicts of interest with respect to the research, authorship, and/or publication of this article. The research was made possible through a grant from the California HealthCare Foundation.