Abstract
Recent attention has focused on comparative effectiveness research (CER) to determine the clinical effectiveness and safety as well as the economic impact of health care interventions, including health care reform legislation that provides for new funding to develop the infrastructure to support CER. The demand for CER will significantly impact the competitive landscape for many sectors of the health care industry. The development of convincing and credible evidence to support products and services has become a critical competitive issue. In some cases, it has become a requirement for new health technologies to gain market access, achieve and sustain premium pricing, and achieve targeted levels of market acceptance. This article is intended for senior business management in the health care industry to facilitate efforts to make strategic decisions about the level and type of clinical and economic evidence needed to achieve market success in this new landscape. This article focuses primarily on the strategy for medical technology in the prism of CER.
Rationale for comparative effectiveness research
In 2007, the Congressional Budget Office (CBO) issued a report addressing the effectiveness of medical treatments. 1 Their report focused on the burden of escalating health care expenditures for government and private payers as well as disparities in health care spending within the US compared with other countries. Greater expenditures were not associated with improved health outcomes 1 The CBO suggested US medical practitioners adopted more costly interventions in the absence of rigorous efficacy data. The CBO concluded that better information on costs, risks, and benefits of different therapies was essential in order to change the practice patterns of health care delivery in the US and decrease health care costs. 1
The CBO analysis was followed by two reports by the Institute of Medicine (IOM) in 2008 2 and 2009. 3 Similar conclusions were made by the IOM, including that the US has the most expensive health care worldwide with expenditure not correlating with improved outcomes, and a notable wide regional variation in patterns of clinical care.2,3 The IOM report also recommended a priority list of 100 research areas where comparative evidence for diagnosing, treating, or monitoring specified medical conditions were lacking. This list will most likely serve as a foundation for future focus and funding efforts by US Government and private payer organizations.
Definitions of comparative effectiveness research
The call to action by the IOM established the momentum for comparative effectiveness research (CER) in the US health care reform efforts. The diversity in definitions of CER is attributable, in part, to variations in the perspectives of multiple stakeholders. 4 In Europe, CER is often associated with evidence developed in the context of formal health technology assessment (HTA), which may include an explicit consideration of cost-effectiveness. 5 In the US, the construct of HTA was formally established by the Office of Technology Assessment in 1978 and defined as ‘a multidisciplinary field of policy analysis, studying the medical, economic, social, and ethical implications of the development, diffusion, and use of health technology.’6,7 According to this definition, technology includes medications, devices, procedures, and systems of organization for the delivery of health care. 7 The process of HTA is intended to evaluate effectiveness, appropriateness, and cost in order to provide decision makers with information to inform policy options.
The global perspective on HTA varies significantly due to differences in culture, history, politics, and methods of financing. Canada, Europe, and other parts of the world have established agencies charged with responsibility for HTA activities, including evaluations of the cost-effectiveness of specific technologies. The United Kingdom’s National Institute for Health and Clinical Excellence is the most mature example of a centralized HTA body that provides guidance, sets quality standards, and manages a national database to improve health. 8 Other national bodies have multiple agencies that reflect various regional or bureaucratic divisions with different responsibilities that fall into the domain of HTA. For example, in Germany, the German Institute of Medical Documentation and Information (DIMDI) establishes research priorities, (HTA at DIMDI) 9 while the Institute for Quality and Efficiency in Health Care (IQWiG) conducts technology assessments and the Federal Joint Decision Making Body (G-BA) makes coverage and reimbursement decisions and establishes treatment guidelines.
Use of the term HTA is less common in the US where there is a greater emphasis on CER as well as evidence-based medicine (EBM), outcomes research, and effectiveness research. 7
Prominent organizations in the US including the Agency for Healthcare Research and Quality, the IOM Committee on CER Priorities, and the Federal Coordinating Council for Comparative Effectiveness Research (FCC-CER) have established definitions for CER.3,10,11 The IOM defines CER as:
‘the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition, or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.’3,7
Despite the differences in definitions for HTA, EBM, and CER, they share a common emphasis on research that yields clinical and economic evidence of the comparative benefits, risks, and values of alternative therapeutic interventions. These results are intended to provide information to clinicians, purchasers, policy-makers, patients, and other stakeholders that will improve the quality and safety of health care at the individual and population level. 12 Such information is considered essential to translate new research findings into clinical practice, improve patient outcomes, speed the provision of health care interventions with proven benefit, control costs, and deliver ‘the right treatment to the right patient at the right time.’13,14 However, practitioners should be mindful that even within HTA assessments there are differences in assessments between medical devices and pharmaceuticals (incremental technology gains, inability to blind in certain surgical scenarios, etc.). 15
The development of a CER program for the US
The American Recovery and Reinvestment Act of 2009 (ARRA) was a significant federal commitment that allocated $1.1 billion to support CER and established the FCC-CER to complement ongoing research conducted in the public and private sectors through the support and development of data and research methods, provision of training for researchers, and timely dissemination of results to health care provides, patients, and other stakeholders. 11
Following this investment, the Patient Protection and Affordable Care Act of 2010 established CER as the central feature of efforts undertaken by federal agencies to improve health care with oversight of federal CER activities assigned to the Patient-Centered Outcomes Research Institute (PCORI). 16 PCORI will be funded going forward from the new health care legislation, with estimates as much as $400–500 million per year.
Stakeholders affected by CER
The 2008 IOM report noted that multiple individuals/entities contribute to, and are affected by, health care decision-making. These include:
Health care providers making decisions about therapeutic interventions for individual patients. Consumers selecting health plans. Patients or their caregivers making treatment choices. Payers and employers making decisions about the type and scope of services to be covered and reimbursed. Professional medical organizations developing clinical practice guidelines. Regulatory agencies responsible for the assessment of new drugs or devices, and public health programs developing population-based interventions.
Decisions made by these various stakeholders will be based upon knowledge that is explicit, systematic, and replicable with an emphasis on determination of the type of services that are most effective in real-world settings with real-world patients. Such patients will include those with comorbid health conditions, diverse risk factors, variations in adherence rates to therapeutic interventions, stage and severity of disease, variations in health care coverage, and demographic characteristics. In addition, factors related to the health care setting, level of training of health care providers, and the timing and quality of interventions can have an impact on effectiveness.
Implications of CER for medical technology companies
The emphasis on CER and the increased demand for evidence to improve health care decision-making, clinical outcomes, and the value of health care interventions has created new challenges for the health care industry. Business lines now face critical decisions regarding the amount and timing of investments in CER initiatives. Depending on the specific type of products or services offered, individual companies will be challenged to balance the evidence needs established by regulatory, reimbursement, and market stakeholders in diverse therapeutic areas, clinical settings, geographic regions, and patient populations. Medical technology companies are now required to determine market success and assess the value of evidence-based claims. This will require trade-offs between minimizing cost and development cycle time while maximizing the market value that comes from incremental investments in evidence development (i.e. disruptive, highly utilized costly technology will require more evidence than a replication of a technology already on the market).
Furthermore, competing potential investments in evidence development must be prioritized. For example, medical technology companies will need to weigh the potential gains and losses associated with more costly, long-term, large studies that are expected to have a large impact compared with multiple, smaller studies that can be conducted at lower cost, in less time, with fewer subjects, and at potentially lower risk to patients and other customers. Evidence development alternatives (observational studies, claims mining, retrospective studies, etc.) that are lower in cost do exist and may address the same needs as the “gold standard” of the randomized controlled trial (RCT) in some circumstances (Methods and data sources for CER section).
In addition, medical technology companies must develop the expertise to anticipate and respond to information from clinical and economic studies, including comparative effectiveness (CE) studies that are publicly funded or sponsored by competitors. Such responses will have to address the interpretation of results that may negatively affect a specific company’s initiatives for utilization, reimbursement, or evidence development. The optimal strategy may include trade-offs between ‘defensive’ and ‘offensive’ investments in CER. Specifically, defensive investments may address evidence gaps in response to other published studies, while offensive strategies will rely on collection and dissemination of evidence from head-to-head comparisons to improve positioning of specific products or services.
Furthermore, large companies may face the potential for strategic conflicts across business units. For example, CER will require studies that evaluate treatments, products, or services targeted to specific patient sub-populations to identify the right intervention for the right patient at the right time. This focus will have implications for alternative interventions developed by different business units within the same company. In addition, comparative studies based on head-to-head comparisons of treatments or products are likely to be conducted by external parties, which will require each company to determine their level of support for, and involvement in, such studies and determine the best response to findings from such studies, particularly if the results are not favorable to the company’s product or service.
Strategic decisions about CER studies must also take into consideration regional and global variations in evidence needs and priorities that will be determined by diverse factors such as the differential epidemiologic burden of specific health conditions, access to health care services, and variations in the demographic and clinical characteristics of patient populations. While some core evidence will have value in all markets, allocation of research budgets will have to take into account regional differences in evidence needs, standards, and opportunities within the context of a global business milieu.
Additional expertise, changes in standard processes used by industry for product and business planning, and development of new external relationships will also be needed to support the development of the strategic evidence required by CER. Of particular importance will be expertise to plan and execute studies, as well as disseminate findings to address the evidence needs of clinicians, regulatory agencies, payers, patients, and other stakeholders. Effective dissemination of evidence-based results will support regulatory approval, support payer coverage, increase market access, and promote clinical adoption. Process and organizational capabilities will be essential to efficiently coordinate these efforts. In addition, there will be an imperative to build and sustain external relationships with investigators and other opinion leaders to ensure that investments in evidence development have maximum impact. There has never been a time when it will be more important to collaboratively and effectively work across business units toward common goals in evidence development that will benefit each company as a whole. A summary of potential strategies to achieve this type of collaborative initiative are briefly described below.
First, it is essential to realize that evidence strategy is an integral part of the product creation process. Importantly, evidence strategy should be initiated early in the product creation process, beginning with an evaluation of the evidence needed to make specific product claims (clinical, mechanical, and economic). The development of evidence for new claims close to product launch is not always possible or desirable, especially when complex, time-consuming studies are needed. Therefore, early planning efforts should identify the optimal study design to respond to specific evidence claims. Claims that must be based on complex studies enrolling large number of patients and following these patients for long periods of time should be identified and initiated early in the process of product development. This may also include evidence mapping, which would include review of various geographical needs, or if studies could be completed in one geography to support the needs of a different geography (example: CE marking a technology in Europe and initiating a study for strategic reasons while a US trial is being planned). Furthermore, a particular evidence strategy may be associated with a different investment profile and different product development cycle time and cost metrics. Thus, early consideration and proactive decision-making about the various types of evidence that is desired and needed to make specific product claims are the key elements of the evidence strategy.
Finally, evidence strategy should encompass all claims and value propositions needed for regulatory approval, favorable reimbursement and pricing, and market uptake. The evidence needs (value proposition) of various stakeholders and the claims desired to address these needs should be considered together. It is no longer sufficient to obtain regulatory approval and clearance first and then address claims support following approval. Consolidated evidence needs should be addressed in the context of benefits for patients, caregivers, and customers, including clinical outcomes, patient preferences, care continuum, clinical setting and workflow, and cost avoidance or improvement. The business case for evidence development should be explicit and priorities established for competing evidence needs and responses. This will include decisions regarding expected study outcomes and consideration of return on the resource investment (ROI). The business case should consider the full range of options for clinical and economic studies.
It is also likely that additional organizational capabilities may be needed to respond more effectively to demands for evidence. Of particular importance will be expertise in areas related to the design and conduct of robust studies, as well as dissemination of evidence. This includes, but is not limited to, biostatistical knowledge, Health Economics and Reimbursement (HER), publication planning, trial design, medical writing, scientific communications, and medical informatics. Strong collaborations between marketing, regulatory, HER, and clinical personnel will optimize the ability of an organization to launch a proactive response for evidence development. Internal organizational strengths and capabilities will be further enhanced by ongoing external relationships such as those with academic medical centers, key opinion leaders, professional societies, and independent investigators. Other partnerships that may enhance an organization’s ability to respond to the demand for evidence can include other stakeholders who share an interest in demonstrating improved clinical and economic outcomes, such as payers, professional organizations, patient advocacy groups, or other industry participants.
Guidelines for assessment of evidence needs
Strategic framework for industry investment in CER
Despite numerous challenges, the demand for CER and demonstrations of the strategic value of evidence present positive business opportunities, particularly for organizations that anticipate evidence requirements and develop a proactive response plan. Substantial efficiencies in product development and research cycle time and costs can emerge from long-range, systematic planning. For example, evidence desired by multiple stakeholders can be collected in a single, well-designed study that generates both clinical and economic outcomes data.
Industry leadership must assess evidence needs in order to determine the business imperative and rationale for studies to generate evidence that responds to the informational needs of diverse stakeholders. Lines of business face critical decisions about the timing and amount to invest in studies to generate evidence of various types. This section briefly reviews concepts to be considered in the process of decision-making about allocation of resources to support decisions about research needs. A strategic framework to support decisions for the investment in evidence development studies is presented in Figure 1.
Strategic planning framework.
There are two key components of a business case for evidence development including (1) economic value of evidence and (2) level of investment required to generate evidence. The economic value of evidence depends on the impact of potential new claims, the expected financial benefit of establishing those claims, and the attractiveness of the market opportunity. Factors that contribute to decisions regarding the level of investment required to generate evidence include the evidence needs of key stakeholders and the type and cost of resources required to develop and disseminate evidence-based findings.
Key questions to assess the business case for evidence development
Positioning on the strategic planning framework reveals different clinical and market contexts that suggest varying evidence development strategies and investment options (Figure 2). In general, business cases located in the upper two quadrants of the framework (high economic value of evidence) support more attractive investment opportunities compared with those located in the two lower quadrants (low economic value of evidence). Specifically, location in the upper left quadrant implies a higher potential ROI from a targeted evidence development strategy that captures high value while not incurring high or unnecessary investment of resources. Investments should be targeted to development of evidence that addresses the needs of key stakeholders as well as coordinated with pricing and marketing strategies to achieve maximum impact. Additional investments in evidence development may also be required to ensure that the long-term strategic advantage is sustained. Studies that fall into the upper right quadrant require a high level of investment but this is potentially offset by the potential for high yield in terms of evidence-based claims. A higher premium on planning and coordination is essential for studies that fall into this quadrant. Thus, business leaders must develop a comprehensive plan that generates new evidence to demonstrate clinical effectiveness and cost-effectiveness for well-defined patient populations.
Range of potential evidence development strategies and investment options.
Evidence development strategies located in the lower left quadrant imply minimal investments, to meet the requirements for market success, which typically is focused on authorization by regulatory agencies. The lower right quadrant also implies a focus on regulatory approval, but also that the substantial investment in evidence required should be repurposed wherever possible. Due to the highest costs associated with evidence development in this quadrant, changes in market dynamics should be closely monitored. Furthermore, business teams should avoid investment in studies that address stakeholder evidence needs but have an unacceptable expected ROI.
Methods and data sources for CER
Ultimately, CER is expected to have significant implications in three primary areas. First, findings will better inform the current process of health care decision-making by diverse stakeholders. Second, the investment in translation of CER results into clinical practice has the potential to improve both the quality and value of health care for all populations in diverse health care settings. Third, CER is considered essential for the support of high-value innovation and the practice of personalized medicine.11,13,16 Distinguishing features of CER include comparisons of alternatives that are most relevant to real-world clinical practice settings. This means that CER studies should represent the treatment(s) typically offered to patients in typical settings and should not include comparisons with placebo unless no intervention is an acceptable option in current clinical settings. In addition, patients included in CER should be representative of those most likely to receive the intervention in real-world clinical settings; this is a major difference from trials currently conducted to obtain approval of new products or devices by the Food and Drug Administration and other regulatory bodies.11,13,16 In addition, CER should identify patient-specific characteristics that are associated with differential responses to therapeutic interventions. These might include age, gender, comorbidities, severity of illness, genetic history, and other demographic or clinical characteristics that might affect the outcome associated with a diagnostic test or therapeutic intervention. Further, CER should include all outcomes that are important to patients such as the frequency and severity of side effects, tolerability, preferences for treatment regimens, and quality of life. When appropriately conducted, CER has the potential to yield information about cost-effectiveness of interventions by providing high-quality comparative outcome data. 16
RCTs have long been considered the gold standard to evaluate the efficacy of a product compared with placebo. However, the results from RCTs may not be representative of the effectiveness of therapies delivered in real-world clinical settings to real-world patient populations. Furthermore, RCTs that compare new products with existing therapies are rare.17,18 CE studies include assessments of health outcomes attributed to alternative medical treatments or services that are of particular interest to payers, providers, patients, and other stakeholders. Typically, CER studies are designed to answer questions about treatment in ‘real-world’ medical settings. In contrast, clinical trials conducted for regulatory approval frequently do not incorporate comparisons with therapeutic alternatives, known as active comparators. In addition, such trials typically have exclusion criteria that limit the study population in order to minimize variations in response to treatment. These criteria increase internal validity while decreasing external one, as patients studied in clinical trials may differ significantly from the ‘real-world’ patient populations.
In addition, CER studies are expected to provide information about the relative effectiveness of different interventions, such as pharmacologic treatments versus surgical procedures or competing behavioral, care delivery, or policy interventions. 19 For device and diagnostic manufacturers, the distinction between comparative studies and more traditional study designs can be critical because effectiveness is often a function of extrinsic factors, such as how the device or diagnostic is used. Understanding the challenges and limitations of CE studies is necessary for the development of an effective evidence strategy and to support evaluation of the results of studies conducted by competitors and other health service researchers.
Forms of comparative effectiveness research defined by the federal coordinating council for comparative effectiveness research
The study types described in Table 3 are likely to have very different costs and time required for completion. The choice of study type should reflect careful consideration of the level of investment required and the value of evidence that is likely to be obtained. In many cases, the best choice that delivers the most value will address a specific evidence need and may not be the most costly. For example, systematic reviews of clinical and economic literature are typically conducted in the context of HTAs. Manufacturers can conduct and publish such reviews to critically assess the published evidence either to demonstrate effectiveness or to highlight an unmet medical need for patients treated with existing alternatives. Such a study is almost certainly less costly than a new decision analysis or observational study and may better highlight comparative evidence based on from multiple separate studies.
Clinical research for regulatory approval typically demonstrates safety and clinical efficacy, but it does not necessarily establish real-world effectiveness or demonstrate value in comparison with other options. Comparative clinical trials, particularly those that evaluate real-world use can address the evidence needs of payers and customers and potentially position a product for market success. The investment required to develop this type of comparative evidence must be balanced against the expected impact on the market. Similarly, economic studies may serve as an important source of information about evidence of value. Prior to product launch, decision analyses may provide information to support decisions about pricing strategy and identify potential evidence gaps. In one example, the Medicare Evidence Development and Coverage Advisory Committee reviewed screening CT colonography for coverage, but found that while the procedure was clinically effective, it would need to be less expensive in order to justify coverage. Additional criticisms included a lack of evidence demonstrating long-term benefit and the potential for risk associated with radiation exposure. 28 This case confirms that development of evidence demonstrating cost-effectiveness prior to coverage determination decisions may be a worthwhile investment. Going forward, more evidence will be required to change coverage policies.
Numerous methodological challenges arise from efforts to develop real-world evidence of effectiveness. Potential changes in study methods are a significant concern to pharmaceutical and device manufacturers, given the substantial investments in ongoing and future studies. Recent work has addressed these challenges with a number of methodological innovations, including designs for both interventional and database-oriented studies, data collection methods, statistical and modeling techniques. In addition, a significant component of the funding from the ARRA administered the AHRQ has been applied to improve methods for conducting CER. One particular approach that has received considerable attention is pragmatic or practical clinical trials (PCTs).29,30 Key features of PCTs are inclusion of clinically relevant alternative interventions, inclusion of a diverse population of study participants, recruitment of participants from heterogeneous practice settings, and data collection that includes a variety of health outcomes.
Additional study designs that may inform CER include outcome studies and cost-of-illness research. Outcomes studies are designed to evaluate the effects of a treatment in terms of patient outcomes, including adverse events, ability to function, and improvements in disease state. Cost of illness research is designed to demonstrate the impact of an illness or treatment for an illness on monetary cost. It has also become accepted practice for investigators to use secondary databases or patient registries, such as those available from state Medicaid programs, state-wide hospital databases, or the Department of Veterans Affairs to evaluate initiation, adherence, continuity of product use, and clinical outcomes regarding effectiveness, safety, and cost, including disparities in these measures associated with specific population groups. 31 However, the potential for confounding and bias is significant for analyses of administrative or other types of observational data. Strategies to minimize these threats include propensity scoring, inverse probability weighting, instrumental variables, and use of direct or indirect methods to accumulate and synthesize CE information.31,32
The International Society for Pharmacoeconomics and Outcomes Research Health Science Policy Council has established a task force to recommend state-of-the-art analytic techniques to achieve statistical control of potential sources of confounding for non-randomized studies based on secondary databases. 33 Recommended strategies include stratification analysis before multivariable modeling, multivariable regression including model performance and diagnostic testing, propensity scoring, instrumental variable, and structural modeling techniques. Residual confounding refers to confounding that has not been completely controlled, which increases the likelihood that confounding effects of some factors may alter the observed treatment-outcome effect. It is essential to assess residual confounding and estimate its effects with methods such as sensitivity analyses. 33
Conclusions
Requirements for evidence from multiple stakeholders have changed and continue to be modified. Decisions about the clinical and economic evidence needed for market success will primarily depend on each product’s particular situation, but all businesses will likely be influenced by key trends, including increasing state and national budget constraints. Fiscal crises and growing budget deficits are likely to impose additional pressure on limiting the growth in health care spending, particularly for spending to develop new health technologies. There are also increasing evidence requirements for effectiveness from payers worldwide, particularly for products with a large overall budget impact on payers and for products with less costly substitutes. In the US, concerns about the public health and safety of the 510k Premarket Notification process could result in changes that increase evidence requirements for US regulatory approval, with a significant potential impact on teams developing evidence strategy for products in development. There are also likely to be additional requirements imposed on diagnostic and therapeutic interventions. For example, imaging is increasingly being used not only for diagnosis, but also for planning and conducting medical and surgical therapeutic interventions. Imaging systems used in such a context may be subjected to additional rigorous evidence requirements that are more typical for Class III devices. Furthermore, as clinician hospitals and other medical settings increase their adoption of integrated electronic health records due, in part, to the financial positive and negative incentives imposed by ARRA, the value of integration with those systems will increase. Along with the growing use of clinical decision support systems, this could improve productivity and lower the marginal cost with each additional user. This changing context is likely to affect the cost-effectiveness of every manufacturer’s products and services, and CER is likely to play a fundamental role in measuring these changes.
Footnotes
Funding
This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Conflict of interest
The authors declare there are no conflicts of interest.
Author's Biographies
Karen K. Giuliano, RN, PhD is a Principal Scientist in Outcomes Research and Director of Corporate Business Development at Philips Healthcare.
Michael Ferguson is currently the Director of Health Economics and Comparative Effectiveness Research at Boston Scientific. At the time of manuscript preparation, Michael Ferguson, PhD was the Director of Clinical Outcomes, medical affairs, at Philips Healthcare.
Erica Silfen, MD, is Senior Vice President and Chief Medical Officer for Philips Healthcare.