What has health technology assessment ever done for us?

Introduction

Health technology assessment (HTA) has been one of the most popular tools of policy on health care reimbursement, pricing and purchasing over the past 30 years. With the aim of informing coverage decisions by means of multidisciplinary evaluation of evidence, HTA has the backing of those advocating evidence-based decisions. Public and private consultancies spread the message of HTA around the world,^1,2 as do international organizations such as the World Health Organization, ³ the World Bank, ⁴ European Union ⁵ and professional networks such as HTAi. ⁶

Many governments have been seduced by HTA’s promise of being able to improve the equitable allocation of resources in health care. Since the late 1980s, most of Western Europe as well as Australia, New Zealand, Canada and several US payer organizations have established HTA agencies. Some Asian (e.g. Taiwan), Latin American (e.g. Brazil) and Central and Eastern European (e.g. Poland) countries have followed in the early 2000s. Other countries, however, have hesitated. So what has HTA to offer to decision-makers in countries without institutionalized HTA, especially in low and middle income countries?

Vague benefits

HTA is especially important in resource-tight contexts because opportunity costs of misdirected resources are higher, in relative terms, than in rich economies.^7,8 However, policy evaluation of HTA is practically non-existent and empirical evidence of the impact of HTA on either health outcomes or spending is scarce.^9–11 One review found only four published studies. ¹² Most focus on outputs (e.g. number of HTA reports produced) and the extent to which conclusions of HTA reports are followed by decision-makers ¹⁰ – leading an observer to conclude that ‘the available knowledge to assess the effectiveness of HTA is just a bunch of “case series” and “case reports,” with little external validity and usually surrogate outcomes’. ¹³ We know next to nothing about impact on health outcomes. ⁹ One of the most comprehensive country-level evaluations excluded health outcomes ‘due to methodological limitations’. ¹⁴ Similarly, a recent report on the United Kingdom’s HTA programme did not assess outcomes. ¹⁵ Except for an industry-commissioned report,¹⁶ improved access to care as a result of HTA is not explicitly mentioned in any reviews.⁹

Evaluations of economic impact are mixed. An early Canadian study found projected annual savings between $16 and $27 million,^17,18 while an Austrian study concluded HTA recommendations had led to a ‘significant’ reduction in expenditure, but deemed precise quantification impossible.^14,19 Far from creating savings, one review noted that guidance issued by the United Kingdom’s NICE led to an increase in spending. ¹⁰ Meanwhile, the immediate and medium-term budgetary consequences of establishing an HTA body in low income countries have not been investigated. ⁹ Overall, the empirical benefits of HTA are unclear. We know little about how HTA affects patients’ health and access to care, and little about what it does to public budgets.

Clear costs

For a field with ‘assessment’ in its name, the absence of empirical evaluation is surprising. Policy evaluation is a complex exercise, in this case, further complicated by the lack of an agreed approach to the design and implementation of HTA or, for that matter, HTA methodology. Such variation might explain why much of the HTA literature focuses on the use of HTA by decision-makers, the assumption being that impact can only be measured under ideal conditions, where expertise reigns unhindered by politics. However, this is of little relevance to policy-makers looking for success stories and opportunities to draw lessons from other countries.

The upfront costs, on the other hand, are relatively clear. Setting up an HTA agency is not cheap: Germany’s IQWiG has an annual budget of EUR 13 million (USD 14.8 million); the Polish AOTMiT EUR 3.5 million and Belgium’s KCE EUR 10 million. ²⁰ The latter represents about 30% of the annual budget of a large university hospital in the Czech Republic. ²¹ Further, there is a concern that HTA might increase expenditure. An independent agency with decision-making powers may prioritize evidence of cost-effectiveness over budget impact more than the ministry of health or payers, leading to the provisions of technologies that would have otherwise been denied reimbursement. Even a purely advisory body may make implicit rationing difficult for decision-makers and bring unwanted attention to a lack of funds or inefficiencies in the health system. Likewise, introducing a HTA agency could destabilize the practice of delaying reimbursement decisions, common to some resource-tight countries. While all of these consequences may be good news for patients, they are unlikely to be attractive to policy-makers focused on short-term health gains and cost-containment.

HTA is also not easy to implement: creating new agencies and adjusting pricing and reimbursement processes requires considerable legislative effort. In addition, many low income countries will first need to train sufficient numbers of HTA experts. Both the financial costs and the effort might well be worth it if policy-makers are convinced HTA can deliver on its promises.

Modernizing mission

Beyond the uncertain effects of HTA on health systems, an additional promise of HTA concerns its consequences for decision-making styles and cultures and for social justice. Some evaluations suggest, in line with Weiss’ enlightenment conceptualization of the knowledge-policy relationship, ²² that HTA acts by changing mindsets rather than immediately determining actions.^12,14 Institutionalized HTA marks a departure from the opaque and arbitrary pricing and reimbursement practices customary in many countries. It is a departure underpinned by normative and epistemic beliefs in the superiority of evidence-based decisions, independence of expert input, transparency and the inclusiveness of stakeholders. ²³ These are linked to the rise of evidence-based medicine ²⁴ and, perhaps more generally, good governance and the trend of expert decision-making. ²⁵ However, none of these principles guarantee improvements of health systems. Whether such policy-making leads to better outcomes than ‘muddling through’ incremental adjustments, has been a debate for decades, ²⁶ and good governance probably attracts more critics than advocates. ²⁷ However, for some decision-makers, a focus on evidence, transparency and inclusiveness represent values in themselves.

An equally compelling promise of HTA (and equally difficult to quantify) is its potential to increase procedural justice in allocative decisions. ²⁸ Here, too, much depends on the details of the institutional setup of HTA – Daniels and van der Wilt, for instance, argue that a deliberative element is necessary for HTA to produce legitimate and fair decisions. ²⁹ However, HTA’s proposal to reduce the arbitrariness of decisions touches directly on ambitions of equity.

For many low and middle income countries, the combination of theoretical and normative arguments for HTA offers a powerful modernizing vision for their health systems. For example, a 2014 World Health Assembly resolution mentions concern for greater efficiency and the need for evidence-based policy-making as reasons for encouraging HTA. ³⁰ This makes HTA hard to dismiss, despite an absence of empirical evidence of its effects. Other policies, from international reference pricing to risk-sharing agreements or implicit rationing, may be easier to implement and fare better at containing costs, but none offer as complex a promise as HTA. This makes HTA currently a policy without direct alternatives, potentially attractive to policy-makers around the world for many different reasons.