Abstract
A systematic review of studies of the cost-effectiveness of telemedicine and telecare was undertaken from 1990 until September 2010. Twelve databases were searched, using economic evaluation terms combined with telemedicine terms. The search identified 80 studies which were classed as full economic evaluations; the majority (38) were cost-consequence analyses. There were 15 cost-effectiveness analyses (CEA) and seven cost-utility analyses (CUA). In the period January 2004 to September 2010 there were 47 studies. Eleven were CEA and seven were CUA. Economic tools are being increasingly used for telemedicine and telecare studies, although better reporting of the methodologies and findings of the economic evaluations is required. Nonetheless, the results of the review were consistent with previous findings, i.e. there is no further conclusive evidence that telemedicine and telecare interventions are cost-effective compared to conventional health care.
Introduction
Economic evaluations provide information about whether health-care technologies represent an efficient use of resources by comparing the costs and benefits of one health-care technology with another. In the case of telemedicine, it is usually compared with the conventional system of delivering health care. As health-care budgets are limited, resources should be allocated to technologies where the ratio of incremental costs to incremental benefits lies within a given cost-effectiveness threshold. This ratio (or threshold) represents the society's willingness to pay for an additional unit of health.
There are a number of guidelines for appraising the quality of economic evaluation studies. The standard checklist for critical appraisal is the one by Drummond and colleagues. 1 Others include: guidelines for economic submissions to the British Medical Journal, 2 Ever's criteria list 3 and Chiou's grading system. 4 Using economic evaluation guidelines will enable readers to make comparisons in their own settings and ensure that results are transferable and generalisable.
A few papers have described frameworks for the economic evaluation of telemedicine services. McIntosh and Cairns 5 suggested a cost-consequences framework, but they also noted some of the difficulties of evaluating telemedicine, such as the constantly changing technology, the lack of appropriate study design to manage the frequently inadequate sample sizes, the inappropriateness of conventional techniques of economic evaluation and the problem of valuing health and non-health outcomes. In contrast, Sisk and Sanders 6 proposed a cost-effectiveness framework and also noted some difficulties in evaluating telemedicine such as the system having multiple uses and joint costs that are difficult to apportion; and the rapidity of technological change.
Studies looking at the cost-effectiveness of telemedicine and telecare have not provided any conclusive evidence as to whether telemedicine is cost-effective. For example, after a systematic review of the cost-effectiveness of telemedicine studies, Whitten and colleagues concluded ‘that there is no good evidence that telemedicine is a cost-effective means of delivering health care’. 7 Roine et al. concluded that the ‘evidence regarding the effectiveness or cost-effectiveness of telemedicine is still limited’. 8 More recently, Bergmo found that ‘the majority of the economic evaluations (of telemedicine) were not in accordance with standard evaluation techniques’. 9
The aims of the present paper were to conduct a systematic review and to see how the evidence has changed over time and to assess whether the studies have adhered to the guidelines for reporting economic evaluations. 1
Methods
A systematic review was conducted to identify all published studies reporting the cost-effectiveness of telemedicine and telecare interventions (hereinafter referred to as telemedicine). The inclusion criteria related to including full economic evaluations 1 (cost-minimisation (CMA), cost-consequences (CCA), cost-effectiveness (CEA), cost-utility (CUA), cost-benefit analyses (CBA)) of telemedicine interventions. The exclusion criteria were cost-only or effectiveness-only analyses, telephone-only or email-only based studies, language other than English, articles where one project led to multiple reporting of the same results and articles published before 1990.
Search strategy
The following databases were searched from their inception date until 9 September 2010: Medline; Embase; Cinahl; EconLit; PsycInfo; ISI Databases (Science Citation Index; Social Science Citation Index; Arts and Humanities Citation Index); Database of Abstracts of Reviews of Effectiveness; NHS Economic Evaluation Database; Health Technology Assessment Database; and the Cochrane Databases. The search terms included economic evaluation terms combined with telemedicine terms (Table 1).
Example of search strategy
Data extraction
Once all abstracts were retrieved they were exported into the citation software package (Endnote Windows Version 6. Thomson ISI, Philadelphia PA, USA) and any duplicates were deleted. All abstracts were then read for relevance based on the inclusion and exclusion criteria. Relevant full-text articles were then obtained and a data extraction form was set up to retrieve information from the articles. The Drummond et al. checklist was followed, but not in a strict manner, because of other matters which were also important.
Results
In total, 2583 abstracts were retrieved after removing duplicates. After the exclusion criteria were applied, 173 articles were obtained and subject to a full review. Ninety-three studies were excluded as they did not meet the inclusion criteria (see Figure 1). Summaries of the 80 studies remaining are provided in the Appendix
Studies eliminated from or selected for the review after applying inclusion and exclusion criteria
The studies examined the use of telemedicine in various areas of health, e.g. there were eight studies each in dermatology and cardiology. Telemedicine was compared to the conventional method in 61 studies and in 15 studies telemedicine was compared to two alternatives. Forty studies were conducted in North America and 32 studies were from Europe. Nearly half of the studies were published in telemedicine journals (n = 38). The year 2004 was the one in which the most articles on the cost-effectiveness of telemedicine (n = 12) were published. The majority of studies evaluating telemedicine interventions were conducted over a two-year period or less (n = 56); 11 studies did not report the period.
Telemedicine equipment and consultations
Forty-three studies reported the equipment lifetime (varying from 2 to 10 years; mode = 5 years) and/or depreciation rate (varying from 2.5% to 8%). The number of telemedicine clinics/consultations was reported in 24 studies. In addition, 23 studies also reported the time duration for telemedicine and only 15 of these reported the time durations for both alternatives. Twenty-eight studies reported the transmission mode for the telemedicine consultations: 18 studies used real-time communication, eight studies used store-and-forward communication and two studies used both methods.
Overall methods of cost-effectiveness
The methods used included 18 CMA, 38 CCA, 15 CEA, seven CUA and two CBA. Only 44 studies informed the reader of the viewpoint of the analyses: 12 were from a societal perspective and 22 studies were from the health-care perspective. The benefits varied from using health state utilities such as QALYs gained (n = 7), to generic measures such as the SF-12 (n = 8). Other studies used a range of outcome measures such as mortality, clinical outcomes such as the number of cases averted, patient satisfaction or health scales. Some studies investigated benefits in terms of transfers and/or the number of hospitalisations avoided.
Only eight studies used discounting for costs. This is reasonable as 43 studies reported a time frame of less than or equal to a year. Thirty-eight studies applied sensitivity analyses to assess the robustness of the results. The overall conclusions across all studies provided by the authors showed no general agreement whether telemedicine was cost-effective compared to conventional means.
Change in the cost-effectiveness of telemedicine over time
In the period January 2004 until September 2010, there were 47 studies on the cost-effectiveness of telemedicine interventions. Eleven studies were CEA and seven were CUA; six studies used discounting; 23 studies undertook sensitivity analyses; and 14 studies reported incremental cost-effectiveness ratios (ICERs). This demonstrates that economic tools are increasingly being used for evaluations of telemedicine.
Discussion
Cost-effectiveness studies are needed to help define the appropriate scope and application of telemedicine interventions in different settings and to provide information to the reader and the decision-maker on whether the use of these interventions in various health areas represents good value for money. 10
Matters arising from the literature search
Eight matters arose from the literature search:
Inadequate details about study design and methodologies. One-third of the studies (n = 28) were randomised controlled trials and the rest were observational studies (3 studies used a before and after design). As the majority of studies were non-randomised, there is potential for selection bias and thus the cost and effect results presented by each study might be biased. Therefore, steps should be undertaken to ensure that selection bias is minimised in the analysis of non-randomised telemedicine studies. Second, 56 studies were undertaken for a period of less than two years, thereby not allowing an assessment of the longer-term effect on patients, on costs or outcomes. As some of the studies were pilot services, the costs and benefits may not reflect the true costs and benefits when the service comes into routine use. For example, most pilot studies were for diagnostic or consultation purposes and the longer term follow-up was not usually measured. Therefore, generalising the study findings to other settings may not always be possible. Third, most of the studies identified did not give adequate details about their study design or provide enough information on how costs (and/or outcomes) were collected, calculated and reported. If studies report the appropriate methodology, it would then be easier for the reader to understand how the author(s) came to their results and conclusions. For example, Tsitlakidis et al.
11
compared telemedicine to referring patients directly and found that the mean duration of a telemedicine consultation was 15 min compared with 40 min for a direct referral. The authors did not explain whether it was the telemedicine intervention itself that allowed for this substitution of time saving. Study perspective. The study perspective should be clearly stated, as it allows the reader to decide whether the information provided in the article is sufficient to answer the study question. Nearly half of the articles (n = 36) did not explicitly report the viewpoint of the analysis. Thus it is hard to make judgements as to whether the conclusions reached by the authors are appropriate. Small sample sizes. Only 65 studies (81%) reported patient numbers and 25 of these studies (39%) had a sample size of 100 patients or less. Studies with small sample sizes will have low statistical power to detect differences in health outcomes and the difference between the two groups may not always be due to the telemedicine intervention. Choice of alternatives. Most studies described the telemedicine intervention explicitly, but costs and outcomes in relation to the comparator were not reported in sufficient detail to make proper comparisons. For example, Bracale and colleagues
12
compared telemedicine with the alternative of patient transfer by boat or by helicopter ambulance. They described how the telemedicine services were established and how they worked, but no further details were given about the comparator. Type of economic evaluation adopted for the telemedicine study. Eighteen studies (23%) conducted a CMA, in which the costs of the telemedicine intervention were compared to the alternative. These studies assumed that the outcomes were equivalent or were assumed to be established without any proper scientific evidence. For example, Armstrong et al.
13
conducted a CMA of teledermatology compared to conventional care. However, they did not provide any evidence to show that outcomes were identical. Studies otherwise can simply be cost studies with no claim to being an economic evaluation. Unless outcomes can be proved to be identical, then the economic evaluations should take place in the form of a CCA, CEA, CUA or CBA. The type of economic evaluation undertaken depends on the context of the study, the comparator(s) and the outcome measure. Costs. Costs can be categorised as: direct medical costs which are costs that usually fall on the health service; direct non-medical costs, which are costs incurred by patients; and indirect costs, which are usually classed as lost productivity. All studies except one reported direct medical costs. Twelve studies were from a societal perspective (all reported direct medical costs). In addition, 10 of the 12 studies reported direct non-medical costs and out of those 10 studies, six also provided information on indirect costs. For any economic evaluation the concept of opportunity cost is essential. The viewpoint of the analysis is important so that the appropriate resource use and costs are identified for the telemedicine service and the comparator for measurement and valuation purposes. The information provided in the articles varied from detailed cost analyses to simply mentioning some of the costs (i.e. cost components were not exhaustive). Therefore caution should be taken when using cost estimates from previous analyses as the studies are not explicit in reporting their methodology and results and allocation of costs may not be accurate. The inconsistency about the costs which were collected and reported in these studies was confirmed by Hailey and colleagues
14
who said ‘the costs included varied significantly between studies, so that the comparison of the cost estimates was not feasible in many cases’. The studies also made assumptions in the calculation of telemedicine equipment costs such as the expected lifetime of the equipment, the discount rate which varied from country to country, and the maintenance charges, which differed depending on length of contracts. The cost assumptions made in these studies may not reflect how telemedicine services are used in practice. For economic evaluations, the large capital outlay of telemedicine applications is converted into an annual cost over the expected lifetime of the equipment. This allows a comparison of the annual cost of the systems without being biased against a system which has a larger capital outlay, but a longer life.
5
Telemedicine costs are largely determined by the scale and utilisation of the service. For example, if more patients use a telemedicine service, the costs for the service should be lower. Benefits. Most studies did not report information on benefits in such a way that would enable a CEA to be performed. The benefits of telemedicine may take a while to appear and may also be difficult to quantify. The few studies that did report outcomes used surrogate measures such as travel or length of stay avoided, rather than clinical improvement. In economic terms, these measures are not regarded as benefits; instead they should be classed as averted; costs. Some studies used non-health benefits as outcome measures, such as improvement in quality of care through improved treatment, faster and more accurate diagnosis, transfer of skills and knowledge, speed of service, improved training and education, and reassurance. The question is how do you identify, measure and most importantly value these non-health benefits and should they be included in the evaluation? McIntosh and Cairns
5
commented ‘this would require a valid and reliable instrument which is sensitive enough to detect beneficial changes in the process of care of value to the patient’. The different types of benefits in telemedicine studies can be categorised as:
A) Improved access to health care. Improved access to health care through telemedicine means that diagnoses can be made more quickly and this may translate not only into better health for the patient, but also in the reduced need for transfers and hospital stays. Sicotte and colleagues
15
compared paediatric telecardiology with conventional care. The authors said that access to telemedicine provided correct and earlier diagnosis of cases and this in turn eliminated unnecessary travel and provided reassurance.
B) Patient satisfaction. Aspects of patient satisfaction that are usually evaluated include: convenience, acceptability and willingness to use telemedicine in the future, and this enables decision-makers to understand more about patients' experience of using telemedicine, which may be important for the future adoption of telemedicine. The 14 studies which used patient satisfaction as an outcome measure, the patients' documented great satisfaction with telemedicine.
C) Effectiveness (including quality of life measures). Effectiveness measures include clinical outcomes which are the results of the interventions used to diagnose (i.e. screening) and manage patients (i.e. treatment plans). For example, Whited et al.
16
compared a digital ophthalmology system with clinic-based examinations to detect diabetic retinopathy and there were three outcomes: number of true positive cases detected, number of patients requiring laser treatment and number of cases of severe vision loss averted. Various instruments are used to measure quality of life such as: generic measures, which measure overall quality of life (e.g. SF-12); utility measures, which give an indication of value placed upon quality of life (e.g. EQ-5D); and monetary measures, which values benefits in terms of currency using ‘willingness to pay techniques’. For example, Jacklin and colleagues
17
compared conventional outpatient consultations with teleconsultations and the authors used the SF-12 as an outcome measure and Castillo-Riquelme et al.
18
used a utility measure, quality-adjusted life years (QALYs) to measure health outcomes when comparing the cost-effectiveness of alternative methods of screening for retinopathy of newborns. Incremental approach. An incremental approach allows the decision maker to see the additional cost and/or benefit of the intervention compared to the alternative and only 16 studies provided an ICER. For example, Sicotte and colleagues
15
estimated the ICER as C$3488 per patient journey avoided. This outcome may be useful to clinical decision-makers; however, it is not meaningful to health service administrators. Unlike the cost per QALY ratio,
19
such a measure does not allow for comparisons across different diseases, treatments or patient groups and in turn decide whether this would result in net costs or savings to the health service. Furthermore, the majority of studies stated whether a telemedicine service was cost-effective or not; however, the authors did not define what they meant by cost-effective or provide a cost-effectiveness threshold.
General discussion
The present paper updates previous systematic reviews. There is no further evidence that telemedicine interventions are cost-effective compared with the conventional health care. The methods used by investigators to conduct and report their work need to be improved to ensure that economic evaluations of telemedicine conform to standard checklists. 1–4
The cost-effectiveness of telemedicine depends not only on the service being evaluated, its comparator, the perspective, patient group and sample size, type of economic analysis and how the costs and outcomes are measured, but also on the take-up rate and the usage of the service. As telemedicine services move into routine use other factors may need to be considered such as the sustainability of the telemedicine activity and the organisation of health services. 10 The present review shows that telemedicine is a broad term and if further reviews or analyses are undertaken then they need to be categorised, such as real-time versus store-and-forward telemedicine, or home versus hospital-based telemedicine. For this reason, it would be unrealistic to attempt to make broad generalisations about the cost-effectiveness of telemedicine. 10
As mentioned, the cost-effectiveness of telemedicine may also vary because of differences in subspecialty (i.e. telecardiology compared with teledermatology) or by use of technology (i.e. videoconferencing compared to internet-based applications). For example, the present review identified eight studies each for cardiology and for dermatology which used telemedicine. The majority of telecardiology studies were not randomised (n = 7) compared to four teledermatology studies; and telecardiology used either a CCA or a CEA, whereas teledermatology used all types of economic evaluations except a CUA. Three studies for teledermatology were from a societal perspective whereas none were from a societal perspective for telecardiology; and finally, only two studies for each application reported their results in terms of ICERs. Even when looking at these few studies within these subspecialties it is hard to make substantial conclusions regarding their cost-effectiveness as there were variations, for example, in the study design and perspective. This was re-iterated in a systematic review by Wade et al. 20 who concluded that real-time telemedicine was cost-effective for home care, but not for local delivery of services between hospitals and primary care.
Finally, the present review identified a few points which are relevant when articles are being assessed for publication: research teams do not always have health economists on their team for the economic evaluations; journals are often restrictive on word length, preventing the detailed reporting of methodology and findings; and publication relies on completed data analysis, which may take a few years from the end of the fieldwork. 10
In conclusion, the results from the present review found that over time there has been an increased use of economic tools for evaluation of telemedicine. However, there is still a need for telemedicine studies to be transparent in reporting their methodologies and results. Decision-makers must be cautious about applying the results of such assessments to their own circumstances. Economic evaluations of telemedicine need to be conducted in accordance with general standards for health economic evaluations.
Footnotes
Acknowledgements
I thank Peter Auguste for helping to review the abstracts and Dr Emma Frew and Sabina Sanghera for their helpful comments. This review formed part of my PhD thesis.