Retrospective Return on Investment Analysis of an Electronic Treatment Adherence Device Piloted in the Northern Cape Province

Abstract

Objective: The return on investment (ROI) for utilizing the SIMpill electronic treatment adherence solution as an adjunct to directly observed treatment short-course (DOTS) is assessed using data from a 2005 pilot of the SIMpill solution among new smear-positive tuberculosis (TB) patients in the Northern Cape Province. The value of this cost minimization analysis (CMA), for use by public health planners in low-resource settings as a precursor to more rigorous assessment, is discussed. Materials and Methods: The retrospective analysis compares the costs and health outcomes of the DOTS-SIMpill cohort with DOTS-only controls. Hypothetical 5-year cash flows are generated and discounted to estimate net present values (NPVs). Results: Comparison between the DOTS-SIMpill pilot cohort and DOTS-only supported controls, for a hypothetical implementation of 1,000 devices, over 5 years, demonstrates positive ROI for the DOTS-SIMpill cohort based on improved health outcomes and reduced average cost per patient. The net stream is shown to be positive from the first year. Discounted NPV is ZAR 3,255,256 (US$ 493,221) for a cohort that would have started mid 2005 and ZAR 3,747,636 (US$ 487,339) starting mid 2010. This is an ROI of 23% over the 5-year period. Conclusion: The addition of electronic treatment adherence support technology can help to improve TB outcomes and lower average cost per patient by reducing treatment failure and the associated higher cost and burden on limited resources. CMA is an appropriate initial analysis for health planners to highlight options that may justify more sophisticated methods such as cost effectiveness analysis or full cost benefit analysis where a preferred option is immediately revealed. CMA is proposed as a tool for use by public health planners in low-resource settings to evaluate the ROI of treatment adherence technology postpilot and prior to implementation.

Introduction

Tuberculosis (TB) is a growing problem in South Africa.¹ In 2008 the estimated incidence of all forms of TB was 948 cases per 100,000 people and the estimated incidence of new sputum smear-positive pulmonary TB cases has increased from 141 per 100,000 people in 1997 to 358 per 100,000 in 2007.² This increase in disease burden has occurred despite an increase in total expenditure on TB control from approximately US$ 1,000 per patient in 2005 to US$ 2,000 per patient in 2008.³

Directly observed treatment short-course (DOTS) regimen is the cornerstone of controlling TB. One of the aims of DOTS is to ensure patient medication compliance. Approximately 10%–15% of patients in South Africa default on treatment and treatment success rates remain low.^4,5 Failure of compliance is associated with the development of multiple drug-resistant TB (MDR-TB), the prevalence of which increased from 1.8% in 2004⁶ to 9.6% in 2011.⁷

Facilitating cost-effective delivery of healthcare services is a key objective of telemedicine in developing countries.⁸ Emerging good practice suggests that a combined approach, integrating human “face-to-face” support with technology that improves communication, may provide advantages for TB-control programs. Short message service (SMS) technology has been used to support patient adherence in a number of TB-related projects.⁹ Smart pill boxes have also been used to attempt to improve compliance. Of eight different devices described in a review of “Smart-Pillboxes” designed to improve medication compliance,¹⁰ one included a capability of uploading data to an online database. None of the eight devices provided for real-time communication to treatment supporters in the event that the patient persistently failed to open the pill bottle at the correct time.

The SIMpill^® system (Simpill Pty Ltd., Western Cape, Cape Town, South Africa) is a cellular telephone SMS-based medical adherence support (MAS) system developed in South Africa. It consists of a device that attaches to a standard pill bottle or blister pack and communicates with a Web-based application by SMS every time the patient opens the bottle or removes a tablet from the blister pack. This is taken as a surrogate marker of the patient taking their medication. The solution enhances existing SMS-alert systems by only sending SMSs when patients do not take the medication (open the bottle or use the blister pack) as prescribed.¹¹ A patient motivated to “fake” adherence, however, can cheat the system.

Caregivers discuss medication schedules with patients and agree reasonable time windows within which each dose should be taken. This information is entered into a Web-based application that stores a profile of each patient and collects information about medication events each time the MAS device is opened. By comparing medication events with expected time windows, the application detects noncompliance and provides appropriate SMS notifications to patients and caregivers in real time. Patients require only a few minutes of direction to ensure that they are able to operate the device correctly.

The Web-based application collects and reports compliance information. Patients and approved healthcare professionals or analysts can be given access to real-time information regarding medication use and compliance through a private secure account on a Web site. Healthcare providers can monitor utilization patterns in real time to help identify interventions that might support adherence, make changes to patients' medication schedules, and generate compliance reports. The SMS notifications can be complemented with email notifications.

If medication is taken outside the prescribed times, an SMS is sent to the patient advising of the mistake and reminding the patient when the next prescribed medication event should take place. If the SMS reminder fails to prompt the patient to take their medication, an SMS can be sent to a person identified by the patient when they registered with the program, such as a family member or home-based caregiver. Additional escalation levels can be added if necessary and an SMS sent to the professional nurse responsible for supervising home-based caregivers. This allows prioritization of patients requiring additional support. The system monitors medication refills and additional SMSs can be sent to the pharmacy or clinic to advise that the patient will soon need a new prescription.

Habituation, the decreased behavioral responsiveness caused by repetitive stimulus,¹² is avoided by not sending an SMS if medication is taken as prescribed. Kenyan investigators sending scheduled SMSs to improve antiretroviral treatment adherence site habituation as the reason that weekly SMS reminders improved adherence by 13%, whereas daily reminders resulted in no improvement.¹³

In 2008, the South African National TB Treatment Program budget was US$ 7.2 per capita while the total cost for TB control was US$ 12 per capita. Treatment adherence in South Africa is poor, resulting in retreatment and the development of MDR-TB and more recently, extensively drug-resistant TB (XDR-TB).^14,15 This further increases treatment costs.

In 2005 a limited pilot project was undertaken at a clinic in the Northern Cape Province, using the MAS device as an addition to standard DOTS. Cost effectiveness of the pilot was never assessed. The local department of health is now considering implementing an MAS solution in the expectation of improving outcomes and managing costs.

Before considering wider implementation of this type of electronic treatment adherence support, the related costs and benefits, if any, need to be calculated. E-health economic assessment is complex.^16,17 Telemedicine economic evaluations remain rare¹⁸ and few are conducted in accordance with standard evaluation techniques.¹⁹ Challenges to adequate economic evaluation include frequently inadequate sample sizes, complexities of valuating health and nonhealth outcomes, and rapidly changing technology.²⁰ No literature has been found that proposes or evaluates a structured method of comparing the return on investment (ROI) obtained from conventional DOTS with that achieved by integrating DOTS with real-time SMS adherence support.

The study examines the costs and health outcomes of a pilot implementation of an MAS solution and compares it with a control group supported by conventional DOTS methods only. The appropriateness of the ROI method for evaluating MAS solutions in a developing world setting, where data are often limited, is discussed.

Background

To avoid confusion, terminology is defined.

Cost effectiveness analysis (CEA) expresses results in a single dimension such as “per lives saved” or “per pain free day” to enable direct comparison.²¹

Cost-utility analysis (CUA) aims to overcome the one-dimensional limitations of a CEA by using utility-based outcome units to compare different interventions.^22,23 Utilities are cardinal values assigned to health states²⁴ based on a scale of 0 to 1, where 0 represents death and 1 perfect health. Utility can be expressed in “quality adjusted life years.”

Cost-minimization analysis (CMA) is used to compare two interventions and identify which is least costly. It is applicable when interventions are expected to have similar outcomes.²⁵ CMA is less complex than CEA, which includes both costs and outcomes of treatments and is sufficient if the outcomes for each intervention are equivalent. Since few procedures are likely to have the same outcome, CMAs are considered to be of limited use.²⁶

Cost benefit analysis (CBA) is a highly flexible method of economic evaluation²⁷ that has been described as “the methodological foundation for turning theory into a pragmatic evaluation tool.”²⁸ It aims to measure both costs and consequences in monetary units.²¹ Approaches such as the Cost-Effects-Utility-Benefits model of Drummond et al.²⁹ combine elements of all the previous methods. The ROI analysis considers the impact of an investment.

The application of CEA, CUA, or CBA can be challenging in the developing world setting where accurate patient follow-up is often difficult or incomplete and adequate data are not readily available.³⁰ To overcome this, this study utilizes CMA as a precursor to a more definitive study. CMA is used to arrive at a cost comparison between patients using the MAS and the controls. ROI assessment calculates discounted break-even analysis and net present value (NPV). Direct and indirect costs and benefits are included. Intangible costs and benefits are not considered. Health outcomes are assessed to establish whether extension to CEA will be of value.

Until recently, few reports have evaluated the use of telemedicine in live projects, despite considerable literature describing the potential value of telemedicine in developing countries.³¹ A review of telemedicine experience in the developing world emphasizes that cost effectiveness is a fundamental aspect of the introduction of telemedicine and cautions that very few studies were found in which the topic was addressed.³² This study proposes evaluation of costs and health outcomes as a precursor to more extensive analysis, utilizing data that are usually readily available and a method that allows for a less cumbersome though useful assessment to be conducted. Expansion to full CEA or CBA, which requires considerably more data and expertise, will be appropriate for selected studies.

Methods

A pilot study utilizing the MAS device was undertaken at the Betty Gaetsewe Clinic located near Kimberly in the Northern Cape Province of South Africa in 2005. The Northern Cape is one of nine provinces in South Africa. It has a relatively stable population recorded as 2.3% of the country in 2005 and 2.2% in 2010³³ served by 34.3 doctors per 100,000 people.³⁴ Data were obtained from officials of the Northern Cape Department of Health and Betty Gaetsewe Clinic.

Twenty-four outpatients were enrolled in the MAS study group, the maximum number permitted by the funding allocated to the pilot by the Department of Health. All were new smear-positive TB patients who commenced the 6-month treatment regimen on first line anti-TB medication during 2005. Entry to the study was voluntary and subjects were sequentially enrolled during 1 month. Illiterate patients, reinfected patients, and patients without mobile phones were excluded. The average number of patients seen per month during 2005 was 21.

Frequency matching was utilized to match the MAS pilot participants with controls in a 4:1 ratio. Controls were selected from new smear-positive TB patients who presented to Betty Gaetsewe Clinic and commenced the 6-month treatment on first line anti-TB medication during the time that the MAS pilot was running. It was not possible to match the month in which the MAS cohort started treatment since all the eligible patients presenting during the month were enrolled in the pilot. Patient records for the months during the time that the MAS pilot was running were examined to identify controls who were frequency matched for TB treatment, local clinic, and gender and age.

Six of the study group died of acquired immune deficiency syndrome (AIDS)–related illness during the study. An attempt was made to identify controls who had died during treatment, to match for this potential confounder; however, none of the controls identified were found to have died. Clinic personnel believed that this was because the folders of patients who died are removed from the clinic, stored elsewhere, and because of the elapsed time could no longer be retrieved. The analysis was therefore split into two scenarios as follows: (1) scenario one included all 24 MAS patients and their 96 controls and (2) scenario two excluded the six MAS patients who died, along with their 24 matched controls, thereby reducing the cohorts to 18 MAS patients and their 72 controls.

Costs were calculated for the treatment of new smear-positive drug-sensitive patients, retreatment of drug-sensitive patients, and treatment of MDR-TB. For the purpose of this evaluation, XDR-TB was not considered since at the time of the study there was only one case in the entire Northern Cape Province and none at Betty Gaetsewe Clinic.

Financial values are shown in South African Rand (ZAR) and United States Dollar (US$). Exchange rates were US$ 1=ZAR 6.60 (June 1, 2005) and US$ 1=ZAR 7.69 (June 1, 2010). Statistical analysis of the differences in outcomes between the groups was conducted using two-tailed Fisher's exact test with alpha set at 5%.

The data are applied to two hypothetical implementations, each utilizing 1,000 MAS devices to support DOTS with new smear-positive TB patients. One hypothetical implementation starts mid 2005 and the other mid 2010. Discounted break-even analysis is utilized to determine the duration of implementation required until the net revenue stream is positive and to derive the NPV for the hypothetical implementation. The NPV is a technique used to determine today's value of a proposed intervention. It is the sum of discounted net benefits over time. These net benefits are the sum of a series of cash flows, both incoming and outgoing, discounted to convert future benefits into present values.³⁵

Results

The demographics of the subjects and their controls in scenarios one and two are shown in Table 1.

Table 1.

Participant Demographics

	SCENARIO ONE		SCENARIO TWO
	MAS PILOT	CONTROL GROUP	MAS PILOT	CONTROL GROUP
n	24	96	18	72
Age (years)	37.9 (SD 16.22)	36.4 (SD 14.85)	36.4 (SD 16.99)	35.2 (SD 15.41)
% male	54.2%	54.2%	50.0%	50.0%

MAS, medical adherence support; SD, standard deviation.

Costs

The costs are determined by direct comparison of the costs of each group of patients. The average costs per patient in each group are summarized in Table 2.

Table 2.

Summary of Average Cost per Patient (Values in South African Rand)

	SCENARIO ONE		SCENARIO TWO
	MAS PILOT	CONTROL GROUP	MAS PILOT	CONTROL GROUP
Medication	296	986	296	1,026
Lab tests	60	165	60	179
Hospitalization	0	2,117	0	2,117
Healthcare personnel costs	2,021	2,659	2,021	2,713
MAS device	1,200	0	1,200	0
MAS device support	1,080	0	1,080	0
	4,657	5,927	4,657	6,035
Total	(US$ 706)	(US$ 898)	(US$ 706)	(US$ 914)

Healthcare personnel costs are derived by multiplying the average time each personnel type allocates per TB patient by the total cost-to-employer for each personnel type.

For the calculation of projected costs, the cost of the MAS device is based on a rental model thereby eliminating concerns about the potential lifespan of a device. Although new smear-positive patients are put on a standard 6-month treatment regimen, the possibility of delays due to battery changes and maintenance results in the average utilization per device taking more than 6 months. For the purposes of this study the average number of clients serviced by one device per year was set at 1.71. This is calculated by assuming that each device will serve a TB patient for 6 months, after which 1 month will be allowed for the device to be serviced, repaired, and prepared for the next patient.

The average medication costs for the different drug-sensitivity profiles at the time of study were ZAR 295.58 (US$ 44.78) for drug-sensitive new smear-positive patients, ZAR 1,732.17 (US$ 262.45) for retreatment of drug-sensitive patients, and ZAR 27,117.28 (US$ 4108.68) for treatment of MDR-TB. The large treatment cost difference between TB and MDR-TB patients is the dominant cost driver that results in cost savings when MDR-TB cases are reduced, despite the additional technology investments required.

The retreatment rate varied considerably between MAS and control cohorts. No subjects in the MAS group required retreatment, while 16.7% of the control group of scenario one and 19.4% of the scenario two controls required retreatment. This becomes significant because retreatment requires the patient to undergo an additional 9 months of treatment, with intramuscular Streptomycin injections being administered daily at the local clinic. The additional cost of the medication, as well as the human resource allocation, is an important element of the costing analysis.

MDR-TB is more expensive to treat than TB that is susceptible to either first line or second line (with the addition of Streptomycin) treatment regimens. A costing line is therefore also allocated to the MDR rate. No MAS cohort patients developed MDR-TB. Since patient-specific data were not available on progression of the control subjects to MDR, this was estimated using the MDR incidence of 1.7%³⁶ reported by government officials for the region in which the study took place and used for projected costing. The prevalence of MDR-TB has risen in the Northern Cape from 164 cases in 2005 to 482 in 2010.³⁷ It appears that good adherence among new smear-positive cases may be protective against MDR-TB in much the way that adherence was found to ensure that patients did not require retreatment.

For the purpose of this evaluation, XDR-TB was not considered.

Effects

The effects (health outcomes) evaluated were the smear conversion rate after 2 months of treatment, the treatment outcome after completing treatment (where negative sputum in the last month of treatment was regarded as treatment success or cure) and the MDR-TB rate. For both of the participant scenarios, the MAS pilot group was compared with the corresponding control group using the Fisher's exact two-tailed test. The results are summarized in Table 3.

Table 3.

Treatment Outcomes

	SCENARIO ONE			SCENARIO TWO
	MAS PILOT	CONTROL GROUP	p	MAS PILOT	CONTROL GROUP	p
Smear conversion rate	62.5%	38.4%	0.0403	83.3%	43.10%	0.0031
Cure rate	75.0%	32.3%	0.0003	100%	34.72%	0.0001

Break-Even Analysis

Break-even analysis is used to determine the theoretical point in time from starting a hypothetical implementation of 1,000 MAS devices, to the time that the deployment will have generated benefits equal to cost (T _R=T _C). Present value tables are used to discount annual figures and inflation is set at 8% per annum. The figures used for the break-even analyses are shown in Tables 4 and 5, for hypothetical implementations starting in 2005 and 2010, respectively.

Table 4.

Break-Even Analysis, 2005–2010 (Values in South African Rand)

YEAR	2005	2006	2007	2008	2009
MAS device rental	1,400,000	1,512,000	1,632,960	1 763,597	1,904,685
MAS device support	1,260,000	1,360,800	1,469,664	1 587,237	1,714,216
Cost to implement SIMpill	2 660,000	2,872 800	3,102,624	3 350,834	3,618,901
Savings per 1,000 patients	3,274,896	3,536,888	3,819,839	4 125,426	4,455,460
Net cash flow	614,896	664,088	717,215	774,592	836,559
Discounted costs	2,660,000	2,736,000	2,814,171	2,894,576	2,977,279
Discounted savings	3,274,896	3,368,464	3,464,706	3,563,698	3,665,518
Discounted net cash flow	614,896	632,464	650,535	669,122	688,239
Net stream	614,896	1,247,360	1,897,895	2,567,017	3,255,256
ROI net stream as % of discounted costs	23%	23%	23%	23%	23%
Net present value (in 2005 for a 5-year implementation) 3,255,256
Break-even analysis
Cumulative costs	2,660,000	5,532,800	8,635,424	11,986,258	15,605,159
Cumulative savings	3,274,896	6,811,784	10,631,622	14,757,048	19,212,508
Break-even analysis (discounted)
Cumulative cost	2,660,000	5,269,333	7,832,584	10,354,180	12,838,403
Cumulative savings	3,274,896	6,487,413	9,643,195	12,747,693	15,806,178
Values from present value tables	(i=0.05)	—	—	—	—
(1/(1+i))^t	—	0.952380952	0.907029478	0.863837599	0.822702475

ROI, return on investment.

Table 5.

Break-Even Analysis, 2010–2014 (Values in South African Rand)

YEAR	2010	2011	2012	2013	2014
MAS device rental	2,057,059	2,221,624	2,399,354	2,591,302	2,798,606
MAS device support	1,851,353	1,999,462	2,159,419	2,332,172	2,518,746
Cost to implement SIMpill	3,908,413	4,221,086	4,558,773	4,923,474	5,317,352
Savings per 1,000 patients	4,811,897	5,196,848	5,612,596	6,061,604	6,546,532
Net cash flow	903,484	975,763	1,053,824	1,138,130	1,229,180
Discounted costs	3,062,344	3,149,839	3,239,835	3,332,401	3,427,613
Discounted savings	3,770,247	3,877,968	3,988,767	4,102,732	4,219,953
Discounted net cash flow	707,903	728,129	748,933	770,331	792,340
Net stream	707,903	1,436,032	2,184,965	2,955,296	3,747,636
ROI net stream as % of discounted costs	23%	23%	23%	23%	23%
Net present value (in 2005 for a 5-year implementation)	3,747,636
Break-even analysis
Cumulative costs	3,908,413	8,129,498	12,688,271	17,611,745	22,929,098
Cumulative savings	4,811,897	10,008,745	15,621,341	21,682,945	28,229,477
Break-even analysis (discounted)
Cumulative cost	3,908,413	6,066,357	9,017,317	11,920,322	14,780,301
Cumulative savings	4,811,897	7,468,680	11,101,795	14,675,871	18,196,973
Values from present value tables	(i=0.05)	—	—	—	—
(1/(1+i))^t	0.783526166	0.746215397	0.71068133	0.676839362	0.644608916

Break-even analysis is of little relevance in this study, which shows a positive net stream from the first year of implementation for both the 2005 and 2010 hypothetical starting points. There is therefore no break-even point. The break-even analysis and discounted break-even analysis for each hypothetical implementation are shown in Figures 1 –4.

Fig. 1.

2005–2009 break-even analysis.

Fig. 2.

2005–2009 discounted break-even analysis.

Fig. 3.

2010–2014 break-even analysis.

Fig. 4.

2010–2014 discounted break-even analysis.

Net Present Value

NPV of the hypothetical deployment of 1,000 MAS devices is the sum present position of the time series of cash flows for the period. NPV>0 is regarded as a promising prospect and NPV<0 an unjustified investment. For this analysis the lifespan of an MAS project is estimated to be 5 years and present value tables are used to discount annual figures.

For a theoretical implementation of 1,000 units in the Northern Cape Province, the net stream for both the 2005 and the 2010 starting points was positive from the first year of implementation. Discounted NPV for the hypothetical implementation starting in 2005 was ZAR 3,255,256 (US$ 493,221) while starting in 2010 resulted in a discounted NPV of ZAR 4,747,636 (US$ 487,339). This is an ROI of 23% over the 5-year period.

These calculations assume that the incidence of MDR-TB remained at 1.7% for the duration of the projects and that there is no extreme drug-resistant TB. Since these rates both escalate after 2005, incorporation of these additional variables would increase the NPVs.

Discussion

Full economic assessments of e-health interventions, using methods such as CEA or CBA, are difficult³⁸ and analysis should be performed from several perspectives, that of the patient, funder, society, healthcare provider, and insurer.³⁹ This study looks only from the perspective of the health service provider, in this case the government. The results suggest that there is a greater ROI when the MAS solution is used in combination with DOTS than when using DOTS alone. This can be measured in the health outcomes as well as in the average cost per patient borne by the health authorities.

The possibility of using the solution to help caregivers to prioritize patients with poor adherence, thereby freeing up additional caregiver time to be utilized by other healthcare programs, such as support for people living with AIDS, is not evaluated in this study. It may be reasonable to assume that this advantage would enhance the ROI achieved when implementing the MAS solution and further study to evaluate this aspect of the solution would be of value.

XDR-TB was not included in this analysis. The number of cases has since increased nationally from 74 in 2004 to 419 in 2006⁴⁰ and inclusion will become relevant in a current evaluation of the MAS solution, as a complication that increases when adherence is poor. Inclusion of XDR-TB in the costing model will further enhance the benefits of the MAS solution. The treatment of XDR-TB is estimated to cost four to five times more than treating MDR-TB.⁴¹

The improved treatment adherence that the MAS solution appears to support is likely to have benefits beyond those evaluated in this study. The public health benefits of improved cure rates are an overall decrease in disease incidence. In the case of infectious disease such as TB, improved cure rates will reduce the risk to citizens of contracting TB in their communities due to exposure to organisms coughed up by patients with active pulmonary disease. This will be of particular significance for people living with human immunodeficiency virus.

The study proposes a methodology whereby public health planners in the developing world can use relatively accessible data to evaluate whether a potential investment in treatment adherence technology will yield positive returns on the investment and whether further analysis using CEA or CBA is justified.

Economic assessment was not considered when the pilot study was implemented in 2005. This causes obvious limitations. It is not a randomized control study, has a small sample size dictated by the budget made available for the pilot, and there is limited possibility to generalize the findings to evaluation of other e-health applications or other countries. Discount rates are not discussed and time delay between the pilot and the analysis may reduce the accuracy of some cost estimations. Expansion of this study to CEA or CBA maybe justified. This will address the impact on caregivers, costs of change, optimism bias, sensitivity analysis, and risk assessment to arrive at a full assessment of the potential value of the MAS.

Despite these limitations, the CMA findings may be of value to public help health planners contemplating economic evaluation of e-health pilot projects in low-resource settings to make more informed decisions when allocating scarce resources.

Footnotes

Acknowledgments

The authors wish to thank the personnel of Betty Gaetsewe Clinic for their assistance with data collection. These dedicated professionals serve patients suffering from TB in the Northern Cape Province with passion and humility.

We also wish to acknowledge the support and motivation provided by the Northern Cape Department of Health Director of TB Services, Ms. Phyllis Baitsiwe.

Disclosure Statement

No competing financial interests exist.

References

Wood

, Lawn

, Johnstone-Robertson

, Bekker

. Tuberculosis control has failed in South Africa—time to reappraise strategy. S Afr Med J, 2011; 101:111–114.

World Health Organization. WHO Report 2009—Global Tuberculosis Control. www.who.int/tb/publications/global_report/2009/annex_3/en/index.html. 2011 January 10.

World Health Organization. WHO Report, Global Tuberculosis Control. South Africa. Geneva: WHO, 2009.

World Health Organization. WHO Report 2006 Global Tuberculosis Control: Surveillance, Planning, Financing. Geneva: WHO, 2006.

World Health Organization. WHO Report, Global Tuberculosis Control: Epidemiology, Strategy, Financing. 2009. www.who.int/tb/publications/global_report/2009/en/index.html. 2011 January 10.

World Health Organization. Global TB Database. www.who.int/tb/country/global_tb_database. 2011 January 10.

Streicher

, Müller

, Chihota

, Mlambo

, Tait

, Pillay

et al. Emergence and treatment of multidrug resistant (MDR) and extensively drug-resistant (XDR) tuberculosis in South Africa. Infect Genet Evol, 2011[Epub ahead of print]10.1016/j.meegid.2011.07.019.

Kifle

, Mbarika

, Tsuma

, Wilkerson

, Tan

. TeleMedicine Transfer Model for Sub-Saharan Africa. 41st Hawaii International Conference on System Sciences, Waikoloa, HI, 2008; 244.

bridges.org Evaluation of the on Cue Compliance Service Pilot, Testing the use of SMS reminders in the treatment of Tuberculosis in Cape Town, South Africa. 2005. www.bridges.org. 2011 January 10.

10.

Naditz

. Medication compliance–helping patients through technology: Modern “Smart” pillboxes keep memory-short patients on their medical regimen. Telemed E Health, 2008; 14:875–880.

11.

Green

. A novel approach to improving adherence to TB treatment. The Essential Drugs Monitor. Geneva: WHO, 2003; 8–9.

12.

Eisenstein

, Eisenstein

. A behavioral homeostasis theory of habituation and sensitization: II. Further developments and predictions. Rev Neurosci, 2006; 17:533–557.

13.

Pop-Eleches , Thirumurthy

, Habyarimana

, Zivin

, Goldstein

, Walque

et al. Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: A randomized controlled trial of text message reminders. AIDS, 2011; 27:825–834.

14.

South African Department of Health. National Tuberculosis Management Guidelines. Pretoria, South Africa: South African Department of Health, 2009.

15.

World Health Organization. Multidrug and extensive drug-resistant TB (M/ZDR-TB) 2010. http://whqlibdoc.who.int/publications/2010/9789241599191_eng.pdf. 2011 January 10.

16.

Mars

. eHealth economics: What are we doing about it? Global Telehealth 2010. Perth, Western Australia, 2010; 77.

17.

Stroetmann

, Jones

, Dobrev

. Evaluating the economic impact of eHealth applications—approach and method. Bonn: empirica Communication and Technology Research, 2006.

18.

Dávalos

, French

, Burdick

, Simmons

. Economic evaluation of telemedicine: Review of the literature and research guidelines for benefit-cost analysis. Telemed E Health, 2009; 15:933–948.

19.

Bergmo

. Can economic evaluation in telemedicine be trusted? A systematic review of the literature. Cost Effectiveness Resour Allocation, 2009; 7:18.

20.

McIntosh

, Cairns

. A framework for the economic evaluation of telemedicine. J Telemed Telecare, 1997; 3:132–139.

21.

Kumar

, Williams

, Sandy

. How do we evaluate the economics of health care? Eur J Orthod, 2006; 28:513–519.

22.

Robinson

. Economic evaluation and healthcare–cost-utility analysis. Br Med J, 1993; 307:859–862.

23.

Cunningham

. An introduction to economic evaluation of health care. J Orthod, 2001; 28:246–250.

24.

O'Brien

, Kay

, Fox

, Mandall

. Assessing oral health outcomes for orthodontics–measuring health status and quality of life. Commun Dent Health, 1998; 15:22–26.

25.

Robinson

. Economic evaluation and healthcare–what does it mean? Br Med J, 1993; 307:670–673.

26.

Robinson

. Economic evaluation and healthcare–costs and cost-minimization analysis. Br Med J, 1993; 307:726–728.

27.

Robinson

. Economic evaluation and healthcare–cost-benefit analysis. Br Med J, 1993; 307:924–926.

28.

EHR IMPACT. Methodology for evaluating the socio-economic impact of interoperable EHR and ePrescribing systems Methodology for evaluating the socio-economic impact of interoperable EHR and ePrescribing systems. 2008. www.ehrimpact.eu/downloads/documents/EHRI_D1_3_Evaluation_Methodology_v1_0.pdf. 2011 January 10.

29.

Drummond

, Sculpher

, Torrance

, O'Brien

, Stoddart

. Methods of economic evaluation of healthcare programmes, 3rd. New York: Oxford University Press Inc., 2005.

30.

Söderlund

, Lavis

, Broomberg

, Mills

. The costs of HIV prevention strategies in developing countries. Bull World Health Organ, 1993; 71:595–604.

31.

Wootton

. Telemedicine support for the developing world. J Telemed Telecare, 2008; 14:109–114.

32.

Wootton

, Bonnardot

. In what circumstances is telemedicine appropriate in the developing world? JRSM, 2010; 103:513.

33.

Statssa. www.statssa.gov.za. 2011 March 15.

34.

Northern Cape Department of Health. Annual Report 2009/2010. Kimberley: Northern Cape Department of Health, 2010.

35.

Henderson

. Health Economics and Policy, 4th. Mason: Cengage Learning, 2009.

36.

South African Department of Health. Tuberculosis Strategic Plan for South Africa, 2007–2011. Pretoria, South Africa: South African Department of Health, 2007.

37.

National Institute for Communicable Disease. Use of laboratory statistics for monitoring the introduction of a new line probe assay and for surveillance of MDR- and XDR-Tuberculosis. Commun Dis Surveill Bull, 2010; 8:54.

38.

Doherty

, Govender

. The cost-effectiveness of primary care services in developing countries: A review of the international literature. Disease Control Priorities Project, The World Bank, 2004.

39.

Walker

, Fox-Rushby

. Critical review of economic evaluations of communicable disease interventions in developing countries. Health Econ, 2000; 9:681–698.

40.

Reuter

. Taking XDR TB seriously in South Africa. SA Fam Pract, 2007; 49:3.

41.

Ndjeka

theglobalfund.org Drug-resistant TB in South Africa: Issue & response. 2010. www.theglobalfund.org/documents/regional_meetings/southafrica2010b/RMSA_2Session9NorbertDoHSA_Presentation_en.pptx. 2011 January 14.