Comparative performance of seven long-running telemedicine networks delivering humanitarian services

Abstract

Seven long-running telemedicine networks were surveyed. The networks provided humanitarian services (clinical and educational) in developing countries, and had been in operation for periods of 5–15 years. The number of experts serving each network ranged from 15 to 513. The smallest network had a total of 10 requesters and the largest one had more than 500 requesters. The networks operated in nearly 60 countries. The seven networks managed a total of 1857 cases in 2011, i.e. an average of 265 cases per year per network. There was a significant growth in total activity, amounting to 100.3 cases per year during the 15 year study period. In 2011, network activity was 50–700 teleconsultations per network. There were clear differences in the patterns of activity, with some networks managing an increasing caseload, and others managing a slowly reducing caseload. The seven networks had published a total of 44 papers listed in Medline which summarized the evidence resulting from the delivery of services by telemedicine. There was a dearth of information about clinical and cost-effectiveness. Nevertheless, the services were widely appreciated by referring doctors, considered to be clinically useful, and there were indications that clinical outcomes for telemedicine patients were often improved. Despite a lack of formal evidence, the present study suggests that telemedicine can provide clinically useful services in developing countries.

Introduction

Recent reviews of e-health applications have concluded that evidence in relation to improved patient outcomes, and evidence for cost-effectiveness, is rather thin on the ground.^1,2 Applications which facilitate care at a distance, i.e. telemedicine, can have particular value in low- and middle-income countries, and telemedicine has been used for many years to support doctors in the developing world.³ Several networks exist, which provide services in different settings and in different ways. However, there is little published evidence about the effectiveness of such networks, and – assuming that it would be desirable to do so – there is no way of deciding what the best method of expanding their activities would be.

Evidence-based decision-making requires performance measurement. Thus in order to draw conclusions about which telemedicine networks are successful a method of evaluating them is required. In previous work, we have proposed a framework for performance evaluation.⁴ The framework was based on observations of three long-running telemedicine networks which deliver humanitarian services. Although there are many possible indicators and metrics that might be relevant, five measures for each of the three main user groups appear to be sufficient⁴ (the three main user groups being the requesters, e.g. doctors or other health workers requesting management advice about difficult cases, the coordinators who manage the caseload and the network, and the experts who respond to electronic referrals). In addition, from the societal perspective, information about clinical- and cost-effectiveness is also required. The performance measures are summarized in Table 1.

Table 1

Performance measures (from Wootton et al.⁴)

Perspective	Performance measure
Requester
	1. rate of query arrival (new cases)
	2. proportion of failed queries
	3. time to first reply from an expert
	4. quality of replies
	5. ease of system usage
Coordinator
	1. rate of query arrival
	2. time required
	3. resources available
	4. feedback from experts/feedback on patient outcomes
	5. ease of system usage
Expert
	1. rate of requests received (for those experts who received queries)
	2. time required to answer
	3. relevance to own expertise
	4. feedback on patient outcomes
	5. ease of system usage
Society
	1. clinical effectiveness
	2. cost-effectiveness
	3. integration into the health care system, e.g. involvement of local people

The present work forms part of a wider study in which the framework for performance evaluation was applied to a larger number of networks. Details of the study design and additional information about the experience of the networks, including governance, medical liability, patient confidentiality, sustainability factors, challenges and future plans have been published elsewhere.⁵ The aim of the present report was to summarize the performance of the networks, to draw conclusions about the effectiveness of the services provided, and to make suggestions about how these might be evolved in the future.

Methods

The telemedicine networks included in the study were identified according to the following criteria:

they delivered clinical and/or educational services through telemedicine

they were long-running networks, defined as those which had been in operation for five years or more

they delivered humanitarian services, defined as those designed to save life and alleviate suffering, and which are delivered without condition (e.g. pro bono).

Long-running telemedicine networks delivering humanitarian services were identified by the authors based on their personal knowledge of the field. Telemedicine networks were excluded from the analysis in the case of no reply to initial enquiry.⁵

Data collection

Information for the study was collected by questionnaires (in English) sent to a contact person at each telemedicine network. The first questionnaire was sent in September 2011, to collect general information about the networks, including purpose, duration of operation, activities provided, model of operation, and numbers of requesters, experts and coordinators. A second questionnaire was sent in October 2011. This was designed to collect specific data about network performance, together with other more detailed characteristics of the networks. Each questionnaire contained 20 questions. Additional short questionnaires were sent in early 2012 to obtain information about the total number of cases managed during 2011, and further information about network operation.

Effectiveness

The information provided by the networks in response to the questionnaires included the total number of scientific papers published and details of the scientific papers published in Medline. Since the best quality evidence comes from papers published in the peer-reviewed literature, the latter information was assessed for evidence of clinical- and cost-effectiveness.

Only papers listed in Medline and dealing with telemedicine services were included in the analysis. To summarize their contents, one author extracted information from each published paper on the following topics: (i) study design; (ii) clinical effectiveness; (iii) cost-effectiveness. Study design was assessed using the nine-point scale traditionally employed to measure the strength of the evidence.⁶ The other topics were graded subjectively and assigned a quality score based on the amount of information provided about them. Finally, the results were reviewed by all authors and changed by consensus where necessary. Hence, the papers from each network were reviewed by researchers from all the other networks, which increased the consistency of the results.

Results

A total of nine telemedicine networks delivering humanitarian services were identified initially. Two of the nine telemedicine networks were excluded from further consideration, one because it had apparently ceased operating when contacted, and another because of no reply. Seven long-running telemedicine networks were therefore included in the present study. The persons responding on behalf of each network, together with general information regarding the activity of the networks, are shown in Table 2.

Table 2

General information regarding the telemedicine networks

	Africa Teledermatology Project	ITM Telemedicine*	Pacific lsland Health Care Project	Partners Online Specialty Consultations	RAFT**	Swinfen Charitable Trust	Teletrauma
Person responding on behalf of the network	Coordinator	Coordinator	Coordinator	Coordinator	Coordinator	Coordinator	Coordinator
Main purpose	Teledermatology in Africa	HIV advice for healthcare workers in Africa	Second opinions for Pacific Islanders	Second opinions for rural clinics in Cambodia	Education of healthcare staff in French-speaking African countries	Second opinions for doctors in developing countries	Trauma advice in the Ukraine
First year of operation	2007	2003–2004	1997	2001	2001	1999	2000
Activities provided	Clinical and educational	Clinical and educational	Clinical and educational	Clinical	Clinical and educational	Clinical	Clinical and educational
Number of requesters	Approximately 50	Approximately 400	More than 300	10	More than 500	403	50
Number of requesting sites	15	Approximately 80	11	4	More than 50	399	7
Number of countries	13	42	9	1	15	58	5
Number of experts	25	Approximately 20	100	30	50	513	15
Number of expert sites	6	6	1	14	20	502	5
Number of countries	3	5	1	1	15	22	3
Number of coordinators	0	2	1	2	2	2	3
Management of requests and selection of experts	Made by requesters	Made by a coordinator	Made by a coordinator	Made by both requesters and a coordinator	Made by a coordinator	Made by a coordinator	Made by a coordinator

*ITM, Institute of Tropical Medicine, Antwerp

**RAFT, Réseau en Afrique Francophone pour la Télémédecine

The median duration of operation of the seven networks studied was 11 years (range 5–15 years). All the networks provided clinical teleconsultations for humanitarian purposes, and five of them also conducted educational activities. The smallest network had a total of 10 requesters and the largest one had more than 500 requesters. The number of referring sites ranged from 4 to 399, and they were located in from 1 to 58 different countries. The number of experts serving each network ranged from 15 to 513. The experts were based in a number of sites ranging from 1 to 502, which were located in from 1 to 22 different countries. The management of the requests and the selection of the experts responsible for answering them was done by a coordinator in most of the seven networks. In one network, this activity was done entirely by the requesters themselves, while in another network the requesters were supported by a coordinator in the process, Table 2. The different organizational models are summarized in Table 3.

Table 3

Organizational models

	Africa Teledermatology Project	ITM Telemedicine*	Pacific lsland Health Care Project	Partners Online Specialty Consultations	RAFT**	Swinfen Charitable Trust	Teletrauma
Mode of operation*	Open (distributed)	Targeted	Targeted	Targeted	Targeted	Targeted	Targeted
Funding of experts	The consultants are volunteers	The consultants are volunteers	The consultants are volunteers	The consultants are volunteers	The consultants are volunteers	The consultants are volunteers	The network pays for their time
Funding of experts	The consultants are volunteers	The consultants are volunteers	The consultants are volunteers	The consultants are volunteers	The network pays for their time	The consultants are volunteers	The network pays for their time
Funding of coordinators	The coordinators are volunteers	The network pays for their time	Medical Director is a volunteer	The network pays for their time	The coordinators are volunteers	The coordinators are volunteers	The network pays for their time
Funding of coordinators	The coordinators are volunteers	The network pays for their time	Medical Director is a volunteer	The network pays for their time	The network pays for their time	The coordinators are volunteers	The network pays for their time
Legal status	University body	Health service body	Government body	Commercial organisation	University body	Charity registered under UK law	Health service body

*For telemedicine networks with a clinical purpose, there are two fundamentally different modes of operation which correspond to the different ways in which an expert is chosen to reply to a query. In the "targeted" mode of operation, a query is directed to a specific expert for reply. In the " open" or "distributed" mode, a query is directed to a group of experts, of whom all, some or none may reply⁴

Network performance

The seven networks managed a total of 1857 cases in 2011, i.e. an average of 265 cases per year per network. There was a significant growth in total activity, amounting to 100.3 cases per year during the 15 year study period (P<0.001), see Figure 1. In 2011, network activity in the individual networks ranged from about 50 to over 700 teleconsultations per year, see Figure 2. There were clear differences in the patterns of activity, with some networks managing an increasing caseload, and others managing a static or reducing caseload, see Table 4. The mean caseloads (averaged over five years) ranged from 57 to 313 cases/year.

Figure 1

Total caseload managed by the networks (the solid line is the linear regression). The number of networks operating each year is shown below

Figure 2

Numbers of clinical consultations in each network

Table 4

Activity trend in each network for the years 2007–2011. The trend was calculated as the linear regression of each network's annual caseload on time

	Average caseload (five-year average) (cases/year)	Five-year trend (cases/year)
Africa Teledermatology Project	259	43
ITM Telemedicine	128	−22
Pacific Island Health Care Project	185	−2
Partners Online Specialty Consultations	89	0
RAFT	313	143
Swinfen Charitable Trust	275	49
Teletrauma	57	2

Little quantitative information was available for the educational activities provided by the telemedicine networks. Only the RAFT network collected data, see Figure 3. The ITM telemedicine network also offered online learning and web-based discussion forums, but the contact-hours could not be quantified.

Figure 3

Hours of tele-education provided by the RAFT network

The performance of the seven networks is summarized in Table 5. For ease of comparison, most of the performance measures are referred to the year 2010. The mean number of requests for consultation in 2010 was 209 (range 75–422). That is, the rate of query arrival to the network ranged from 6.2 to 35.2 cases per month. The number of unanswered replies to requests in 2010 was very low, and the highest percentage of unanswered replies to the 2010 caseload was 3%. One of the main reasons why these cases could not be processed was the inadequate quality of the information submitted by the requesters. That is, the unanswered cases were not related to the availability of appropriate experts, but to deficiencies in the submissions by the referrers.

Table 5

Performance of the telemedicine networks in 2010

	AfricaTeledermatologyProject	ITMTelemedicine	Pacific lslandHealth CareProject	Partners OnlineSpecialtyConsultations	RAFT	SwinfenCharitableTrust	Teletrauma
Number of requests	422	106	199	80	350	232	75
Rate of query arrival to the network (new cases/month)	35	9	17	7	29	19	6
Number of unanswered requests	0	0	0	0	Approximately 10	3	0
Percentage of unanswered requests in the annual caseload	0	0	0	0	Approximately 3	1	0
Average time to first reply to the requests (hours)	24	24	5.6	72	24	24.2	12
Possibility of a dialogue with the expert after the initial response	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Outcome data about individual patients available for the expert*	Sometimes	Sometimes	Often	Sometimes	Sometimes	Sometimes	Sometimes
Number of cases with feedback on outcomes	25	Less than 10	All	25	Approximately 50	Approximately 20	10
Percentage of cases with feedback on outcomes	6	<10	100	31	14	<1	13

*The possible responses to the multiple-choice question were: Never; Sometimes; Often; Always

The average time to first reply to the requests is an important performance measure which affects the value of the clinical teleconsultations to the referrer. In the previous 12 months of operation, most networks had an average time to first reply of about 24 hours (range 5.6–72 hours). Another performance measure regards the possibility, for the requesters, of conducting a dialogue with the experts. This facility was available in all seven telemedicine networks. On the other hand, the experts only sometimes had access to outcome data about individual patients. In particular, the number of times that feedback was provided on outcomes in 2010 ranged from all cases managed by the network to only a few of them.

Clinical and cost-effectiveness

The seven networks had published a total of 44 papers listed in Medline which summarized their experience in the delivery of services by telemedicine.^7–50 However, there was a dearth of quantitative information about clinical- and cost-effectiveness, Table 6. Of the 44 papers, only 11 reported results regarding clinical outcomes, and no papers reported a formal cost-effectiveness analysis. The majority of papers were anecdotes or case reports, together with non-controlled clinical series. Overall, the quality of the scientific output was classified as poor to average.⁵ Performing a formal evaluation of the clinical outcomes and the cost-effectiveness would require a higher strength of evidence, as assessed by the standard scale for study design.⁶

Table 6

Clinical and cost-effectiveness of the telemedicine networks, based on their published papers

	AfricaTeledermatologyProject	ITMTelemedicine	Pacific lslandHealth CareProject	Partners OnlineSpecialtyConsultations	RAFT	SwinfenCharitableTrust	Teletrauma
Number of scientific papers published in Medline	2	3	12	4	7	14	2
References	7,8	9–11	12–23	24–27	28–34	35–48	49,50
Evidence of clinical effectiveness	0	0	1	2	0	6	2
Evidence of cost-effectiveness	0	0	0	0	0	0	0
Non-controlled clinical series (Level VIII)	2	2	8	4	0	11	2
Anecdotes or case reports (Level IX)	0	1	4	0	9	3	0

Discussion

The present study reports the characteristics of seven long-running networks which deliver humanitarian services by telemedicine. The networks were of different sizes, used different models of operation and provided different services (although all of them provided clinical teleconsultations). The reasons why the caseload of some networks was rising while that of other networks was falling are not yet fully understood. It may be a consequence of the use of small-scale organizational models, and it might be that improved collaboration between networks in future will improve the situation. Furthermore, there was no obvious relation between the model of organization employed by the networks and their performance, see Tables 3 and 4.

Despite the differences in their organization, there were similarities in the performance of the networks: all provided rapid responses to clinical requests, and all offered the possibility of further dialogue between requester and expert. Although there had been relatively few user surveys, those that had been conducted all showed high levels of user satisfaction.^9,40 Thus, it appears that there is no single common successful organizational model. That is, different models are associated with sustainability (since the telemedicine networks had all been in operation for 5 years or more) and lead to similar performance. In future, a comparison between long-running telemedicine networks (i.e. the ones included in the present paper) and other telemedicine networks which have ceased operation, might provide a better understanding of which are the most successful organizational models.

Although all networks were able to report some of the performance data suggested in the framework, few of them had been able to obtain outcomes data routinely, and the published work contained generally weak evidence for clinical effectiveness. None of the networks had published evidence of cost-effectiveness, as would be considered normal in other areas of health care, and there were no reports which complied with current guidelines on health economics studies. On the other hand, it is worth noting that an absence of evidence for cost-effectiveness does not prove that the operations are not cost-effective. Despite the weak evidence for effectiveness, an extensive corpus of published work provided evidence that the services were widely appreciated by referring doctors, considered to be clinically useful, and there were indications that clinical outcomes for telemedicine patients were often improved.

All of the networks held concerns about long-term funding (data reported elsewhere⁵), despite their apparent sustainability to date. Published information on cost-effectiveness would make it easier to secure funding through conventional sources, which emphasizes the importance of collecting performance data of the kind suggested in the framework.⁴ Other measures which might improve sustainability include better mechanisms for cooperation between networks.⁵¹

Limitations

The present study had certain limitations. For example, there may be other long-running telemedicine networks around the world which also deliver humanitarian services. However, we believe that the ones studied include the main active networks. Other limitations concern the study design. For example, we examined only successful networks, and arguably, experience from unsuccessful networks may be equally informative. Also, the experience of the telemedicine networks was reviewed by the persons responding on behalf of the networks. This might represent a potential reporting bias.

Conclusion

Despite a lack of formal evidence, the present study suggests that telemedicine can provide clinically useful services in developing countries. Since the potential pool of patients is very large, we suggest the right strategy at present should be one of cautious expansion. That is:

continued support for current networks;

introduction of new networks, based on similar models to those employed by the networks described here;

a programme of research designed to secure formal evidence of clinical- and cost-effectiveness.

Future research should aim to obtain formal evidence of clinical effectiveness, which implies the need for patient follow-up. This is not easy in the settings in which the networks operate. Future research should also aim to obtain evidence of cost-effectiveness, which is likely to require controlled trials. Again, this is not likely to be easy to do in practice. However, if convincing evidence of cost-effectiveness can be obtained, then telemedicine in the developing world could take its place alongside measures of proven effectiveness, such as vaccination, and be funded for large-scale implementation.

Footnotes

Acknowledgements

We are grateful to our colleagues in the various networks for their willingness to share the data about network operation.

References

Piette

, Lun

, Moura

Impacts of e-health on the outcomes of care in low- and middle-income countries: where do we go from here?

Bull World Health Organ 2012;90:365–72

Black

, Car

, Pagliari

, The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med 2011;8:e1000387

Wootton

, Patil

, Scott

, Ho

(eds). Telehealth in the Developing World . London: RSM Press, 2009. Available from http://www.idrc.ca/EN/Resources/Publications/Pages/IDRCBookDetails.aspx?PublicationID=57 (last checked 3 May 2012)

Wootton

, Vladzymyrskyy

, Zolfo

, Bonnardot

. Experience with low-cost telemedicine in three different settings. Recommendations based on a proposed framework for network performance evaluation. Glob Health Action 2011;4. doi: 10.3402/gha.v4i0.7214

Wootton

, Geissbuhler

, Jethwani

, Characteristics of nonprofit, long-running telemedicine networks. Bull World Health Organ 2012;90:341–7

Jovell

, Navarro-Rubio

. [Evaluation of scientific evidence]. Med Clin 1995;105:740–3

Kaddu

, Soyer

, Gabler

, Kovarik

. The Africa Teledermatology Project: preliminary experience with a sub-Saharan teledermatology and e-learning program. J Am Acad Dermatol 2009;61:155–7

Weinberg

, Kovarik

. The African Teledermatology Project: providing access to dermatologic care and education in sub-Saharan Africa. Pan Afr Med J 2009;3:16

Zolfo

, Bateganya

, Adetifa

, Colebunders

, Lynen

. A telemedicine service for HIV/AIDS physicians working in developing countries. J Telemed Telecare 2011;17:65–70

10.

Zolfo

, Lynen

, Dierckx

, Colebunders

. Remote consultations and HIV/AIDS continuing education in low-resource settings. Int J Med Inform 2006;75:633–7

11.

Zolfo

, Arnould

, Huyst

, Lynen

. Telemedicine for HIV/AIDS care in low resource settings. Stud Health Technol Inform 2005;114:18–22

12.

Person

. Pacific Island Health Care Project: early experiences with a web–based consultation and referral network. Pac Health Dialog 2000;7:29–35

13.

Ruess

, Uyehara

CFT

, Shiels

, Digitizing pediatric chest radiographs: comparison of low–cost, commercial off–the–shelf technologies. Pediatr Radiol 2001;31:841–7

14.

Belnap

, Freeman

, Hudson

, Person

. A versatile and economical method of image capture for telepathology. J Telemed Telecare 2002;8:117–20

15.

Person

, Hedson

, Gunawardane

. Telemedicine success in the United States Associated Pacific Islands (USAPI): two illustrative cases. Telemed J E Health 2003;9:95–101

16.

Park

, Ruess

, O'Connor

, Hussain

, Oshiro

, Person

. Internet consultations from a remote Pacific island: impact of digitized radiologic images on referral decisions. J Digit Imaging 2004;17:253–7

17.

Hensel

, Person

, Schaefer

, Burkhalter

. An internet-based referral/consultation system for the U.S.-associated Pacific Islands: its contribution to orthopedic graduate medical education at Tripler Army Medical Center. Mil Med 2005;170:214–8

18.

Meza-Valencia

, de Lorimier

, Person

. Hirschsprung disease in the U.S. Associated Pacific Islands: more common than expected. Hawaii Med J 2005;64:96–8,100–1

19.

Person

. The Pacific Island Health Care Project: easing the cancer burden in the United States Associated Pacific Islands. Pac Health Dialog 2004;11:243–7

20.

Person

. The Republic of Palau and the Pacific Island Health Care Project (PIHCP). Pac Health Dialog 2005;12:132–40

21.

Abbas

, Person

. The Pacific Island Health Care Project (PIHCP); experience with rheumatic heart disease (RHD) from 1998 to 2006. Hawaii Med J 2008;67:326–9

22.

Bush

, Ruess

, Jack

, Person

. Adrenal insufficiency secondary to tuberculosis: the value of telemedicine in the remote diagnosis of Addison's disease in Ebeye, Republic of the Marshall Islands. Hawaii Med J 2009;68:8–11

23.

Batts

, Thompson

, Person

. Late presentation of diaphragmatic hernia in a Pacific Island pediatric population. Hawaii Med J 2009;68:59–61

24.

Kedar

, Ternullo

, Weinrib

, Kelleher

, Brandling-Bennett

, Kvedar

. Internet based consultations to transfer knowledge for patients requiring specialised care: retrospective case review. BMJ 2003;326:696–9

25.

Heinzelmann

, Jacques

, Kvedar

. Telemedicine by email in remote Cambodia. J Telemed Telecare 2005;11(Suppl 2):44–7

26.

Brandling-Bennett

, Kedar

, Pallin

, Jacques

, Gumley

, Kvedar

. Delivering health care in rural Cambodia via store-and-forward telemedicine: a pilot study. Telemed J E Health 2005;11:56–62

27.

Kvedar

, Heinzelmann

, Jacques

. Cancer diagnosis and telemedicine: a case study from Cambodia. Ann Oncol 2006;17(Suppl 8):37–42

28.

Geissbuhler

, Ly

, Lovis

, L'haire

. Telemedicine in Western Africa: lessons learned from a pilot project in Mali, perspectives and recommendations. AMIA Annu Symp Proc 2003:249–53

29.

Bagayoko

, Muller

, Geissbuhler

. Assessment of Internet-based tele-medicine in Africa (the RAFT project). Comput Med Imaging Graph 2006;30:407–16

30.

Geissbuhler

, Bagayoko

, Ly

. The RAFT network: 5 years of distance continuing medical education and tele-consultations over the Internet in French-speaking Africa. Int J Med Inform 2007;76:351–6

31.

Bagayoko

, Niang

, Traoré

, Bediang

, Naef

, Geissbuhler

. Deploying portable ultrasonography with remote assistance for isolated physicians in Africa: lessons from a pilot study in Mali. Stud Health Technol Inform 2010;160:554–8

32.

Bediang

, Bagayoko

, Geissbuhler

. Medical decision support systems in Africa. Yearb Med Inform 2010:47–54

33.

Bediang

, Bagayoko

, Raetzo

, Geissbuhler

. Relevance and usability of a computerized patient simulator for continuous medical education of isolated care professionals in sub-saharan Africa. Stud Health Technol Inform 2011;169:666–70

34.

Bagayoko

, Anne

, Fieschi

, Geissbuhler

. Can ICTs Contribute to the efficiency and provide equitable access to the health care system in Sub-Saharan Africa? The Mali experience. Yearb Med Inform 2011;6:33–8

35.

Wootton

. Design and implementation of an automatic message-routing system for low-cost telemedicine. J Telemed Telecare 2003;9(Suppl 1):44–7

36.

Wootton

, Menzies

, Ferguson

. Follow-up data for patients managed by store and forward telemedicine in developing countries. J Telemed Telecare 2009;15:83–8

37.

Alverson

, Swinfen

, Transforming systems of care for children in the global community. Pediatr Ann 2009;38:579–85

38.

Patterson

, Swinfen

, Azzo

, Taha

, Wootton

. Supporting hospital doctors in the Middle East by email telemedicine: something the industrialized world can do to help. J Med Internet Res 2007;9:e30

39.

Wootton

, Youngberry

, Swinfen

. Referral patterns in a global store-and-forward telemedicine system. J Telemed Telecare 2005;11(Suppl 2):100–3

40.

Wootton

, Youngberry

, Swinfen

. Prospective case review of a global e-health system for doctors in developing countries. J Telemed Telecare 2004;10(Suppl 1):94–6

41.

Swinfen

, Swinfen

, Youngberry

, Wootton

. A review of the first year's experience with an automatic message-routing system for low-cost telemedicine. J Telemed Telecare 2003;9(Suppl 2):63–5

42.

Swinfen

, Swinfen

. Low-cost telemedicine in the developing world. J Telemed Telecare 2002;8(Suppl 3):63–5

43.

Graham

, Zimmerman

, Vassallo

, Telemedicine – the way ahead for medicine in the developing world. Trop Doct 2003;33:36–8

44.

Jakowenko

, Wootton

. An analysis of the images attached to referral messages in an email-based telemedicine system for developing countries. J Telemed Telecare 2006;12(Suppl 3):49–53

45.

Vassallo

. A guide to sending e-mail telemedicine referrals. Trop Doct 2003;33:34–5

46.

Vassallo

, Swinfen

, Wootton

. Experience with a low-cost telemedicine system in three developing countries. J Telemed Telecare 2001;7(Suppl 1):56–8

47.

Patterson

, Hoque

, Vassallo

, Farquharson Roberts

, Swinfen

. Store-and-forward teleneurology in developing countries. J Telemed Telecare 2001;7(Suppl 1):52–3

48.

Vassallo

, Hoque

, Roberts

, Patterson

, Swinfen

. An evaluation of the first year's experience with a low-cost telemedicine link in Bangladesh. J Telemed Telecare 2001;7:125–38

49.

Vladzymyrskyy

. Four years’ experience of teleconsultations in daily clinical practice. J Telemed Telecare 2005;11:294–7

50.

Vladzymyrskyy

. Our experience with telemedicine in traumatology and orthopedics. Ulus Travma Acil Cerrahi Derg 2004;10:189–91

51.

Wootton

, McGoey

. Who shall coordinate the coordinators? Facilitating the work of telemedicine networks which provide humanitarian services. J Telemed Telecare 2012;18:63–5