Exploring the ambivalent evidence base of mobile health (mHealth): A systematic literature review on the use of mobile phones for the improvement of community health in Africa

Abstract

Background

Africa is labelled the world's fastest-growing ‘mobile region’. Considering such growth and the fragility of the continent's healthcare, mHealth has flourished. This review explores mHealth for community health in Africa in order to assess its still ambivalent evidence base.

Methods

Using PubMed, Web of Science, OvidSP and Google Scholar, a systematic review was conducted of one decade (2005–2015) of peer-reviewed literature on mHealth in Africa. Data analysis focused on qualifications of success and failure. Impact evaluations of project assessments (n = 65) were complemented with general analyses/overviews of mHealth's effectiveness (n = 35).

Results

Review of these texts reveals ambivalence in the appraisal of mHealth; essentially, the critical stance in general analyses/overviews is absent from project assessments. Especially weak evidence concerning sustainability and scalability is stressed in overviews. Project assessments are more optimistic. Their analysis suggests a causal connection between simplicity and success. Effective interventions are thus characterized by straightforward design and modest objectives. Greatest impediments of impact are general technology-related issues and intervention inappropriateness due to insufficient understanding of beneficiaries and specific context of use (circumstantial complications).

Conclusion

Distinguishing between these two categories of complications helps to break the deadlock that marks the mHealth debate and add nuance to claims that mHealth's evidence base is weak. Constructive realism – rather than unfounded optimism or pessimism without nuance – should guide the design of interventions. Besides anticipative of technology-related complications, such realism must lead to either basic interventions or to smart mHealth shaped by deep understanding of the context of implementation.

Keywords

mHealth Africa mobile phones community health evidence base impact assessment

Introduction

Within the previous decade, Africa has witnessed the fastest growth in mobile phone subscribers in the world, estimated at 955 million in 2015.¹ Decreasing costs of technology and telecommunication along with expanding network coverage, primary causes of such growth rates, have also stirred interest in the potential of mobile phones as devices for development. Besides agriculture and education, health care in particular is considered a realm in which improved communication and innovative mobile applications can prove a cost-efficient route to progress. Accordingly, mobile health (mHealth) – defined by the WHO² as ‘medical and public health practice supported by mobile devices, such as mobile phones, patient monitoring devices, personal digital assistants (PDAs), and other wireless devices’ – flourishes. It has become the focus of a wide range of projects across the region that aim to address crucial inadequacies (e.g. inaccessibility and lack of information) that cripple rural African health care in particular.

Unfortunately, great enthusiasm for mobile technology is not backed by conclusive evidence of the (significant) impact of mHealth projects. In fact, despite of the availability of a growing body of research on mHealth in Africa, insight in the complexities of cost-benefit, scalability, transposability, continuity and technology adoption remains limited. Particularly striking is the apparent lack of advance that surfaces if early and more recent reflections on mHealth are compared. Indicative is the conclusion by Aranda-Jan et al.³ in one of the latest literature reviews on mHealth in Africa. According to Aranda-Jan et al., ‘evidence [concerning the success of mHealth] remains poor […] and questions regarding impact, scalability, increase coverage, cost-effectiveness and sustainability […] are yet to be addressed’ (p.188).³ Such a conclusion resonates with a much earlier observation of Kaplan⁴ that ‘convincing evidence regarding the overall cost-effectiveness of mobile phone “telemedicine” is still limited and good-quality studies are rare’ (p.12).⁴

Apart from being remarkably similar despite of the 8-year time-lapse, these two relatively critical observations concerning the evidence base of mHealth are representative of the lack of enthusiasm that characterizes the tone of most literature studies and conceptual/thematic pieces included in this systematic review. In contrast, most of the reviewed project assessments report positive results from mHealth interventions. Although virtually all texts do identify obstacles that hinder implementation, project assessments tend to emphasize successes and potential and positive impact despite the critical challenges that are (in contrast) highlighted in literature studies and conceptual/thematic articles. Virtually all reviewed project assessments concern pilots or small-scale interventions. Examples of upscaling and broad-based interventions can be found throughout the continent (particularly interesting are open-source applications like CommCare and Medic Mobile). Unfortunately, despite indications of their success in the grey literature, comprehensive research on their effectiveness at large is currently not available in peer-reviewed work. As such, they are not included here.

The aim of this review, which includes 65 project assessments and 35 conceptual/thematic articles and literature reviews, is to assess a body of scholarly literature in order to explore and explain, and eventually move beyond, the unconstructive ambiguity that marks the evidence base of 10 years of mHealth in Africa. It thus provides a systematic analysis of reported success factors as well as obstacles, with an emphasis on the latter. The particular selection of reviewed articles indicates the need for an exclusive focus. mHealth is an extremely dynamic and variegated field, with applications in data collection and disease surveillance, patient follow-up and medication adherence, communication and information for health care workers (HCWs), health promotion and disease prevention and remote monitoring. Including the entire diversity of these application categories would severely complicate opportunities for comparison.

Instead, the current review makes a careful selection to include only those application categories that are immediately related to the improvement of health care delivery services, and consequently, the enhancement of what can be called community health. This does not imply the prioritization of certain categories. Yet, comparing the assessments of mHealth as ‘remote monitoring’ or ‘data collection’ with those of mHealth ‘health promotion’ or ‘disease prevention’ will serve to obscure rather than aid systematic review. Excluded are mHealth applications for ‘remote monitoring’ and ‘data collection’, as these – albeit in different ways – both imply an outward flow of information (community –> externals). Hence, they do not immediately address patients or beneficiaries or aim to increase local knowledge and trigger behaviour change, and will only indirectly improve health outcomes. Furthermore, this review addresses mHealth through examining community health projects that use mobile (feature- or smart) phones. Studies concerning other types of mobile devices, such as PDAs and tablets, are excluded from this review. The reason for this is the highly dynamic nature of mHealth. As PDAs have already become outdated, the use of tablets is still so much in its infancy that there is insufficient material to allow proper evaluation. This means the focus in this review is entirely on assessments of the use of mobile phones to support the immediate improvement of the health status of communities in (generally rural) African areas where medical facilities are scarce and underequipped. It includes interventions/ activities pertaining essentially primary health care, addressing common medical needs, health practices and challenges of a population deprived of adequate and accessible services. Labelled ‘community health’, this focus includes the application of mHealth for the purpose of: (a) patient follow-up and medication adherence, (b) communication and information for health care workers and (c) health promotion and disease prevention.

The review also includes overviews of mHealth research (literature studies) and more general contemplations of the field and subfields of mHealth and its core concepts (conceptual/thematic articles). Apparent differences in the evaluation of mHealth's potential – the optimism of project assessment versus the critical perspectives that dominate literature studies and conceptual/thematic articles – will be analysed and explained in the final section of this review. Moving beyond such ambivalence in the evidence base, the article concludes that a distinction should be made between technology-related complications that relate to the technical nature of the interventions, and circumstantial complications that involve the multifaceted context of implementation. It is argued that, in order for mHealth to realize its potential, persistent naivety concerning common technology-related complications (e.g. connectivity issues, erratic power supply) should first be eradicated. However, more fundamental to the ultimate valuation of mHealth are circumstantial complications. Project assessments below indicate that most successful interventions are the ones least affected by contextual challenges because of their simple design and modest objectives. Complex interventions that aim for more intricate attitudinal and behavioural change prove ineffective as a result of a lack of detailed insight into, for instance, local power dynamics, socio-cultural peculiarities and communicative conventions. Comprehensive assessments are direly needed to boost the impact of these ambitious interventions. Yet, contrary to what is argued elsewhere, looking for transposable recipes for change by means of an intensification of existing (randomized controlled trial (RCT)-dominated) research strategies will not suffice. Instead, the review indicates in-depth understanding should be gained through qualitative and participatory analysis of context specific factors that determine the success of mHealth at a given time and place.

Materials and methods

An electronic systematic literature search was conducted using PubMed, Web of Science and Journals@OvidSP. Search terms used included “mHealth OR m-health AND Africa” and “Mobile phones OR cell phones OR smartphones AND health AND Africa”. In addition to these search engines, Google Scholar was used to check for missed articles that met the inclusion criteria. The searches were limited to articles published in English between 2005 and 2015. The searches were performed in 2015 by one author. All duplicated articles were removed automatically using Mendeley and a manual revision was done for verification. From the total search results, all abstracts were screened, and studies were selected for full-text review. Full-text articles were searched manually in digital sources, and studies were excluded when access to full-text articles was not available. To avoid selection bias, the two authors carried out the full-text article review and any difference in the selection was discussed and papers selected accordingly. Exclusion criteria were: not about Africa exclusively, article not peer-reviewed, and non-mHealth implementation (e.g. telemedicine, other types of eHealth). Because this review concerns contributions of mobile devices to community health exclusively, articles about mHealth as remote monitoring and data collection have been excluded. Except for project protocols, all study designs (RCTs, pilot project, literature reviews etc.) were included.

Results

From a total of 1110 search results (after removing duplicates) 159 studies were selected for full-text review, of which 99 studies were included in the review according to the inclusion criteria. On the basis of peer review of the original manuscript, one final project assessment was added, resulting in 100 included studies, published between 2005 and 2015. This encompassed 65 project assessments – 64 that solely deal with mobile phones and one that includes tablets – as well as 16 literature reviews, and 19 conceptual or thematic articles. The project assessments were further classified into topics according to the type of intervention: ‘Patient follow-up and medication adherence’ (n = 27), ‘Communication and information for HCWs’ (n = 22) and ‘Health promotion and disease prevention’ (n = 16). When a project assessment fitted two or more categories, the authors selected the dominant and classified the project as such. Figure 1 presents a flowchart detailing the inclusion/exclusion process. A large number of project assessments were pilot studies or even pre-trial investigations. Furthermore, most of the RCTs were implemented at community level and not yet scaled up. Hence evidence presented on effectiveness is limited and the long-term impact is unclear. Nonetheless, conclusions tend to stress the feasibility of projects and their potential to aid healthcare delivery in Africa. The following sections will discuss the findings of the project assessments classified per type of intervention. The discussion will assess the obstacles and barriers to mHealth for community health that seem most pertinent.

Figure 1.

Inclusion/exclusion flowchart.

1. Patient follow-up and medication adherence

Table 1 shows an overview of studies on patient follow-up and medication adherence. mHealth implementations to improve patient follow-up and medication adherence were the most common among the reviewed project assessments (n = 27). These assessments range from pre-trial informative studies to RCTs to in-depth post-trial analyses. They report findings from 10 different countries, on 10 different ways of using mobile phones to help manage eight different conditions. As the character of the studies depends on project stage, we have classified them based on the status of the intervention: pre-trial (n = 8), pilots (n = 5) and post-trial (n = 14).

Table 1.

Overview of studies on patient follow-up and medication adherence (n = 27).

APPLICATION CATEGORY: PATIENT FOLLOW-UP AND MEDICATION ADHERENCE (n = 27)
AUTHOR	YR	TYPE	COUNTRY	PHONE USE	CONDITION	RESULTS
Pre-trial (n=8)
5. Coleman	‘14	Qualitative	South Africa	Monitoring	Geriatrics	Feasible; advise implementation
6. Smillie et al.	‘15	Qualitative	Kenya	2-way text, call	HIV/AIDS	Feasible, yet challenges
7. Mall et al.	‘13	Qualitative	South Africa	Text only	Mental hlth.	Feasible; advise implementation
8. Hwabamungu and Williams	‘10	Qualitative	South Africa	Various	HIV/AIDS	Feasible, yet challenges
9. Haji et al.	‘14	PAR	Tanzania	Graphic Texts	TB	Feasible; advise implementation
10. Mbuagbaw et al.	‘14	Mixed	Cameroon	Text only	HIV/AIDS	Feasible, yet challenges
11. Bigna et al.	‘14	Quantitative	Cameroon	Text, call	HIV/AIDS	Feasible, yet challenges
12. Otieno et al.	‘14	Quantitative	Kenya	Text only	Malaria	Feasible; advise implementation
Pilot(n=5)
13. Hoffman et al.	‘10	Quantitative	Kenya	Life Monitoring	TB	Feasible
14. Kliner et al.	‘13	Quantitative	Swaziland	‘Flashing’	HIV/AIDS	Insignificant
15. Modrek et al.	‘14	Quantitative	Nigeria	Text only	Malaria	Significant
16. Dean et al.	‘12	Qualitative	South Africa	SMS group	HIV/AIDS	Feasible, yet challenges
17. Schwartz et al.	‘15	Qualitative	South Africa	Text, call	HIV/AIDS	Feasible
Post-trial (n=14)
18. Constant et al.	‘14	RCT	South Africa	Text only	Abortion	Significant
19. Constant et al.	‘15	RCT	South Africa	Questionnaire	Abortion	Inadequate
20. Raifman et al.	‘14	RCT	Ghana	Text only	Malaria	Significant
21. Lester et al.	‘10	RCT	Kenya	Text only	HIV/AIDS	Significant
22. Pop-Eleches et al.	‘11	RCT	Kenya	Text only	HIV/AIDS	Significant
23. Siedner et al.	‘14	RCT	Uganda	Text only	HIV/AIDS	Challenges
24. Kunutsor et al.	‘10	Quantitative	Uganda	Text, call	HIV/AIDS	Generally positive results
25. Mbuagbaw et al.	‘13	Quantitative	Cameroon	Two-way text	HIV/AIDS	Significant
26. Odigie et al.	‘12	Quantitative	Nigeria	Helpline	Cancer	Significant
27. Seidenberg et al.	‘12	Quantitative	Zambia	Text only	HIV/AIDS	Significant
28. Reid et al.	‘14	Quantitative	Botswana	Text only	HIV/AIDS	Generally positive results
29. Kop, van de, et al.	‘12	Quantitative	Kenya	Two-way text	HIV/AIDS	Generally positive results
30. de Tolly and Constant	‘14	Qualitative	South Africa	Various	Abortion	Generally positive results
31. Leon et al.	‘15	Qualitative	South Africa	Text only	Hypertension	Generally positive results

Pre-trials (n = 8)

Most pre-trial studies are investigations into the feasibility and acceptability of proposed interventions. Half of these studies were qualitative studies, relying on, for instance semi-structured interviews to investigate the possibilities of text message reminders for elderly people,⁵ HIV patients,⁶ or mental health patients⁷ to take their medication. Another concerned the ‘adoption prospects’ of mHealth applications in an antiretroviral (ARV) clinic.⁸ Another study⁹ was a cross-sectional user requirement study of graphic texts to support TB patients in their treatment. This research used a Participatory Action Research model, including ethnographic research. A mixed-method study¹⁰ explored the possibilities of community ownership over a text-messaging programme to improve adherence to antiretroviral therapy (ART). Two studies employed quantitative methods. The first was an investigation into the barriers of using mobile phones for paediatric HIV care.¹¹ The second was a cross-sectional study of the feasibility of text messaging to improve treatment adherence of children with malaria.¹² All these studies indicated a general^5,7,9 to high¹² feasibility of the proposed implementation. However, four of them also outlined foreseen challenges. For instance, language illiteracy was pointed out in two studies as an important barrier to text-based reminders.^10,11 Also, studies mentioned concerns about privacy^6,8 and responsibility for costs.^8,10 The study which included clinic workers’ perspectives⁸ found that rather than mobile phones, clinic workers preferred to have more training, staff, financial support, volunteers and community involvement. In this study, 34 out of 55 clinic workers said they were not willing to contribute financially to the proposed project.

Pilots (n = 5)

Out of the five pilot studies, three used quantitative methods. One of these¹³ tested a mobile direct observation system to monitor TB patients’ medication adherence. It concludes that both patients and health professionals consider it a viable option. However, further research is needed to see if it indeed improves medication adherence and to explore the cost effectiveness. Another study¹⁴ used ‘flashing’ (i.e. generating missed calls) to improve HIV patients’ attendance for follow-up at the clinic. This intervention did not improve attendance rate at all. In contrast, a third pilot study,¹⁵ using text message reminders, did improve adherence to anti-malarial drugs.

The two qualitative pilot studies both concerned (prevention of) mother-to-child HIV transmission.^16,17 The first evaluated an interactive SMS support group, consisting of young mothers and a clinician, and measured overall satisfaction with the intervention. The second one assessed the combination of text messages and calls from a clinician as a highly feasible and acceptable form of maternal support. This study found that emotional and motivational support is most frequently cited as the greatest participation benefit.

Summarizing, it can be seen that four out of five pilots showed positive results, either in the form of (high) feasibility or acceptability^13,16,17 or in the form of significant adherence improvement.¹⁵ Nevertheless, the interactive support-group intervention¹⁶ highlights frustration with malfunctioning phones and confidentiality challenges due to the frequent sharing of phones. It also poses the question of how it can be assured that the intervention, after scaling up, will include the most stigmatized women. The only study with negative results¹⁴ fails to provide a solid explanation for this. Rather, it suggests that non-adherence despite flashing-reminders might be related to ‘cultural beliefs’.

Post-trials (n = 14)

Out of 14 post-trial studies, six were RCTs. Two of these concerned the monitoring of medical abortion in South Africa. The first¹⁸ evaluated whether text messages to women who were about to have a medical abortion can reduce anxiety and better prepare them for symptoms. The results are significant. The second RCT¹⁹ established whether women having a medical abortion could self-assess whether their abortion was ‘complete’ using an interactive mobile phone questionnaire. This, however, did not predict all cases needing additional treatment at follow-up. One RCT²⁰ evaluated text reminders for anti-malarial treatment. It showed significant results. The other three RCTs concerned HIV care. Two of them^21,22 indicate significant improvement of ART adherence due to SMS reminders. The third study²³ provides more complicated findings, suggesting for instance that literacy is a robust predictor of success.

There were six post-trial studies that were not RCTs yet deployed a quantitative research design. Two of these were cross-sectional studies.^24,25 Except for one, all were concerned with HIV care. The exception concerned a helpline meant to enhance communication between cancer patients and their doctors. The helpline proved very successful.²⁶ Another study²⁵ initiated a two-way communication system which allowed people with HIV to express needs as well as gratitude by responding to text messages sent to them to improve medication adherence. A study in Zambia found that, in the diagnosis of infant infection with HIV, turnaround times could be reduced by using an automated notification system based on texting.²⁷ Another study,²⁴ which contacted HIV patients via voice or text to remind them about their missed clinic appointments, also shows positive results. The last two quantitative studies evaluated the acceptability and barriers of SMS reminders to enhance ART adherence among patients who had previously participated in an RCT.^28,29 Both studies indicated general satisfaction but various preferences with respect to content, frequency and timing of text messages.

The two qualitative post-trial studies were in-depth analyses of the experiences and responses of RCT participants.^30,31 The first³⁰ interviewed patients of the earlier mentioned medical abortion studies.^18,19 They conclude that SMS messages¹⁸ were highly acceptable, although more women had worried about phone privacy than estimated at baseline. The self-assessment questionnaire¹⁹ was successfully completed by 86.3% of the women. The second study³¹ conducted focus groups and individual interviews to explore experiences with a text-based adherence support intervention on blood pressure control. Patients highly valued the SMS content, the respectful tone, the delivery experience (timing of reminders and frequency) and the relational aspect of trial participation (feeling cared for).

In conclusion, most post-trials have generally positive outcomes that reveal the feasibility and potential of mHealth projects for patient follow-up and medication adherence. Only two studies emphasize (literacy) challenges²³ or inadequacies¹⁹ over potential. With respect to the barriers, privacy again turned out to be a challenge, as well as the importance of assessing the context and piloting.³⁰ Some studies found that patients preferred voice messages above text messages because of illiteracy.²⁴ With respect to cultural and gender issues, the study that set up a helpline²⁶ reports that three out of four married women felt that having their husband speak to the doctor made it easier to arrange for follow-up visits to the clinic, because women cannot travel to the clinic without permission of their husbands. Such permission was difficult to obtain prior to the intervention. The same study also reports that half of all participants had to borrow a phone to be able to call the doctor. With respect to the content and frequency of text messages, one study²⁰ suggests that, although a simple text message reminder can increase adherence to anti-malarial treatment, additional information included in messages does not have a significant impact. A second study²⁸ points out that although most patients were willing to receive SMS reminders to attend clinic appointments, only few expressed enthusiasm for daily SMS reminders to improve ART adherence. Finally, one study²⁹ argues that subsequent controlled trials in expanded settings and in-depth qualitative research are required to confirm and elucidate findings concerning determinants of (lasting) participant engagement.

Synthesis

Considering its primary objectives of reminding and informing, ‘patient follow-up and medication adherence’ can be regarded the most successful and straightforward application category. The reviewed articles suggest that, especially in the form of basic and immediate instruction – for example, text messages that simply remind patients to take their medication or attend clinics – this kind of application is realistic and effective. Complicated manifestations of patient follow-up and medication adherence seem less impactful. Evidence indicates that the success of interventions that, for instance, involve the execution of more complex tasks is limited at most. Illustrative is the failed attempt to have post-abortion women fill out an interactive mobile phone self-assessment questionnaire to see whether the abortion is indeed complete.

Challenges mentioned regardless the level of task complexity include (in order of prominence): illiteracy, securing privacy and confidentiality (due to phone sharing), financing (unwillingness of beneficiaries to contribute) and (mal)functioning of technology. Differences between pre-trial, pilots and post-trials are small. Optimism dominates all three types of assessments, although challenges are most carefully outlined in (predominantly qualitative) pre-trial studies.

2. Communication and information for (community) health care workers

Table 2 provides an overview of studies on communication and information for HCW. Studies on communication and information for HCW (n = 22) report findings from 10 different countries. Again, a classification into pre-trial (n = 5), pilots (n = 6), and post-trial (n = 11) will be used to discuss their results. In contrast to the studies on patient follow-up and medication adherence, where the use mobile phones for one-way texting was most common, text-only applications are a minority here (n = 5). The majority of the project assessments (n = 12) concern smartphone applications or interactive SMS networks such as RapidSMS or FrontlineSMS. These pieces of software are designed to monitor, train or communicate with community health workers (CHWs) (n = 11), conventional HCW (including clinic staff) (n = 8) or drug sellers (n = 3).

Table 2.

Overview of studies on communication and information for health care workers (n = 22).

APPLICATION CATEGORY: COMMUNICATION AND INFORMATION OF HEALTH CARE WORKERS (n = 22)
AUTHOR	YR	TYPE	COUNTRY	PHONE USE	CONDITION	RESULTS/IMPACT
Pre-trial (n = 5)
32. Chang et al.	‘13	Mixed	Uganda	App	HIV/AIDS	Feasible with changes to software
33. Little et al.	‘13	Mixed	Ethiopia	App	Maternal	Feasible yet concerns
34. Jo et al.	‘14	PC modelling	Uganda	Various	Neonatal	Greatest potential mHealth in MNH
35. Li et al.	‘13	Theoretical modelling	Burundi,	Text only	Malaria	Too many challenges (in Burundi)
35. Li et al.	‘13	Theoretical modelling	Zimbabwe	Text only	Malaria	Feasible (in Zimbabwe)
36. Pérez et al.	‘12	Ethnography	South Afr.	App	Unspecified	Feasible with changes to software
Pilot (n = 6)
37. Campbell et al.	‘14	Participatory	Malawi	SMS platform	General	Very positive; significant change
38. Lemay et al.	‘12	Mixed	Malawi	SMS platform	Reproductive	Very positive; significant change
39. Mahmud et al.	‘10	Quantitative	Malawi	SMS platform	General	Very positive; significant change
40. Ngabo et al.	‘12	Quantitative	Rwanda	SMS platform	MNH	Very positive; significant change
41. Armstrong et al.	‘12	Mixed	Botswana	Edu. software	General	Not feasible without modifications
42. Littman-Quinn et al.	‘13	Unspecified	Botswana	Edu. software	Female hlth.	Technological /cultural challenges
Post-trial (n = 11)
43. Friedman et al.	‘15	RCT	Ghana	Text only	Diarrhoea	Insignificant
44. Tomlinson et al.	‘13	RCT	South Afr.	App	General	Challenges
45. Zurovac et al.	‘11	RCT	Kenya	Text only	Malaria	Significant
46. Chang et al.	‘11	Mixed	Uganda	Two-way text and call	HIV/AIDS	Statistically insignificant yet positive qualitative results
47. Neupane et al.	‘14	Mixed	South Afr.	App	General	Acceptable yet challenges
48. Noordam et al.	‘15	Quantitative	Malawi,	App	Child health	Challenges
48. Noordam et al.	‘15	Quantitative	Zambia	App	Child health	Challenges
49. Tumusiime et al.	‘14	Quantitative	Uganda	SMS platform	Child health	Feasible
50. Zurovac et al.	‘12	Quantitative	Kenya	Text only	Malaria	Cost-effective
51. Dusabe et al.	‘13	Quantitative	Tanzania	Two-way text	Reproductive	Acceptable yet challenges
52. Jones et al.	‘12	Qualitative	Kenya	Text only	Malaria	Acceptable yet challenges
53. Shao et al.	‘15	Qualitative	Tanzania	App	Child health	Challenges

Pre-trials (n = 5)

Two out of five pre-trial analyses are mixed-methods studies.^32,33 Another employs computer-based modelling.³⁴ A third uses a literature framework.³⁵ A final pre-trial concerns an ethnographic study.³⁶ The first mixed-method study³² is a usability study that aimed to design user-friendly smartphone applications for community health workers (CHWs). It argues that unrestricted use of smartphones generated a strong sense of ownership and empowerment among the health workers. This is said to work as a strong motivator. The second mixed-method study³³ investigates the acceptability of CHWs using smartphone applications to enhance communication, clinical decisions, and to receive alerts. The aim was to improve patient care at a community-based HIV/AIDS clinic. This study reports general enthusiasm for mHealth. However, it also raises interesting concerns about possible negative consequences. For instance, research participants worried about security issues – such as potential theft – that make the CHWs insecure about walking at night. Another issue raised was the danger of CHW tasks exceeding certain limits. One CHW expressed concern that technology would be detrimental to the human side of their work, stressing that ‘physical sharing is very important’. Interestingly, participants expressed ideas on what needed to be done at a community level to prepare for the introduction of a mHealth intervention to minimize undesirable outcomes. Finally, some participants demonstrated unrealistic expectations of mHealth capabilities.

A third study³⁴ uses modelling software to identify priority areas for maternal and neonatal health services. This study suggest that skilled birth attendance and increased facility delivery as targets for mHealth strategies are likely to provide the biggest mortality impact relative to other intervention scenarios. A fourth study³⁵ compares the feasibility of a text message intervention for malaria in Burundi and Zimbabwe. It argues that while it is feasible in Zimbabwe, there are too many challenges in Burundi.

A fifth study³⁶ is an ethnographic usability study of a smartphone-based app that allows medical virtual volunteering. It shows that only with ‘real’ users, operating in the intended context of use, will usability problems be adequately detected. For this purpose, ethnographic research proved valuable. For instance, respondents in this study claimed they managed to finish the tasks, while the researchers observed that some had significant problems and could, in fact, not complete the tasks.

Pilots (n = 6)

Four out the six pilots targeted enhanced communication between (remote) CHWs and other medical staff.^37-40 The other pilots two aimed to improve knowledge of clinicians.^41,42

All four studies targeting enhanced communication distributed mobile phones to CHWs in an interactive text message-based network, using RapidSMS⁴⁰ or comparable interactive SMS networks.^37-39 The results are very positive: impressive changes in the operational capacity of the hospital, saved fuel and time-consuming trips, higher resolution in data reporting, and expanded healthcare delivery capacity.^37-39 in addition, CHWs benefitted from more trust and respect, and increased status in their communities, resulting in self-confidence and improved quality of care.^37,38,40 On the basis of participatory evaluation, one study³⁷ reports that CHWs had transformed into active information agents, recipients as well as providers of information, well connected to colleagues, beneficiaries, supervisors, and district health facilities. The only challenges that surfaced in these studies were telephone maintenance and lack or limited access to electricity.⁴⁰

The remaining two pilots targeting clinicians show less positive results. The first⁴² reflects on Botswana's first telemedicine and mobile learning programmes. This pilot study addresses numerous technical and social challenges, including malfunctioning phones, unreliable IT infrastructure, accidental damage to mobile devices, and cultural misalignment between IT and healthcare providers. It suggests that strategic goals of mHealth programmes should be aligned with those of national health policies and educational systems, and stresses that initiatives must be owned and led by local stakeholders. The other study⁴¹ tests txt2MEDLINE among clinicians. It shows how clinicians' use of this service is severely undermined because of some key issues in the application design.

Post-trials (n = 11)

Both the RCTs^43-45 and the mixed-methods studies^46,47 present findings on attempts to improve the quality of care of CHWs (with one exception⁴² where drug sellers are the ‘beneficiaries’). The quantitative studies^48-50 are concerned with the design and implementation of mHealth projects. The two qualitative studies^51,52 present perceptions and experiences of CHWs participating in quantitative studies. The one remaining article⁵⁰ is a cost-effectiveness study of one of the RCTs,⁴⁵ written by the same authors.

The RCT which targets drug sellers did not find indications of behaviour change.⁴³ Another RCT⁴⁵ succeeded in improving CHWs’ adherence to malaria treatment guidelines, and proved cost effective.⁵⁰ Findings from other studies emphasized that most challenges were not technical but related to the (mal)functioning of health care systems, such as poor motivation caused by low salaries, limited professional and career development, and lack of incentives.^44,47 One study⁴⁴ argued that mobile phones will not replace the need for the ‘human face’ of support as well as the need for CHW supervision. One mixed-method study⁴⁶ showed insignificant statistical results yet qualitative indications of improvements in patient care and additional benefits such as a sense of empowerment among CHWs and improved CHWs–patient communication.

Like the RCTs, the three quantitative studies also stress non-ICT-related barriers to implementation of mHealth projects. For instance, they mention programme delivery characteristics that were not easily addressed by the additional ICT component⁴⁸ and the need for training and follow-ups which are necessary for the intervention to work.⁵¹ One study⁴⁹ emphasizes the willingness of CHWs to learn and develop new skills as something that may contribute to improved health care. The qualitative studies show a general acceptability of mHealth projects. One study⁵² calls for more research into the effects of specific SMS characteristics (e.g. frequency, delivery, and number of messages) and shows how participants expressed the view that SMS could not be used to replace training and seminars. The second study⁵³ again shows how – even if participants appeared motivated – there was a low uptake of technology because of the (mal)functioning of health systems and related barriers such as a lack of staff, lack of medicine, and lack of financial motivation.

Synthesis

Research on ‘communication and information for health care workers’ shows more ambiguous results than research on ‘patient follow-up and medication adherence’. The presence of non-technological barriers is more apparent in this application category. A crucial challenge concerns the fact that beneficiaries in Africa – that is, local medical staff – are positioned within generally inadequate or dysfunctional health care systems. Rather than merely individual practice, success of the intervention thus depends on the way in which these systems are organized, are able to facilitate change and, for instance, affect motivation of those involved.

Within this application category, a research type-related difference in project valuation surfaces. Whereas pilot studies show comparatively positive results, post-trial assessments indicate the presence of complications alongside accomplishments. Similar to ‘patient follow-up and medication adherence’, such difference correlates with the complexity of the intervention (rather than type of study). For instance, a pilot assessing an attempt to simply improve communication between remote HCWs and clinic-based medical staff (RapidSMS) reports straightforward successes. Yet, a post-trial research on the effectiveness of an intervention meant to improve knowledge, build competence and realize behaviour change proved challenging as a result of the complexity of aspired change and the impact of systemic flaws (e.g. lack of staff, limited treatment facilities).

3. Health promotion and disease prevention

Table 3 provides an overview of studies on health promotion and disease prevention. Studies on health promotion and disease prevention (n = 16) are mostly concerned with the potential of text messages or interactive text quizzes to improve health knowledge and behaviours. These studies were conducted in seven different countries. They are also classified into pre-trials (n = 2), pilots (n = 6), and post-trials (n = 8).

Table 3.

Overview of studies on Health promotion and disease prevention (n = 16).

APPLICATION CATEGORY: HEALTH PROMOTION AND DISEASE PREVENTION (n = 16)
AUTHOR	YR	TYPE	COUNTRY	PHONE USE	CONDITION	RESULTS
Pre-trial (n = 2)
54. Jennings et al.	‘13	Qualitative	Kenya	Text only	HIV/AIDS	Feasible yet challenges
55. Akinfaderin-Agarau et al.	‘12	Qualitative	Nigeria	Text and quiz	Reproductive	Significant challenges
Pilot (n = 6)
56. Chib et al.	‘12	Quantitative	Uganda	SMS campaign	HIV/AIDS	Insignificant
57. Vahdat et al.	‘13	Quantitative	Kenya	SMS campaign	Reproductive	General positive
58. L'Engle et al.	‘13	Quantitative	Tanzania	SMS campaign	Reproductive	General positive
59. Wakadha et al.	‘13	Quantitative	Kenya	Text only	Immunization	General positive
60. MacLeod et al.	‘12	Mixed	Ghana	Various	MNH	Challenges
61. Larsen-cooper	‘14	Mixed	Malawi	Various	MNH	Significant challenges
Post-trial (n = 8)
62. de Tolly et al.	‘12	RCT	South Africa	Text only	HIV/AIDS	Significant
63. Odeny et al.	‘15	RCT	Kenya	Two-way text	HIV/AIDS	Significant
64. Lund et al.	‘10	RCT	Tanzania	Text only	Maternal	Significant
65. Danis et al.	‘10	Quantitative	Uganda	SMS quiz	HIV/AIDS	Generally positive
66. Lepper, de et al.	‘13	Quantitative	Uganda	SMS quiz	General	Generally positive
67. Crawford et al.	‘14	Quantitative	Malawi	Text, voice	MNH	SMS preferred method
68. Lau et al.	‘14	Mixed	South Africa	Text only	MNH	Insignificant
69. Chib et al.	‘13	Modelling	Uganda	SMS campaign	HIV/AIDS	Insignificant

Pre-trials (n = 2)

Both pre-trials were qualitative and exploratory studies into the contextual factors influencing SMS interventions. The first⁵⁴ explores the potential of text messages to HIV-positive women and their partners as a means to prevent mother-to-child transmission. This study found general acceptability of the gender-tailored SMS and voice calls. However, not all types of information proved equally acceptable, and participants preferred personalized messages above generic ones. In addition, health workers emphasized the continuous need for person-to-person counselling coupled with, rather than replaced by, telecommunication. The second study⁵⁵ explored the barriers and limitations experienced by young women with respect to the use of their phones to access sexual and reproductive health services in Nigeria. Crucial barriers were cost of service, confidentiality and anonymity concerns due to requests for socio-demographic information, poor marketing and publicity, socio-cultural beliefs and expectations, power relations with male partners, and poor infrastructural/network quality. Considering these barriers, the authors expect that socio-cultural challenges will be the most difficult to overcome, especially considering the fact that young girls require permission from gatekeepers to use mobile phones and, consequentially, access the service.

Pilots (n = 6)

Out of six pilots, four were quantitative studies^56–59 assessing the effectiveness of SMS messages or campaigns in providing health information. Of these, only the text-based HIV/AIDS campaign⁵⁶ reports very negative results. The project failed to increase knowledge levels on a sufficient scale, with only one-fifth of participants responding to any of the SMS questions. In addition, the programme's intended audience – those who do not have correct HIV/AIDS knowledge – were not those most likely to receive the correct knowledge through this campaign, thus reinforcing existing knowledge divides within the population. In contrast, using SMS to provide contraception information to young people⁵⁷ or family planning information⁵⁸ and as reminders for timely immunization⁵⁹ were assessed more positively. Besides addressing technical issues,⁵⁹ important recommendations were to implement mHealth as complementary to traditional health programming and not as stand-alone-interventions,⁵⁷ and to extensively engage husbands for immunization-related interventions.⁵⁹

The two remaining papers were mixed-methods studies.^60,61 The first⁶⁰ concerns the MoTeCH initiative in northern Ghana; a complex system comprising several different processes and the engagement with multiple target groups. Aimed to improve community-based primary health care, MoTeCH sends weekly educational voice messages to pregnant women and young mothers. In addition, nurses send patient data to the MoTeCH system, which is then aggregated to generate monthly reports to district health managers. The authors conclude that (mainly) software changes are needed for the programme to work well. For MoTeCH to be able to address all concerns, ‘its design and implementation requires nothing less than a systemic change in Ghana's existing health information system process’.⁶⁰

The last pilot⁶¹ concerned a mixed-methods exploration of the potential of working with community volunteers as intermediaries to extend access to mHealth for individuals with no phone. Borrowing the volunteers' phones, beneficiaries could call a toll-free hotline or receive text or voice messages on reproductive, maternal, and new-born health topics. Although the service was used by people without personal cell phones, there were significant challenges. Solar panels used to charge the phones did not function, and 70% of the low-cost phones broke down. Volunteers had received inadequate training and some did not know how to use the phones. In addition, motivation to partake (as volunteer) dropped significantly as a result of a lack of (financial) incentives. Finally, beneficiaries said it was inconvenient to travel to the volunteer, who often lived far away.

Post-trials (n = 8)

The post-trials include three RCTs that all used text messages and show significant impact. Two of them used texting to improve (infant) HIV testing and prevention of mother-to-child HIV transmission.^62,63 Infant HIV testing in particular increased, while overall return rates for postpartum visits remained low.⁶³ In addition, it was found that the amount of text messages that were sent did not have a statistically significant effect.⁶² The third RCT⁶⁴ managed to significantly increase skilled delivery attendance among urban women. However, it did not reach rural women.

Three quantitative studies^65-67 present more diverse findings. Two of them employed an interactive SMS quiz design. One study⁶⁵ aimed to explore the viability of SMS as a user interface technique for untrained users, and assessed the potential impact of social factors, such as literacy, on participation rates. While, for instance, the impact of literacy remains uncertain, findings show that SMS messages can be used effectively with untrained users. However, the authors believe that it may be possible to help participants to learn more quickly how to respond correctly to the system by modifying the feedback it sends. A second study⁶⁶ was a cohort study which aimed to assess the response patterns of participants in free SMS health quizzes. Results indicated that the most important factor influencing response time and participation rate was the reliability of the network (provider). Quiz topics also influenced both participation rates and response time. The third study⁶⁷ determined the difference in delivery success of health messages through SMS, voice to personal phones, and voice retrieved from a community phone. Findings indicated that SMS was the preferred delivery method because of lower cost, higher delivery success, and higher levels of intended or actual behaviour change.

The only mixed-methods study,⁶⁸ exploring the use of text messages by pregnant women, was hindered by a substantial loss to follow-up, and failed to improve antenatal health knowledge. Nevertheless, there was some evidence of (self-reported) healthier behaviours during pregnancy.

The final study is by the same authors of the pilot on HIV/AIDS campaign that was negatively assessed.⁵⁶ In their post-trial evaluation⁶⁹ the authors present interesting lessons learned. They argue that socio-cultural, informational, and economic vulnerabilities associated with gender, illiteracy, poverty, and stigma were not considered in the SMS programme design. However, gender patterns, for instance, are a key driver of the HIV epidemic in Uganda, as women are subject to male-dominated relationships. The relationship between food insecurity, as an outcome of extreme poverty, and increased sexual risk-taking by women is an example of economic vulnerability. However, women who tend to be more affected by the stigma associated with being HIV-positive might not have benefitted from SMS campaigns due to their lower mobile phone ownership. Consequently, the campaign may have failed to reach some of the most vulnerable segments of the population (e.g. lowest-income females). Also, quiz design motivated recipients with the correct HIV/AIDS knowledge to respond (and thus become eligible for free HIV screening). In doing so, however, the more vulnerable sections of the population, such as those with limited knowledge, remained unreached. Another weakness concerned the dependency on only one telecommunication provider. Because of this exclusive partnership, subscribers of other providers were automatically ineligible. The authors recommend that, in order to address socio-cultural and informational vulnerabilities, future SMS interventions should opt for a participatory approach and include the community in designing campaigns.

Synthesis

Assessments of interventions in the realm of ‘health promotion and disease prevention’ report generally positive results. An exception concerns the pre-trials; they stress the difficult task of overcoming socio-cultural challenges. Pilots and post-trial studies are more optimistic. However, also these indicate that simplicity is a key determinant of success. Most effective is one-way text messaging (e.g. messages urging vaccination, HIV/AIDS testing or clinic delivery). Interventions that moved beyond simple instruction, or for instance require a systemic change in existing health information systems, proved less effective. Particularly interesting is the problematic distribution of information – for the sake of behavioural change – by means of SMS. Attempts to build knowledge or create awareness by means of, for instance, quizzes proved ineffective as participation is often limited to people who are already informed and aware. Reaching the most vulnerable segments of populations (e.g. women, illiterate, poor) requires project initiators to tackle the issue of access and put much more effort in the design of content. The reviewed articles unfortunately provide little indication of successful recipes to do so.

Discussion

Most of the mHealth project assessments in Africa included in this review report successes and stressed positive outcomes, either in terms of acceptability and feasibility or in terms of actual health outcomes. In fact, out of 65 project assessments, 46 (71%) concerned an ultimately positive project evaluation. Nine studies emphasized challenges and the urgent need for modification, and six studies classified the impact as ‘insignificant’. The remaining four studies showed different (neutral) results such as ‘SMS was preferred delivery method’. However, nearly all of these studies acknowledge the presence of obstacles impeding the successful implementation of mHealth interventions. Some of these obstacles concerned technological or infrastructural challenges such as poor access to network and/or electricity, malfunctioning phones, or a general unreliable ICT environment. More fundamental, however, were often-mentioned obstacles pertaining to human factors or the context of implementation, such as high illiteracy rates, confidentiality issues due to phone sharing, beneficiaries’ poor motivation caused by, for instance, low salaries, a lack of incentives and/or high work pressure, and broader health system challenges such as a shortage of adequately trained staff, and failure to integrate the new technology within existing systems of care. Unlike in most of the project assessments, it is these obstacles that feature most prominently in the conceptual/thematic articles (n = 19) and literature reviews (n = 16) that were included in this review. Although these overview studies do recognize the potential of mHealth, they almost unanimously conclude that there is insufficient evidence to be confident about the scalability and sustainability in particular of the majority of mHealth projects. These conceptual/thematic articles and literature studies will be further discussed below. Their inclusion in this systematic review is crucial, as they help further uncover key challenges that hamper full realization of the potential of mHealth. Table 4 provides an overview of conceptual/thematic articles.

Table 4.

Overview of conceptual/thematic mHealth studies.

CONCEPTUAL/THEMATIC ARTICLES (n = 19)
AUTHOR(S)	YR	THEME/CONTENT	CONCLUSION
4. Kaplan	‘06	General	No evidence base
70. Akter and Ray	‘10	General	Major barriers, weak evidence base
71. Chib	‘13	General	Major barriers, weak evidence base
72. Folaranmi	‘14	General	Major barriers, weak evidence base
73. Kahn et al.	‘10	General	No evidence base
74. Mechael	‘09	General	No evidence base
75. Mechael et al.	‘12	General	Major barriers, weak evidence base
76. Norris et al.	‘13	Mental health in South Africa	No evidence base
77. Waldman and Stevens	‘15	Reproductive health in South Africa	Major barriers, weak evidence base
78. Thirumurthy and Lester	‘12	Behaviour change	Evidence for mHealth as BCC tool
79. Zurovac et al.	‘12	Malaria	Major barriers, weak evidence base
80. Speciale and Freytsis	‘12	Midwifery	Major barriers, weak evidence base
81. DeRenzi et al.	‘09	Health care workers	Major barriers, weak evidence base
82. Leon et al.	‘12	Scaling up	Major barriers, weak evidence base
83. Tomlinson et al.	‘13	Scaling up	No evidence base
84. Mburu et al.	‘13	Conceptual framework	Major barriers, weak evidence base
85. Nacinovich	‘11	Definition mHealth	Major barriers, weak evidence base
86. O'Connor and O'Donoghue	‘15	Contextual barriers to mHealth	Major barriers, weak evidence base
87. Sanner et al.	‘12	MHealth typology	Major barriers, weak evidence base

Of the conceptual/thematic articles included in this review, seven address the general problems and general potential of mHealth.^4,70-75 Four articles focus on the potential of mHealth for a specific condition or field of care: mental health,⁷⁶ sexual and reproductive health,⁷⁷ HIV care⁷⁸ and malaria.⁷⁹ Two articles assess specific target groups: midwifes⁸⁰ and field-based HCW.⁸¹ Two other articles concern the scaling-up of mHealth projects.^82,83 One article offers a conceptual framework for designing mHealth solutions,⁸⁴ one article is concerned with defining mHealth,⁸⁵ another reflects on contextual barriers,⁸⁶ and a final article provides a mHealth typology.⁸⁷

Strikingly, the majority of these studies express grave concerns about the current weak evidence base of mHealth. In fact, the dominant overall conclusion is that there is yet insufficient proof of the effectiveness of mHealth.^{4,73,74,76,83} Interestingly, most barriers that are highlighted go beyond mere technological challenges and are related to:

Broader systemic challenges,^72,82 such as inadequate training to support HCW, conflicting health system priorities or difficulties integrating technology with existing information systems.

Political issues,^70,77,80,86 such as insufficient political commitment and support, or disagreements between stakeholders with different agendas.

Financial issues,^72,74,75 usually a lack of funding.

Logistical and infrastructural issues,^74,80,82,86 such as a weak ICT environment with limited implementation capacity, or inconsistent access to electricity.

Cultural issues,^4,74,80,86 such as high illiteracy rates, hierarchical access to technology, phone sharing which complicates confidentiality, (gendered) power dynamics.

The central message is that mHealth should not be perceived as a magic bullet or stand-alone intervention which can resolve existing problems;^71,72,74,81 neither can it be used to replace the role of conventional health care providers.^71,72,76 Another repeatedly emphasized concern is that context matters, and that it is important to understand the (social, cultural, political and even communicative/linguistic) dynamics of phone use.^71,81 There is too often a gap between how a programme was originally designed to work and how it functions in the field.⁸¹ Hence there is need for mHealth projects which take an integrated and user-centred approach, designed with a specific target group in mind rather than being technology driven.^70,71,80

Unsurprisingly, researchers thus (continue to) call for more research to provide mHealth with its much-needed evidence base: more RCTs,⁷⁸ qualitative research,^71,79 cost-effective evaluations,^4,72,79 and impact evaluations.^73,74,79

Table 5 shows an overview of mHealth literature studies. Out of the 16 literature studies included in this review, seven reviewed specific/single-purpose mHealth projects: chronic diseases,⁸⁸ non-communicable diseases,⁸⁹ disease control,⁹⁰ maternal health,⁹¹ HIV,⁹² new-born health,⁹³ and behaviour change communication.⁹⁴ Five literature studies were reviews on mHealth in general.^3,95–98 Another three studies targeted CHWs,^99–101 and one final study focused on the influence of mHealth on gender.¹⁰²

Table 5.

Overview of mHealth literature studies.

LITERATURE STUDIES (n = 16)
AUTHOR(S)	YR	THEME/CONTENT	CONCLUSION
3. Aranda-Jan et al.	‘14	General	Evidence base is weak, more research needed
88. Beratarrechea et al.	‘14	Chronic disease	Evidence base is weak, more research needed
89. Bloomfield et al.	‘14	Non-communicable diseases	Evidence base is weak, more research needed
90. Deglise et al.	‘12	Disease control	Evidence base is weak, more research needed
91. Noordam et al.	‘11	Maternal health	Evidence base is weak, more research needed
92. Sharma et al.	‘12	HIV/AIDS	Evidence base is weak, more research needed
93. Tamrat, Kachnowski	‘12	MNH	Evidence base is weak, more research needed
94. Gurman et al.	‘12	Behaviour change	Evidence base is weak, more research needed
95. Betjeman et al.	‘13	General	Evidence base is weak, more research needed
96. Chib et al.	‘14	General	Evidence base is weak, more research needed
97. Goel et al.	‘13	General	Evidence base is weak, more research needed
98. Hall et al.	‘14	General	Evidence base is weak, more research needed
99. Braun et al.	‘13	Community health workers	Evidence base is weak, more research needed
100. Källander et al.	‘13	Community health workers	Evidence base is weak, more research needed
101. O'Donovan et al.	‘14	Health care workers	Evidence base is weak, more research needed
102. Jennings and Gagliardi	‘13	Gender	Evidence base is weak, more research needed

All 16 studies conclude that, despite some promising initial results at a small scale, there is not enough evidence to be confident about the scalability and sustainability of most mHealth ventures. Consequently, and like the conceptual/thematic articles, also all these studies stress the urgent need for rigorous research, in particular impact evaluations^{91,94,98,99,101,102} cost-effectiveness studies^3,90,98,99 and (qualitative) research into context of implementation.^{92–94,96,101}

Barriers to implementation often mentioned mostly concern policy-related and cultural or contextual issues. With regards to policy-related challenges, authors stress a dependency on funding, unclear health system responsibilities, unreliable infrastructure, and a lack of evidence on cost-effectiveness.³ Yet, they also observe a gap between what application designers do in practice and what the governments consider priorities.⁹⁸ One study¹⁰⁰ argues that, in this respect, a ‘national ownership’ of mHealth cannot be overemphasized.

In line with the conceptual/thematic articles, literature reviews also stress that particularly crucial to the success and failure of mHealth projects are cultural and contextual issues,^{92,94,96,99,101} and the careful integration of mHealth into the existing healthcare system.^91,94,99 Projects have proven to be most successful when they have been adjusted to fit the local context and language, when they are backed by a government with an existing mHealth strategy and an interest in creating a system to integrate mHealth, and when they have been developed and implemented by public–private partnerships (e.g. participation of local private service providers).^91,94 Absolutely crucial in this respect is to opt for a user-centric design in which, more than currently, there is a focus on the complex process of adoption.^94,99,101 and an effort to collaborate with local partners.^91,94

Conclusion

(Potential) benefits of mHealth projects are stressed in most project assessments discussed in this review. Despite significant limitations and challenges that surface in the majority of these texts, by and large they are characterized by an overall sense of optimism. Such optimism is in stark contrast to virtually all conceptual/thematic articles and literature studies, which continue to stress the weak evidence base of mHealth.

Systematic review of 10 years of mHealth research (2005–2015) – with an exclusive focus on community health in Africa – suggests that this contrast can be explained as the result of the difference in focus and objective of the various scholarly assessments. Project-based studies generally show an (inevitably) limited scope, not exceeding the confines of the project as defined, as well as a limited timeframe (mainly surrounding trial phase). Also, they are more likely to be biased towards validation as a result of project affinity. Such affinity is absent in conceptual/thematic articles and literature studies. They provide an aerial perspective instead, and move beyond narrowly defined project objectives in order to detect tendencies and contemplate and categorize the strengths, weaknesses, opportunities and threats that characterize practices in the field of mHealth.

All but one of these conceptual/thematic articles and literature studies (n = 34) conclude that mHealth's evidence base is at least too weak to legitimize existing confidence in mobile phones as (cost-)effective instruments of rapid advance in African health care. In particular, these texts report a lack of understanding of, or a serious reason to doubt, the sustainability and scalability of mHealth interventions. Even though project assessments report improved communication and behaviour change, the overview studies stress the presence of flaws and structural challenges that prevent broad implementation and undermine the sustainability of change.

The systematic review of research on mobile phones for community health above indicates these flaws and challenges concern two ‘categories of complication’. Distinguishing these two categories, recognizing their explanatory value in the assessment of mHealth's potential in Africa, allows us to move beyond the optimist/critic divide and add some crucial nuance to the observation that evidence remains scarce. Essentially, a distinction should be made between technology-related complications of a universal kind, and circumstantial complications that are context (time and place) specific. Technology-related complications concern the technical nature of mHealth as a developmental phenomenon: the practical difficulties that one encounters as the intervention involves the distribution of complex and vulnerable devices that require a reliable technological infrastructure to function properly. Across Sub-Sahara Africa, realization of the potential of mHealth projects is undermined by similar ‘phenomenic’ complications such as: poor network service, inconsistent access to electricity, malfunctioning phones, and a lack of resources necessary to secure maintenance and incessant subscription. The persistence of technology-related complications as one of the reasons for project failure seems naïve and suggests that, at the point of (pilot) implementation, it is too often unclear whether or how general technological preconditions are locally met. A solid pre-project assessment would at least allow more effective anticipation of technology-related challenges.

Once adequately addressed, and with the inescapable advance of information and communication technology (e.g. Google's Project Link and Project Loon), these technology-related complications – prominent in current reviews – should ultimately not determine our verdict on mHealth. Rather, the focus should be on what works and what does not work once technology-related challenges are effectively tackled. This involves a contemplation of the impact of circumstantial complications that depend on the wider context of implementation: the local organization of education, political conditions and, for instance, the socio-cultural dynamics that shape interactions, communicative practices and adaptation trajectories. As shown above, the relevance of such circumstantial complications is widely recognized. However, possibly due to the techno-centric orientation of most mHealth literature, their actual impact as determinants of success/failure seems underestimated. It is insufficient merely to acknowledge the fact that context matters. Syntheses above are illustrative. Project assessments within all three intervention categories show the effectiveness of mHealth projects depends on either (a) the success of rare efforts to address circumstantial complications, or (b) the possibility to avoid (many of) these complications through simple design and limited objective. Most successful are clearly those interventions that aim for communicative convenience (the earlier mentioned Medic Mobile initiative in Kenya is an interesting example that should be thoroughly assessed), simple instruction or notification (e.g. SMS reminders), especially if beneficiaries are experienced phone users. In order for more complicated interventions to work and thus realize development through attitudinal and behaviour change, a detailed understanding of the multifaceted context of implementation is indispensable.

In conclusion, as far as the future of mHealth in Africa is concerned, a constructive realism – rather than unfounded optimism or pessimism without nuance – should guide the design of interventions. Besides being anticipative of the harmful consequences of technology-related complications, such realism should either translate into modest aspirations and the mere facilitation of basic communication and information, or into a smarter kind of mHealth shaped by a thorough understanding of circumstantial complications. The latter inevitably requires an increased investment in research. Yet, unlike what has been concluded elsewhere, that does not just mean intensifying existing (RCT-dominated) research strategies, and looking for transposable recipes for success. Rather, it means expanding our quest for understanding in order to include careful qualitative and participatory (cyclical) explorations of exactly those local opportunities and threats that determine the realization of the undeniable potential of mHealth.

Footnotes

Contributorship

JdK and EK researched literature and conceived the study. EK wrote the first draft of the manuscript. JdK reviewed, added to, and edited the manuscript. Both authors approved the final version of the manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

Not applicable.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Guarantor

JdK.

Peer Review

This manuscript was reviewed by two individuals who have chosen to remain anonymous.

References

Ericsson Mobility Report 2015. Stockholm: Ericsson, 2015, http://www.ericsson.com/res/docs/2015/mobility-report/emr-nov-2015-regional-report-African-africa.pdf (accessed 27 June 2016).

World Health Organization (WHO). mHealth: New horizons for health through mobile technologies Geneva: WHO, 2011, http://www.who.int/goe/publications/goe_mhealth_web.pdf (accessed 27 June 2016).

Aranda-Jan

Mohutsiwa-Dibe

Loukanova

. Systematic review on what works, what does not work and why of implementation of mobile health (mHealth) projects in Africa. BMC Public Health 2014; 14: 188.

Kaplan

. Can the ubiquitous power of mobile phones be used to improve health outcomes in developing countries? Global Health 2006; 2: 9.

Coleman

. Medication adherence of elderly citizens in retirement homes through a mobile phone adherence monitoring framework (Mpamf) for developing countries: A case study in South Africa. Indian J Pharm Educ Res 2014; 48(3): 6–11.

Smillie

Van Borek

van der Kop

. Mobile health for early retention in HIV care: A qualitative study in Kenya (WelTel Retain). African J AIDS Res [Internet] 2015; 13(4): 331–338. Available from: http://www.tandfonline.com/doi/full/10.2989/16085906.2014.961939.

Mall

Sibeko

Temmingh

. Using a treatment partner and text messaging to improve adherence to psychotropic medication: A qualitative formative study of service users and caregivers in Cape Town, South Africa. Afr J Psychiatry 2013; 16(5): 364–370.

Hwabamungu

Williams

. M-health adoption and sustainability prognosis from a care givers' and patients' perspective. Annu Res Conf South African Inst Comput Sci Inf Technol 2010; 1: 123–131.

Haji HA, Suleman H, Rivett U, et al. Mobile graphic-based communication: Investigating reminder notifications to support tuberculosis treatment in Africa. In: Zhang Y, Yao G, He J, Wang L, Smalheiser NR, Yin X (eds) Health Information Science Third International Conference, HIS 2014, Shenzhen, China, 22–23 April 2014. LNCS 8423. Springer, pp.204–211.

10.

Mbuagbaw

Bonono-Momnougui

R-C

Thabane

. A framework for community ownership of a text messaging programme to improve adherence to antiretroviral therapy and client-provider communication: A mixed methods study. BMC Health Serv Res 2014; 14: 441.

11.

Bigna

Noubiap

Plottel

. Barriers to the implementation of mobile phone reminders in pediatric HIV care: A pre-trial analysis of the Cameroonian MORE CARE study. BMC Health Serv Res 2014; 14: 523.

12.

Otieno

Githinji

Jones

. The feasibility, patterns of use and acceptability of using mobile phone text-messaging to improve treatment adherence and post-treatment review of children with uncomplicated malaria in western Kenya. Malar J 2014; 13: 44.

13.

Hoffman

Cunningham

Suleh

. Mobile direct observation treatment for tuberculosis patients: A technical feasibility pilot using mobile phones in Nairobi, Kenya. Am J Prev Med 2010; 39(1): 78–80.

14.

Kliner

Knight

Mamvura

. Using no-cost mobile phone reminders to improve attendance for HIV test results: A pilot study in rural Swaziland. Infect Dis Poverty 2013; 2: 12.

15.

Modrek

Schatzkin

De La Cruz

. SMS messages increase adherence to rapid diagnostic test results among malaria patients: Results from a pilot study in Nigeria. Malar J 2014; 13: 69.

16.

Dean

Makin

Kydd

. A pilot study using interactive SMS support groups to prevent mother-to-child HIV transmission in South Africa. J Telemed Telecare 2012; 18(7): 399–403.

17.

Schwartz

Clouse

Yende

. Acceptability and feasibility of a mobile phone-based case management intervention to retain mothers and infants from an Option B + program in postpartum HIV care. Matern Child Health J 2015; 19: 2029–2037.

18.

Constant

de Tolly

Harries

. Mobile phone messages to provide support to women during the home phase of medical abortion in South Africa: A randomised controlled trial. Contraception 2014; 90(3): 226–233.

19.

Constant

de Tolly

Harries

. Assessment of completion of early medical abortion using a text questionnaire on mobile phones compared to a self-administered paper questionnaire among women attending four clinics, Cape Town, South Africa. Reprod Health Matters 2015; 22(44 Suppl 1): 83–93.

20.

Raifman

JRG

Lanthorn

Rokicki

. The impact of text message reminders on adherence to antimalarial treatment in northern Ghana: A randomized trial. PLoS One 2014; 9(10): e109032–e109032.

21.

Lester

Ritvo

Mills

. Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): A randomised trial. Lancet 2010; 376(9755): 1838–1845.

22.

Pop-Eleches

Thirumurthy

Habyarimana

. Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: A randomized controlled trial of text message reminders. AIDS 2011; 25(6): 825–834.

23.

Siedner

Santorino

Haberer

. Know your audience: Predictors of success for a patient-centered texting app to augment linkage to HIV care in rural Uganda. J Med Internet Res 2014; 17(3): e78.

24.

Kunutsor

Walley

Katabira

. Using mobile phones to improve clinic attendance amongst an antiretroviral treatment cohort in rural Uganda: A cross-sectional and prospective study. AIDS Behav 2010; 14(6): 1347–1352.

25.

Mbuagbaw

Thabane

Ongolo-Zogo

. Opening communication channels with people living with HIV using mobile phone text messaging: Insights from the CAMPS trial. BMC Res Notes 2013; 6: 131.

26.

Odigie

Yusufu

LMD

Dawotola

. The mobile phone as a tool in improving cancer care in Nigeria. Psychooncology 2012; 21(3): 332–335.

27.

Seidenberg

Nicholson

Schaefer

. Early infant diagnosis of HIV infection in Zambia through mobile phone texting of blood test results. Bull World Health Org 2012; 90(5): 348–356.

28.

Reid

MJA

Dhar

Cary

. Opinions and attitudes of participants in a randomized controlled trial examining the efficacy of SMS reminders to enhance antiretroviral adherence: A cross-sectional survey. J Acquir Immune Defic Syndr 2014; 65(2): e86–e88.

29.

van der Kop

Karanja

Thabane

. In-depth analysis of patient-clinician cell phone communication during the WelTel Kenya1 antiretroviral adherence trial. PLoS One 2012; 7(9): e46033–e46033.

30.

de Tolly

Constant

. Integrating mobile phones into medical abortion provision: Intervention development, use, and lessons learned from a randomized controlled trial. J Med Internet Res 2014; 16(1): e5.

31.

Leon

Surender

Bobrow

. Improving treatment adherence for blood pressure lowering via mobile phone SMS-messages in South Africa: A qualitative evaluation of the SMS-text Adherence SuppoRt (StAR) trial. BMC Fam Pract 2015; 16: 80.

32.

Chang

Njie-Carr

Kalenge

. Perceptions and acceptability of mHealth interventions for improving patient care at a community-based HIV/AIDS clinic in Uganda: A mixed methods study. AIDS Care 2013; 25(7): 874–880.

33.

Little

Medhanyie

Yebyo

. Meeting community health worker needs for maternal health care service delivery using appropriate mobile technologies in Ethiopia. PLoS One 2013; 8(10): e77563.

34.

Labrique

Lefevre

. Using the lives saved tool (LiST) to model mHealth impact on neonatal survival in resource-limited settings. PLoS One 2014; 9(7): e102224–e102224.

35.

Li Y, Yan Y, Du L, et al. Health Information Science 2013; 7798: 250–257. Available from: http://dl.acm.org/citation.cfm?id=2467876.2467898.

36.

Pérez

Pla

. Remote usability study on mHealth app VirTelMed in a South African setting. Int J Computer Appl 2012; 59(6): 28–34.

37.

Campbell

Schiffer

Buxbaum

. Taking knowledge for health the extra mile: Participatory evaluation of a mobile phone intervention for community health workers in Malawi. Glob Health Sci Pract 2014; 2(1): 23–34.

38.

Lemay

Sullivan

Jumbe

. Reaching remote health workers in Malawi: Baseline assessment of a pilot mHealth intervention. J Health Commun 2012; 17(suppl 1): 105–117.

39.

Mahmud

Rodriguez

Nesbit

. A text message-based intervention to bridge the healthcare communication gap in the rural developing world. Technol Health Care 2010; 18(2): 137–144.

40.

Ngabo

Nguimfack

Nwaigwe

. Designing and implementing an innovative SMS-based alert system (RapidSMS-MCH) to monitor pregnancy and reduce maternal and child deaths in Rwanda. Pan Afr Med J 2012; 13: 31.

41.

Armstrong

Liu

Seymour

. Evaluation of txt2MEDLINE and development of short messaging service-optimized, clinical practice guidelines in Botswana. Telemed J E Health 2012; 18(1): 14–17.

42.

Littman-Quinn

Mibenge

Antwi

. Implementation of m-health applications in Botswana: telemedicine and education on mobile devices in a low resource setting. J Telemed Telecare 2013; 19(2): 120–125.

43.

Friedman

Woodman

Chatterji

. Can mobile phone messages to drug sellers improve treatment of childhood diarrhoea? A randomized controlled trial in Ghana. Health Policy Plan 2015; 30(suppl 1): i82–i92.

44.

Tomlinson

Rotheram-Borus

Doherty

. Value of a mobile information system to improve quality of care by community health workers. S Afr J Inf Manag 2013; 15(1): 1–9.

45.

Zurovac

Sudoi

Akhwale

. The effect of mobile phone text-message reminders on Kenyan health workers' adherence to malaria treatment guidelines: A cluster randomised trial. Lancet 2011; 378(9793): 795–803.

46.

Chang

Kagaayi

Arem

. Impact of a mHealth intervention for peer health workers on AIDS care in rural Uganda: A mixed methods evaluation of a cluster-randomized trial. AIDS Behav 2011; 15(8): 1776–1784.

47.

Neupane

Odendaal

Friedman

. Comparing a paper based monitoring and evaluation system to a mHealth system to support the national community health worker programme, South Africa: An evaluation. BMC Med Inform Decis Mak 2014; 14: 69.

48.

Noordam

George

Sharkey

. Assessing scale-up of mHealth innovations based on intervention complexity: Two case studies of child health programs in Malawi and Zambia. J Health Commun 2015; 20: 1–11.

49.

Tumusiime

Agaba

Kyomuhangi

. Introduction of mobile phones for use by volunteer community health workers in support of integrated community case management in Bushenyi District, Uganda: Development and implementation process. BMC Health Serv Res 2014; 14(Suppl 1): S2.

50.

Zurovac

Larson

Sudoi

. Costs and cost-effectiveness of a mobile phone text-message reminder programmes to improve health workers' adherence to malaria guidelines in Kenya. PLoS One 2012; 7(12): e52045.

51.

Dusabe

Nnko

Changalucha

. Phone text message referral intervention. J Telemed Telecare 2013; 19(2): 2–4.

52.

Jones

COH

Wasunna

Sudoi

. “Even if you know everything you can forget”: Health worker perceptions of mobile phone text-messaging to improve malaria case-management in Kenya. PLoS One 2012; 7(6): e38636.

53.

Shao

Rambaud-Althaus

Swai

. Can smartphones and tablets improve the management of childhood illness in Tanzania? A qualitative study from a primary health care worker's perspective. BMC Health Serv Res 2015; 15: 135.

54.

Jennings

Ong'ech

Simiyu

. Exploring the use of mobile phone technology for the enhancement of the prevention of mother-to-child transmission of HIV program in Nyanza, Kenya: A qualitative study. BMC Public Health 2013; 13: 1131.

55.

Akinfaderin-Agarau

Chirtau

Ekponimo

. Opportunities and limitations for using new media and mobile phones to expand access to sexual and reproductive health information and services for adolescent girls and young women in six Nigerian states. Afr J Reprod Health 2012; 16(2): 219–230.

56.

Chib

Wilkin

Ling

. You have an important message! Evaluating the effectiveness of a text message HIV/AIDS campaign in Northwest Uganda. J Health Commun 2012; 17(Suppl 1): 146–157.

57.

Vahdat

L'Engle

Plourde

. There are some questions you may not ask in a clinic: Providing contraception information to young people in Kenya using SMS. Int J Gynecol Obstet 2013; 123(Suppl 1): e2–e6.

58.

L'Engle

Vahdat

Ndakidemi

. Evaluating feasibility, reach and potential impact of a text message family planning information service in Tanzania. Contraception 2013; 87(2): 251–256.

59.

Wakadha

Chandir

Were

. The feasibility of using mobile-phone based SMS reminders and conditional cash transfers to improve timely immunization in rural Kenya. Vaccine 2013; 31(6): 987–993.

60.

Macleod

Phillips

Stone

. The architecture of a software system for supporting community-based primary health care with mobile technology: The Mobile Technology for Community Health (MoTeCH) initiative in Ghana. Online J Public Health Inform 2012; 4(1): 1–17.

61.

Larsen-Cooper

Bancroft

Toole

. Where there is no phone?: The benefits and limitations of using intermediaries to extend the reach of mHealth to individuals without personal phones in Malawi. Afr Pop Stud 2015; 29(1): 1628–1641.

62.

de Tolly

Skinner

Nembaware

. Investigation into the use of short message services to expand uptake of human immunodeficiency virus testing, and whether content and dosage have impact. Telemed J E Health 2012; 18(1): 18–23.

63.

Odeny

Bukusi

Cohen

. Texting improves testing: a randomized trial of two-way SMS to increase postpartum prevention of mother-to-child transmission retention and infant HIV testing. AIDS 2014; 28(15): 2307–2312.

64.

Lund

Hemed

Nielsen

. Mobile phones as a health communication tool to improve skilled attendance at delivery in Zanzibar: A cluster-randomised controlled trial. BJOG 2012; 119(10): 1256–1264.

65.

Danis CM, Ellis JB, Kellogg WA, et al. Mobile phones for health education in the developing world: SMS as a user interface. In: Proceedings of the First ACM Symposium on Computing for Development. Proc ACM DEV’10. New York: ACM, 2010, 1–9. Available from: http://dl.acm.org/citation.cfm?id=1926197.

66.

de Lepper

Eijkemans

MJC

van Beijma

. Response patterns to interactive SMS health education quizzes at two sites in Uganda: A cohort study. Trop Med Int Health 2013; 18(4): 516–521.

67.

Crawford

Larsen-Cooper

Jezman

. SMS versus voice messaging to deliver MNCH communication in rural Malawi: Assessment of delivery. Glob Health Sci Pract 2014; 2(1): 1–12.

68.

Lau

Cassidy

Hacking

. Antenatal health promotion via short message service at a Midwife Obstetrics Unit in South Africa: A mixed methods study. BMC Pregnancy Childbirth 2014; 14: 284.

69.

Chib

Wilkin

Hoefman

. Vulnerabilities in mHealth implementation: A Ugandan HIV/AIDS SMS campaign. Glob Health Promot 2013; 20(1 Suppl): 26–32.

70.

Akter

Ray

. mHealth – An ultimate platform to serve the unserved. Yearb Med Inform 2010, pp. 94–100.

71.

Chib

. The promise and peril of mHealth in developing countries. Mob Media Commun 2013; 1(1): 69–75.

72.

Folaranmi

. mHealth in Africa: Challenges and opportunities. Perspect Public Health 2014; 134: 14–15.

73.

Kahn

Yang

Kahn

. “Mobile” health needs and opportunities in developing countries. Health Aff 2010; 2(2): 252–258.

74.

Mechael

. The case for mHealth in developing countries. Innov Technol Govern 2009; 4(1): 103–118.

75.

Mechael

Nemser

Cosmaciuc

. Capitalizing on the characteristics of mHealth to evaluate its impact. J Health Commun 2012; 17(Suppl 1): 62–66.

76.

Norris

Swartz

Tomlinson

. Mobile phone technology for improved mental health care in South Africa: Possibilities and challenges. S Afr J Psychol 2013; 43(3): 379–388.

77.

Waldman

Stevens

. Sexual and reproductive health and rights and mHealth in policy and practice in South Africa. Reprod Health Matters 2015, pp. 1–10.

78.

Thirumurthy

Lester

. M-health for health behaviour change in resource-limited settings: Applications to HIV care and beyond. Bull World Health Org 2012; 90(5): 390–392.

79.

Zurovac

Talisuna

Snow

. Mobile phone text messaging: Tool for malaria control in Africa. PLoS Med 2012; 9(2): e1001176.

80.

Speciale

Freytsis

. MHealth for midwives: A call to action. J Midwifery Womens Health 2013; 58(1): 76–82.

81.

DeRenzi

Borriello

Jackson

. Mobile phone tools for field-based health care workers in low-income countries. Mt Sinai J Med 2011; 78(3): 406–418.

82.

Leon

Schneider

Daviaud

. Applying a framework for assessing the health system challenges to scaling up mHealth in South Africa. BMC Med Inform Decis Mak 2012; 12: 123.

83.

Tomlinson

Rotheram-Borus

Swartz

. Scaling up mHealth: Where is the evidence? PLoS Med 2013; 10(2): e1001382–e1001382.

84.

Mburu S, Franz E and Springer T. A conceptual framework for designing mHealth solutions. In: Proceedings of the 3rd ACM MobiHoc Workshop on Pervasive Wireless Healthcare. New York: ACM, 2013, pp.31–36.

85.

Nacinovich

. Defining mHealth. J Commun Healthcare 2011; 4(1): 1–3.

86.

O'Connor

O'Donoghue

. Contextual barriers to mobile health technology in African Countries: A perspective piece. J Mob Technol Med 2015; 4(1): 31–34.

87.

Sanner

Roland

Braa

. From pilot to scale: Towards an mHealth typology for low-resource contexts. Health Policy Technol 2012; 1(3): 155–164.

88.

Beratarrechea

Lee

Willner

. The impact of mobile health interventions on chronic disease outcomes in developing countries: A systematic review. Telemed J E Health 2014; 20(1): 75–82.

89.

Bloomfield

Vedanthan

Vasudevan

. Mobile health for non-communicable diseases in Sub-Saharan Africa: A systematic review of the literature and strategic framework for research. Global Health 2014; 10: 49.

90.

Deglise

Suggs

Odermatt

. SMS for disease control in developing countries: A systematic review of mobile health applications. J Telemed Telecare 2012; 18(5): 273–281.

91.

Noordam

Kuepper

Stekelenburg

. Improvement of maternal health services through the use of mobile phones. Trop Med Int Health 2011; 16(5): 622–626.

92.

Sharma

Agarwal

Horvath

. Mobile phone text messaging for promoting adherence to antiretroviral therapy in patients with HIV infection. WHO Reprod Heal Libr Geneva World Heal Org 2012; 3(4): CD009756. Available from: http://apps.who.int/rhl/hiv_aids/cd009756_sharmap_com/en/.

93.

Tamrat

Kachnowski

. Special delivery: An analysis of mhealth in maternal and newborn health programs and their outcomes around the world. Matern Child Health J 2012; 16(5): 1092–1101.

94.

Gurman

Rubin

Roess

. Effectiveness of mHealth behavior change communication interventions in developing countries: A systematic review of the literature. J Health Commun 2012; 17(Suppl 1): 82–104.

95.

Betjeman

Soghoian

Foran

. mHealth in Sub-Saharan Africa. Int J Telemed Appl 2013; 2013: 482324.

96.

Chib

van Velthoven

Car

. mHealth adoption in low-resource environments: A review of the use of mobile healthcare in developing countries. J Health Commun 2014; 20: 37–41.

97.

Goel

Bhatnagar

Sharma

. Bridging the human resource gap in primary health care delivery systems of developing countries with mHealth: Narrative literature review. JMIR mHealth uHealth 2013; 1(2): e25.

98.

Hall

Fottrell

Wilkinson

. Assessing the impact of mHealth interventions in low- and middle-income countries – what has been shown to work? Glob Health Action 2014; 7: 25606.

99.

Braun

Catalani

Wimbush

. Community health workers and mobile technology: A systematic review of the literature. PLoS One 2013; 8(6): 4–9.

100.

Källander

Tibenderana

Akpogheneta

. Mobile health (mHealth) approaches and lessons for increased performance and retention of community health workers in low- and middle-income countries: A review. J Med Internet Res 2013; 15(1): e17.

101.

O'Donovan

Bersin

O'Donovan

. The effectiveness of mobile health (mHealth) technologies to train healthcare professionals in developing countries: A review of the literature. BMJ Innov 2014; 1(1): 33–36.

102.

Jennings

Gagliardi

. Influence of mHealth interventions on gender relations in developing countries: A systematic literature review. Int J Equity Health 2013; 12: 85.