Are we on the same page? Exploring stakeholders’ shared mental models of mobile health in rural Nigeria

Abstract

Despite the benefits promised by mobile health, the introduction of these solutions is often met with resistance from various stakeholders. This article adopts a shared mental model approach to unearth the current perceptions, concerns, and mentalities of key stakeholders engaged in the provision of healthcare in Nigeria. These include policy makers, academics, healthcare professionals, and health information systems developers. Interviews and focus groups were used to examine stakeholders’ views across three mental models: (1) the technology, (2) processes, and (3) the team. Our investigations reveal disparities in stakeholders’ existing mental models and their perceptions of the proposed mobile health solution. We argue that fostering a common understanding of mobile health, as well as elucidating an improved understanding of processes and team behaviours, will mitigate the risk of resistance among stakeholders involved in the design and delivery of community healthcare services and culminate in a positive attitude towards new mobile health solutions among these stakeholders. We highlight the need to enhance communication and training from national to rural levels to promote complementary mental models and positively influence team performance.

Keywords

eHealth healthcare providers mHealth mobile health shared mental model theory stakeholder perceptions

Introduction

In recent years, a large number of projects have been commissioned by non-government organisations (NGOs), governments, and other funding bodies to explore the potential of mobile health (mHealth) in supporting decision making in healthcare, thereby improving healthcare delivery and consequently health outcomes in developing countries.^1,2 Existing research on mHealth has largely focused on the improvements in healthcare services supported by mHealth tools,^3,4 the improvement of healthcare interactions between rural healthcare workers and community members,^5,6 and on more general improvements in mHealth scope, efficiency, and quality.^7–9 Despite this increased investment and growing interest among academic researchers, many of the promised benefits remain unrealised in this context.

There are three core academic issues which hinder the potential of mHealth in developing countries, namely, (1) the paucity of research to guide future projects, (2) problems reaching sustainability, and (3) a lack of theoretical explanations. First, there exists a limited body of empirical mHealth research in the context of developing countries.¹⁰ More specifically, there is a dearth of research which investigates the impact of these technologies on health outcomes, workflows in healthcare delivery, and the challenges facing implementation of these technologies.¹ This is problematic as the implementation of mHealth is not without its challenges and the introduction of new solutions is often met with resistance.¹¹ There is a need to understand the factors driving and inhibiting the intentions to use and sustained usage of mHealth in developing countries.¹² The second issue which hinders the potential of mHealth is the enduring issue of ‘pilotisis’.¹² This is defined as the brief introduction of mHealth in a pilot, feasibility or Randomised Controlled Trial study, which fail to be scaled up and sustained in the long-term. The issue of sustainability continues to plague mHealth initiatives in developing countries,¹³ with many mHealth solutions abandoned.¹⁴ Third, the limited body of knowledge borrows heavily from technology adoption literature.¹⁰ While the existing Information Systems (IS) literature provides adaptable theories, the unique and complex nature of healthcare as a context has led to calls for unique theorising.¹⁵ Furthermore, the developing world context is likely to present new challenges to stakeholders’ attitudes towards, and intentions to use, mHealth solutions and to theory development as a result. It is argued that the status quo of theorising mHealth in developing countries is primarily ‘explanatory’ in nature. Subsequently, existing research falls short of providing predictions, causal explanations, and explicit prescriptions on how to implement mHealth in developing countries. It is thus argued that there is a need to explore the efficacy of existing theories and constructs in this context, while also delving deeper into the unique challenges facing the introduction of mHealth solutions in developing countries.

This study focuses on the perceptions of stakeholders engaged in healthcare delivery in Enugu State, Nigeria. Existing literature demonstrates that the vast majority of research conducted in developing countries is often pursued at an individual level of analysis or for particular target groups. While this is important to examine, it is argued that more work needs to be performed at a group/team level to ensure collaboration in driving IT projects forward. Yet, little research has been conducted within the mHealth domain from a developing country perspective on the group/team level, as opposed to other levels of analysis. Researching at a group/team level is important in the context of this study as many mHealth projects in developing countries are undertaken through teams/various stakeholders to ensure their success. As a concept, stakeholders refer to ‘those groups without whose support the organisation would not exist’.¹⁶ IS scholars have leveraged a stakeholder perspective,^16–18 in part to understand the process requirements involved in system design¹⁹ and also as a means to manage conflicts or diverging interests that could contribute to project abandonment.¹⁸ Stakeholders are considered a team as they share a common goal and can accomplish what individuals cannot do alone, which in this study is the potential successful introduction of mHealth in Nigeria.

In an effort to provide a strong foundation for future research, the article utilises a shared mental model (SMM) approach to understand the current shared mentality of the many stakeholders involved in delivering healthcare at the community level. By understanding existing mentalities, this study seeks to offer empirically supported, theoretical explanations and predictions on the challenges facing mHealth. We argue that such insights are imperative to developing approaches to foster shared mentalities and improve attitudes towards mHealth thereby enabling the realisation of the promised benefits.

The structure of the article is as follows: First, the context to the study is outlined. The article proceeds with a review of the existing literature. This study argues for the efficacy of utilising the SMM as a theoretical foundation for exploring stakeholders’ views, namely those stakeholders involved in the design and delivery of healthcare services in the community. We present and qualitatively explore a number of propositions. The implications from the qualitative findings are discussed along with a brief overview of the next steps in this research.

Context of the study: healthcare in Nigeria

This research is focused on the community level in Nigeria, specifically Nsukka. Nsukka is 1 of 17 Local Government Areas (LGAs) in Enugu State, Nigeria. Nationally, each LGA is divided into between 15 and 20 political wards. An LGA must not encompass more than 20 towns. If an LGA grows beyond this number, towns are forced to merge. The National Primary Healthcare Development Agency is part of this arrangement and is responsible for implementing the Ward Health System. Health wards were established based on a World Health Organization (WHO) recommendation to better delineate the boundaries of community health districts.²⁰ These were implemented by mapping to the existing electoral wards under the LGA. In 2007, the Enugu State Government provided free medical care for children under five in an effort to tackle the high mortality rate; however, accessibility in some rural areas remains challenging.²¹

Integrated community case management (iCCM) of childhood illness is a set of paper-based guidelines developed by the WHO and UNICEF. iCCM is a standardised approach to the provision of healthcare services (i.e. a service that provides medical treatment and care to the public using paper-based processes) to children, under the age of 5, in rural areas of developing countries. The underlying premise behind iCCM is to reduce child morbidity and mortality rates in developing countries.²² The guidelines are adapted on a per-country basis, dependent on the most prevalent cause of child mortality in that region. Research has reported, however, that the use of the paper-based guidelines at the point of care are criticised for causing misdiagnosis and mistreatment.²³ While iCCM guidelines are adopted in many developing countries, the Nigerian healthcare system (at time of data collection) did not employ iCCM. Instead, primary healthcare (PHC) workers, that is, rural/community healthcare workers whose main responsibility is to deliver healthcare to dwellers in small villages in need of assessment and referral, used what is known locally in Enugu as ‘Standing Orders’ when delivering healthcare services.

As part of the funded research project, namely IMPACT (usIng Mobile Phones for Assessing, Classifying and Treating sick children), we designed and developed an electronic iCCM based solution to explore the potential of mHealth to improve community healthcare delivery, particularly in the assessment and diagnosis of children under 5 years for diseases such as malaria, cholera, and diarrhoea. The proposed mHealth solution ‘represents the use of mHealth applications for the diagnosis of specific conditions’.²⁴ The electronic mobile application digitises the existing paper-based approach, which employs the exact terminology established by WHO and UNICEF. The ‘IMPACT app’ can be used on Android-based smart devices (e.g. phones and tablets) at the point of care. Following the existing iCCM paper-based guidelines, a clinical decision support model underpins the app which, based on the input by the user, presents a recommended diagnosis and treatment to the PHC worker for consideration. This app is completely novel to PHC workers in Nigeria. As a result, PHCs are faced with many challenges: access to smart devices, understanding iCCM, using the application correctly so it does not hinder patient care, understanding the outcomes presented and appropriate reaction to the response (i.e. recommended diagnosis and treatment) provided to the user. The ‘IMPACT app’ will be used to support PHC in five wards in Nsukka LGA, illustrated in Figure 1. This application offers a low-cost mHealth solution, using robust, secure, cloud-based data storage in a rural community, where the reliability of technical infrastructure is a challenge.

Figure 1.

Primary healthcare system, Enugu State, Enugu (snapshot of Nsukka LGA).

Literature review

The existing literature was reviewed with two aims: (1) to establish current knowledge levels on the factors influencing attitudes towards mHealth in developing countries and (2) to ascertain the status quo of theory in this context. This approach was conducted to ensure the shortcomings of existing research within this domain are addressed in this article. Two constructs commonly tested in technology adoption studies are of interest, namely, attitudes and behavioural intentions. Attitude refers to an individual’s positive or negative views towards the proposed mHealth solution and is formed based on the individual’s cognitive assessment of the benefits and consequences related to use of the solution.²⁵ Behavioural intentions are represented by intention to use the new mHealth solution upon its introduction, described as an individual’s internal subjective judgement of the probability they will perform the behaviour in question.²⁶ The article is particularly concerned with the attitudes and behavioural intentions of stakeholders surrounding the mHealth solution. It is important to understand attitudes and behavioural intentions as they culminate in eventual adoption behaviour,^25,26 with positive attitudes and intentions indicating acceptance and negative attitudes and intentions fostering resistance towards the solution.

Empirical studies

Several studies in the area of health technology implementation adopt models such as the technology adoption model (TAM) or the unified theory of acceptance and use of technology (UTAUT).^27,28 The findings of these studies support the role of many predictors with some exceptions. For instance, perceived ease of use and the perceived views of referent others (social influence) positively influenced adoption intentions in Taiwan, Thailand, and Morocco.^6,29,30 Performance expectancy or an individual’s perception that technology will be useful in their work activities significantly influenced health professionals’ intentions to use health technology in Thailand and Taiwan,^6,29 but did not influence intentions in Canada.³¹

The literature discusses a number of additional barriers facing mHealth in developing countries such as prohibiting effects of perceived fit,³² infrastructural barriers and cultural barriers,⁹ resistance, and confusion over the benefits and aims of the proposed technical intervention³³ on stakeholders’ attitudes and behavioural intentions. Several areas in the developing world context have been identified for empirical examination including the unique privacy and security challenges in these countries, changes to processes brought by mHealth, and the distribution of power and control among different stakeholders.¹ Due to the many possible predictors of mHealth acceptance, our aim is to identify the factors impacting attitudes towards and intentions to use mHealth solutions in Nigeria.

With the exception of Ezenwa and Brooks,³³ the majority of existing studies focus on the views and perceptions of one group of stakeholders such as doctors or patients. The importance of understanding the views of multiple stakeholders has repeatedly been highlighted.^11,34,35 In the developing country context, and indeed in Nigeria, there are numerous stakeholders who must be considered when implementing a mHealth solution, that is healthcare workers (e.g. doctors, nurses, community health workers), patients, systems developers (e.g. systems designers, software developers), and facilitators (e.g. department/ministry of health, district health offices).²⁴ Furthermore, stakeholders provide rich insights into the variety of perceived benefits and challenges associated with mHealth solutions, especially in a developing world context.³⁵

Theoretical trends

There are a number of notable trends in the mHealth literature in developing countries. First, a number of studies fail to adopt a guiding theory or to engage in unique theorising.³⁵ This gap is particularly evident in the Nigerian context with only one existing study utilising existing theory. The study in question, conducted by Ezenwa and Brooks,³³ adopted Actor Network Theory to provide insights into the factors driving stakeholders’ perceptions of an IS to aid decision making in maternal care. Second, many studies focusing on mHealth in the context of developing countries utilise technology adoption theories such as TAM and UTAUT. While these studies provide interesting empirical insights, they engage in little new theorising. This is problematic as theory should allow researchers to generalise results beyond the confines of a particular context, but this generalisability is noticeably absent in mHealth research within developing countries.³⁶ Many existing studies tend to engage in explanatory theorising leading to calls for studies which go beyond to provide deeper insights and actionable predictions.³⁷

To combat these issues, researchers need to move beyond employing traditional theories to understand the many barriers that could influence the successful usage of mHealth¹² and work towards understanding why ‘pilotisis’ remains prominent in developing countries.³⁷ To achieve such insights, there is also a heightened need to explore the views of the various stakeholders involved in the delivery of healthcare services with regard to the compatibility of new technologies with existing processes, values, and work style.³² This article thus utilises a SMM approach to explore the mentality of stakeholders responsible for the healthcare delivery at the community level.

Conceptual model and propositions

Mental models can be described as individuals’ internal representations of their external environment which are influenced by the individual’s experiences, views, and perceptions.³⁸ While mental models are held by individuals, at the team level, SSMs are described as complementary cognitive representations of situations and concepts that are shared by team members.³⁹ In the team environment, SMM theory proposes that a shared mentality among team members can improve team performance.⁴⁰ However, if team members possess conflicting or inaccurate mental models, this could have drastic impacts on decision making, team coordination, and performance.^41,42 It is thus imperative to develop complementary mental models among a team to ensure the team performs to maximum effectiveness.⁴³ In the health context, team performance translates into efficient provision of healthcare services, adherence to guidelines, utilisation of mHealth interventions, and strengthened relationships between healthcare workers and supervisors.⁴⁴

The potential of SMMs to create an adaptive and dependable work environment has been highlighted in a number of contexts including healthcare.⁴⁵ Much of the existing SMM research focuses on the role of the team’s internal processes. For example, the team completes the work they are assigned to do and they know alternate strategies to complete a task, in producing positive outcomes⁴⁶ with many studies tested on teams in military, information technology, and engineering contexts.⁴¹ A number of recent studies have utilised SMM theory in the health context focusing on discipline or location constrained teams such as emergency departments or intensive care units in developed world contexts, for example, Hysong et al.³⁹ and Undre et al.⁴⁷ No study to date has utilised SMM to focus on health teams in developing countries. Furthermore, there have been calls for studies which investigate the influence of SMMs on improving the provision of healthcare through investigating the mental models of key stakeholders.⁴³ This article begins to answer this call and by exploring the team in terms of the stakeholders engaged in delivering healthcare services in rural communities in Nigeria. For this study, the team includes PHC workers and other stakeholders such as representatives from the state Ministry of Health (MoH), key academics (clinical), healthcare professionals, and health information systems (HIS) developers. For the purpose of this study, our focus was on the various stakeholder groups that represent the inter-organisational and inter-professional boundaries that are responsible for the provision of healthcare services⁴³ at the community level. For the purpose of this study, these stakeholders are referred to as ‘the team’.

Existing literature notes that team members are likely to hold several mental models⁴⁸ with four shared models commonly discussed. The first two models are task-orientated focusing on the technology and the process, whereas the latter two models are team-orientated focusing on the people involved.⁴¹ These models align with the many questions that must be answered when introducing a new mHealth solution, such as will it work effectively; is the mHealth app available in other languages; can it be easily integrated; is it socially and culturally acceptable?⁴⁹ The need to focus on multiple models is apparent in the context of this study due to the complexity of healthcare delivery. Moreover, the introduction of mHealth rarely stands independently; it is ‘embedded in a system of complementary technologies and requires three key elements for success: (1) people, (2) process, and (3) technology’.⁴⁹ Therefore, it is important to explore stakeholders’ mentality across these aspects to determine the likelihood of success for a new mHealth intervention. The next section considers the core mental models. As team mental models change over time, for each model,³⁸ we are concerned with team members’ current perceptions pertinent to a new mHealth solution. We argue that understanding current mental models can provide insights needed to develop interventions which develop team member’s mental models upon the implementation of the mHealth solution.

Mental model 1: the technology model

The technology model originally termed the equipment model requires that team members have a common understanding of the technology used to complete work processes and its functions.^40,50 This model focuses on team members’ knowledge of the technology’s functions, abilities, and limitations.⁴⁰ Thus, it is important to first establish team members’ understanding of the current technology used to deliver healthcare services and their initial attitudes towards the new technological solution, namely the IMPACT app. This provides insights into whether different stakeholders have a shared, accurate understanding of existing technologies, the functionality, and the limitations of current methods. In this study, we are focused on stakeholders’ awareness of the existence of these technologies, their perceptions of the performance of these technologies, and their views on the wider societal/public benefits of these technologies.

A review of mHealth adoption in low- and middle- income countries⁵¹ highlights that slow adoption and use of mHealth by members in the community is due to a lack of awareness surrounding the availability of different technological solutions. While a myriad of mHealth solutions are available to community members in developing countries, albeit a lack of awareness may exist, it is imperative that mHealth performs well (i.e. do not cause any medical misdiagnosis and mistreatment).⁵² The technology must operate appropriately (i.e. not malfunction) and provide the necessary functionality to various stakeholders.⁵³ Research argues that stakeholders’ perceptions of mHealth must satisfy a given requirement in society (i.e. must also deliver wider societal/public benefits as opposed to only providing end user benefits).⁵⁴ Therefore, heightened awareness among different stakeholders regarding the benefits from a societal perspective may foster positive attitudes and greater intentions towards using mHealth in developing countries. If stakeholders are aware of current solutions, perceive that these solutions are performing well, and believe these solutions provide benefits, they will express positive views towards these existing technologies and positive views towards the introduction of new mHealth solutions. This positive disposition towards technology is likely to lead to positive attitudes towards a new mHealth application.

P1. Stakeholders’ current technology mental model (awareness, technological performance, societal benefits) will influence their attitudes and adoption intentions towards the IMPACT application.

Mental model 2: the task model

The task mental model requires that team members have a common understanding of the processes/tasks of the team.⁵⁵ This model encompasses team members’ views on the tasks of the team, challenges facing the team, and contextual/environmental factors.⁵⁰ SMMs are dynamic and alter upon changes to tasks or roles.⁴⁵ The first task model captures team members’ current perceptions of the tasks and processes involved in healthcare delivery. Indeed, in a study investigating healthcare workers’ mental models of clinical practice guideline implementation, workers in high-performing facilities exhibited a shared understanding of guidelines while those in lower performing facilities did not.³⁹ In addition, the values and attitudes towards what ‘to do’ and ‘not to do’ pertaining to information processing and communication,⁵⁶ via mHealth, must be expressed to all team members to ensure that care is coordinated.

The contextual/environmental factors distinguishing developing countries from their western developed counterparts are argued to have an impact on end users’ perceptions.⁵⁷ A review of the literature reveals that attitudes and adoption intentions towards mHealth in developing countries are often influenced by external factors which are predominantly contextual in nature.⁵⁸ The working environments of PHCs are extremely challenging due to the limited resources available. The current task mental model explores the team’s understanding of current data collection processes to determine (1) their task processes, (2) their views on the efficacy of these processes, and (3) their understanding of contextual/environmental factors. It is argued that current mental models will influence stakeholders’ views on the whether the IMPACT app can improve these processes. For instance, if the team believe current paper-based data collection processes are effective, they may not see the value of digitising data collection.

P2. Stakeholders’ current task mental model (task processes, task efficacy, contextual/environmental challenges) will influence their attitudes towards and intentions to use the IMPACT application.

Mental model 3: the team model

The two team-orientated models require that team members have a shared understanding of the roles of team members, responsibilities, communication flows, and the strengths and weaknesses of team members.⁴⁰ These two models are amalgamated in this article and describe a common understanding of team roles/responsibilities, team skills/abilities and motivation, which are reported to potentially influence the behaviour of teams.⁵⁰

While existing research often focuses on teams based in one location with similar roles, for example, nurses,⁵⁹ the team in this study includes PHC workers and stakeholders such as government officials and academics (those involved in the design and implementation of healthcare delivery in the community). Thus, these team members may vary in their perceptions of the roles of PHC workers, and the challenges and barriers they face. Despite this added complexity, the importance of a shared understanding of the team’s roles cannot be understated in this context, as a shared understanding of the responsibilities of PHC workers is paramount in evaluating the effectiveness of the team in fulfilling these roles and responsibilities. Indeed, a study exploring the perceptions of nurses and physicians found many discrepancies between nurses’ and physicians’ understanding of how their responsibilities are distributed.⁶⁰ This study highlights the importance of educating team members on the roles and responsibilities of all parties engaged in healthcare delivery in this context.

Healthcare providers with high self-efficacy (i.e. computer skills) are found to have greater intentions to utilise and adopt mHealth.⁶¹ In contrast, low self-efficacy individuals tend to avoid tasks and situations that they believe exceed their capabilities.⁶² Furthermore, research⁶³ argues that for teams to be effective, they must be motivated to work collaboratively. This is imperative in a healthcare context as team members must depend on each other to deliver healthcare services.⁶⁴ This point is further reinforced by research which demonstrates that the dynamic of the group and their motivations have an impact on usage intentions of IS.⁶⁵

As the various stakeholders have diverse viewpoints,⁴³ it is important to explore their views on the skills of team members, current motivation levels, and current perceptions of performance. It is argued that these perceptions will influence stakeholders’ perceptions towards the IMPACT app; if health leaders believe that PHC workers currently lack the skills needed to complete their tasks, are unmotivated, and underperforming, they may express negative attitudes towards its introduction and likelihood of success.

P3. Stakeholders’ current team mental model (roles/responsibilities, team skills, and motivation) will influence their attitudes towards and intentions to use the IMPACT application.

These propositions are illustrated in the conceptual model in Figure 2.

Figure 2.

Current mental model.

Methodology

This study is part of a larger research project, the aim of which is to explore the feasibility of introducing a mHealth solution, namely the IMPACT app, in Enugu State, Nigeria. The project consists of two phases of data collection. This first phase follows a multi-method approach combining two qualitative data collection methods with secondary data gathered from various sources in the Enugu State community health system. This phase explores the current mental models of key stakeholders involved in the provision of healthcare services and the influence of these models on stakeholder attitudes of and intentions to use the IMPACT app. Qualitative methods were chosen to describe and explain the complexities of real-life situations which may not be captured through experiment or survey research.⁶⁶ This article focuses on the first phase of this study, where we targeted the direct users (PHC workers) of the solution and the key decision makers who hold the power to influence the project. During this phase the indirect users of the mHealth solution, that is, children and parents/guardians, were not interviewed. Data collection procedures for phase one are illustrated in Figure 3. Ethical approval to conduct this study was granted by research ethics committees at University College Cork and Parklane Teaching Hospital, Enugu State University of Science and Technology.

Figure 3.

Data collection process.

Data collection took place during a week-long visit to Enugu State in September 2016. The research project team engaged with ‘the team’ of a selection of key stakeholders via interviews, presentations, and focus groups. Purposive sampling was followed to identify key informants based on criteria devised by the research team.⁶⁷ These were largely determined based on positional and/or reputational methods at state level and in the target communities.⁶⁸ For interview participants, the criteria were as follows: (1) currently employed in the health sector and/or engaged in community health research, (2) aware of existing data collection policies, and (3) knowledgeable of the current status of healthcare delivery in Enugu State. Interviews were conducted with those responsible for the provision of healthcare services in the community. Study participants included representatives from the Ministry of Health (MoH), Enugu State University of Technology (ESUT) academics collaborating with the IMPACT research project team, PHC workers from five rural communities including Edem, Ibagwa, Alor-uno, Okpuje, and Okutu (near Nsukka 58 km from Enugu City), and HIS developers, that is, individuals responsible for the development and implementation of the mHealth solution. PHC workers (this is local terminology for community healthcare workers in Nigeria) are responsible for delivering healthcare services in rural communities; PHCs are the direct users of the IMPACT solution.

Interviews were conducted at various locations [including Nsukka Health Centre (district), Edem-Ani Health Centre (community), Parklane Teaching Hospital, and ESUT.] Focus groups were conducted with PHC workers ranging in experience levels from the five different communities and medical students studying at a university in Enugu State, Nigeria. Focus groups were held in a town Health Centre and a local university. As part of the focus group sessions, the research project team presented this collaborative research project comprising a team of IS researchers from a university in Western Europe and academics from a university in Enugu State. The purpose of the IMPACT app was outlined followed by a presentation of the app and group discussion. Table 1 details the data collection effort during this visit.

Table 1.

Data collection overview.

Data collection	Roles	Type	Description	Duration
Interviews (6 interviews)	Commissioner for Health, Enugu State	Health Leader	Main decision maker for Enugu State on delivery of healthcare services. Background – trained doctor, spent time practicing medicine in the US.	45 min
	Head of Department of Community Medicine	Academic1	Collaborator on IMPACT project, academic lead on project for ESUT, trained community health doctor.	60 min
	Provost – College of Medicine	Academic2	Collaborator on IMPACT project, provided resources and support for the project on the ground including transportation and made staff available to support all meetings and focus groups.	40 min
	Permanent Secretary to the Governor	Administrative	Advisor to the State Governor. Provided a broad overview of community healthcare delivery in Enugu State and created opportunities to identify/contact interviewees.	40 min
	District Health Information Systems Expert	Health IT Expert	Over 20 years’ IS expertise, lead HIS on recent Enugu state implementation of mHealth for health record data capture using a WML-based solution.	40 min
	Clinical Academic, Dept. Community Medicine with experience managing development projects	Academic3	Over 30 years’ experience as a doctor (epidemiologist). Director of Public Health Services from 1991 to 2004. Had been involved in several large scale NGO projects on child health, health/stroke initiative, HIV, prior to IMPACT.	40 min
Presentations followed by Focus Groups	Paediatricians, Medical Students, nurses (51 attendees)	Health professional	ESUT Trainee medical students with a variety of experiences and expertise.	90 min
Presentations followed by Focus Groups	PHC Workers, Officers in Charge (of rural health facilities) and Community Chiefs (41 attendees)	Health professionalCommunity Representatives–well respected and influential in their respective communities.	All had at least 2 years training as PHC workers in the community. Many of this group have bachelor’s degrees, for example, nursing, pharmacy. Chiefs represented the interests of their communities, they had experience of a recent funded NGO mHealth project.	90 min
Field notes	NA	NA		30 pages

IMPACT: using mobile phones for assessing, classifying and treating sick children; ESUT: Enugu State University of Technology; IS: information systems; HIS: health information systems; NGO: non-government organisations; PHC: primary healthcare; NA: not applicable.

It is important to note that the academics interviewed are located at a teaching hospital in Enugu State. This research was completed in collaboration with the Department of Community Medicine in Enugu, Nigeria; a group which is actively involved in the development and delivery of projects supporting the community health initiatives in this area.

Data analysis

As mental models are internal representations of events, they cannot be directly measured.³⁸ Furthermore, the existing SMM literature offers no consensus on how to best measure mental models.⁵⁹ Thus, to capture team mental models, this study utilised open questioning techniques to explore participants’ perceptions related to the mental models, with supplementary closed questions used to ascertain their attitudes towards the IMPACT app and adoption intentions. Interviews and focus groups were recorded with participants’ permission and transcribed verbatim. Hand-written field notes were also typed to add further context and richness to the transcripts. Using a qualitative analysis approach,⁶⁹ the data were analysed using broad ‘seed categories’ representing the SMMs of technology, process, and team. These categories⁷⁰ were broken down into current and new mental models and included several aspects of each mental model. For example, the technology mental model included awareness of existing technologies, perceived performance, and perceived benefits. Seed categories also included attitudes towards the application and adoption intentions. These categories were reviewed in iterations by team members to reach conclusions on the mental models of stakeholders and derive recommendations for strengthening these mental models.

Results

This section provides a synthesis of the findings, which are grouped in accordance with the three broad mental models outlined in the literature.

Technology mental model

The teams’ current technology mental model was investigated focusing on (1) awareness of existing technologies, (2) perceived performance of these technologies, and (3) perceived benefits. The majority of the team were aware of existing systems such as the mobile phones used by PHC workers to upload monthly patient health record summary reports from the clinic to a centralised district health information systems database. However, there were some differences in perceptions of the current performance of these phones. For example, one Health Leader expressed the opinion that these phones effectively captured data at the community level. In contrast, many other members of the team noted challenges associated with the performance of these phones. Academic1 stated that these systems often do not work, due to the system not being available for use, lack of electricity to charge the mobile phones, and poor access to an Internet connection, thus leading PHC workers to revert to paper-based data collection methods. Additional challenges noted by the Health IT Expert included loss of phones, damage to phones, loss of trained personnel, and technical issues such as Internet access and cost of accessing these technical resources. Sustainability of technology projects was highlighted by all of the academics and the Health IT Expert as a serious issue facing the continuance of mHealth use. Team members were very positive regarding the benefits of these technologies.

The link between the technology mental model and initial perceptions towards the IMPACT app was apparent in both the interviews and focus groups. Health professionals and academics expressed largely positive attitudes towards the new solution. They felt the solution could enable PHC workers to treat more patients and improve the quality of care offered to patients in the community. In addition, it was reported by one stakeholder that ‘Enugu is health friendly we have this free matter now, which wants to deliver everything that’s free, a lot of free facilities to ensure that the citizens are healthy’ (Health Leader). The Health IT Expert believed the data collection enabled by these phones could result in many benefits such as disease surveillance for the wider community. Using mHealth was further argued to ‘give us a lot more exposure and it would empower us to contribute better to our community’ (Academic1). These stakeholders perceived that the introduction of mHealth within their communities would have many benefits to society. Conversely, those stakeholders working on the ground (i.e. PHC workers) had different views arguing that community members may believe and trust the mHealth more than the PHC’s advice, even if the mHealth is flawed to some extent – ‘technology is clearly not equal to medicine and face–to-face healthcare delivery’ (PHC worker).

Task mental models

Team members’ current task mental model encompasses perceptions of existing tasks and processes, the efficacy of these processes, and existing challenges from a contextual/environmental perspective. The academics and the Commissioner for Health outlined the many tasks associated with healthcare planning, budgeting, policymaking, financing, data collection, and analysis. As part of the focus group discussion, PHC workers described the tasks they face every day, including assessing children, administering drugs, and filling out forms. These are perceived as time-consuming activities. The academics and the PHC workers detailed the myriad of challenges facing healthcare delivery in the community in Enugu including resourcing issues, shortage of trained workers, issues related to the poor treatment of PHC workers including issues with late payment, issues with current data collection methods, and incompleteness of data at the state level. In contrast, one Health Leader believed that current processes were effective: ‘We have national data where those data are stored for reference, comparative analysis, and record purposes’. The different views highlight a lack of consensus between the task mental model of the health leader and of other stakeholders.

The core task discussed related to assessing and treating children. PHC workers outlined the challenges associated with the current guidelines or ‘Standing Order’ (existing approach for delivering healthcare services) for patient assessment. This detailed book of guidelines for classification and treatment of adults and children used by PHC workers at rural facilities acts as a paper-based decision support tool for patient assessment. The team members indicated that using the ‘Standing Order’ was very time-consuming and prevented them from doing more important tasks such as immunisations and providing antenatal support. During one visit to a rural health facility at Edem-Ani, the standing order book could not be located. The PHC workers at the facility were surprised that the book could not be found and the academics on the team were shocked that this was the case. This is a significant indicator of the issues around adherence to existing paper-based guidelines.

During the group discussion, PHC workers stated that they believed the IMPACT app would improve processes related to the assessment, classification, and treatment of children by increasing the speed at which they could treat children. In theory, they perceived the app would afford them more time to do other tasks while on duty, and support them to undertake uniform assessment of the sick children visiting their healthcare facility. Other stakeholder groups including the academics interviewed and the student paediatricians/nurse focus group expressed complementary views noting that IMPACT app could overcome existing challenges and improve data collection and sharing processes by digitising manual processes, as well as improving the quality of data collected.

One academic (with a clinical background), having seen firsthand the challenges facing PHC workers on the ground in the rural community, noted that the IMPACT app could improve the flow of health data and improve the effectiveness and efficacy of healthcare delivery at the community level. This is exemplified in the following quote: ‘I think it is the future in the practice of medicine so it is a welcome development to better increase efficiency of our healthcare delivery system’ (Academic3). Among health leaders, the IMPACT app was viewed as a welcome improvement in the existing system as opposed to a necessity.

In terms of the contextual/environmental challenges facing IMPACT, interviewees discussed similar challenges such as training; availability of electricity, network, and Internet access; sustainability; achieving buy-in from PHC workers; and illustrating credibility of the technology to support PHC workers in their roles. Despite the consensus on the existence of these challenges, there were differences expressed in terms of what challenges would be the most difficult to overcome and could thereby hinder the success of IMPACT. The Health Leader and one senior administrator expressed the belief that technical issues related to the availability of electricity and Internet access were not a significant concern and could be easily overcome. However, PHC workers, Academic1, and Academic2 believed these issues were critical and must be addressed in order to successfully implement mHealth in the community.

Team mental models

The team mental model focused on roles/responsibilities, skills, and the motivation of the team. There was general consensus among stakeholders that PHC workers were qualified to complete their roles. Fears of technology replacing PHC workers were reduced when the Health Leader expressed that ‘if the app or the use of the app decreases the number of employees required to run a health centre, then we can employ them elsewhere’ (Health Leader). It was noted, however, that PHC workers often do not show up to work, can be unproductive while at work, do not always adhere to guidelines and occasionally cease using the existing technologies available. While some viewed this behaviour as insubordination, others noted that this was justified as many PHC workers were awaiting payment for many months. One PHC worker acknowledged issues with adherence to treatment guidelines during the focus group discussion. They explained that this is due to resourcing and not lack of motivation on their part

Due to the shortage of staff, we do not have time to be taking it step-by-step, so, sometimes, you check the temperature, the pulse, you observe the child, but to go in details of asking all those questions, you will not have time because you may be alone on duty. (PHC Worker)

To overcome this lack of adherence, it was proposed to ‘promote PHC adherence to standard diagnosis and treatment practices by monitoring performance, this may be enhanced through the app, we have to give them [PHC] the impression that they are being monitored you know’ (Academic3). Unfortunately, the issue of demotivated PHC workers in the community was mentioned by most team members. The lack of motivation is primarily due to the lack of resources available to PHC workers.

Stakeholders expressed positive views on the potential of PHC workers to utilise technology and applications. In terms of the knowledge and skills of team members, there was general consensus that PHC workers had the appropriate skill set to employ mHealth as part of delivering healthcare services in the community. In addition, it was noted (Academic2) that ‘Luckily most of Nigerians, even in the rural community use very complicated phones’. During the PHC focus groups, many participants showed the research team downloaded apps that they were trying to integrate into their existing practices to capture patient respiratory rates. PHC workers were highly critical of these apps, they perceived some to work well and they identified those that were inconsistent and inaccurate based on their tacit knowledge.

Through using smartphone devices, stakeholders perceived that they had the appropriate skill set to potentially use mHealth at the point of care. Nonetheless, the need for training was emphasised by every interviewee and was discussed during all of the focus group discussions. Team members believed that training would be imperative to ensure workers were adequately equipped to adopt the new technology. In addition to receiving training on the app, a number of PHC workers called for more general training in the use of smartphone devices and other mHealth apps. Team members highlighted the importance of political support and community support to ensure the success of the application. The need to motivate PHC workers to use the application and promote continued use over time was highlighted by both academics and health leaders. Furthermore, academics and health leaders noted the mHealth solution would enable the monitoring and continuous supervision of PHC workers, which they believed would motivate workers via the threat of punishment. Academic1 commented ‘We have to give them the impression that they are being monitored. Because if we fail to follow up, some of them might relax and stop using the app’. This view is in stark contrast to the views outlined by PHC workers who viewed the IMPACT application as a tool that could be used to help them better carry out their duties in the community.

Discussion

Researchers in the mHealth domain have highlighted the importance of collaboration and shared views between government officials, healthcare systems providers, community healthcare workers, private organisations, and universities to ensure the success of mHealth initiatives.⁷¹ To achieve this collaboration in the context of community care in Enugu, this study utilises a SMM approach to explore the existing technology, task, and team mental models of key stakeholders regarding the IMPACT app. Leveraging our findings on these existing mental models, we seek to identify paths to reaching SMMs. We argue that a comprehensive training programme involving all stakeholders is paramount to contribute to communicating and developing shared and accurate mental models among healthcare providers.

Existing technology MM (conflicting)

There are current projects underway in Enugu which involve digital data collection. While all stakeholders had a shared awareness of the existence of these projects, some disparities existed in the perceived performance of these technologies with the Commission for Health adopting a positive view and those engaged on the ground, that is, PHC workers, detailing challenges which hinder use of mHealth. Indeed, the Commissioner for Health perceived the proposed IMPACT app as merely an additional improvement on the infrastructure that is currently in place and available to PHC workers in Enugu. For community healthcare workers this technology represented a radical improvement on existing solutions. Our study supports recent research, which highlights the lack of an agreed mHealth policy⁵¹ and the widely diverse views about the current state of the community healthcare system compared with frontline PHC workers on the ground in Enugu. These disparate views are a cause for concern, if stakeholders believe that current technologies are ineffective; they may hold concerns around the reliability of new solutions such as the proposed IMPACT app. Furthermore, if remote stakeholders such as health officials hold an inaccurate vision of the existing technologies, they may fail to comprehend the importance of new solutions or indeed the training needed for end users to master new solutions, which may differ largely from existing solutions in terms of interface and functionality. Conflicts were also present in perceptions of the benefits of these technologies. It seems that PHC workers often lacked an understanding of the broader implications around using mHealth, particularly in terms of data collection for resource planning and disease surveillance. Research on team mental models highlights the importance of team members having a shared and accurate view as a precursor to effective performance.⁴¹ We argue that these conflicts must be addressed to ensure all stakeholders understand the broader benefits of the technology and hold accurate views of existing and new technological solutions. This can foster positive attitudes towards new technologies and help incentivise adoption.

Existing task MM (conflicting)

The existing task mental model is predominately conflicting. Some sharedness was evidenced in stakeholders’ descriptions of important tasks and processes. However, there was strong conflict between views on the efficacy of existing processes with the Health Leader seemingly unaware of the existing challenges. Recent studies continue to illustrate poor adherence of PHC workers with current standing order practices.⁷² The PHC workers provided amble reasoning for their lack of adherence including understaffing, lack of resources, non-payment for work, and low morale as a result. While these reasons are understandable, it seems health officials are unaware of the stresses placed on PHC workers and how these challenges impact their ability to fulfil tasks.

The different stakeholder groups discussed different challenges. For example, PHC workers highlighted the additional time needed to adhere to treatment guidelines while the academics involved in this study highlighted incomplete data resulting in poor data quality as an enduring issue in the community healthcare service at present. The SMM literature stresses the importance of SMMs in understanding the roles and expectations of different team members.⁴⁰ As team members currently lack an understanding of the tasks and challenges other members, the ability of the team to set reasonable expectations for other team members is limited, and the effectiveness of the time is limited as a result. Our findings illustrate the need to ensure that stakeholders understand how their work processes impact other groups involved in the health system and the challenges facing each group. If decision makers are aware that staff shortages reduce application adherence, they may seek to address these issues through the reprioritization and reallocation of essential resources. Moreover, recent research found that electronic systems improved the completeness of patient assessment in Tanzania with healthcare workers preferring the electronic format finding it easier to use and promote greater adherence to Integrated Management of Childhood illness (IMCI) guidelines (note that CCM guidelines are a briefer version of IMCI) when assessing children using the technology compared with paper assessments.⁷³ Thus, the mHealth application has the potential to overcome the challenges facing different stakeholders around data quality and reducing the pressure on workers. Again, we stress the importance of ensuring all team members are informed of the existing tasks of all members and how any new systems alter these tasks.

Existing team MM (conflicting)

Sharedness was expressed in terms of team members’ qualifications to complete their roles and their ability to acquire new skills. However, there were stark differences in team members’ views regarding current motivation levels. PHC workers noted that low motivation levels stemmed from lack of resources, time pressures, and lack of payment. The academics interviewed lacked an understanding of the pressures facing PHC workers but were aware of issues with motivation and noted the need to try motivate PHC workers. Existing research demonstrates the potential of technology to alleviate some of these issues around healthcare delivery in rural areas in developing countries.⁷³ Our findings highlight the clear need to educate all stakeholders of the pressures facing other stakeholders, particularly to better understand the challenges facing healthcare professionals operating in the community. By ensuring all stakeholders understand the pressures facing other team members, they are better positioned to assess the current performance of team members and make decisions to improve this performance. Indeed, we argue if all stakeholders understood the roles of other members and the potential of the mHealth solution to assist with team members’ tasks and overcome data quality issues, the likelihood of successfully introducing the solution would be greater.

SMMs among team members have the potential to improve team performance in a variety of contexts.³⁹ However, mental models among team members should not only be similar, they must also be accurate.⁵⁹ This study builds upon the small number of SMM studies in the health context⁷¹ to explore the mental models of stakeholders in community healthcare delivery in Enugu, Nigeria. The interviews and focus groups provide strong indications that existing mental models are lacking in both sharedness and accuracy. It is imperative to address these conflicts in existing mental models to foster a shared understanding prior to introducing a new technology to increase the likelihood of success.

While some sharedness existed among stakeholders across each mental model, there are many conflicting viewpoints which may prove challenging when piloting and eventually implementing the IMPACT solution, as the new technology will lead to changes around technology used in healthcare delivery, processes undertaken by the team, and team performance. It is likely the introduction of the new technology will also result in changes around these three mental models related to the technology, task, and team. There is an apparent need to develop interventions which aim to improve the shared mentality across the models of technology, task, and the team. Interventions such as training have been explored in other contexts.⁴¹ Moreover, in the context of team mental models, training is the most commonly applied team-level intervention.⁴¹

Training is viewed as vital to the successful implementation of a new mHealth intervention as end users must understand how to put the technology into effective use.⁴⁹ Numerous researchers in the context of mHealth in developing countries highlight the importance of training healthcare workers to successful implementation with training programmes ranging from a few hours to a week in length and covering the functions of the mobile device in general, overview of the software, configuration and troubleshooting.³ While training of end users is imperative, it is not sufficient to address conflicts in existing mental models. We agree with previous assertions on the importance of including end users in the introduction and development of a new mHealth solution,³ but also propose that other stakeholders should be involved in the implementation process beyond the initial decision making to training in the solution.

As part of the electronic health capacity building effort on the ground in Enugu State, a training programme will be designed to develop a SMM among all stakeholders regarding the IMPACT app, processes, and team. This comprehensive training programme will have two aims. First, conflicts in existing mental models will be addressed by providing an overview of the overall healthcare delivery situation in Enugu, current technologies and challenges, processes involved in healthcare delivery at all levels, and current team member roles and barriers. The second aim will focus on fostering shared and accurate mental models across stakeholders regarding the IMPACT app by providing in-depth training on using the application, information on troubleshooting, detailing how the application alters work processes and addresses challenges at the various levels of healthcare such as data quality issues and time and resourcing issues facing healthcare workers. This training programme will involve ‘cross-role’ training to all stakeholders in a bid to create a shared, accurate understanding of the roles of all team members. This approach has been found to be effective in encouraging team members to update and converge their mental models.⁴¹

By fostering a shared mentality across these three core mental models, we believe the likelihood for the successful implementation of the IMPACT app will be much higher. The findings from the first stage of the study illustrate the links between mental models and attitudes. Thus, by pursuing complementary, accurate mental models, these positive attitudes expressed by stakeholders such as health leaders may manifest into strong explicit support during implementation, and positive adoption intentions expressed by end users will manifest into actual adoption and consistent usage. The proposed model for stage two of the study is outlined in Figure 4 and will be tested to further extend SMM theory in the area of mobile health in developing countries.

Figure 4.

Current and new mental models.

The model illustrated in Figure 4 aims to address conflicts in team members’ existing mental models and foster complementary models regarding the IMPACT app across the areas of technology (relevance, functioning, benefits to all), task (improvements to processes), and the team (understanding of the challenges and roles of all team members and supporting and emphasising the important role that PHC workers play in the provision of community healthcare).

Conclusion

Using a qualitative approach, the aim of this research is to investigate the current perceptions, concerns, and mentalities of key stakeholders towards a new mHealth solution in Enugu, Nigeria. Interviews and focus groups were conducted with stakeholders to elicit the teams’ views across three mental models: (1) the technology, (2) processes, and (3) the team.

A number of limitations must be noted. First, the study seeks to capture the views of the healthcare delivery team at the community level but does not include the views of patients or guardians. We acknowledge that patients are an invaluable stakeholder cohort but the primary patients who would be impacted by the introduction of the application are young children (under the age of 5), given the ethical implications associated with involving this vulnerable group in a study such as this the team decided to focus the study from a healthcare provision perspective. Second, this study focuses on one rural area of Nigeria: Enugu State. It could be argued that the mental models expressed here may contrast to the views in other areas of Nigeria and other developing countries. To combat this potential limitation, the study develops a deep and rich understanding of the context and accepts that the transferability of the findings to other contexts can only be explored through future research.⁷⁴ This article provides a robust starting point for future research.

Decision support mHealth solutions such as the IMPACT app have great potential to improve the quality of care delivered by lower cadres of health workers.³ In order to realise these benefits, this study explores the SMMs of stakeholders invested in healthcare delivery in the community in rural Nigeria. This article makes both theoretical and practical contributions. SMM theory is widely used to understand teamwork and collaboration in organisational settings such as teams in military, IT, and engineering contexts.⁴¹ However, it is only more recently that SMM is being used to understand teams operating within the healthcare context.⁴⁵ From a theoretical perspective, this study uses SMM theory to explore the teams’ current perceptions of mHealth and the provision of healthcare in Enugu. The SMM lens provides an opportunity to understand the shared mentality, or lack thereof, regarding mHealth among the stakeholders serving the health needs of the rural community. Considering SMM within the context of perceptions regarding attitudes to mHealth illuminates the convergence and divergence of stakeholder’s knowledge and beliefs across the healthcare ecosystem. The study also addresses the lack of empirical evidence on the mentalities of stakeholders in a community healthcare setting in a developing country. By understanding the mentality of stakeholders, an approach to create accurate complementary mental models regarding the technology, processes, and team can be developed. Finally, by adapting SMM, the study provides an alternative approach to traditional technology adoption models for understanding how different factors can impact stakeholders’ views towards a new mHealth solution.

The model for shared mentality in mHealth illustrated in Figure 2 needs to be assessed and further refined, to include further detail, across the three aspects of people (team), process (task), and technology. To develop this research stream further, the effectiveness of the training programme on altering existing mental models and creating SMMs regarding the new solution represents an important area of exploration. In addition, future research may look to combine the mental models of stakeholders with the role of common predictors of technology acceptance popularised in the technology adoption literature across contexts including mHealth such as perceived usefulness of the technology, perceived ease of use, facilitating conditions, and social influence.⁷⁵ Indeed, training programmes should encompass elements which seek to impact end users’ and other stakeholders’ perceptions of these factors.

The study makes important practical contributions. We will seek to address conflicts in existing mental models and develop consistent mental models pertaining to the IMPACT app through a comprehensive training programme implemented on the ground in Enugu, Nigeria. The training will focus on both healthcare workers and other stakeholders engaged in healthcare delivery. In terms of the former, it provide hands on practical training to PHC workers to use the IMPACT app and to gather valuable and important feedback on the usefulness and usability of the app while assessing patients in the field. The proposed targeted training will focus on educating PHC workers on the use of mHealth in general, leveraging existing frameworks to support the evaluation of mHealth apps. This approach will empower PHC workers to test and use existing available low cost solutions to do their jobs in the community.³ Most importantly, it will provide PHC workers new skills, renewed confidence, and support to discharge their roles in community health during a time when the payment of public service salaries are delayed. This training signals important investment in human capital.⁷³ The training will also seek to foster SMMs regarding the new mHealth solution across all stakeholders. The cross-training approach can lead to the development of inter-dispositional knowledge, which ultimately means all team members are cognisant of each other’s roles and responsibilities and as a result are more accurate in their perceptions of team members’ performance and the processes which may be altered by implementing a new technology.⁴¹ In summary, the study provides actionable insights to aid those stakeholders, including developers, researchers, and healthcare professionals, responsible for delivering healthcare services in the community in overcoming the barriers to implementation and scaling of new solutions.⁷⁶

By harnessing the insights from this study, we can develop a training programme which fosters a shared mentality among team members as a means of supporting more effective communication, collaboration, and shared understanding regarding the use of mHealth to assess sick children. Improved team performance through the provision of a long-term training and development strategy has the potential to translate into more efficient delivery of healthcare services, adherence to health guidelines, utilisation of mHealth interventions, strengthened relationships between PHC workers and other team members,⁴⁴ improvement of healthcare interactions between rural healthcare workers and community members,^5,6 and improved alignment between the health leaders’ perceptions of community healthcare services and the actual experiences of healthcare delivery on the ground in rural communities. Further exploration of stakeholder SMM in this context should serve to inform the ongoing training, technology, and change management needs of all stakeholders and better ‘assist individuals and groups in “working out” what the new system means for them and their work’.⁷⁷

Footnotes

Acknowledgements

We extend our thanks to the primary healthcare workers we met in Nsukka and the staff at the College of Medicine (ESUT) for supporting this research.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work is funded by the Irish Research Council, grant no. ‘New Horizons REPRO/2015/116’.

ORCID iD

Ciara Heavin

References

Al Dahdah

Du Loû

Méadel

. Mobile health and maternal care: a winning combination for healthcare in the developing world? Health Policy Techn 2015; 4(3): 225–231.

GSMA. The GSMA mHealth tracker, https://www.gsma.com/mobilefordevelopment/m4d-tracker/mhealth-deployment-tracker (2013, accessed 12 September 2017).

Agarwal

Perry

Long

, et al. Evidence on feasibility and effective use of mHealth strategies by frontline health workers in developing countries: systematic review. Trop Med Int Health 2015; 20(8): 1003–1014.

Rowe

de Savigny

Lanata

, et al. How can we achieve and maintain high-quality performance of health workers in low-resource settings? The Lancet 2005; 366(9490): 1026–1035.

Akter

D’Ambra

Ray

. User perceived service quality of m-Health services in developing countries. In: Proceedings of the 18th European conference on information systems, Pretoria, South Africa, 7–9 June 2010.

Kijsanayotin

Pannarunothai

Speedie

. Factors influencing health information technology adoption in Thailand’s community health centers: applying the UTAUT model. Int J Med Inform 2009; 78(6): 404–416.

Chib

Lwin

Ang

, et al. Midwives and mobiles: using ICTs to improve healthcare in Aceh Besar, Indonesia. Asian J Commun 2008; 18(4): 348–364.

DeRenzi

Findlater

Payne

, et al. Improving community health worker performance through automated SMS. In: Proceedings of the 5th international conference on information and communication technologies and development, Atlanta, GA, 12–15 March 2012. New York: ACM.

Varshney

. Mobile health: four emerging themes of research. Decis Support Syst 2014; 66: 20–35.

10.

Hossain

. Assessing m-Health success in Bangladesh: an empirical investigation using IS success models. J Enterprise Inform Manag 2016; 29(5): 774–796.

11.

Klöcker

Bernnat

Veit

. Stakeholder behavior in national eHealth implementation programs. Health Policy Techn 2015; 4(2): 113–120.

12.

Cohen

Coleman

Abrahams

. Use and impacts of e-Health within community health facilities in developing countries: a systematic literature review. In: Proceedings of the European conference on information systems (ECIS’2015), Münster, 26–29 May 2015.

13.

Kahn

Yang

Kahn

. ‘Mobile’ health needs and opportunities in developing countries. Health Aff 2010; 29(2): 252–258.

14.

Heeks

. Health information systems: failure, success and improvisation. Int J Med Inform 2006; 75(2): 125–137.

15.

Agarwal

Gao

DesRoches

, et al. Research commentary – the digital transformation of healthcare: current status and the road ahead. Inform Syst Res 2010; 21(4): 796–809.

16.

Pouloudi

. Aspects of the stakeholder concept and their implications for information systems development. In: Proceedings of the 32nd annual Hawaii international conference on systems sciences (HICSS-32), Maui, HI, 5–8 January 1999. New York: IEEE.

17.

Ahn

Skudlark

. Resolving conflict of interests in the process of an information system implementation for advanced telecommunication services. J Inf Technol 1997; 12(1): 3–13.

18.

Pan

GSC

. Information systems project abandonment: a stakeholder analysis. Int J Inform Manage 2005; 25(2): 173–184.

19.

Sharp

Finkelstein

Galal

. Stakeholder identification in the requirements engineering process. In: Proceedings of the 10th international workshop on database and expert systems applications, Florence, 3 September 1999. New York: IEEE.

20.

Abosede

Campbell

Olufunlayo

, et al. Establishing a sustainable ward health system in Nigeria: are key implementers well informed. J Community Med Health Educ 2012; 2: 164.

21.

Uzochukwu

Chinenye

Ogochukwu

, et al. Free maternal and child health services in Enugu State, South East Nigeria: experiences of the community and health care providers. Int J Med Health Sci Res 2015; 2(9): 158–170.

22.

World Health Organization (WHO). Integrated community case management of malaria, http://www.who.int/malaria/areas/community_case_management/overview/en/ (2016, accessed 28 July 2018).

23.

Bryce

Victora

Habicht

, et al. Programmatic pathways to child survival: results of a multi-country evaluation of Integrated Management of Childhood Illness. Health Policy Plan 2005; 20(suppl. 1): i5–i17.

24.

Eze

Gleasure

Heavin

. Mobile health solutions in developing countries: a stakeholder perspective. Health Syst 2020; 9(3): 179–201.

25.

Dinev

Albano

, et al. Individuals’ attitudes towards electronic health records: a privacy calculus perspective. In: Gupta

Patel

Greenes

(eds) Advances in healthcare informatics and analytics. Cham: Springer, 2016, pp. 19–50.

26.

Ajzen

Fishbein

. Understanding attitudes and predicting social behavior. Englewood Cliffs, NJ: Prentice Hall, 1980.

27.

Davis

. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quart 1989; 13: 319–340.

28.

Davis

Bagozzi

Warshaw

. User acceptance of computer technology: a comparison of two theoretical models. Manage Sci 1989; 35(8): 982–1003.

29.

Chang

I-C

Hwang

H-G

Hung

W-F

, et al. Physicians’ acceptance of pharmacokinetics-based clinical decision support systems. Expert Syst Appl 2007; 33(2): 296–303.

30.

Bennani

Oumlil

. IT acceptance by nurses in Morocco: application of a modified unified theory of acceptance and use of technology. IBIMA Bus Rev 2014; 2014: 849383.

31.

Ifinedo

. Technology acceptance by health professionals in Canada: an analysis with a modified UTAUT model. In: Proceedings of the 2012 45th Hawaii international conference on system sciences, Maui, HI, 4–7 January 2012. New York: IEEE.

32.

Mengesha

Garfield

Kebede

, et al. Preliminary assessment of Ethiopian physicians taste for Telemedicine: the case of Black Lion Hospital, Addis Ababa, Ethiopia, 2014, https://aisel.aisnet.org/ecis2014/proceedings/track09/6/

33.

Ezenwa

Brooks

. Using mobile technology to facilitate the health agenda in Nigeria: issues and opportunities. In: Proceedings of the UK Academy for information systems conference, London, 19 March 2013.

34.

Currie

. TEMPEST: an integrative model for health technology assessment. Health Policy Techn 2012; 1(1): 35–49.

35.

O’Connor

Ryan

Hardy

, et al. Stakeholders perspectives on paper-based and electronic clinical decision support systems in Malawi Africa. J Decis Syst 2016; 25(Suppl. 1): 410–420.

36.

Chib

. The promise and peril of mHealth in developing countries. Mobile Media Commun 2013; 1(1): 69–75.

37.

O’Connor

Eze

Heavin

. The status quo of IS conference publications on theorising eHealth in developing countries. In: Proceedings of the 37th international conference on information systems (ICIS’2016), Dublin, 11–14 December 2016.

38.

Jones

Ross

Lynam

, et al. Mental models: an interdisciplinary synthesis of theory and methods. Ecol Soc 2011; 16: 46.

39.

Hysong

Best

Pugh

, et al. Not of one mind: mental models of clinical practice guidelines in the Veterans Health Administration. Health Serv Res 2005; 40(3): 829–848.

40.

Cannon-Bowers

Salas

Converse

. Shared mental models in expert team decision making. In: Castellan

(ed.) Individual and group decision making: current issues. Hillsdale, NJ: Lawrence Erlbaum Associates, 1993, pp. 221–246.

41.

Mohammed

Ferzandi

Hamilton

. Metaphor no more: a 15-year review of the team mental model construct. J Manage 2010; 36(4): 876–910.

42.

Espinosa

. Shared mental models: accuracy and visual representation. In: Proceedings of the 7th Americas conference on information systems (AMCIS), Boston, MA, 3–5 August 2001. Boston: AIS, pp. 2102–2107.

43.

Evans

Baker

. Shared mental models of integrated care: aligning multiple stakeholder perspectives. J Health Organ Manag 2012; 26(6): 713–736.

44.

Thondoo

Strachan

Nakirunda

, et al. Potential roles of Mhealth for community health workers: formative research with end users in Uganda and Mozambique. JMIR Mhealth Uhealth 2015; 3(3): e76.

45.

McComb

Simpson

. The concept of shared mental models in healthcare collaboration. J Adv Nurs 2014; 70(7): 1479–1488.

46.

Johnson

Lee

, et al. Measuring sharedness of team-related knowledge: design and validation of a shared mental model instrument. Hum Resource Dev Int 2007; 10(4): 437–454.

47.

Undre

Sevdalis

Healey

, et al. Teamwork in the operating theatre: cohesion or confusion? J Eval Clin Pract 2006; 12(2): 182–189.

48.

Klimoski

Mohammed

. Team mental model: construct or metaphor? J Manage 1994; 20(2): 403–437.

49.

Lewis

Hodge

Gamage

, et al. Understanding the role of technology in health information systems. Pac Health Dialog 2012; 18(1): 144–154.

50.

Lim

Klein

. Team mental models and team performance: a field study of the effects of team mental model similarity and accuracy. J Organ Behav 2006; 27(4): 403–418.

51.

Aamir

Ali

Boulos

MNK

, et al. Enablers and inhibitors: a review of the situation regarding mHealth adoption in low- and middle-income countries. Health Policy Techn 2018; 7(1): 88–97.

52.

Sako

Karpathiou

Wickramasinghe

. Data accuracy in mHealth. In: Wickramasinghe

Troshani

Tan

(eds) Contemporary consumer health informatics. Cham: Springer, 2016, pp. 379–397.

53.

O’Connor

O’Reilly

. Examining the infusion of mobile technology by healthcare practitioners in a hospital setting. Inform Syst Front 2018; 20(6): 1297–1317.

54.

O’Connor

Heavin

Gallagher

, et al. Understanding community health workers’ intentions to participate in mHealth studies within developing countries. Behav Inform Technol 2017; 36(7): 745–753.

55.

Mathieu

Heffner

Goodwin

, et al. The influence of shared mental models on team process and performance. J Appl Psychol 2000; 85(2): 273–283.

56.

Davenport

Prusak

. Information ecology: mastering the information and knowledge environment. Oxford: Oxford University Press, 1997.

57.

O’Connor

O’Donoghue

. Contextual barriers to mobile health technology in African countries: a perspective piece. J Mobile Technol Med 2015; 4: 31–34.

58.

Chigona

Nyemba

Metfula

. A review on mHealth research in developing countries. J Community Informat 2013; 9(2): 1011–1019.

59.

Burtscher

Manser

. Team mental models and their potential to improve teamwork and safety: a review and implications for future research in healthcare. Safety Sci 2012; 50(5): 1344–1354.

60.

McComb

Lemaster

Henneman

, et al. An evaluation of shared mental models and mutual trust on general medical units: implications for collaboration, teamwork, and patient safety. J Patient Saf 2017; 13(4): 237–242.

61.

Oyelami

Okuboyejo

Ebiye

. Awareness and usage of Internet-based health information for self-care in Lagos State, Nigeria: implications for healthcare improvement. J Health Inform Develop Countr 2013; 7(2): 165–177.

62.

Compeau

Higgins

. Computer self-efficacy: development of a measure and initial test. MIS Quart 1995; 19: 189–211.

63.

Janz

Colquitt

Noe

. Knowledge worker team effectiveness: the role of autonomy, interdependence, team development, and contextual support variables. Pers Psychol 1997; 50(4): 877–904.

64.

Patel

Pratt

Patel

. Team processes and team care for children with developmental disabilities. Pediatr Clin North Am 2008; 55(6): 1375–1390, ix.

65.

Lee

Bharosa

Yang

, et al. Group value and intention to use – a study of multi-agency disaster management information systems for public safety. Decis Support Syst 2011; 50(2): 404–414.

66.

Yin

. Case study research design and methods (Applied social research methods series), vol. 5. 3rd ed. Thousand Oaks, CA: SAGE, 2013, pp. 1–312.

67.

Kemper

Stringfield

Teddlie

. Mixed methods sampling strategies in social science research. In: Tashakkori

Teddlie

(eds) Handbook of mixed methods in social & behavioral research. Thousand Oaks, CA: SAGE, 2003, pp. 273–296.

68.

Knoke

. Networks of elite structure and decision making. In: Wasserman

Galaskiewicz

(eds) Advances in social network analysis: research in the social and behavioral sciences (Sage focus editions), vol. 171. Thousand Oaks, CA: SAGE, 1994, pp. 274–294.

69.

Ågerfalk

Fitzgerald

. Outsourcing to an unknown workforce: exploring opensourcing as a global sourcing strategy. MIS Quart 2008; 32: 385–409.

70.

Miles

Huberman

. Qualitative data analysis: an expanded sourcebook. Thousand Oaks, CA: SAGE, 1994.

71.

Westli

Johnsen

Eid

, et al. Teamwork skills, shared mental models, and performance in simulated trauma teams: an independent group design. Scand J Trauma Resusc Emerg Med 2010; 18: 47.

72.

Nimpagaritse

Korachais

Nsengiyumva

, et al. Addressing malnutrition among children in routine care: how is the Integrated Management of Childhood Illnesses strategy implemented at health centre level in Burundi? BMC Nutr 2019; 5: 22.

73.

Mitchell

Hedt-Gauthier

Msellemu

, et al. Using electronic technology to improve clinical care – results from a before-after cluster trial to evaluate assessment and classification of sick children according to Integrated Management of Childhood Illness (IMCI) protocol in Tanzania. BMC Med Inform Decis Mak 2013; 13: 95.

74.

Lincoln

Guba

. Naturalistic inquiry, vol. 75. Beverly Hills, CA: SAGE, 1985.

75.

Alam

Hoque

, et al. Factors influencing the adoption of mHealth services in a developing country: a patient-centric study. Int J Inform Manage 2020; 50: 128–143.

76.

Källander

Tibenderana

Akpogheneta

, et al. Mobile health (mHealth) approaches and lessons for increased performance and retention of community health workers in low- and middle-income countries: a review. J Med Internet Res 2013; 15(1): e17.

77.

Klecun

Lichtner

Cornford

. e-Literacy in health care. Stud Health Technol Inform 2014; 205: 843–847.