A Mixed Methods,Cluster-Randomised Study Protocol to Design and Test WhatsApp Group-Based HIV Microlearning for Rural South African Healthcare Workers

Abstract

Treatment of HIV in South Africa is informed by national guidelines, which are regularly updated. Ongoing training of healthcare workers is vital for optimal patient care, but is challenging, due to human, financial and infrastructural constraints. There is a need for innovative solutions to meet training and support needs. With 93.2% of South African adults using WhatsApp, it is an ideal platform for training, even in the most remote clinics. The primary aim of the study is to design, test and evaluate the effect of WhatsApp group-based HIV training on nurses’ and community health workers’ knowledge. The secondary aims are to assess uptake, acceptability, and feasibility of the intervention; and to explore and describe the changes in prescribing, comparing the intervention group to the control group. A pragmatic, mixed-methods, parallel-group cluster-randomised study design will be used. This paper details the design – and rationale behind it – of the study. A critical realist stance was chosen, with outcomes measured using questionnaires, pre- and post-intervention; WhatsApp group interaction; focus groups; and folder reviews. Inferential analysis will be conducted using Stata, descriptive analysis using Excel™ and qualitative data using template analysis with NVivo™. Rigor of the study will be ensured using relevant points from the CASP and COREQ checklists, considering the study’s theoretical underpinning, design and aims. Reporting will be guided by the CONSORT and SRQR guidelines.

Keywords

qualitative research design mixed methods methodology microlearning WhatsApp

Background

South Africa had an HIV prevalence of 12.7% in 2022, which translates to 7.8 million people living with the disease (Human Sciences Research Council, 2023). The country has the largest HIV programme in the world (Unitaid, 2019), with 91% of people over 15 years old on treatment in 2022 (Human Sciences Research Council, 2023). Treatment of HIV is informed by national guidelines, which are regularly updated. Ongoing training, or continuing professional development (CPD), of healthcare workers (HCWs) is vital for optimal patient care.

In South Africa there are many logistical challenges to ongoing training: the country covers an area of 1.2 million km² with 4184 health facilities in 2018 (Health Systems Trust, 2020), many in hard-to-reach, rural settings.

Rural clinics are often run almost single-handedly by nurses, supported by community health workers (CHWs), with weekly, fortnightly or monthly visits from a doctor. In South Africa, HIV training has traditionally been face-to-face at centralised points (3–5 days), with a more recent move to online CPD. Distance and a lack of resources – human, financial and infrastructural/technological – reduce uptake of both of these methods (Feldacker, Jacob, et al., 2017; Feldacker, Pintye, et al., 2017).

Going Mobile

There is a need for innovative solutions to meet training and support needs. Social media is recognised as a way to promote research, collaboration, and academic advancement (Shah et al., 2023). WhatsApp, a free and accessible platform, has been shown to be the preferred social media platform by nurses (Ajuwon et al., 2018; Torrejon et al., 2020). With 100% cell phone penetration in 2022 (ICASA, 2022) and 93.2% of South African 16–64 year olds using WhatsApp in 2021 (Staff Writer, 2021), it is an ideal platform for training, even in the most remote clinics.

Keeping it Short, Sweet, and Interactive

Time is a precious commodity in busy clinics. Microlearning is defined as an educational experience that is focused, short and effective, usually driven by mobile devices, social connectedness and time scarcity (Torgerson, 2021). Essentially, microlearning encompasses teaching using ‘bite-sized’ information chunks: each lesson clearly explaining one concept (Kohnke, 2021). Its efficacy lies in interactivity and simplicity of design (Baumgartner, 2013) and it’s been endorsed within the health professions for student learning, training and CPD (De Gagne et al., 2019).

Microlearning has shown efficacy and satisfaction for learners across a range of health professions and social media platforms, including diagnostic evaluations for nurses using Instagram images (Tennyson et al., 2022); quiz-based microlearning on COVID for physicians, nurses, and allied health professionals (Triana et al., 2021); 3-min videos on paediatric emergency procedures for medical students (Cheng et al., 2017); and case-based mobile messaging for pharmacy students (Wang et al., 2018). Haghighat et al. (2023) used microlearning via WhatsApp and an online learning management system to improve final year nursing students’ trauma care knowledge. They found that the intervention improved knowledge and 1-month retention of knowledge, with high levels of satisfaction from participants. In their multi-country, multi-health profession study of microlearning for pharmacovigilance, Hegerius et al. (2020), also reported high levels of satisfaction with their 3- to 5-min video format, but participants requested practical exercises.

One of the regularly cited requests in studies of digital learning is for interaction between trainers and participants, and participants among themselves (Hegerius et al., 2020; Lee, 2021; Reeves et al., 2017; Tudor Car et al., 2019). The new generation of digital learners seek a humanistic approach with an online community, feedback and collaboration (Han et al., 2019).

Post-COVID, the scope of digital learning changed rapidly as educators across levels of education sought creative ways to have active learning on digital platforms (Chen et al., 2022). Tennyson et al. (2022), recommend digital microlearning as “an excellent innovative strategy to facilitate a sense of interconnectedness among nurse practitioner students engaged in distance-based learning.” Tudor Car et al. (2019), in their systematic review of digital education on clinical practice guidelines, proposed that more interactivity and engagement may improve educational gains. Developing platforms that encourage interaction in a ‘live’ online setting is key.

Using Cases to Teach

Case-based learning is a well-established teaching method and, while its effect on learning compared to other methods is inconclusive, it has been shown to be engaging, motivating and enjoyed by teachers and students (Thistlethwaite et al., 2012). Shah et al. (2023) used case-based, asynchronous microlearning via an app to train nurses on antimicrobial stewardship in a small pilot study and found improvements in knowledge and participants’ confidence, with 89% saying they thought it an effective teaching tool.

Lack of Thorough Evaluation of Innovative Training Styles

In their scoping review of 17 studies on using microlearning in health professions education, De Gagne et al. (2019) concluded that it has the potential to change education delivery and improve learning but found that there is a paucity of data on higher level learning outcomes and studies including adult learners.

The Kirkpatrick Model is an extensively used tool to evaluate training at four levels: (1) reaction – is the training favourable, engaging and relevant; (2) learning – does the training improve knowledge, skills, confidence and attitude; (3) behaviour – is what is learnt applied in the workplace; and (4) results – does training result in the desired outcome (Brooks et al., 2016; De Gagne et al., 2019).

Many previous WhatsApp-based and microlearning interventions have only been evaluated at level 1 and 2, leaving data on level 3 and level 4 scarce (Coleman & O'Connor, 2019; De Gagne et al., 2019; Tennyson et al., 2022).

Similarly, numerous other reviews of using social media in medicine have found benefits for communication and training, but an overarching need for rigorous research on its effectiveness with strong theoretical foundations, to explore its full potential (Chan & Leung, 2018; Coleman & O'Connor, 2019; Katz & Nandi, 2021).

The Gap

The call for recognition, research and development in the field of microlearning as a training tool for HCWs was amplified by the COVID pandemic (Dahiya & Bernard, 2021; Mak et al., 2021). The need for rigorous research on innovative, accessible, time-conscious, and effective training is clear.

Aims and Objectives

The primary aim of the study is to design, test and evaluate the effect of WhatsApp group-based HIV training on nurses’ and community health workers’ (CHW) knowledge in a remote area of South Africa. The primary outcome will be a change in knowledge compared to a control group (who do not receive training).

The secondary aims are to assess uptake, acceptability, and feasibility of the intervention; and to explore and describe the changes in prescribing, comparing the intervention group to the control group.

The specific objectives are listed in Table 1.

Table 1.

Specific Objectives.

• To determine knowledge changes through knowledge testing of nurses and CHWs: test score change from baseline to immediately after training of intervention clusters; test score change from baseline to three months after training of intervention compared to control clusters

• To determine knowledge retention through knowledge testing of nurses and CHWs three months after the intervention: changes in test score of intervention cluster from immediately after training to three months after

• To describe the uptake of WhatsApp-based training for nurses and CHWs: proportion of those offered the training who participated

• To describe the participation in the training: proportion of participants in ‘live’ group and content viewed at timepoints up to two weeks after the live session; and retention: proportion who dropped out during the training period

• To describe the acceptability of WhatsApp-based training for nurses and CHWs: quantitative and qualitative analysis of closed- and open-ended questions in the online knowledge questionnaire, focus group discussions and WhatsApp group interactions

• To describe the feasibility of WhatsApp-based training for nurses and CHWs: quantitative and qualitative analysis of closed- and open-ended questions in the online knowledge questionnaire, focus group discussions and analysis of the WhatsApp groups

• To explore and describe the changes in prescribing/clinical care due to the training intervention: changes in incorrect prescribing of the learning points in the training, comparing intervention clusters to control groups for a year before the intervention to a year after; through folder reviews

Theoretical Framework

For the purposes of the qualitative analysis of this study, the critical realist stance that there is a reality, but our understanding of that reality is influenced and limited by context, human practices and social forces was followed (Braun & Clarke, 2022; King & Brooks, 2017c).

Two minor theories were used to guide the design of the study – the all-encompassing Unified Theory of Acceptance and Use of Technology (UTAUT) (Marikyan & Papagiannidis, 2023) and the more pragmatic framework used by Asiimwe et al. (2012), based on Jeng’s Usability Assessment of Academic Digital Libraries: Effectiveness, Efficiency, Satisfaction, and Learnability (Jeng, 2005).

The Unified Theory of Acceptance and Use of Technology (UTAUT) has been used widely, across sectors including technological tools for nurses (Barchielli et al., 2021; Su & Chao, 2022; Winckler, 2022) however, many studies cited it, but didn’t use the model (Dwivedi et al., 2019), as is the case here. The modelling itself was not used but its driving factors were used to inform the questionnaire questions and focus group semi-structured interview guide.

While Jeng’s (2005) model was aimed at digital libraries, its suggested factors affecting usability were adapted by Asiimwe et al. (2012) to inform the framework to assess the feasibility, accessibility and use of point-of-care malaria tests in Uganda. This adaption – and further adaptions thereof – have been used widely to guide the qualitative testing of usability across a range of point-of-care interventions in LMICs including South Africa (de Vos et al., 2023), Zambia (Ansbro et al., 2015), and Thailand (Bancone et al., 2022).

The above studies were used to test the usability of point-of-care interventions, but the factors affecting usability were applicable to this study’s WhatsApp-based intervention. We used the usability model to inform the definition and methods of our study to measure the effectiveness, acceptability and feasibility of the study (Table 2).

Table 2.

Usability Factors and Study Definitions Based on Asiimwe et al.’s (2012) Model.

Usability factor	Definitions in study context
Learnability	Ability to understand the lesson and absorb the knowledge
Willingness	Willingness of HCWs to take part in the training
Satisfaction	Feeling that the training is convenient and/or enjoyable
Efficiency	Ability to attend and/or read the messages of the training
Effectiveness	The change in HCW knowledge and in patient care

Methods

Design

A pragmatic, mixed-methods, parallel-group cluster-randomised study (Figure 1). Quantitative analysis will be used to determine the effectiveness of WhatsApp-based training on increasing knowledge and altering prescribing behaviour. Both quantitative and qualitative analysis will be used to describe uptake, acceptability and feasibility of the intervention as a training tool.

Figure 1.

Study flowchart.

Study Setting

The study will be conducted in four predominantly rural districts in the Eastern Cape province of South Africa (Figure 2). Adult HIV prevalence in the province was 18.8% in 2022 (Human Sciences Research Council, 2023).

Figure 2.

Map of study district and towns, in South Africa.

Cluster randomisation (Hayes & Moulton, 2017) by town will be used to minimise inter-HCW contamination. Stratification across clusters will be done by number of HCWs and patients attending facilities, to obtain two equally sized groups and minimise variance between groups (Rutterford et al., 2015). Clusters will be randomly allocated within each stratum by the study statistician. Due to the nature of the intervention, blinding is not possible.

Study Population, Sampling and Recruitment

Healthcare Workers for Training

Clinics in this area are staffed by nurses, CHWs and counsellors, with weekly, fortnightly or monthly visits from a shared doctor. Few have an on-site pharmacist. Doctors and pharmacists are excluded from the study to minimise contamination across clinics.

The study is powered to 90% to detect a 2-point difference in mean HCW knowledge, accounting for the cluster design of the study (ICC = 0.01) with a sample size of 24 clusters (12 in each arm, with an average of three HCWs in each cluster) to give 5% significance. Fifty primary care clinics in 30 towns (clusters) will be included.

Sampling for the training intervention is purposive, with all CHWs, counsellors and nurses in the facilities invited to participate during recruitment visits to the clinics and flyers left for those not there on the day, with the option to join the study via WhatsApp.

Healthcare Workers for Focus Groups

Sampling for the focus groups, post-intervention, is purposive, to get rich data relevant to the training (Vasileiou et al., 2018). All HCWs who complete the training will be asked if they are willing to participate in a focus group during the online post-intervention questionnaire, and then convenience sampling will be used to pick the clinics that will be included, considering distance, number of agreeable participants and time.

While methodologies like grounded theory require sampling to ‘saturation’ or ‘data adequacy’ (Morse & Clark, 2019), there has been much debate about the concept within qualitative study (Vasileiou et al., 2018), especially when using thematic analysis (TA), which relies largely on reflexivity: each researcher may capture different nuances from the data collection and analysis (Braun & Clarke, 2019). More recent sample size discussions have recommended that sample size should be guided by the study characteristics e.g., epistemological/theoretical approach, nature of the investigated phenomenon, aims and scope, data quality and researcher skills/experience (Vasileiou et al., 2018). For the purposes of this study, Dey’s definition of ‘theoretical sufficiency’ – data collection stops when the researcher has reached a sufficient understanding – will be used (Braun & Clarke, 2019).

Patients for Folder Reviews

Patient folders from the population affected by the learning points in the training will be reviewed (Table 3).

Table 3.

Study Population for Folder Reviews.

• Adults on antiretrovirals who have/have not received TB preventive therapy

• Dosing of dolutegravir in TB-/HIV-coinfected adults on rifampicin

• Dosing of metformin in patients on dolutegravir

• Dosing of dolutegravir in patients on carbamazepine and/or phenytoin and/or phenobarbitone

• Dolutegravir use in women of childbearing potential

• Use of Gene Xpert™ in HIV-infected women booking at the antenatal clinic

Previous prescription reviews have shown rates of incorrect prescriptions due to drug-drug interaction rates of 18.7-43.0%, adjusted mean 20.0% (Agu et al., 2014; Evans-Jones et al., 2010; Kigen et al., 2011; Kuemmerle et al., 2021; Schlaeppi et al., 2020; Seden et al., 2015). Crude population calculations were used to establish population sizes for measurable outcomes (Chiwandire et al., 2021; Epilepsy South Africa, 2020; Human Sciences Research Council, 2019; South African National Department of Health, 2019). Using an expected 10% effect size for sampling calculations, at 80% power, 5% significance, with an ICC of 0.03, between 30 and 40 patient folders per cluster (if 24-26 clusters), will be needed.

Every effort will be made to review as many folders as possible, across all clinics, with the available resources (cost, time and human). If resource constraints force it, clinics for folder review will be randomised to include a smaller number of clusters in the folder reviews. This may mean it will be under-powered but as this is a proof-of-concept, secondary objective, this is considered to be reasonable.

Inclusion and Exclusion Criteria

Participants in the training intervention and focus group sessions, by nature of their working status/qualification, will be ≥18 years old and not considered to be part of any vulnerable population within the context of this study. Inclusion criteria are listed in Table 4.

Table 4.

Inclusion Criteria.

Training intervention	Focus groups	Folder reviews
• CHWs, counsellors and nurses	• Participation in all WhatsApp group sessions of the training	• Adult patients on ART
• Training will be conducted in English, so will require English comprehension	• Focus groups/interviews will be conducted in English, so will require English comprehension
• Access to a phone with WhatsApp
• Informed consent	• Informed consent

Ethics

Ethical approval has been granted for the project through the University of Cape Town’s Human Research Ethics Committee (HREC 491/2022) and the Eastern Cape Health Research Committee (EC_202209_003).

Patient-level consent for folder reviews is not feasible and there is no risk to patients, as no personal identifiers will be included in the reporting. This is a common challenge with cluster randomised studies which resulted in the development of the Ottawa Statement in 2012 (Taljaard et al., 2013). In accordance with recommendations 6 and 7 of the Ottawa Statement, consent was obtained at provincial level.

Training Intervention

Design of the lessons – six for nurses, four for CHWs – was guided by principles outlined in Corbeil, Khan & Corbeil’s Microlearning in the Digital Age: The Design and Delivery of Learning in Snippets (2021); Crompton’s Handbook of mobile learning (2013); Bates’ Teaching in a digital age (2019); BC’s 10+ years’ experience with interacting with nurses on the hotline; and her 5 years’ experience doing a weekly case study on the hotline’s Facebook page, which she manages. Learning points are based on the National HIV guidelines (South African National Department of Health, 2020) and the lessons were reviewed by a professor of pharmacology and tested in a pilot study. For CHWs, lessons include who to refer to the nurse.

WhatsApp groups will be made for each of the control and intervention groups, divided by profession – CHWs and counsellors together in one group; nurses in the other. Initially, all four groups will receive cell phone data to facilitate access to online questionnaires and WhatsApp groups. Information messages with links to the baseline online questionnaire will be sent and the online questionnaires will remain open for three weeks. Groups will then be told whether they are in the intervention group (receive training immediately) or control group (receive training at the end of the study).

Each lesson consists of cases and questions, sent in divided messages to encourage interaction, followed by the correct answer in both text and voice note. The training will be conducted ‘live’ in WhatsApp groups: 10–15-min, case-based microlearning lessons within the routine lunch break between 13:00 and 14:00. An introduction session will be given, followed by the lessons twice a week over three weeks (nurses) or twice a week over two weeks (CHWs).

Fifteen minutes before the beginning of each session, a ‘housekeeping’ page (Figure 3) will be sent to the WhatsApp groups. The eight points on conduct within the group were guided by the draft social media ethical guidelines of the Health Professions Council of South Africa (2019); barriers reported in previous studies (Mars & Scott, 2016; Woods et al., 2019) and review by an ethics expert.

Figure 3.

Housekeeping rules for training intervention on WhatsApp.

Outcomes

Data will be collected using multiple methods, to evaluate the four levels of the Gold Standard Kirkpatrick’s Levels of Training Evaluation (Champagne, 2016). The outcomes (Table 5) will be intervention-related changes in knowledge and prescribing behaviour compared with changes in matched control groups. The knowledge change immediately after the training intervention (three weeks) will compare only the intervention group to its baseline knowledge. This was done to prevent questionnaire fatigue in the control group.

Table 5.

Primary and Secondary Outcomes.

Primary outcome

• Improved knowledge of HCWs

Secondary outcomes

• Retained knowledge of HCWs, three months after training intervention

• Uptake, engagement, participant experience of feasibility and acceptability, retention

• Improved prescribing practices

Data Collection

Data will be collected in four ways:

1. Questionnaires, pre- and post-intervention: quantitative and qualitative

2. WhatsApp group interaction: quantitative and qualitative

3. Focus group/interviews: qualitative

4. Folder reviews: quantitative

Questionnaires

The questionnaires were designed, completed and stored on LimeSurvey (Community Edition; Version 5.6.10 + 230313). The first section is basic demographic information including town, profession, age, gender and years of experience.

The second section tests knowledge – ten multiple choice questions for nurses; seven for CHWs. The knowledge questions were based on those from the questionnaire disseminated in a previous study testing HCW knowledge of DTG’s drug-drug interactions (Chisholm et al., 2022), which was reviewed by HIV researchers within the Division of Clinical Pharmacology, and the AIDS and Society Research Unit at the Centre for Social Science Research, both at the University of Cape Town, to ensure questions were relevant, phrased to minimise bias and to assess reliability and validity.

The final section asks about mentorship and, in the post-intervention questionnaire, questions on the usability (acceptability and feasibility) of the intervention (Table 6). At the end of both the post-intervention and 3-month post-intervention questionnaires, the open-ended question, ‘If you have any suggestions, comments, or complaints about the course, please put them here. Your input is very valuable and will be used to improve the training’, was asked, with a free text answer option.

Table 6.

Questionnaire Questions to Measure Usability of the Intervention. Answer Options: Yes/No.

Factors	Questionnaire questions
Learnability	Did you find the WhatsApp-based training easy to understand?
	Did you find the WhatsApp-based training easy to use?
	Do you think that this training will change how you do your day-to-day duties, treating clients on ARVs?
Willingness	Did you find the WhatsApp-based training easy to use?
	Did you enjoy the WhatsApp-based training?
	Would you prefer non-interactive WhatsApp-based training, i.e. info being sent as a message, with no interaction in the group?
	Would you participate in this kind of learning if it were a weekly session throughout the year?
Suitability	Did you find the WhatsApp-based training useful?
	Would you prefer face-to-face training rather than WhatsApp-based training?
	Do you think that this training will change how you do your day-to-day duties, treating clients on ARVs?
Satisfaction	Did you find the WhatsApp-based training useful?
	Did you find the WhatsApp-based training easy to use?
	Did you enjoy the WhatsApp-based training?
	Would you prefer non-interactive WhatsApp-based training, i.e. info being sent as a message, with no interaction in the group?
	What were the challenges you experienced with the WhatsApp-based training?
Efficiency	Did you find the WhatsApp-based training convenient?
Efficiency	Were you able to access all ‘live’ sessions of the WhatsApp-based training at 13:00?
Effectiveness	Do you think that this training will change how you do your day-to-day duties, treating clients on ARVs?

The URL link to the online questionnaires will be sent in the WhatsApp groups, and will remain open for three to four weeks, with regular reminders sent to complete them. They will be administered at baseline (intervention and control groups); immediately after the training intervention (intervention group); and three months after the training (intervention and control groups). Multiple choice answers are set to change order at random, to minimise the risk of sharing of answers and/or memorisation.

WhatsApp Group Interaction

Engagement of participants in sessions will be measured using WhatsApp functionalities. WhatsApp Business allows the admin to see to whom messages have been delivered, and who has seen them.

Due to potential disruptions of loadshedding – South Africa has an energy crisis which has resulted in regular, scheduled periods of the electricity being cut off each day – this will not necessarily have to be during the live session. Each participant’s having ‘seen’ the messages in each lesson will be defined as them having received the lesson. This will be measured during the lesson, one hour after, 24 hours after, at the end of the week, and two weeks after the training.

All participant interaction will also be collected. Each day’s messages will be exported from WhatsApp and saved in a password-protected folder.

Focus Groups

Traditionally, in health service and medical education research, focus groups were used to guide the development of research questionnaires, but there has been a move toward their use to get the ‘voice’ on the design, application and evaluation of curricula (Barbour, 2005; Jaschinski & De Villiers, 2008; Lam et al., 2001).

Barbour (2010) describes focus groups as “coming into their own” when the aim is to generate research data relating to new developments or procedures, that requires contributions from everyone to answer ‘why?’ (and ‘why not?’), and to get different perspectives from the same group. The nature of the training intervention – essentially, a group activity – makes focus groups the ideal way to gain insight into the views and experiences of the healthcare workers of the WhatsApp-based training of the study.

Focus groups – with a maximum of ten participants – will be conducted in a convenient, quiet and private location, either at the clinic or in the town where the clinics are. The researcher will conduct the focus groups and an assistant will be present in the meetings. With participant permission, they will be recorded and the recordings will be saved in a password-protected folder. Questions are semi-structured (Annexure 1).

Folder Reviews

To measure changes in prescribing behaviour, retrospective folder reviews will be done in both intervention and control groups at the end of the study. The 48 weeks before the intervention will be compared to the 48 weeks after the intervention. The period of clinic visits for recruitment, prior to the intervention, will be excluded, to minimise confounding.

In the case of drug-drug interaction knowledge, proportion of incorrect prescriptions will be measured in three groups:

• Patients on metformin (correct: maximum 500 mg bd)

• Patients on rifampicin (correct: 50 mg dolutegravir bd)

• Patients on antiepileptics (correct: no phenytoin/phenobarbitone; if carbamazepine, 50 mg dolutegravir bd)

For measuring changes in prescription on dolutegravir use in women of childbearing potential, the use of GeneXpert™, and TB preventive therapy, we will review folders of:

• Women aged 18–45 years started on ART (dolutegravir -based regimens)

• Pregnant women booking for the first time (GeneXpert™ performed)

• All adult patients started on ART (TB preventive therapy)

Data collected will include prescriber (doctor/nurse); patient data (age, gender, pregnancy); clinical data (HIV diagnosis date, ART initiation date; co-morbidities); record of Gene Xpert™; and drug data (dolutegravir start date, NRTI backbone, dosing/dispensed number of dolutegravir, TB treatment, metformin and antiepileptic drugs, other co-prescribed drugs).

Data Analysis

Quantitative Data

At individual HCW level, descriptive analyses will be performed using Excel 365 (Microsoft Version 2309). Proportions and frequencies will be used to describe participants who get correct answers at each follow-up timepoint.

Inferential data analysis will be done using Stata Statistical Software: Release 15.1 (College Station, TX: StataCorp LLC) using expanded linear mixed-effects regression model adjusted for age, gender, years of experience, stratification, and clustering, with the primary analysis being of HCW total knowledge score. Mean differences will be estimated with their respective 95% confidence intervals.

For the patient level analyses, the same methods will be applied, while adjusting to account for the clustering effect. This inflation factor is often referred to as the ‘design effect’ (Campbell et al., 2000). T-test values will be divided by the square root of the design effect ((1 + (n – 1) r), where n is the average cluster size and r is an estimate of the intracluster coefficient (ICC). The effect of the shared sessional doctor exposure is a factor that will be adjusted for in the analysis.

Descriptive Analysis of WhatsApp Data

Text and emoji interaction within the training sessions – that was not direct answering of case-based learning point questions – will be extracted for descriptive quantitative analysis using NVivo (Version 14, released 2023).

Uptake, participation, and interaction will be described using proportions. For uptake, the proportion who participated will be calculated using the formula:

Uptake = \frac{Number of healthcare workers who signed up for the study}{N u m b e r o f h e a l t h c a r e w o r k e r s a t t h e c l i n i c s}

For participation in the training, the proportion will be calculated at each timepoint – during the lesson, one hour after, 24 hours after, at the end of the week, and two weeks after the training – using the formula:

Participation = \frac{Number of participants who have ‘ see n' messages at timepoint x}{Number of participants who signed up}

For interaction within the training sessions, counts of participant’s interaction will be counted through line-by-line coding.

All emoticons will be counted in NVivo and used to create a descriptive word cloud on https://www.worditout.com/. Studies, both in non-clinical and clinical settings have shown that emojis are increasingly used to improve communication, add emotion, modulate intensity, contextualise and upgrade/downgrade text, and signal closing or opening of interactions (Halverson et al., 2023; Sampietro, 2019).

Descriptive Analysis of Questionnaire Data

Proportions will be reported for the survey questions assessing acceptability and feasibility (Table 6).

Qualitative Data

Qualitative data, drawn from the online questionnaires, WhatsApp group discussions and focus groups will be coded using NVivo (QSR International Pty Ltd. Version 12, 2018) and analysed thematically.

WhatsApp group discussions will be exported and saved in Excel spreadsheets with personal identifiers removed. Focus groups will be transcribed verbatim by BC. Themes will be drawn out using template analysis and reported descriptively using NVivo.

Template analysis is a ‘generic’ TA approach that allows the researcher to choose the methodology and philosophy that best suits the needs of the study (King & Brooks, 2017b). King and Brooke’s method of template analysis is described as a midpoint between coding reliability approaches (e.g., that commonly used in the grounded theory paradigm) and the reflexive approach, sharing the methods of theme creation of coding reliability but using the qualitative philosophy of the reflexive approach (Byrne, 2022).

The method has been used widely, across a diverse range of research areas (King & Brooks, 2017b), including psychology research (Brooks et al., 2015), sports science (Schneider et al., 2023), primary care guideline implementation (McKillop et al., 2012), and postgraduate medical education (Fokkema et al., 2013) and is suitable within the critical realist framework.

King and Brooks (2017b) situate template analysis within (limited) realist paradigms, where causal explanations are sought, by recommending three points to consider:

1. Template analysis can make use of a priori themes, drawing on existing theory or previous evaluation: in this case, data from the online questionnaires.

2. Reflexivity is vital to develop themes that are not just due to researcher subjectivity.

3. The reflexivity – and realist paradigm – does not rely on technical checks such as inter-rater reliability, but rather relies on researchers constantly reflecting on possible threats to interpretation quality and how to minimise them.

Template analysis has a seven-step process (Figure 4) but is flexible in that it is an iterative process, and emphasises the need to repeatedly go back to the data – quality checks – to support and organise constructed themes (King & Brooks, 2017a). It uses hierarchical coding with the creation of a flexible template that is added to, redefined, and deleted from, throughout the process (Brooks et al., 2015; Keidser et al., 2019).

Figure 4.

The seven steps of template analysis (King & Brooks, 2017a).

The use of a priori themes can be particularly useful in mixed methods studies like this one, where data from the quantitative portion – in this case, the post-intervention questionnaire – can be used to inform those themes (King & Brooks, 2017a). Template analysis allows researchers to take assumptions from existing data without them restricting the analytic process (Fokkema et al., 2013). The flexibility of the method fits within the pragmatic design and research questions of the study. NVivo will be used throughout the analysis process, to ensure a comprehensive audit trail.

Ensuring Qualitative Rigor

Qualitative analysis exists at the intersection of the researcher, data and, importantly, contextual interpretation (Braun & Clarke, 2022) and, as such, it has very different quality evaluation requirements to quantitative analysis, a subject of ongoing debate (King & Brooks, 2017a; Yadav, 2022) that is beyond the scope of this paper. By virtue of the diversity of paradigms in qualitative research, a single set of quality criteria is impossible and they need to be reflexive (Yadav, 2022).

The most important aspect of ensuring quality is to acknowledge the diversity of qualitative research and to be explicit and transparent about methods used, engaging with the theoretical and philosophical assumptions underlying the research procedures (Braun & Clarke, 2021; Kahlke, 2014; Liu, 2022), a concept often left out of research practice (King & Brooks, 2017c; Lipscomb, 2008; Prestwich et al., 2014).

Numerous checklists for ensuring qualitative rigor have been drawn up over the past 50 years, intersecting each other in some cases, and contradicting in others (Yadav, 2022). The development of a specific list of criteria for ensuring good quality qualitative research is controversial, but they are, invariably, helpful, especially to researchers new to the field.

Critical realist research needs to, instead of relying on standardised checks, be aware of the interpretation of quality and establish a plan to address it (Maxwell, 2011). King and Brooks (2017a) back Symon and Cassell’s recommendation that researcher’s should identify relevant criteria and explicitly justify them, which is echoed by Yadav (2022), who recommends that novice researchers use a combination of the available checklists guided by the particular study paradigm.

A recent review of the criteria for good qualitative research briefly described strategies used to meet quality criteria by researchers including Lincoln and Guba, and Merriam and Tisdell (Yadav, 2022). Relevant points from Yadav’s (2022) synthesis of strategies and from two checklists – the 10-question qualitative research Critical Appraisal Skills Programme (CASP) checklist, designed and developed by experts based on JAMA’s user guides (CASP, 2018); and the Consolidated Criteria for Reporting Qualitative Research (COREQ), a 32-item checklist for interviews and focus groups (Tong et al., 2007) – were used to develop a framework to ensure the rigor of this study, taking into account its theoretical underpinning, design and aims. The strategies were grouped by Yadav’s (2022) three suggested strategy groups: credibility, transferability, and dependability and confirmability (Table 7).

Table 7.

Strategies to Enrich Trustworthiness in the Context of the Study (Yadav, 2022).

	Strategy	Key points in the context of this study
Credibility	Triangulation^a	Qualitative data (focus groups, open-ended questionnaire questions, WhatsApp discussion) with quantitative data (closed-ended questionnaire answers)
	Collecting data to saturation	Data collection until reach ‘theoretical sufficiency’ – when the researcher has reached a sufficient understanding
	Reflexivity	Use of a ‘reflexivity diary’ throughout process
	Peer review	Discussion with PhD supervisors to limit bias
	Prolonged engagement	In-person recruitment visits to facilities, WhatsApp interaction throughout training, in-person focus groups, multiple engagements with data before analysis
Transferability	Maximum variation	Focus groups sampled to include speakers from both dominant languages in the region (isiXhosa and Afrikaans); highly rural, small town and larger town
	Typical sampling	By virtue of including only nurses and CHWs on a subject pertaining to their work
	Thick description	Direct quotes will be used throughout the reporting
Dependability and confirmability	Audit trail	Procedures, decisions and reflections detailed throughout
	Peer debriefing	External audit by pharmacist colleague at HIV Hotline uninvolved in the study
	Triangulation^a	Methodological and data triangulation
	Reflexivity	Transparency of research domain, question, methodology, data collection, data analysis, and in the writing and presentation of findings

^aTriangulation: analysing a research question from more than one perspective (Amin et al., 2020).

Reporting Results

Reporting will be guided by two sets of guidelines. Firstly, the CONSORT 2010 updated guidelines for reporting parallel group randomised trials (Schulz et al., 2010), with its two relevant extensions: cluster randomised trials (Campbell et al., 2012) and randomised trials of nonpharmacologic treatment (Boutron et al., 2008). Secondly, the Standards for Reporting Qualitative Research (SRQR), a 21-point guide collated by a group with experience in health professions education will be consulted (O’Brien et al., 2014).

Footnotes

Acknowledgements

Professor Carl Lombard and Mr Lovemore Mapahla for their help with the statistical design of the study; Ms Melissa Wallace, for her contribution to the qualitative sections; the healthcare workers in the study clinics, who are the participants; the National HIV & TB Healthcare Worker Hotline.

Declaration of conflicting interests

The author(s) declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The work reported herein is made possible through BC’s funding by the South African Medical Research Council through its Division of Research Capacity Development under the Bongani Mayosi National Health Scholars Programme from funding received from the Public Health Enhancement Fund/South African National Department of Health. An educational grant from Aspen will cover the research costs of the study, excluding BC’s salary. The sponsors had no role in guidelines discussion and manuscript preparation and finalisation. The content herein is the sole responsibility of the authors and does not necessarily represent the official views of the SAMRC or Aspen.

Ethical Statement

ORCID iD

Briony Sue Chisholm

Data Availability Statement

Data sharing not applicable to this article as no datasets were generated or analysed during the current study.

References

Agu

K. A.

Oqua

Adeyanju

Isah

M. A.

Adesina

Ohiaeri

S. I.

Ali

P. N.

Ekechukwu

Akpakwu

A. A.

Sani

Omeh

I. O.

King

R. C.

Wutoh

A. K.

(2014). The incidence and types of medication errors in patients receiving antiretroviral therapy in resource-constrained settings. PLoS One, 9(1), Article e87338. https://doi.org/10.1371/journal.pone.0087338

Ajuwon

Pimmer

Odetola

Gröhbiel

Oluwasola

Olaleye

(2018). Mobile Instant Messaging (MIM) to support teaching practice: Insights from a nurse tutor program in Nigeria. Malawi Medical Journal, 30(2), 120–126. https://doi.org/10.4314/mmj.v30i2.12

Amin

M. E. K.

Nørgaard

L. S.

Cavaco

A. M.

Witry

M. J.

Hillman

Cernasev

Desselle

S. P.

(2020). Establishing trustworthiness and authenticity in qualitative pharmacy research. Research in Social and Administrative Pharmacy, 16(10), 1472–1482. https://doi.org/10.1016/j.sapharm.2020.02.005

Ansbro

É. M.

Gill

M. M.

Reynolds

Shelley

K. D.

Strasser

Sripipatana

Tshaka Ncube

Tembo Mumba

Terris-Prestholt

Peeling

R. W.

Mabey

(2015). Introduction of syphilis point-of-care tests, from pilot study to national programme implementation in Zambia: A qualitative study of healthcare workers' perspectives on testing, training and quality assurance. PLoS One, 10(6), Article e0127728. https://doi.org/10.1371/journal.pone.0127728

Asiimwe

Kyabayinze

D. J.

Kyalisiima

Nabakooza

Bajabaite

Counihan

Tibenderana

J. K.

(2012). Early experiences on the feasibility, acceptability, and use of malaria rapid diagnostic tests at peripheral health centres in Uganda-insights into some barriers and facilitators. Implementation Science, 7, 5. https://doi.org/10.1186/1748-5908-7-5

Bancone

Gilder

M. E.

Win

Gornsawun

Penpitchaporn

Moo

P. K.

Archasuksan

Wai

N. S.

Win

Aung

K. K.

Hashmi

Hanboonkunupakarn

Nosten

Carrara

V. I.

McGready

(2022). Technical evaluation and usability of a quantitative G6PD POC test in cord blood: A mixed-methods study in a low-resource setting. BMJ Open, 12(12), Article e066529. https://doi.org/10.1136/bmjopen-2022-066529

Barbour

R. S.

(2005). Making sense of focus groups. Medical Education, 39(7), 742–750. https://doi.org/10.1111/j.1365-2929.2005.02200.x

Barbour

R. S.

(2010). Focus groups. In The Sage handbook of qualitative methods in health research (pp. 327–352). Sage Publications Ltd. https://doi.org/10.4135/9781446268247

Barchielli

Marullo

Bonciani

Vainieri

(2021). Nurses and the acceptance of innovations in technology-intensive contexts: The need for tailored management strategies. BMC Health Services Research, 21(1), 639. https://doi.org/10.1186/s12913-021-06628-5

10.

Bates

(2019). Teaching in a digital age ( Opentext

B. C.

(Ed.), (2nd ed.). ISO. https://pressbooks.bccampus.ca/teachinginadigitalagev2/

11.

Baumgartner

(2013). Educational dimensions of microlearning - towards a taxonomy for micolearning. In Bruck

Sedlaczek

(Eds.), Designing microlearning experiences - building up knowledge in organisations and companies. University Press. https://portfolio.peter-baumgartner.net/files/pdf/2013/Baumgartner_2013_Educational_Dimensions_for_MicroLearning.pdf

12.

Boutron

Moher

Altman

D. G.

Schulz

K. F.

Ravaud

CONSORT Group . (2008). Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: Explanation and elaboration. Annals of Internal Medicine, 148(4), 295–309. https://doi.org/10.7326/0003-4819-148-4-200802190-00008

13.

Braun

Clarke

(2019). To saturate or not to saturate? Questioning data saturation as a useful concept for thematic analysis and sample-size rationales. Qualitative Research in Sport, Exercise and Health, 13(2), 201–216. https://doi.org/10.1080/2159676X.2019.1704846

14.

Braun

Clarke

(2021). One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qualitative Research in Psychology, 18(3), 328–352. https://doi.org/10.1080/14780887.2020.1769238

15.

Braun

Clarke

(2022). Thematic analysis: A practical guide. Sage Publications Ltd.

16.

Brooks

Jeskins

Monagle

Pratchett

Young

(2016). Kirkpatrick’s four levels of evaluation model. In Practical tips for developing your staff (pp. 23–26). Facet. https://doi.org/10.29085/9781783301812.012

17.

Brooks

McCluskey

Turley

King

(2015). The utility of template analysis in qualitative psychology research. Qualitative Research in Psychology, 12(2), 202–222. https://doi.org/10.1080/14780887.2014.955224

18.

Byrne

(2022). A worked example of Braun and Clarke’s approach to reflexive thematic analysis. Quality and Quantity, 56(3), 1391–1412. https://doi.org/10.1007/s11135-021-01182-y

19.

Campbell

Marion

Mollison

Jill

Steen

Nick

Grimshaw

Jeremy

Eccles

Martin

(2000) (In this issue). Analysis of cluster randomized trials in primary care: a practical approach. Family Practice, 17(2), 192–196. https://doi.org/10.1093/fampra/17.2.192

20.

Campbell

M. K.

Piaggio

Elbourne

D. R.

Altman

D. G.

CONSORT Group . (2012). Consort 2010 statement: Extension to cluster randomised trials. BMJ British Medical Journal, 345(7441). Article e5661. https://doi.org/10.1136/bmj.e5661

21.

CASP . (2018). CASP qualitative checklist. Retrieved 12 September 2022, from. https://casp-uk.net/images/checklist/documents/CASP-Qualitative-Studies-Checklist/CASP-Qualitative-Checklist-2018.pdf

22.

Champagne

(2016). Evaluation instrument creation basics and blended evaluation items and sample tools. In Kirkpatrick

J. D.

(Ed.), Kirkpatrick’s four levels of training evaluation (pp. 76–108). ATD Press.

23.

Chan

W. S.

Leung

A. Y.

(2018). Use of social network sites for communication among health professionals: Systematic review. Journal of Medical Internet Research, 20(3), Article e117. https://doi.org/10.2196/jmir.8382

24.

Chen

Ayoob

Desser

T. S.

Khurana

(2022). Review of learning tools for effective radiology education during the COVID-19 era. Academic Radiology, 29(1), 129–136. https://doi.org/10.1016/j.acra.2021.10.006

25.

Cheng

Y. T.

Liu

D. R.

Wang

V. J.

(2017). Teaching splinting techniques using a just-in-time training instructional video. Pediatric Emergency Care, 33(3), 166–170. https://doi.org/10.1097/pec.0000000000000390

26.

Chisholm

B. S.

Swart

A. M.

Blockman

(2022). South African healthcare workers' knowledge of dolutegravir's drug-drug interactions in the first year of its rollout: A cross-sectional online survey. Journal of the International AIDS Society, 25(3), Article e25885. https://doi.org/10.1002/jia2.25885

27.

Chiwandire

Zungu

Mabaso

Chasela

(2021). Trends, prevalence and factors associated with hypertension and diabetes among South African adults living with HIV, 2005-2017. BMC Public Health, 21(1), 462. https://doi.org/10.1186/s12889-021-10502-8

28.

Coleman

O'Connor

(2019). The role of WhatsApp® in medical education; a scoping review and instructional design model. BMC Medical Education, 19(1), 279. https://doi.org/10.1186/s12909-019-1706-8

29.

Corbeil

Khan

Corbeil

(2021). Microlearning in the digital age: The design and delivery of learning in Snippets (1st ed.). Taylor & Francis. https://doi.org/10.4324/9780367821623

30.

Crompton

(2013). Handbook of mobile learning. Routledge. https://www.researchgate.net/publication/263852116_A_historical_overview_of_mobile_learning_Toward_learner-centered_education

31.

Dahiya

Bernard

(2021). Microlearning: The future of CPD/CME. J Eur CME, 10(1), Article 2014048. https://doi.org/10.1080/21614083.2021.2014048

32.

De Gagne

J. C.

Park

H. K.

Hall

Woodward

Yamane

Kim

S. S.

(2019). Microlearning in health professions education: Scoping review. JMIR Med Educ, 5(2), Article e13997. https://doi.org/10.2196/13997

33.

de Vos

Daniels

Gebengu

Mazzola

Gleeson

Piton

Mdingi

Gigi

Ferreyra

Klausner

J. D.

Peters

R. P. H.

(2023). Usability of a novel lateral flow assay for the point-of-care detection of Neisseria gonorrhoeae: A qualitative time-series assessment among healthcare workers in South Africa. PLoS One, 18(6), Article e0286666. https://doi.org/10.1371/journal.pone.0286666

34.

Dwivedi

Y. K.

Rana

N. P.

Jeyaraj

Clement

Williams

M. D.

(2019). Re-Examining the unified theory of acceptance and use of technology (UTAUT): Towards a revised theoretical model. Information Systems Frontiers, 21(3), 719–734. https://doi.org/10.1007/s10796-017-9774-y

35.

Epilepsy South Africa (2020). Facts about Epilepsy. Retrieved 23 April 2022. https://epilepsy.org.za/content/uploads/2020/07/002-Brochure-Facts-about-Epilepsy.pdf.

36.

Evans-Jones

J. G.

Cottle

L. E.

Back

D. J.

Gibbons

Beeching

N. J.

Carey

P. B.

Khoo

S. H.

(2010). Recognition of risk for clinically significant drug interactions among HIV-infected patients receiving antiretroviral therapy. Clinical Infectious Diseases, 50(10), 1419–1421. https://doi.org/10.1086/652149

37.

Feldacker

Jacob

Chung

M. H.

Nartker

Kim

H. N.

(2017). Experiences and perceptions of online continuing professional development among clinicians in sub-Saharan Africa. Human Resources for Health, 15(1), 89. https://doi.org/10.1186/s12960-017-0266-4

38.

Feldacker

Pintye

Jacob

Chung

M. H.

Middleton

Iliffe

Kim

H. N.

(2017). Continuing professional development for medical, nursing, and midwifery cadres in Malawi, Tanzania and South Africa: A qualitative evaluation. PLoS One, 12(10), Article e0186074. https://doi.org/10.1371/journal.pone.0186074

39.

Fokkema

J. P.

Teunissen

P. W.

Westerman

van der Lee

van der Vleuten

C. P.

Scherpbier

A. J.

Dörr

P. J.

Scheele

(2013). Exploration of perceived effects of innovations in postgraduate medical education. Medical Education, 47(3), 271–281. https://doi.org/10.1111/medu.12081

40.

Guidelines for Good Practice in the Health Care Professions: Ethical Guidelines on Social Media . (2019). Guidelines for good practice in the health care professions: Ethical guidelines on social media. https://www.hpcsa.co.za/Uploads/Professional_Practice/Conduct_%26_Ethics/Ethical2_Guidelines_on_Social_Media.pdf

41.

Haghighat

Shiri

Esmaeili Abdar

Taher Harikandee

S. S.

Tayebi

(2023). The effect of micro-learning on trauma care knowledge and learning satisfaction in nursing students. BMC Medical Education, 23(1), 622. https://doi.org/10.1186/s12909-023-04609-2

42.

Halverson

C. M. E.

Donnelly

C. E.

Weiner

Lee

J. L.

(2023). Content analysis of emoji and emoticon use in clinical texting systems. JAMA Network Open, 6(6), Article e2318140. https://doi.org/10.1001/jamanetworkopen.2023.18140

43.

Han

E. R.

Yeo

Kim

M. J.

Lee

Y. H.

Park

K. H.

Roh

(2019). Medical education trends for future physicians in the era of advanced technology and artificial intelligence: An integrative review. BMC Medical Education, 19(1), 460. https://doi.org/10.1186/s12909-019-1891-5

44.

Hayes

Moulton

(2017). Cluster randomised trials (2nd ed.). Taylor & Francis. https://www-taylorfrancis-com.ezproxy.uct.ac.za/books/mono/10.4324/9781315370286/cluster-randomised-trials-richard-hayes-lawrence-moulton

45.

Health Systems Trust . (2020). Health indicators. https://www.hst.org.za/healthindicators

46.

Hegerius

Caduff-Janosa

Savage

Ellenius

(2020). E-learning in pharmacovigilance: An evaluation of microlearning-based modules developed by Uppsala monitoring centre. Drug Safety, 43(11), 1171–1180. https://doi.org/10.1007/s40264-020-00981-w

47.

Human Sciences Research Council . (2019). South African national HIV prevalence, incidence, behaviour and communication survey, 2017. H. Press. https://www.hsrc.ac.za/uploads/pageContent/10779/SABSSM_V.pdf

48.

Human Sciences Research Council . (2023). The sixth South African national HIV prevalence, incidence, behaviour and communication survey. https://sahivsoc.org/Files/SABSSM-FINAL-LAUNCH-presentation-27November2023-1.pdf

49.

ICASA . (2022). The state of the ICT sector report in South Africa. Independent Communications Authority of South Africa.

50.

Jaschinski

De Villiers

M. R.

(2008). Factors influencing the development of practical skills of interns working in regional hospitals of the Western Cape province of South Africa. South African Family Practice, 50(1), 70–70d. https://doi.org/10.1080/20786204.2008.10873676

51.

Jeng

(2005). Usability assessment of academic digital libraries: Effectiveness, efficiency. Satisfaction, and Learnability, 55(2-3), 96–121. https://doi.org/10.1515/libr.2005.96

52.

Kahlke

R. M.

(2014). Generic qualitative approaches: Pitfalls and benefits of methodological mixology. International Journal of Qualitative Methods, 13(1), 37–52. https://doi.org/10.1177/160940691401300119

53.

Katz

Nandi

(2021). Social media and medical education in the context of the COVID-19 pandemic: Scoping review. JMIR Med Educ, 7(2), Article e25892. https://doi.org/10.2196/25892

54.

Keidser

Matthews

Convery

(2019). A qualitative examination of user perceptions of user-driven and app-controlled hearing technologies. American Journal of Audiology, 28(4), 993–1005. https://doi.org/10.1044/2019_aja-19-0022

55.

Kigen

Kimaiyo

Nyandiko

Faragher

Sang

Jakait

Owen

Back

Gibbons

Seden

Khoo

S. H.

USAID-Academic Model for Prevention Treatment of HIV/AIDS . (2011). Prevalence of potential drug-drug interactions involving antiretroviral drugs in a large Kenyan cohort. PLoS One, 6(2), Article e16800. https://doi.org/10.1371/journal.pone.0016800

56.

King

Brooks

J. M.

(2017a). Doing template analysis: A guide to the main components and procedures. In Template analysis for business and management students (pp. 25–46). Sage Publications Ltd. https://doi.org/10.4135/9781473983304

57.

King

Brooks

J. M.

(2017b). Introduction. In Template analysis for business and management students (pp. 1–12). Sage Publications Ltd. https://doi.org/10.4135/9781473983304

58.

King

Brooks

J. M.

(2017c). Philosophical issues when using template analysis. In Template analysis for business and management students (pp. 13–23). Sage Publications Ltd. https://doi.org/10.4135/9781473983304

59.

Kohnke

(2021). Optimizing microlearning materials for mobile learning. In Corbeil

Khan

Corbeil

(Eds.), A multidimensional roadmap for implementing effective microlearning solutions (pp. 80–94). Taylor & Francis.

60.

Kuemmerle

Sikalengo

Vanobberghen

Ndege

R. C.

Foe

Schlaeppi

Burri

Battegay

Paris

D. H.

Glass

T. R.

Weisser

Marzolini

KIULARCO Study Group . (2021). Recognition and management of clinically significant drug-drug interactions between antiretrovirals and co-medications in a cohort of people living with HIV in rural Tanzania: A prospective questionnaire-based study. Journal of Antimicrobial Chemotherapy, 76(10), 2681–2689. https://doi.org/10.1093/jac/dkab254

61.

Lam

Chow Chan (2001). The use of focus group interviews in Asian medical education evaluative research. Medical Education, 35(5), 510–513. https://doi.org/10.1046/j.1365-2923.2001.00898.x

62.

Lee

Y. M.

(2021). Mobile microlearning: A systematic literature review and its implications. Interactive Learning Environments, 31(7), 4636-4651. https://doi.org/10.1080/10494820.2021.1977964

63.

Lipscomb

(2008). Mixed method nursing studies: A critical realist critique. Nursing Philosophy, 9(1), 32–45. https://doi.org/10.1111/j.1466-769X.2007.00325.x

64.

Liu

(2022). Paradigmatic compatibility matters: A critical review of qualitative-quantitative debate in mixed methods research. Sage Open, 12(1), Article 215824402210799. https://doi.org/10.1177/21582440221079922

65.

Mak

Franzosa

Burack

Reinhardt

J. P.

(2021). Research needed on microlearning as a training strategy for CNAs in skilled nursing facilities. Journal of the American Medical Directors Association, 22(12), 2610–2611. https://doi.org/10.1016/j.jamda.2021.07.025

66.

Marikyan

Papagiannidis

(2023). Unified theory of acceptance and use of technology: A review. Retrieved 22 August from. https://open.ncl.ac.uk/

67.

Mars

Scott

R. E.

(2016). WhatsApp in clinical practice: A literature review. Studies in Health Technology and Informatics, 231, 82–90. https://ebooks.iospress.nl/pdf/doi/10.3233/978-1-61499-712-2-82.

68.

Maxwell

(2011). A realist approach to qualitative research. Sage.

69.

McKillop

Crisp

Walsh

(2012). Barriers and enablers to implementation of a New Zealand-wide guideline for assessment and management of cardiovascular risk in primary health care: A template analysis. Worldviews on Evidence-Based Nursing, 9(3), 159–171. https://doi.org/10.1111/j.1741-6787.2011.00233.x

70.

Morse

J. M.

Clark

(2019). The Sage Handbook of current developments in grounded theory. Sage Publications Ltd. https://doi.org/10.4135/9781526485656

71.

O’Brien

B. C.

Harris

I. B.

Beckman

T. J.

Reed

D. A.

Cook

D. A.

(2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine, 89(9), 1245–1251. https://doi.org/10.1097/acm.0000000000000388

72.

Prestwich

Sniehotta

F. F.

Whittington

Dombrowski

S. U.

Rogers

Michie

(2014). Does theory influence the effectiveness of health behavior interventions? Meta-Analysis. Health Psychology, 33(5), 465–474. https://doi.org/10.1037/a0032853

73.

Reeves

Fletcher

McLoughlin

Yim

Patel

K. D.

(2017). Interprofessional online learning for primary healthcare: Findings from a scoping review. BMJ Open, 7(8), Article e016872. https://doi.org/10.1136/bmjopen-2017-016872

74.

Rutterford

Copas

Eldridge

(2015). Methods for sample size determination in cluster randomized trials. International Journal of Epidemiology, 44(3), 1051–1067. https://doi.org/10.1093/ije/dyv113

75.

Sampietro

(2019). Emoji and rapport management in Spanish WhatsApp chats. Journal of Pragmatics, 143, 109–120. https://doi.org/10.1016/j.pragma.2019.02.009

76.

Schlaeppi

Vanobberghen

Sikalengo

Glass

T. R.

Ndege

R. C.

Foe

Kuemmerle

Paris

D. H.

Battegay

Marzolini

Weisser

Kilombero and Ulanga Antiretroviral Cohort KIULARCO study group . (2020). Prevalence and management of drug-drug interactions with antiretroviral treatment in 2069 people living with HIV in rural Tanzania: A prospective cohort study. HIV Medicine, 21(1), 53–63. https://doi.org/10.1111/hiv.12801

77.

Schneider

Matheson

E. L.

Tinoco

Silva-Breen

Diedrichs

P. C.

LaVoi

N. M.

(2023). A six-country study of coaches’ perspectives of girls’ body image concerns in sport and intervention preferences: Template analysis of survey and focus group data. Body Image, 46(1), 300–312. https://doi.org/10.1016/j.bodyim.2023.06.013

78.

Schulz

K. F.

Altman

D. G.

Moher

CONSORT Group . (2010). CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. BMJ, 340, c332. https://doi.org/10.1136/bmj.c332

79.

Seden

Merry

Hewson

Siccardi

Lamorde

Byakika-Kibwika

Laker

Parkes-Ratanshi

Back

D. J.

Khoo

S. H.

(2015). Prevalence and type of drug-drug interactions involving ART in patients attending a specialist HIV outpatient clinic in Kampala, Uganda. Journal of Antimicrobial Chemotherapy, 70(12), 3317–3322. https://doi.org/10.1093/jac/dkv259

80.

Shah

S. S.

Zangla

Qader

M. A.

Chaturvedi

Mannemuddhu

S. S.

(2023). Embracing the (r)evolution of social media and digital scholarship in pediatric nephrology education. Pediatric Nephrology, 39(7), 2061–2077. https://doi.org/10.1007/s00467-023-06251-y

81.

South African National Department of Health . (2019). Primary health care facilities and services. Retrieved 27 April from. https://www.healthestablishments.org.za/Home/Facility

82.

South African National Department of Health . (2020). National consolidated guidelines for the management of HIV in adults, adolescents, children and infants and prevention of mother-to-child transmission. National Department of Health of South Africa and World Health Organisation.

83.

Staff Writer . (2021). The biggest and most popular social media platforms in South Africa, including TikTok. Inside Business. https://businesstech.co.za/news/internet/502583/the-biggest-and-most-popular-social-media-platforms-in-south-africa-including-tiktok/

84.

C. Y.

Chao

C. M.

(2022). Investigating factors influencing nurses' behavioral intention to use mobile learning: Using a modified unified theory of acceptance and use of technology model. Frontiers in Psychology, 13, Article 673350. https://doi.org/10.3389/fpsyg.2022.673350

85.

Taljaard

Weijer

Grimshaw

J. M.

Eccles

M. P.

Ottawa Ethics of Cluster Randomised Trials Consensus Group . (2013). The Ottawa Statement on the ethical design and conduct of cluster randomised trials: Precis for researchers and research ethics committees. BMJ, 346, f2838. https://doi.org/10.1136/bmj.f2838

86.

Tennyson

C. D.

Smallheer

B. A.

De Gagne

J. C.

(2022). Microlearning strategies in nurse practitioner education. Nurse Educator, 47(1). 2, 3. https://doi.org/10.1097/NNE.0000000000001083, https://journals.lww.com/nurseeducatoronline/Fulltext/2022/01000/Microlearning_Strategies_in_Nurse_Practitioner.2.aspx

87.

Thistlethwaite

J. E.

Davies

Ekeocha

Kidd

J. M.

MacDougall

Matthews

Purkis

Clay

(2012). The effectiveness of case-based learning in health professional education. A BEME systematic review: BEME guide No. 23. Medical Teacher, 34(6), e421–e444. https://doi.org/10.3109/0142159x.2012.680939

88.

Tong

Sainsbury

Craig

(2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19(6), 349–357. https://doi.org/10.1093/intqhc/mzm042

89.

Torgerson

(2021). What is microlearning? Origin, definitions, and applications. In Corbeil

Khan

Corbeil

(Eds.), A multidimensional roadmap for implementing effective microlearning solutions (1st ed., pp. 14–31). Taylor & Francis.

90.

Torrejon

Rodriguez-Blanco

Mayer

M. A.

(2020). Use of social media apps by nurses for professional purposes in Catalonia: Cross-sectional study. Studies in Health Technology and Informatics, 270, 786–790. https://doi.org/10.3233/shti200268

91.

Triana

A. J.

White-Dzuro

C. G.

Siktberg

Fowler

B. D.

Miller

(2021). Quiz-based microlearning at scale: A rapid educational response to COVID-19. Med Sci Educ, 31(6), 1731–1733. https://doi.org/10.1007/s40670-021-01406-8

92.

Tudor Car

Soong

Kyaw

B. M.

Chua

K. L.

Low-Beer

Majeed

(2019). Health professions digital education on clinical practice guidelines: A systematic review by digital health education collaboration. BMC Medicine, 17(1), 139. https://doi.org/10.1186/s12916-019-1370-1

93.

Unitaid . (2019). South Africa to introduce state-of-the-art HIV treatment. https://unitaid.org/news-blog/south-africa-to-introduce-state-of-the-art-hiv-treatment/#en

94.

Vasileiou

Barnett

Thorpe

Young

(2018). Characterising and justifying sample size sufficiency in interview-based studies: Systematic analysis of qualitative health research over a 15-year period. BMC Medical Research Methodology, 18(1), 148. https://doi.org/10.1186/s12874-018-0594-7

95.

Wang

Cui

Shi

Duan

Wang

(2018). The use of mobile messaging-based case studies in a pharmacotherapy introduction class in China. Journal of Computer Assisted Learning, 34(5), 526–533. https://doi.org/10.1111/jcal.12257

96.

Winckler

(2022). Not another box to check! Using the UTAUT to explore nurses' psychological adaptation to electronic health record usability. Nursing Forum, 57(3), 412–420. https://doi.org/10.1111/nuf.12686

97.

Woods

Moorhouse

Knight

(2019). A descriptive analysis of the role of a WhatsApp clinical discussion group as a forum for continuing medical education in the management of complicated HIV and TB clinical cases in a group of doctors in the Eastern Cape, South Africa. Southern African Journal of HIV Medicine, 20(1), 982. https://doi.org/10.4102/sajhivmed.v20i1.982

98.

Yadav

(2022). Criteria for good qualitative research: A comprehensive review. The Asia-Pacific Education Researcher, 31(6), 679–689. https://doi.org/10.1007/s40299-021-00619-0