Using a Multicountry Learning Network to Harvest and Rapidly Spread Implementation Knowledge across Programs Aimed to Reduce Mother-to-Child Transmission of HIV and Improve Nutrition: Perspectives and Lessons Learned for Similar Large-Scale Initiatives

Abstract

As countries pursue UNAIDS’s 90-90-90 target for ending the AIDS epidemic, success is dependent on learning how to deliver effective care. We describe a learning network and mechanisms used to foster communication and sharing of ideas and results across 6 countries in the Partnership for HIV-Free Survival. The network used 2 forms of peer exchange, in-person and virtual, and a variety of knowledge management mechanisms to harvest and spread key learning. Key learning included valuable insights on how to design and convene a multicountry learning network, including top enablers of success and practical insights on the network’s value. The network was instrumental in accelerating learning about improving care. Our experience shows the value of creating a quality improvement–driven, multicountry learning network to accelerate the pace of improving care systems. Government ownership and adaptation of collaborative learning efforts to the country context must be considered when designing future networks.

Keywords

multicountry learning implementation science knowledge management HIV nutrition

Background

As countries apply HIV/AIDS guidelines in pursuit of UNAIDS’s 90-90-90 target for ending the AIDS epidemic,^1,2 system implementers increasingly turn to methods that foster continuous learning about what works and what does not in delivering effective care and closing the “know-do gap”³ between evidence-based knowledge and the actual delivery of care. Country success in rapidly applying and scaling up evidence-based protocols and guidelines can be accelerated by their ability to learn from efforts to implement HIV/AIDS care within their own country setting as well as from other countries that are undertaking similar implementation efforts.⁴

Learning exchanges, or peer-to-peer learning (a process where individuals or organizations share information, ideas, expertise, and experiences for a common purpose), are described as a powerful way to share what works in one setting and adapt, replicate, and scale up successful interventions in different settings across regions and nations.^5,6 Practitioners are more likely to learn and adopt ideas from others who have or are going through similar challenges⁵ than from didactic classroom teaching. Although there is increasing interest in learning across countries, and global networks already exist,^7

-11 countries formally communicating and working together to better implement a specific program of common interest using a common implementation approach is highly uncommon.¹²

What Do We Already Know about This Topic?

Learning exchanges, or peer-to-peer learning have been described as a powerful way to share what works in one setting and adapt, replicate, and scale up successful interventions in different settings across regions and nations. Practitioners are more likely to learn and adopt ideas from others who have or are going through similar challenges than from didactic classroom teaching.

Although there is increasing interest in learning across countries, and global networks already exist, countries formally communicating and working together to better implement a specific program of common interest using a common implementation approach was highly uncommon before our work commenced. Previous evidence points to the value of creating a QI-driven, multicountry learning network to tackle other topics. A few studies have demonstrated 1) significant improvement over time among organizations that report sharing information compared to those reporting less sharing of knowledge 2) new policies introduced as a direct result of experiences learned from others and 3) peer exchange as fundamental to building sustainability within countries.

How Does Your Research Contribute to the Field?

Our research further demonstrates the value of creating a QI-driven, multicountry learning network as a system of learning to harness and capture knowledge to better prepare countries, regions or states to tackle current and future challenges. We describe a range of mechanisms that can be used and adapted by others to foster communication and sharing of ideas and results. We provide a replicable, adaptable model others can use to accelerate learning about improving care. We offer valuable lessons learned about the process, offering key implications of using and adapting this type of multicountry learning network for improving program implementation.

Our research reinforces previous studies demonstrating peer exchange as fundamental to improvement over time, building sustainability within countries through shared experience, knowledge, and expertise as well as reinforcing effective QI strategies and spread.

What Are Your Research's Implications toward Theory, Practice, or Policy?

Our research further demonstrates that through cross-organizational learning, health systems can be better prepared to address emerging challenges. We believe the value demonstrated by this multicountry learning network could be replicated by others, leading to the accelerated pace of improving systems of care, whether PMTCT or other areas of care. Our research points to the strength in application of a common QI approach and joint learning toward better clinical outcomes. Already, adaptations of our model and lessons from this work have been integrated into the creation of new multi-country global learning networks.

There is significant value in any system that accelerates the transfer of new implementation knowledge to other countries undertaking similar programming. Our research demonstrates further evidence toward the impact of exchanges leading to system redesign based on ideas and learning from exchanges and the impetus for introducing new policies as a direct result of experiences learned from others.

In addition, the existing PHFS multicountry learning network could serve as a platform for improving other services within these same countries, building on the trust and lasting relationships that have already been developed.

The Partnership for HIV-Free Survival (PHFS) was a coordinated, 6-country implementation intervention to improve survival of infants born to mothers infected with HIV that used quality improvement (QI) methods to deliver better programs for prevention of mother-to-child transmission (PMTCT), maternal, newborn, and child health (MNCH), and nutrition. Other articles in this supplement detail the QI methods and approach used within each country (JIAPAC-18-08-SA-1100, 1103, 1104, 1105, and 1106). In this article, we describe the mechanisms used to foster communication and sharing of ideas and results across PHFS countries, leveraging a multicountry learning network. We describe the range of in-person and virtual learning/knowledge sharing opportunities created and what we learned about the process. We share perspectives on the value of this learning system and reflect on the implications of using this type of multicountry learning network for improving program implementation.

Multicountry Learning Network Description

Setting

The World Health Organization (WHO), the United Nations Children’s Fund, and the US President’s Emergency Plan for AIDS Relief launched the PHFS in March 2013 in partnership with national leaders and technical assistance providers in 6 countries: Kenya, Lesotho, Mozambique, South Africa, Tanzania, and Uganda. Before the launch of the PHFS, none of the countries had centrally coordinated learning mechanisms to harvest and disseminate implementation knowledge gained across their countries or externally. The PHFS established these mechanisms. In addition to the in-country QI approach to learning, the PHFS established a multicountry learning network to offer structured opportunities for exchange of in-country generated implementation knowledge across countries.

Other articles in this supplement describe the context, quantitative data analysis, and results of this learning network. JIAPAC-18-08-SA-1098 describes how these countries were chosen and detail the history, setting, design, and approaches of the PHFS.

Participation

Participarts in the learning network included government leaders and managers from all health-care levels involved (subdistrict, district/regional, and national), country-level and global implementing support partners, and donor agencies. To bring the experience of practitioners and the community served, health workers (community and facility levels) and community members (mothers) participated in specific knowledge exchanges.

Goal

To elicit, create, and disseminate learning across the network, a knowledge management system was established to accelerate the pace of learning toward progress in the effective implementation of PMTCT and nutrition programming by spreading implementation knowledge across the network.

Framework and Approach

The network was designed to draw learning from the underlying core QI implementation approach being used within each country to foster improved performance at a local or district level. The methods used to facilitate learning were based on adaptations of previously applied exchange methods. For instance, the network emphasized rapid adaptive learning, collaborative learning, use of reflective data systems, phased scale-up designs, and planning for sustainability as found in the Institute for Healthcare Improvement’s breakthrough series collaborative improvement approach widely tested and adapted for use in various settings.^13

-16 Specific exchange methods were also based on Wenger’s community of practice model, Dixon’s knowledge management techniques, and other successful social learning models.^4,17,18 Both in-county and multicountry learning were driven by a regular rhythm of gathering and sharing implementation ideas and data (Figure 1) generated by facilities in the participating countries.

Figure 1.

Multicountry learning network: framework for learning.

The implementation knowledge shared across the multicountry learning network followed the cascade of care provided for mothers and infants in antenatal care, at delivery, and postnatally. Teams tracked and shared the ideas being tested to improve these care steps and the methods being applied to implement them with laser focus on harvesting and documentation. Although methods of learning exchange were not novel, this joint focus and formally communicating and working together on a specific program of common interest using a similar QI approach were novel.

Resources for Support, Tracking, Harvesting, and Evaluation

The key technical support partners (Institute for Healthcare Improvement and University Research Co) from the US Agency for International Development Applying Science to Strengthen and Improve Systems Project formed a core knowledge management team to stimulate global learning with the PHFS. They planned, organized, and facilitated all multicountry learning activities with local in-country key contacts and created a support system for continuous learning and knowledge capture. Several tools were designed and used to capture and track country implementation progress, data, learning shared over time, and country-specific learning needs.

Evaluations were used to gather feedback from network participants at the end of each in-person meeting/exchange to assess experience, value, and effectiveness of methods to enhance individual/team learning, determine whether learning objectives were met, and identify whether (and what) learning took place. Qualitative and quantitative data were analyzed and used to inform the continuous design and content of future exchanges.

Ethical Approval and Informed Consent

Ethics approval was not sought, as this is a perspectives piece of a QI initiative to improve the uptake of existing government approved, evidence-based clinical interventions. Given that no new clinical interventions were being introduced and no intervention were introduced that could cause harm, we did not seek institutional review board approval. Only deidentified, aggregate data that were collected were used, and data were used only for learning.

Multicountry Learning Network Activities and Learning Outputs

The network used 2 forms of peer exchange: in-person and virtual. Several types of in-person meetings were held between March 2013 and March 2017: all-country meetings (6 countries participating), regional meetings (3 or 4 countries participating), and country-to-country knowledge-exchange visits (one country team visiting another country). To meet participants’ learning needs, participants were involved in choosing and designing exchange methods that were used. Initial methods were chosen at the launch and then reassessed after 6 months (during a session at regional meetings) and 1 year. Table 1 shows the tool used to assess impact versus cost of potential exchange methods that each country considered. Five types of virtual methods were identified by participants and used between March 2013 and March 2017: webinars, newsletters, listserv discussion group,¹⁹ social media, and an online learning platform. Methods and activities are described below and in greater detail in Table 2.

Table 1.

Tool to Assess Impact versus Cost of Potential Exchange Methods Used by Each Country.^a

	Menu of Learning Exchange Options Proposed and Discussed with Each Country Team
	Low Impact	Medium Impact	High Impact
High cost			In-person all-country meeting 2- to 3-day regional or all-country team meeting
Medium cost			Peer-to-peer knowledge-exchange visits Small team of country reps visit another country’s sites Learn about systems for postnatal care (PNC), data/QI, PMTCT/NACS integration successes and changes being made Exchange visits during learning sessions Above visits planned during learning session dates to maximize impact of learning from multiple facilities/partners
Low cost	Virtual meeting All-country teams convene for scheduled 1-day meeting via virtual platform for facilitated collaborative learning Teams gather collectively in own country, dedicated space Topics for discussion distributed ahead of time Presentations planned and facilitated idea generation Facebook/Twitter chat Scheduled day and time for teams to participate in hour-long facilitated chat Questions posed, ideas shared and live discussion PHFS discussion group Hosted online discussion group for casual Q&A and facilitated topics related to PMTCT/NACS	Smaller peer-to-peer one-on-one-calls Facilitated and scheduled phone discussions on specfic topic across 2 or more teams Smaller subset of reps for increased dialogue Video messaging/learning/site visit Using video cameras and/or recording programs, teams record and share learning from facilities for other teams to access via video One-on-one video interviews with each country team to share learnings, answer questions distributed ahead of time	Hybrid virtual meeting Adding on-site facilitation to virtual meeting option Designated facilitator (from partner teams) is on-site in each country for a dedicated 1- to 2-day virtual learning meeting Facilitated local discussions/breakouts and then connections across to other countries via virtual connections to share ideas and learning Piggyback meeting onto a current meeting Identify upcoming meetings 2 or more country teams will be involved in that are not PHFS related (eg, upcoming HIV, PEPFAR, NACS meetings) Planned facilitated discussion on PHFS—for a few hours or full day; topics distributed ahead

Abbreviations: NACS, nutrition assessment, counseling, and support; PEPFAR, US President’s Emergency Plan for AIDS Relief; PHFS, Partnership for HIV-Free Survival; PMTCT, prevention of mother-to-child transmission; Q&A, question and answer; QI, quality improvement.

^aThis menu of collaborative learning methods was shared with each team at the 1-year mark to (1) assess current exchange methods (chosen by teams themselves at the launch); (2) identify alternative exchange options to use; (3) assess if shared learning goals were being met; and (4) identify emerging learning needs (content) on which to center future exchanges.

Table 2.

Learning Methods, Activities, and Outputs.

Description of Exchange Methods Applied, Key Learning, and Outputs Produced
In-Person or Virtual Exchange Activity, Timing, and Duration	Participants	Purpose and Methods Used for Exchange	Key Areas of Learning and Outputs Produced	Comments on Method
All-country launch meetingMarch 2013, 3 days	6 countriesMultidisciplinary teams and technical partners from all 6 participating countries, global partners, PEPFAR (USAID, CDC, OGAC), WHO, and UNICEF	Purpose: introduce the aim of partnership and joint learning, build relationshipsMethods: presentations, workshop-style large/small group discussion teaching and practice using QI methods and tools, storyboard “show and tell” round robins, breakout sessions for team planning, knowledge cafés on specific topic areas	Alignment around common data-driven QI approach/methods to test strategies for effective PMTCT and nutrition assessment, counseling, and support (NACS) within each country health program Solidified an overall aim of the PHFS Experience and current situation within each country in implementing 2010 WHO guidelines on HIV and infant feeding, links to NACS Building on existing national PMTCT and nutrition programs Defined/refined the roles and responsibilities of members of the country teams and in-country PHFS steering committees (to coordinate partners and work) Initiated a roadmap for the partnership A list of learning exchange methods generated Value of joint learning: clear understanding gained that despite varying contexts, there were many common themes (both barriers and opportunities) and that they could learn how to solve challenges from one another Outputs: A driver diagram (a visual display to help team build their theory of change as to what focal areas were most needed to drive changes to reach PHFS aim) was co-created by country teams. Additionally, a map of the key clinical processes that ensured a combined focus on HIV and nutrition, as well as a relentless focus on combined care for mothers and infants was created. This process map provided a common reference for teams from each country to plan improvement interventions, measure progress and share information about the successes and challenges in the care of the mother–baby unit across the learning network. The driver diagram and process map are depicted and further described by Barker et al in this supplement.	Worked well to unify and build relationships within and across country teams and to create a solid foundation for the network for learning^a
Regional meetingsOctober 2013, 2 daysNovember 2013, 2 days	Southern meeting: South Africa, Mozambique, LesothoEastern meeting: Uganda, Kenya, TanzaniaMoH, district and implementing partner representatives from countries, global partners, USAID, WHO, and UNICEF	Purpose: share implementation progress, data, challenges, knowledge building, exchange, and relationship buildingMethods: presentations, workshop-style large/small group discussion, storyboard rounds, breakout sessions, team planning, knowledge cafés on specific topic areas/questions posed; site visit to a health facility (in Mozambique) to share experiences on the implementation of Option B+.	Discussed early results and changes being made/tested along the PMTCT continuum: Keeping mother–baby pairs in care Implementation of NACS Knowing the HIV status of every mother–infant pair Ensuring optimal antiretroviral therapy (ART) coverage for every mother–infant pair Emphasis on prioritizing areas of urgency (eg, keeping pregnant women and mother–baby pairs in care, ensuring pairs receive standard, routine package of care at every visit)Country teams also reported: Their progress with using QI to improve care delivery along the PMTCT path Plans for enhancing data systems, testing change ideas, and using data to demonstrate change and increase buy-in. Stated value of joint learning: learning exchange helped teams feel supported and proud of accomplishments to date, renewed commitment, and generated a host of ideas and plans to implement.	Worked well for advancing understanding of clinical and improvement knowledge, understanding of learning principles, generating ideas for improving processes and solidifying relationships across countries^a
Knowledge-exchange visitsFebruary 2015, 2 daysJune 2015, 2 days	Two exchanges:Lesotho to UgandaKenya to TanzaniaMoH, district management, facility-based representatives, and supporting implementing partners; community members in Tanzania	Purpose: share effective implementation strategies/practices between country teams, programmatic challenges, and how they are overcomeMethods: classroom-style presentations, small intimate, in-depth discussions, and site visits to improvement sites/health facilities and ministry offices; video-taped interviews were used to capture real-time feedback and learning.	Four areas of key learning emerged from the Uganda and Lesotho exchange: Role of the MoH and implementing partners Coaching and engaging teams Gathering and using data Testing changes along care path and guidance Four areas of key learning emerged from the Tanzania and Kenya exchange: Stakeholder engagement Role of the community Care delivery Institutionalization and scale-up Country delegations also planned how they would apply this learning.Outputs: Two reports on the process, key learning, and value of the exchanges were created:Uganda-Lesotho Knowledge-Exchange ReportTanzania-Kenya Knowledge-Exchange Report	Worked very well for deeper exchange, learning and trust building; highly popular and allowed members to see the work of others in action^a
Final all-country review meetingFebruary 2016, 2 days	6 countries:MoH and implementing partner representatives from all 6 countries (those involved in the design and implementation of PHFS, and who could speak to the results achieved in each country), global partners and funders	Purpose: retrospective review of the PHFS—report results and gather learning across the 6 countriesMethods: presentations, Q&A, roundtable small working groups on specific questions posed, all-group report backs, panel and group discussions, and smaller one-on-one conversations.	Intensive debriefing and data exchange sessions covered progress review from each country on results and changes along the PMTCT continuum, including key changes made, data/results, and activities at facility and community levels to improve PMTCT. Reflections from teams on what worked and what did not with advice to others initiating similar initiatives. Focus on how specific changes were developed and carried out at facility level, and how these evidence-based changes were spread to additional facilitiesFocused discussion on: Best ways to mainstream and embed PMTCT, MNCH, and nutrition learning, changes and QI methods into national PMTCT programs Current and future plans and strategies for scale-up How to sustain the gains made and planning next steps for documenting and spreading learning regionally, nationally and globally Output: Key learning for multiple articles (in this supplement)	Worked well for allowing teams to highlight how they turned what they learned throughout the partnership into programmatic changes and results^a
All-country webinarsMonthly for first year, quarterly thereafterDuration of calls: 1 to 1.5 hours	Representatives from country ministry and local departments of health, in-country supporting implementation partners, global partners, USAID, WHO, and UNICEF	Purpose: share updates, data, and changes being made at facility level; learn from experts about PMTCT, MNCH, nutrition, QI, and scale-up.Methods: country teams presentations (data, changes, lessons learned), facilitated Q&A and group discussion on the topic.	Sharing changes being made along the PMTCT pathway and data/results—including creating data dashboard Lessons from implementation Tools being designed and used (data tracking and reporting tools, mother–baby registers, routine care checklists) Engaging with communities to affect change and partner for improvement Creating change packages, lessons on scale-up Debriefing on updated HIV guidelines and impact on countries Outputs: Webinar recordings, slides, and comments:https://www.usaidassist.org/toolkits/partnership-hiv-free-survival-learning-platform/webinars	Worked well to give teams a platform to share, for keeping teams informed, advancing learning, and connections
Small working group virtual meetingsEvery other month, 6 calls	1-2 M&E representatives from all 6 countries, data experts	Purpose: align thinking and strategies for tracking and sharing data for learningMethods: facilitated discussions, presentations and Q&A	Relevant ways to measure and track improvements for sharing across the partnership Whether a set of common indicators to share for learning purposes was possible or find other ways to converge and share existing indicators Variances in operational definitions of similar indicators across the countries How to avoid creation of new data sets that were different than HMIS indicators already being used Opportunities to test out new indicators along the PMTCT pathway (ie, postnatal) where there were no existing measures Output: tools for measurement and reporting	Worked well for initial alignment. Discontinued after the first year
Electronic newslettersMonthly (15)	Sent to all listserv members	Purpose: share results and learning on a regular basis to keep all informed on each other’s progressContent shared from country teams	Data and changes being tested across the continuum Updates on implementation progress on the ground Case studies sharing deeper progress from each team Stories, experiences, and pictures from teams	Worked well for information push out, keeping teams informed
ListservThroughout initiative	All-country contacts, global steering committee members, and supporting partners	Purpose: foster information and resource sharing and encourage discussion across the countriesMostly used to push information out to countries	Focused sharing of: Monthly updates on progress from teams Resources or articles of interest General questions and responses from teams that related to their work on the ground Summary documents on content harvested from in-country teams Tools teams created to measure and track	Limited use by county teams: Many were hesitant to share on our listserv as this was seen as a formal method requiring approvals
Facebook and Twitter	Open to all teams	Purpose: keep teams connected through quick reminders, shared questions, and foster connections	Tips on running in-country learning sessions Change ideas being shared at learning sessions Resources on tracking and keeping mother–baby pairs in care	Limited use and effectiveness; some noted these sites were not allowed in workplaces, others noted limited access to Internet prevented use
Knowledge harvestsUganda: February 2014January 2015Tanzania: April 2014Lesotho: September 2015	UgandaTanzaniaLesotho	Purpose: pool and analyze data in order to harvest evidence-based changes leading to improved PMTCT/NACS care	Outputs: Creation of guidance materials on tested changes that lead to improved processes of care based on evidence (known as change packages): Improving Retention of Mother–Baby Pairs: Tested Changes and Guidance from Uganda Improving Completeness and Accuracy of Data for Elimination of Mother-to-child Transmission of HIV: Tested Changes and Guidance from Uganda Improving Quality of Services Provided for HIV-positive Mothers and Their Babies at Routine Visits Tested Changes and Guidance from Uganda Tanzania PHFS Implementation Experience and Change Package https://www.usaidassist.org/toolkits/partnership-hiv-free-survival-learning-platform/tools	Worked well^a
Online learning platformSpring 2016	Open source	Purpose: to serve as a repository of results, tools, improvement stories, lessons learned, and evidence-based changes packages created as a result of teams from each country participating in PHFS	To help further understanding of the process and methods used within the PHFS, resources on the platform include country profiles, case studies, guidelines and standards, journal articles and presentations, reports from country learning, tools created as a result, and webinar recordings. Specific topics available: Reduction of transmission of HIV from mother to infant Keeping HIV-positive mothers and infants alive and in care Assessing nutritional status and proper categorization, counseling, treatment, and support of mother–infant pairs Tracking compliance with national PMTCT and maternal/infant health and nutrition guidelines https://www.usaidassist.org/toolkits/partnership-hiv-free-survival-learning-platform	Unsure of effectiveness: limited feedback from participants on usefulness

Abbreviations: CDC, Centers for Disease Control and Prevention; MoH, Ministry of Health; OGAC, US Office of the Global AIDS Coordinator; PEPFAR, US President’s Emergency Plan for AIDS Relief; PHFS, Partnership for HIV-Free Survival; PMTCT, prevention of mother-to-child transmission; Q&A, question and answer; QI, quality improvement; UNICEF, United Nations Children’s Fund; USAID, US Agency for International Development; WHO, World Health Organization.

^aOverall: In-person meetings enabled deeper, direct peer-to-peer learning. However, these face-to-face meetings are expensive, which is likely to be a limiting factor. Virtual exchanges provided constancy of communication between face-to-face sessions and a predictable rhythm of interactions between members. Timing and dosing of activities was considered.

In-Person Meetings (Regional and All-Country)

In March 2013, the PHFS was launched at a 4-day, all-country meeting by high-level ministry officials from the host country, South Africa. Through quantitative and qualitative feedback, participants noted this meeting served to unify and build relationships within and across country teams, creating the start of a network for learning. Country teams identified methods for and committed to knowledge sharing.

Two 2-day regional meetings of eastern and southern countries were held 6 months after the launch, hosted by Uganda and Mozambique. Postmeeting evaluations indicated these meetings provided further understanding of clinical and improvement knowledge, understanding of learning principles, ideas for improving processes, and a method for testing those ideas.

The final all-country meeting was held over 2 days in February 2016, hosted by Tanzania. Participants noted this meeting successfully showcased how teams turned what they learned throughout the partnership into changes and improvements along the PMTCT continuum.

Knowledge-Exchange Visits

Two knowledge-exchange visits took place (Lesotho to Uganda, and Kenya to Tanzania) in 2015. These visits had a heavy emphasis on deeper relationship building through smaller classroom discussions and site visits for real-time, hands-on learning. These visits led to deeper exchanges compared to larger multicountry meetings.

As a direct result of these in-person exchanges, programmatic changes were made in several countries.

Virtual Methods

Between March 2013 and March 2017, 12 webinars were held with a range of attendees from each country in addition to global partners. The majority of webinars (64%) were focused on sharing internal learning among countries. The remaining webinars focused on bringing external learning topics relevant to the PHFS to the country teams (eg, new community learning methods, updated WHO guidelines). During the initial evaluation after 6 and 12 months, all-country teams stressed the value and knowledge gained by participating in webinars, noting they wanted to continue this method of exchange.

Case in Point: Data Sharing

While strong efforts were made to garner MoH commitment from the outset, timely and free-flowing data sharing between countries remained a challenge for some countries due to sensitivity and reluctance without prior ministry-level approvals. The design of the learning network itself needed to adapt to the realities of trying to move data transparently across country borders. The use of a common set of indicators for comparison across countries, codesigned and adapted from existing country indicators and reporting systems, is ideal but very difficult to put into practice.

A listserv was established to provide a regular rhythm of information sharing, with over 96 e-mails/touch points both to and from participants in the first 9 months. Between March 2013 and March 2017, 15 newsletters were shared across the listserv to all participants (reaching 248 subscribers). A total of 120 Twitter and Facebook impressions were made between March 2013 and May 2014. These 2 social media platforms did not see significant growth or usage. After initial testing, both Twitter and Facebook were abandoned in favor of face-to-face communications, webinars, newsletters, and a PHFS mini-website. This online repository, launched in 2016 (https://www.usaidassist.org/toolkits/partnership-hiv-free-survival-learning-platform), helped disseminate learning across the participating countries and to the global community through 1900 pages views in the repository’s first year of operation.

Learning Outputs and Dissemination

Several outputs (products) of learning were created and are described in Table 2 and in-country papers in this supplement. Learning was also disseminated by country teams to the global community at national and international conferences. These outputs of our learning system lend evidence to our disciplined approach to learning and harvesting.

Country Learning: Improving Delivery of Care through Effective Implementation

Participants learned how to implement evidence-based interventions to improve delivery of care, adjusting for the contexts within each country. Countries learned from each other about what worked (and didn’t work) for each of the major care steps in the pathway to increase HIV-free survival and improve MNCH and nutrition of mother–baby pairs. Further details on country-specific learning and results are documented in individual country papers, JIAPAC-18-08-SA-1103, 1104, 1105, and 1106 in this supplement.

Lessons Learned: Designing and Convening a Multicountry Learning Network

We gained valuable insights on how to design a multicountry learning network from challenges we faced as well as successes. Through analysis of feedback from participants, implementing staff, global partners, and facilitators, top enablers of success were gleaned. These are listed in Table 3 and detailed below. We believe these lessons can be applied to other large-scale learning networks.

Table 3.

Top 5 Enablers of Successful Multicountry or Other Large-Scale Learning Networks.

(1) Local ownership and co-design to secure engagement and commitment (including commitment to data sharing)

(2) Clear delineation of roles with a dedicated knowledge management team to nurture learning

(3) Connection before content: nurture ongoing relationship and trust building

(4) Designing for value, ensuring continuous evaluation, and adaptation

(5) Head and heart: a combined data- and story-driven approach to learning

Local Ownership and Co-Design to Secure Engagement and Commitment (Including Commitment to Data Sharing)

The PHFS was designed by global partners and introduced to governments ahead of the launch, an approach that led to delayed traction in some countries and lack of full engagement by others. We learned that local ownership, and in our case ownership by the Ministry of Health (MoH), drives ongoing commitment. For learning network success, participants needed to be committed to the network and regular sharing.¹⁸ A consultative process with country (regional, state, and/or local) leaders around the design from the beginning and their heavy involvement in the set up and launch of a network can secure engagement. This engagement is also crucial to ensuring consistent participation in all sharing exchanges from the start. We recommend that a focal country (eg, MoH) point person(s) should lead, coordinate, and encourage stakeholders and facilitate internal/external sharing. The setup of this role and securing this commitment require a significant amount of time ahead of the launch of this type of undertaking.

One of the challenges faced in convening a multicountry or any large-scale network is who participates, how often, and whether there is a consistent core team participating from each participating group to advance each exchange. We recommend that core country (regional, state, and/or local) team members debrief any new participants ahead of exchanges so time is not taken away to reintroduce concepts and background information. This also underscores the need for proper documentation from the beginning.

An effective learning system also depends on the transparent flow of program data and a shared understanding that data will not be used without an explicit intention of learning for improvement (ie, not for judging/censuring health worker performance). We learned that country commitment to data sharing is needed but will only be given if trust exists (see Case in Point). The difficulties in sharing data for learning are well described.^20,21 JIAPAC-18-08-SA-1098 describes our particular data issues within this supplement.

Clear Delineation of Roles with a Dedicated Knowledge Management Team to Nurture Learning

Inevitably when forming any multicountry or similar large-scale learning network, the sheer number of stakeholders and/or technical supports, coupled with competing needs/priorities, can drown out the ultimate goal of learning/application of knowledge. External technical support teams working together through a coordinated collaborative effort could better support the country (regional, state, and/or local) lead. Delineation of roles shifts the role of external technical supporters from leading to facilitating an intervention like PHFS. In addition, establishing a point person(s) or dedicated team for knowledge management to nurture learning and to sustain ongoing sharing through regular check-ins and coordinating harvesting is critical. We recommend, what we termed as, a “honey-bee” cross-pollination approach be used by this team to personally connect, capture, and spread ideas across participants.

Connection Before Content: Nurture Ongoing Relationship and Trust Building

One of the most telling pieces of advice shared by a country team member is, “The most important thing is building a relationship. People don’t care how much you know, until they know how much you care. They have to trust you first.” The PHFS built relationships and trust needed for increased sharing of ideas and challenges through numerous prelaunch activities, one-on-one nurturing by the knowledge management team, and deeply rooted relationships already built within many of the participating countries with technical partners. For example, we conducted monthly one-on-one phone calls to each country team at the beginning of the initiative in addition to a regular rhythm of group exchange and support to help foster relationships, a sense of connection, and perceived value.

Designing for Value, Ensuring Continuous Evaluation, and Adaptation

Value drives commitment.^22,23,24 We started with a theory and framework for learning to help predict how value could be created and envision the transfer of knowledge. If the value of participating in any large-scale learning is not realized early on, commitment will waver. Creating value through thoughtful co-design of exchange methods with leadership and participants themselves (eg, country team members) was our top priority. We mapped out an initial strategy and process for learning together. Throughout the PHFS, continuous, real-time evaluation of our knowledge-exchange methods and continuous adaptation of our design was needed to meet evolving and emerging learning needs of participants. This responsiveness as well as our disciplined approach to learning was critical for keeping participants motivated and engaged throughout the initiative, taking account of various learning styles, cultural nuances, and potential barriers for sharing knowledge. Recommendations to consider when planning similar knowledge-exchange methods are provided in Table 4.

Table 4.

Top 10 Recommendations for Planning Knowledge Exchanges.

(1) Allow ample time for planning, preparation, permissions, and advanced setup for any type of exchange.

(2) Rotate hosts for exchanges to garner greater support and allow each participant to showcase their progress/lessons (either in-person or virtual).

(3) Choose the right participants for these exchanges based on their ability and likelihood of future influence and action (ensure facility- and community/family/patient-level participants).

(4) Consider where each team is in their implementation journey when deciding what type of agenda would allow for maximum cross-learning to happen.

(5) Craft vibrant exchange agendas to mix up learning techniques to ensure 2-way learning; use creative methods (knowledge cafés, “talk show” panels, peer assists, open space/agenda-free discussions). A mixed-method approach to match various learning preferences should be considered.

(6) Be flexible and allow the agenda to alter and change to match participants’ learning needs on the spot, as areas of interest will always arise once open conversations and connections start to happen.

(7) Account for any potential language and/or personal barriers ahead of time; give time for conversations/debriefs in local languages and smaller one-on-one discussions to eliminate unintentional isolation of participants who may not be comfortable with large group sharing.

(8) Ensure ample in-depth discussion on the “how” and practical skills sharing time at meetings (in-person or virtual) with less structured presentations (eg, provide time for teams to learn how to collect, analyze, and present data graphically).

(9) Factor in time for forward planning by allowing individuals and teams to process learning and turn learning into actionable steps for application postmeeting and determine how they will share what was learned with those not at the exchange.

(10) Create an “all teach all learn” environment so all parties engage together.

Head and Heart: A Combined Data- and Story-Driven Approach to Learning

We found that having a data-driven QI approach as the underlying approach to learning brought about new dimensions of cross-country exchange compared to other learning networks without the intentional QI focus.²⁵ This emphasis on data for learning coupled with knowledge-sharing approaches that centered on storytelling (emphasizing what was happening from a clinical, community, and personal perspective that made the data change, what worked, and what didn’t) provided opportunities for more influential learning.

Findings on Value of the Learning Network

When asked at the final meeting, all respondents agreed the network was highly valuable and that they would be part of this type of partnership and learning network again. Participants noted that the network provided a forum for joint problem-solving, new ideas, and innovation. It provided an opportunity for health system leadership of some countries to learn more deeply about implementation methods with which they had little experience (eg, by seeing how other country governments led and owned the approach). For others, the network reinforced effective QI strategies and allowed more experienced country teams to serve as mentors. Countries highly valued reflection with others on process gaps hindering care of mother–baby pairs. The enhanced perspectives of multiple countries allowed participants to see their own work in new ways and often prompted action or a shift in approach once teams realized their countries were not as far along as others in implementation. Through analysis of direct feedback from participants, the most valued aspects of the learning network were gleaned and are listed in Table 5.

Table 5.

Lessons Learned: Top 5 Most Valued Aspects of a Multicountry Learning Network (with Sampling of Participant Testimony).

(1) Peer learning and support: formal links to other countries, relationships built, trust fostered for deeper sharing (changes being made, results, progress)

➢ Engaging with partners from other countries; listening to other experiences; I feel challenged by successes from other countries.

➢ I have learned new thinking; most valuable was when we were sharing results and tested changes.

(2) Discussion time to debate/confront challenges and build solutions together

➢ It helped me to learn more about the performance of other countries, the challenges they encountered and how they managed to overcome them. It was an eye opener to me that we are really not performing well enough.

➢ Health center visit and rotation of sessions [during Uganda-Lesotho exchange] brought insight to the problems we have and the solutions to consider.

(3) Dedicated time for learning how others used the QI approach and methods

➢ Learning from other countries how they applied QI to improve PMTCT and nutrition aspects of care for mother and baby pairs; harvested change ideas that can be adapted to local context in my country.

➢ Seeing positive QI results from other countries; realizing that though we were coming from different countries, speaking different languages, the QI language was/is understood well by everyone.

➢ Coaching during site visits (Uganda-Lesotho exchange) helped me actualize the coordination of coaching as it occurred in the natural setting that will be applicable to my setting as well.

(4) Ideas to Action: Turning ideas and new ways of thinking into action

➢ Lesotho and Uganda teams both noted that as a direct result of what they learned from Tanzania and Mozambique, they would prioritize and alter ways to involve their communities in QI and mother–baby pair retention efforts.

➢ Lesotho held a team retreat to share learning and plan for immediate application of ideas gained after participating in an exchange with Uganda, altering their approach to QI capability building and MoH leadership.

➢ Tanzania shared learning/ideas they gained from their exchange with Kenya during a national learning network (they created to spread learning more widely).

➢ Site visits (during exchange visit) were insightful, and I will use the exchange visit format between demonstration and spread sites in Uganda so teams are able to share and motivate each other; and further MoH engagement in exchanges.

(5) Accelerating pace of learning

➢ It has been clear (from participating in the exchange) that focusing on improvement aims and not taking for granted each step in providing services can improve services and outcomes within shorter times.

Abbreviations: MoH, Ministry of Health; PMTCT, prevention of mother-to-child transmission; QI, quality improvement.

Additional Design Consideration for a Multicountry or Other Large-Scale Learning Network

Build-in a Formal Evaluation Method of Multicountry Learning Activities from the Start

We believe joint learning accelerated progress toward county-specific aims. Although difficult to quantify the learning network’s impact on pace and implementation effectiveness, participants noted their teams came away with a host of ideas and insights related to technical content and noted they valued the rapid dissemination of evidence and learning by participating in the multicountry learning network. Evaluators of multicountry learning networks note the difficulty in quantifying the “evidence” of cross-country learning,^7,8 and the information that is available is not often accessible for others to learn from.⁶ Although we collected qualitative and quantitative data at discrete points in time to measure value of (participant experience in) individual exchanges and to capture knowledge (learning, ideas) gained by participating, a formal, rigorous evaluation of the multicountry learning component was delayed due to partner and funding constraints. We believe feedback/data collected (on reach, value, and learning) are indicators not just of enhanced experience and learning but also of behavior change as a direct result of participating in the learning exchanges. Without a formal evaluation of the multicountry component, however, conclusions cannot be fully supported. Future studies of learning systems should build in formalized method of evaluating knowledge translation, behavior change, and results as indicated in Kirkpatrick levels of measurement²⁶ or Graham et al Knowledge-to-Action framework.²⁷ Measures could also include (1) value or impact of the engagement continuously over time and/or as reported by the same cohort over time and (2) a comparison between those sites participating in the multicountry learning network component and control sites not participating but still working to improve PMTCT and HIV-free survival. Evaluation of results of the PHFS overall are detailed in JIAPAC-18-08-SA-1102.

Future Applications of a Multicountry Learning Network Model

It has been suggested that by “expanding cross-organizational learning, health systems will be better prepared to address the major medium-term challenges that have emerged from recent QI experience in Africa.”⁶ The PHFS sought to create an expanded system of learning that could harness and capture knowledge to better prepare member countries to tackle current and future challenges. We believe the value demonstrated by this multicountry learning network could be replicated by others, leading to the accelerated pace of improving systems of care, whether PMTCT or other areas of care. Although true attribution cannot be made on direct effects of the multicountry learning component on clinical outcomes, what we can suggest is the strength in application of a common QI approach and joint learning. JIAPAC-18-08-SA-1100 further describes this approach in this supplement, noting similar trends in improvements despite the differences in country contexts, varying indicators, and other external influences. We recommend further studies compare learning systems currently being used in low- to middle-income countries^7,8,28 to highlight which components could work in different settings. Adaptation of collaborative learning efforts to the country context and government ownership needs close attention when designing multicountry learning networks.^3,28 As we discovered, the speed of progress will be affected by the various stages countries are in with respect to approach, government ownership, and buy-in of the value of a learning network.

While costs of establishing a similar model should be studied, our experience and other evidence point to the value of creating a QI-driven, multicountry learning network to tackle other topics.¹⁵ Linnander et al²⁸ demonstrated significant improvement over time among organizations that report sharing information compared to those reporting less sharing of knowledge. The member countries within the Joint Learning Network for Universal Health Coverage placed great value in hearing stories of members redesigning systems, and many have introduced new policies as a direct result of experiences learned from others.⁷ The HEALTHQUAL International’s All Country Learning Network noted their peer exchange as fundamental to building sustainability within countries through shared experience, knowledge, and expertise as well as reinforcing effective QI strategies and spread.⁸

Undoubtedly, there is significant value in any system that accelerates the transfer of new implementation knowledge to other countries undertaking similar programming. In addition, the existing PHFS multicountry learning network could serve as a platform for improving other services within these same countries, building on the trust and lasting relationships that have already been developed. Although new topics would bring new participants, the core foundation of this type of learning has been established within ministries and supporting partners that could benefit from similar work in the future.

Conclusion

The PHFS multicountry learning network was instrumental in accelerating learning about how to improve the delivery of HIV/AIDS, MCNH, and nutrition care within participating countries and in spreading this new implementation knowledge between countries. Local change ideas as well as the more formal collection of successful implementation ideas (change packages) were made available across the 6-country network through the in-person and virtual learning system. The network promoted trust and transparency that allowed teams and individuals to generate dialogue and ideas and reinforced effective methods to mainstream QI approaches to improving HIV/AIDS and nutritional care.

Footnotes

Acknowledgments

Ministries of Health from all 6 countries were leading participants in this multicountry learning network. We would like to acknowledge the role of these government leaders and managers from all health-care levels (subdistrict, district/regional, and national), health workers (community and facility levels), and community members (mothers) as well as technical support partner teams on the ground from all 6 participating countries in sharing insight, knowledge, and lessons learned. In particular, we would like to acknowledge Nneka Mobisson-Etuk for her leadership and role in operationalizing the design of the PHFS and network, Maureen Tshabalala and Sarah Olver for their key implementation roles on the knowledge management team, Angelina Sassi for project management and coordination as well as numerous others who were involved early-on as key members of the knowledge management team.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Anisa Ismail, MPH

References

90-90-90: An Ambitious Treatment Target to Help End the AIDS Epidemic. [Internet]. Geneva: UNAIDS; 2014. http://www.unaids.org/en/resources/documents/2014/90-90-90. Accessed January 17, 2017.

PEPFAR 3.0: Controlling the Epidemic: Delivering on the Promise of an AIDS-free Generation. [Internet]. Washington, DC: U.S. President’s Emergency Plan for AIDS Relief; 2014. https://www.pepfar.gov/documents/organization/234744.pdf. Accessed January 17, 2017.

Pakenham-Walsh

. Learning from one another to bridge the “know-do gap”. BMJ. 2004;329(7475):1189.

Wenger

. Communities of practice and social learning systems. Organization. 2000;7(2):225–246.

Leonard

Kumar

. The art of Knowledge Exchange: A Results-Focused Planning Guide for Development Practices. [Internet]. Washington, DC: World Bank; 2012. https://openknowledge.worldbank.org/handle/10986/11983. Accessed January 17, 2017.

Heiby

. The use of modern quality improvement approaches to strengthen African health systems: a 5-year agenda. Int J Qual Health C. 2014;26(2):117–123.

How knowledge sharing moves countries towards UHC | speaking of medicine. [Internet]. Speaking of medicine. 2017. http://blogs.plos.org/speakingofmedicine/2014/12/12/knowledge-sharing-moves-countries-towards-uhc/. Accessed January 17, 2017.

Agins

Birchard

Palumbo

. All country learning network: a peer-driven knowledge management strategy to build capacity for sustainable national quality management programs in low- and middle-income countries. 2013. http://www.healthqual.org/sites/default/files/ACLN_FINAL.pdf. Accessed January 17, 2017.

Shiffman

. Networks and global health governance: introductory editorial for health policy and planning supplement on the emergence and effectiveness of global health networks. Health Policy Plann. 2016;31(suppl 1):i1–i2.

10.

Works

. Accelerating differentiated care | global initiatives. [Internet]. Icap.columbia.edu. 2017. http://icap.columbia.edu/global-initatives/accelerating-differentiated-care/. Accessed January 17, 2017.

11.

Connecting the World Through Learning. 2006. 1st ed. Global Development Learning Network. http://www.gdln.org/files/gdln_brochure_-_en_0.pdf. Accessed January 17, 2017.

12.

Bardfield

Palumbo

Birchard

Geis

Agins

. Healthqual international All Country Learning Network (ACLN): peer-driven knowledge management strategy and community of practice to build capacity for sustainable national quality management programs in low- and middle-income countries. In: Proceeding of the 10th International Conference on Intellectual Capital and Knowledge Management and Organisational Learning. Kidmore End: Academic Conferences International Limited; 2013. p. 54–XII.

13.

Institute for Healthcare Improvement. The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement. IHI Innovation Series White Paper. Boston, MA: Institute for Healthcare Improvement; 2003.

14.

Murray

Pérez

. Informing and performing: a study comparing adaptive learning to traditional learning. Inform Sci. 2015;18:111–125. http://www.inform.nu/Articles/Vol18/ISJv18p111-125Murray1572.pdf. Accessed March 1, 2019.

15.

Franco

Marquez

. Effectiveness of collaborative improvement: evidence from 27 applications in 12 less-developed and middle-income countries. BMJ Qual Saf. 2011;20(8):658–665.

16.

Wells

Tamir

Gray

Naidoo

Bekhit

Goldmann

. Are quality improvement collaboratives effective? A systematic review. BMJ Qual Saf. 2018;27(3):226–240.

17.

Dixon

. Common Knowledge: How Companies Thrive by Sharing What They Know. Boston, MA: Harvard Business School Press; 2000.

18.

Wenger

McDermott

Snyder

. Cultivating Communities Of Practice: A Guide to Managing Knowledge. [Internet]. Boston, MA: Harvard Business School Press; 2002. http://cpcoaching.it/wp-content/uploads/2012/05/WengerCPC.pdf. Accessed January 17, 2017.

19.

Harris

. Freedom of the (Key) press: internet-based discussion groups. Comput Teach. 1993;52–55.

20.

Alter

Vardigan

. Addressing global data sharing challenges. J Empir Res Hum Res Ethics. 2015;10(3):317–323.

21.

Solberg

Mosser

McDonald

. The three faces of performance measurement: improvement, accountability, and research. Joint Comm J Qual Im. 1997;23(3):135–147.

22.

Groysberg

Lee

Price

Cheng

. The Leader’s Guide To Corporate Culture. Harvard Business Review. 2018. https://hbr.org/2018/01/the-culture-factor. Accessed March 1, 2019.

23.

Kofman

Senge

. Communities of commitment: the heart of learning organizations. In: Chawla

Renesch

, eds. Learning Organizations: Developing Cultures for Tomorrow’s Workplace. Portland, OR: Productivity Press; 1995:14–43.

24.

Edwards

. Democratic Discipline in Learning Communities: Theory and Practice. Lanham, MD: R&L Education; 2011.

25.

Institute of medicine (US) roundtable on evidence-based medicine. In: Olsen

Aisner

McGinnis

, eds. The Learning Healthcare System: Workshop Summary. New Approaches—Learning Systems in Progress. Vol 4. Washington, DC: National Academies Press (US); 2007. https://www.ncbi.nlm.nih.gov/books/NBK53496/. Accessed January 17, 2017.

26.

Kirkpatrick

. Evaluating Training Programs. San Francisco, CA: Berrett-Koehler; 2006.

27.

Graham

Logan

Harrison

. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006;26(1):13e2.

28.

Linnander

McNatt

Sipsma

. Use of a national collaborative to improve hospital quality in a low-income setting. Int Health. 2016;8(2):148–153.