Getting in the Loops: Using Adaptive Design Methods for Fall Armyworm Spodoptera frugiperda (JE Smith) (Lepidoptera: Noctuidae) Mitigation in Kenya,Nepal,and Bangladesh

Welcoming Process-Outputs as New Inputs (Feedback)

Feedback uses the output of a process-step as an input for the next iteration of that step. From the study data, design-looping afforded a kind of feedback that transformed the output-input loop into a “difference which makes a difference” (Bateson, 1972, p. 462) around how the design process operates. Design-looping included, but was not limited to,

Treating feedback as design-looping required stakeholders to welcome feedback as potentially process-transforming and to anticipate and incorporate such a possibility into the design process itself (e.g., in particular, imagining how video assets used in Kenya would remain relevant in other countries).

Respecting Collaborative and Adaptive Learning

Design-looping operationalized USAID’s (2017) collaborative, learning, and adaptive (CLA) framework. This points to times when stakeholders allowed a temporally or managerially prior step to be changed by the above process-outputs going forward. This principled allowance highlights CLA’s adaptive and learning aspects, to the extent that the design-process itself changes its protocols and behavior in light of that output.

From the study data, such adaptive learning—similar to Piaget’s notion of accommodation as a form of adaptation (see von Glasersfeld, 2014)—altered both the process’ concepts as new information emerged as well as the structures that generated those process-concepts in the first place (RQ1). In theory of change terms, accommodation (as adaptive learning) involves alterations not only to what a process has as its goal but also how the goal will be realized (Santana, 1995). It is necessary to emphasize this distinction because how a process is conducted integrally affects stakeholders’ participation (Daré et al., 2014).

Trusting Flattened Decision-Making Hierarchies to Support Participation

From the study data, design-looping strengthened the collaborative aspect of CLA precisely by flattening decision-making hierarchies and affording broader participation. Collaborative work generally implies a less steeply hierarchical, more egalitarian arrangement of agents within a knowledge- and work-production process (Andersson, 2009; Lutomia, 2019). From this study’s data, this meant recognizing collaborators at all point along the design-chain not simply as means or tools to an end but as participants with equal agency within a process (Lutomia, 2019; Naik, 2018). More broadly, the processes of participatory innovation design in this study afforded a sufficient variety of empowered voices, identities, and worldviews to capture both the educational protocol and relevant concerns around its depictions in its design (Bousquet et al., 2014; Douthwaite & Hoffecker, 2017; Lineberry, 2019; Mocumbe, Abbott et al., 2016) (RQ1).

Background

This case focuses on design-side looping during the production of an ICT4D educational animation commissioned to mitigate FAW’s effects. Titled How to Identify and Scout for Fall Armyworm (SAWBO, 2018), part of the rationale for commissioning this animation arose from a need for more effective educational materials and messaging around rapidly expanding FAW infestations in Kenya (Bello-Bravo, Huesing, Boddupalli, et al., 2018). Farm Input Productions Africa noted, “Previously, we had relied on traditional training under a tree with visual aids and practical demonstration. This was effective, but there were two critical gaps in understanding (1) how to distinguish the FAW from other caterpillars and (2) how to scout effectively” (Bello-Bravo & Pittendrigh, 2019b).

In early 2019, FIPS-Africa tested How to Identify and Scout for Fall Armyworm with 2,513 farmers across western and eastern Kenya. While the FAW animation filled the gaps noted above—especially by allowing real-time zooming with the animation otherwise “difficult to show in live-trainings or in a conventional video”—Bello-Bravo and Pittendrigh (2019b) also reports, “For the first time, our farmers really understood the point of scouting and were keen to run to the field and search practically for the caterpillar!”

Design-looping

Part of the success of the animation derives from its media, which employs generically appealing animated imagery (Bello-Bravo et al., 2010) voiced over in the locally most comfortably spoken participant dialect (Bello-Bravo, Dannon et al., 2013) and delivered via ICT in a facilitated discussion context (Bello-Bravo et al., 2019; Bello-Bravo, Nwakwasi et al., 2013). As noted in the Introduction, the animation itself represents a culmination of prior efforts to characterize a FAW scouting protocol (Prasanna et al., 2018), namely:

While the first two steps above involved considerable looping between project stakeholders (e.g., CIMMYT, IITA, and the funding agency) (Bello-Bravo, Huesing, Boddupalli, et al., 2018), the present study focuses on the third step only. We refer to the stakeholders of this “in-house” research phase of SAWBO’s fall armyworm animation as the “production manager,”“script-writers,” and “animators.” Nevertheless, this points to two aspects of scale (RQ3) and bi-directional learning (RQ2), where the “input” to the beginning of a project is already clearly laid out. In this case, the input for the project is one of several possible scientifically validated FAW mitigation techniques, specifically selected for its likely ease-of-use by end-recipients. As such, macroscale discussions and prior work affect the microscale work to be accomplished by narrowly parameterizing it. The critical importance of looping here “builds in” the ability to check back with the macroscale level of decision-making (RQ3) if needed. This is not always the case.

Like many artistic processes, the relationship between the animators (who physically realize the animation itself) and the script-writers (who refine an understanding of the animation’s intent) involves continuous dialog, revisiting, revision, and bi-directionality. After scripting and storyboarding the protocol:

The script-writers [hand] off the script to the animators, who turned the storyboards into animations. … If something’s not clear, [the animators] go back and forth with the script-writers, making suggestions a lot of the time about which direction to go with, which gets into the final animation (Production Manager Emails, 2020)

With respect to how decision-making occurs around which direction to go with:

Basically, it boils down to both of them collaborating to figure out a solution to whatever’s at issue, big or small. It’s not often that I get called in to settle things; we all trust each other to come up with good solutions (Production Manager Emails, 2020, emphasis added).

Other looping process occur informally.

This back and forth goes on formally at work, but also informally, over drinks after hours, through text messages, whenever’s convenient. Communications are constantly going back and forth to get what’s being animated to faithfully reflect the script (Production Manager Emails, 2020).

Here, the script represents the critical site where step-loops between the script-writers and animators intersect and interact. Where this genuinely involved looping, the interactive process meant that the animators’ outputs became inputs for further script modifications and the design process generally; that is, they actively contributed to what was being depicted not just how. This is an in-house version of the scaling capacity to check back with the project initiators noted above (RQ3). This is critical, because “sometimes a script-writer doesn’t even really understand what they have asked for until seeing it in animated form” (Production Manager Emails, 2020). This bi-directional iterative process links to the sense of accommodation as part of adaptive learning with the CLA framework (USAID, 2017).

There’s a lot of back-and-forth initially, roughing in what character frames we can reuse, ideas for modifications, etc. We have a great team, but it’s really about listening to the strengths and suggestions that each person brings and putting them into practice (Production Manager Emails, 2020, emphasis added).

The bi-directionality of feedback also encourages stakeholder buy-in generally. “For us, we have an ethos that it’s merit of the idea proposed, not who says it, that can have decision-making force” (Production Manager Emails, 2020). This increased buy-in helps produce a better end-product (RQ1):

So, no, it’s not like everyone can say anything all the time about anything. In- or out-of-house constraints can limit the range of suggestions, but around things where the floor is open, the best idea wins (Production Manager Emails, 2020).

Building in design-looping can afford “buffers” against later demands to drastically change the work product, such that (1) the cost-effectiveness of (re)producing media, even “from scratch,” is less infeasible, while (2) still assuring that the new effort still aligns the intent and output of the research process with the goals of the innovation overall (RQ1). This means there is less chance of having to “live with” errors, shortcomings, or simply missing newer information because there was no means to change the product; importantly, this will include the ability to scale the product after distribution (RQ3).

A long time ago, we realized the cost-effective advantages of animation over doing live-action video. And when it comes to having to modify (or redo) an animation, those cost-savings really increase. We can usually “recycle” large portions of the already animated material; sometimes, it only involves changing the voice overdubs. It’s a highly flexible strategy (Production Manager Emails, 2020).

At all levels of the design-process (both in-house and in base-touching with the video’s commissioners), trust played a key role: trust that each collaborative partner would follow-through on their parts of the process but also trust in the skills and competencies of each partner (Gambetta, 1988; Lutomia, 2019). While the scientific consensus on best-practices strategies for FAW mitigation highly constrained any variance in the final script, how the script-writer dramatized that consensus (and how animators realize that content) remained open.

Like with technical writing, we were given a pretty strict set of “steps” to present in the animation but had lots of leeway in terms of how we depicted those steps. There was always back-and-forth to make sure we’d stayed on the right track, but they trusted us to get it right (Production Manager Emails, 2020).

Background

This case focuses on design-side looping when translating and retooling the previously completed ICT4D educational animation for use in Nepal. In 2019, a collaborative CIMMYT team in Nepal began to respond to FAW’s then-recent emergence. Aware of SAWBO’s fall armyworm animation, the team requested linguistic and imagistic localization of the video for use in Nepal. Having built-in loops explicitly makes scaling (RQ3) and aligning this request with project goals (RQ1) possible. Once completed, the revised animation was initially shared on YouTube and viewed by 1,500 people and then subsequently aired on Nepalese TV stations during the second half of June 2019 to approximately one million viewers (Bello-Bravo & Pittendrigh, 2019a).

Design-looping

Translation rarely, if ever, involves a mechanical transposition of meaning from one language into another (Ajiboye, 2016; Bello-Bravo et al., 2023), especially when internationalizing learning materials for people with less access to technological, educational, and even linguistic resources (Bello-Bravo, Lutomia, Abbott, Mazur, et al., 2020). As such, one must translate culture, not simply language (Fernández Guerra, 2012), a process that Braçaj (2015) underscores as a most challenging part of translation. In fact, as part of adapting the FAW video for use in Nepal, a request was made to modify the imagery in addition to providing a local translation. Again, by building in loops, meeting this request becomes cost-effectively possible and feasible because resources are already in place (RQ3).

We have learned that translation is never only linguistic. In some contexts, it becomes necessary to make sure the content is both acceptable to the target population and remains accurate. A disease like tuberculosis may not have a specific local name as such, may be lumped together under one malady, or be framed and understood as a spiritual affliction. You cannot always just walk into a village and start promoting “germ theory” and expect people to automatically accept it. It does have to be explained in a way that is correct, but also in a way that the local population will understand it. Both are achievable (Production Manager Emails, 2020).

Dialectally localizing existing SAWBO educational videos is an integral part of SAWBO’s design process and media effectiveness; such translation enhances credibility (Hoffman, 2011). In this case, as in others, the process involved first identifying a local speaker fluent in the video’s original language (English) and the target language (initially Nepali, later others for use in Cambodia and Pakistan). The script and video were shared with the translator, who then drew on culturally local information to linguistically and experientially provide a recorded translation (on a mobile phone). The audio was then dubbed into the original video. This iterative back and forth (RQ3) mirrors the fine-scale iterations between script-writers and animation producers in the previous section.

Building-in and anticipating re-translation as part of design-looping affords widely scaling up the reach of individual animations at decreasing costs per unit (RQ3). Moreover, because generating a new translation of an existing video for use in a new linguistic area can be done quickly and easily (compared to generating a video from scratch), this dramatically increases the timeliness of the response. For example, the response time between the initial 2019 identification of FAW in Nepal (PQPMC, 2019) and the animation’s mass deployment in the second half of June was less than one planting season.

As during the Research phase, the value of trust (Gambetta, 1988) can be even more critical in this context. Project teams often draw on local speakers’ cultural and linguistic expertise with little to no fluency and few to no means for confirming the local translators’ work independently. In the present case, more than one speaker of Nepali was available to conference on the translation and enhance its accuracy by resolving any discrepancies (Gwet, 2014).

This built-in anticipation for culturally and linguistically (re)translating animations for different settings explicitly links to USAID’s (2017) collaborative, adaptive, and learning approach. “We absolutely rely on this kind of local expertise to finish a last-mile of the work that no one else already on the team can do. We couldn’t complete this work without looping them in” (Production Manager Emails, 2020).

Background

The previous cases in Kenya and Nepal involve FAW mitigation efforts during, or at the point of a, dangerously widespread extent of infestation. The present case in Bangladesh explores a preemptive deployment of the ICT4D animation to prevent FAW infestation from reaching dangerously high levels. As of the end of May 2019 in Bangladesh, in a news article summarizing a meeting by leading experts on FAW published by Jahan et al. (2019) on CIMMYT’s website, the authors reported, “Even though the level of infestation in Bangladesh is still relatively light, … crops have already been attacked in 22 districts within just a few months.”

For forestall FAW prophylactically, a team in Bangladesh—consisting of USAID initiative partners, CIMMYT, the Bangladeshi Agricultural Advisory Society (AAS), the Bangladesh Wheat and Maize Research Institute (BWMRI), Department of Agricultural Extension (DAE), and Michigan State University—collaborated to organize 283 showings of SAWBO’s FAW animation to 17,124 farmers over 38 days in six districts in Bangladesh (Bello-Bravo & Pittendrigh, 2019a; MSU, 2020). Besides views of the animation on YouTube, postings on Bangladesh’s sixty-four district websites were increasing by 500 views per day (Bello-Bravo & Pittendrigh, 2019a). In addition to this digital deployment of the animation, government-sponsored “training of trainer” efforts taught and resourced 400 extension agents for redistributing mobile ESD FAW content to communities across Bangladesh. Unlike the case in Nepal, no visual reconfiguration of the animation was needed, only looping for translation (into Bangla and other local dialects).

Design-looping

While SAWBO did not directly participate in the physical deployment of the video, many knowledge loops “with national, regional, and local actors key went into finding ways to make the video go viral” (Production Manager Emails, 2020). “A key challenge was coordinating, in the fastest and most efficient kind of way, identifying channels for getting the animation to the most people” (Production Manager Emails, 2020). These again reflect scaling capacities (RQ3) that help align the product with project goals (RQ1).

You can’t just put messages on blast and then cross your fingers. You need ways to loop back reports that tell you if what you’re sending out is getting to people. With digital means, like website hits, we can see numbers of people accessing the video, but this still doesn’t tell us what they’re doing with the message (Production Manager Emails, 2020).

As part of design-looping that takes outputs as difference-making differences as inputs (Bateson, 1972), this involved learning from end-user insights that changed how the (deployment) process worked (i.e., taking seriously suggestions by participants for times and places to show the video) (RQ2 bi-directional learning). In terms of adaptive learning, this meant taking information from locally informed people as alerts for “upstream” managers around opportunities, channels, and resources that would not have otherwise been visible to managers (RQ2 increased participatory access). This resulted in high message exposure rates (as reported) for comparatively low deployment costs. “The project leaders [who commissioned the animation] had an idea, when they started, what some of the outlets for distributing the animation would be, but there was a process of learning overall what other outlets existed. Local people were key to that” (Production Manager Emails, 2020). This design-looping for deployment allowed goal-setters and deployers to be resilient, flexible, and adaptive (Gunderson, 1999) about meeting the situational complexity of delivery on the ground with a sufficient variety of strategies that matched project goals with project efforts.

At the end of the day, you never really know where opportunities will arrive. You just have to be, I don’t know, open, attentive … keep your eye out for things that might help, even if you didn’t know about them in advance—and then being able to flex with it when you find them. It’s one of the neater things that can happen. Usually really feels like a breakthrough (Production Manager Emails, 2020).

Feedback

Design-looping describes a particular type of organizational infrastructure and attitude toward project outputs that allow those outputs to be taken up as information learned. This learned information has the potential to change either the organization, its processes, its goals, or all of these (Barnaud et al., 2014; Gunderson, 1999). Although this kind of information learned classically describes a scientific attitude toward experimental empiricism (Fischer, 2014; Merton, 1957; Valenzuela & Soriano, 2005), it also reflects the emergent processes of artistic creativity, revision, and reimagining that can occur as part of goals-oriented processes at both the personal and organizational levels (Barreteau et al., 2014; Bruner, 1962; McMahon & Ford, 2013). Richmond (1984) expressly argued that scientific and artistic approaches are “functionally interdependent” (p. 81); indeed, both domains bulk large when designing artistically realized embodiments of scientifically validated best-practices for ICT4D educational animations.

It is necessary to stress this information learned, because organizational feedback is more often used to neutralize or dampen potential needed changes in order to maintain its current behaviors, goals, and processes (Moradi et al., 2021; Munck Af Rosenschöld et al., 2014). Instead, design-looping (as in both art and science) captures a commitment to an end-product that agrees with reality, that matches the observed facts on the ground. In this way, it motivates an organizational commitment to ensure that innovation outcomes ultimately align with intended project goals. This commitment involves an organizational willingness to be “surprised” by a given outcome, for example, to set aside what Gunderson (1999) called a “spurious certitude” in order to adaptively learn and correct any earlier misunderstanding or inadequate understanding of what is needed to arrive at a given organizational goal.

Listening

Such feedback also involves the willingness of project stakeholders (but leaders and managers especially) to listen to and give credence to unexpected and unwelcome information and reports from non-leaders (Andresen & Potter, 2017; Jordan et al., 2017). While this echoes models for global-scale, community-based/participatory change as well (Barnaud et al., 2014; USAID, 2017), it also describes another instance of the accommodation of outputs as inputs that make a difference for collaborative and management processes overall (Folke et al., 2002; USAID, 2017).

Listening manifests clearly in all three cases above: during Creation, “We have a great team, but it’s really about listening to the strengths and suggestions that each person brings and putting them into practice” (Production Manager Emails, 2020); during Translation, “We absolutely rely on this kind of local expertise to finish a last-mile of the work that no one else already on the team can do. We couldn’t complete this work without looping them in” (Production Manager Emails, 2020), and during Deployment, “The project leaders had an idea, when they started, what some of the outlets for distributing the animation would be, but there was a process of learning overall what other outlets existed. Local people were key to that” (Production Manager Emails, 2020). Each of these “moments” reflect where the RCDL processes changed as a result of listening to feedback.

Listening also has affinities with design-based research (DBR). DBR uses actual learning environments to allow researchers an iterative and “systematic tweaking of theoretically inspired aspects of the learning environment” (Barab & Squire, 2004, pp. 156–157). Here, the “theoretically inspired” aspect refers to whatever central learning phenomenon the DBR researcher is examining while iterative (looping) research tweaks that bring to light that phenomenon’s empirical characteristics. In this study, the “theoretically inspired” aspect referred to iteratively tweaking an animation’s esthetic and linguistic characteristics (sometimes based on feedback from “in the field) such that it ideally embodied the FAW-control best-practices protocol.

Abstraction

Conceptually, a basic abstraction had already been accomplished by boiling down the situational complexity of FAW mitigation into an actionable scouting and spraying protocol for farmers (Bello-Bravo, Huesing, Boddupalli, et al., 2018; Prasanna et al., 2018). The task then was to translate that protocol into a culturally and linguistically comprehensible and appropriate ICT4D animation accessible to and useable by end-users (Bello-Bravo & Pittendrigh, 2018; Fernández Guerra, 2012; Hoffman, 2011).

Translating non-generic nouns (like “fall armyworm”) into languages without any available analog presents the challenge of either introducing new vocabulary into the language or attempting to supplant some already-extant generic term (e.g., “worm”) with a more exact term. In this case study’s videos, this was overcome by accompanying the audio description with graphic images of fall armyworm and its damage to maize leaves, consistent with the enhancement media learning.

For multimedia learning in general, this kind of redundancy “is not repetition, but rather … a means for abridging time in reaching one’s objective” (Nardone & Portelli, 2007, p. 928, paraphrasing Bateson et al., 1956) in this case, abridging time by simultaneously presenting information on FAW in multiple sensory channels. However, while the use of multimodal imagery and sound to convey information generally heightens confidence that the correct message will be received and learning gains achieved (Ljubojevic et al., 2014; Mayer, 2002; Moreno & Mayer, 2002), in the present case, the images and words for FAW are likely being linked together for the first time for video recipients. A verbal/visual redundancy is helpful (if not essential) for making this linkage, without which efforts to scout and spray for the specific pest in question cannot occur.

Stance

The “movement” of translation often seems horizontal—from one language to another. This horizontality links to a committed attitude or stance for interpersonally non-hierarchical relations between collaborators, like the equally empowered participants in design-based research and stakeholders within companion modeling (Easterday et al., 2014; Étienne, 2014; Lutomia, 2019; Lutomia et al., 2020; Madela, 2020). In other words, the sorts of techniques, strategies, and mutual respect for cultures that verbal translation needs to negotiate and navigate a text from one language to another become apparent as qualities of interactions between stakeholders in an RCDL theory of change approach for design-looping.

Fundamental principles shared among these several approaches include cross-disciplinarity, the “formalization of a diversity of viewpoints, scientific knowledge considered as one point of view among others in the consultation, [and] priority on the iterative consultation process rather than on its products” (Bousquet et al., 2014, p. 4). This is not a prioritization of process to the detriment of product but a recognition that how one conducts a process affects the outcome (MacLean, 1962; McLuhan, 2009).

Because “every stakeholder in a social system has his (sic) own view of the reality of the system … built up on what he (sic) has experienced during his (sic) life trajectory in the physical and social space” (Bousquet et al., 2014, p. 2), it becomes necessary to respectfully negotiate and navigate a beneficial diversity of worldviews on a project (Bello-Bravo, 2020; Levett-Olson, 2010), to translate the values and intentions of all participants for the benefit of the endeavor. One cannot rely on this to happen fortuitously or automatically but requires a committed stance to it in advance (Daré et al., 2014). This also underscores once more the vital importance of trust (Barreteau et al., 2014) and dialog (Barnaud et al., 2014) among stakeholders—as an open, mutually respectful, bi-directional conversation between the life-domains and experiences of stakeholders (Bakhtin, 1981) rather than a dominating or top-down communicative approach (Fry & Souillac, 2017; Nelson et al., 2017). In the present case, this stance is signaled by, “We have a great team, but it’s really about listening to the strengths and suggestions that each person brings and putting them into practice” (Production Manager Emails, 2020).

Process Buy-In

Beyond the self-evident successes suggested by these statements, they are also notable for an enthusiasm that seems to parallel the Kenyan farmers’ eagerness. While pride at sense of a job well done can powerfully drive team members’ engagement, buy-in, and work commitment (Jansen, 2010), seeing the consequences of one’s contribution to a process can also generate work commitment, group cohesion, and closer alignments of work performance with work policies (Bateson, 1972; Cheng, 2005; Henningsen & Henningsen, 2018). This helps to make work seem meaningful and not alienating (Bailey & Madden, 2016; Soffia et al., 2022). Feedback, listening, and a committed stance to less hierarchical participation by stakeholders help support and empower these kinds of process outcomes (Bailey & Madden, 2016; Barnaud et al., 2014; Daré et al., 2014).

Design-looping powerfully opens up and potentially enables all stakeholders to feel this sense of having contributed a difference that made a difference (Bateson, 1972). One of the arguments for front-end focus grouping and participatory inclusion of end-users in the decision making and design processes for innovations is precisely that they may be more inclined to use the innovation simply because of participating in its development (Cheng, 2005; Mocumbe, 2016; Revere et al., 2014; Sanders, 2008).

Organizational Learning

Looping in this sense enables bi-directional learning contexts not only for managers (when they give credence to, and accommodate, new information or work outputs as inputs) but also for non-leaders (as they realize that their contributions, skills, and insights are not just acknowledged but valued as inputs to the project as a whole) (Naik, 2018; Whiting et al., 2008). As the decision-making hierarchy becomes flattened in collaborative, and especially scientific collaborative settings (Lutomia, 2019), the importance of looping increases because it opens information channels otherwise not available to decision-makers (including critical participatory commitment channels for shepherding any ongoing process-oriented goals and goal-oriented processes). The result—as the hoped-for end of all innovation diffusion (Rogers, 2003)—is an increased likelihood of innovation diffusion efforts concretely matching that innovation’s goals.

While the notion of learning presents an intractable number of definitions (see discussions in Alexander et al., 2009; De Houwer et al., 2013), De Houwer et al.’s (2013) functional definition positions learning as “changes in the behavior of an organism that are the result of regularities in the environment of that organism” (p. 633). Relatedly, Hollingworth (1932) underscores faster response times to repeated (familiar) stimuli as evidence of learning. As such, learning involves not only individual people’s changes of behavior due to experience (De Houwer et al., 2013; Lachman, 1997) but bi-directional learning within the (workplace) environment itself. While the above-mentioned stance of trust, emphases on process, and flattened hierarchies draw on design-looping to empower a willingness to change behavior, processes, and even goals for the work, increased efficiency also can enter into the work process environment when the faster response times to familiar situations occur. “Eventually, pretty much with any project, we get into this flow [of interactions], almost like a habit, that just makes things move forward really smoothly and efficiently. I really like the work then when that’s happening” (Production Manager Emails, 2020).

Ultimately, a willingness to learn (on the part of a person, a process, or an organization) may be the most critical factor in all of this. This willingness can be built in to a design process by a principled commitment to a “priority on the iterative consultation process rather than on its products” (Bousquet et al., 2014, p. 4). For example, while the characterization and utility of feedback loops in engineering, organizational, and psychological contexts are long-established (Ashby, 1957; Bateson, 1972; Horowitz, 1984; Scott & Davis, 2015), unless that process commits to putting feedback to use through a collaborative and adaptive “difference which makes a difference” (Bateson, 1972, p. 462) approach, then the learning called for by USAID’s (2017) CLA will not occur (or will occur only by accident, unintentionally).

Pragmatically, this willingness to learn helps buffer against the unforeseeable. That is, Cooper et al. (2002) acknowledge that because “we have yet to discern how to systematically extract and disseminate management lessons as we move from project to project, and as we manage and execute portfolios of projects” (p. 213), they advocate for “learning to learn” (p. 213). In practice, this means turning the lens of the CLA process (USAID, 2017) on itself. Learning to learn in the present cases of this study involved a principled willingness to reflect on the premises and assumption of any current design process while remaining open and attentive to unanticipated and unforeseeable alternatives. “You just have to be … open, attentive … keep your eye out for things that might help, even if you didn’t know about them in advance” (Production Manager Emails, 2020). This involves a “looking for opportunities” even in crises (Brubaker et al., 2021).