Lifewide or School-Only Learning: Approaches to Addressing the Developing World’s Learning Crisis

Abstract

This cluster randomized controlled trial tested the impact of school-only and lifewide-learning (LWL) approaches to supporting early-grade learning over 2 years in rural Rwanda. We compare school-only and LWL treatments with a business-as-usual control condition and with each other. Schools in both treatment groups received reading materials and teacher training. LWL villages also received support to enrich home and community literacy ecologies. Student reading assessments, administered across 21 sectors (analogous to U.S. school districts), showed that both treatments positively impacted learning. LWL produced a greater impact, particularly in oral comprehension, reading fluency, and reading comprehension. However, nearly one third of the students lacked basic skills at endline, indicating that further efforts are needed to address the learning crisis in the least-developed countries.

Keywords

developing world lifewide learning literacy randomized controlled trial Rwanda

In 1990, for every 100 primary school–age children enrolled in school in the least-developed countries (LDCs), another 71 were not enrolled (World Bank, 2017).¹ By 2013, this ratio had improved dramatically, with only 18 out-of-school primary-age children for every 100 children enrolled. A number of interrelated factors contributed to this enrollment surge, including revised rate-of return analyses embraced by the World Bank (Lockheed, 1990), the Education for All movement (UNESCO, 2000), and the Millennium Development Goals (http://www.un.org/millenniumgoals). With the enrollment crisis abating, however, a “learning crisis” (UNESCO, 2013) has emerged, wherein children across LDCs are failing to learn to read despite several years of school attendance (Gove & Cvelich, 2011).

Virtually all efforts to address this learning crisis in LDCs by specifically targeting children’s reading development have been school-centric, focusing on teacher training, curriculum reform, textbook revisions, and other school-bound factors (Kim, Boyle, Zulkowski, & Nakamura, 2016). Consequently, these efforts have ignored the majority of children’s waking hours, which are spent outside school. This is despite the widespread recognition that children’s lifewide experiences—their experiences both inside and outside school—impact their learning (Hess & Holloway, 1984; Snow, Burns, & Griffin, 1998). In this article, we report the results of a randomized controlled trial in a rural area of one LDC, Rwanda. The trial tested the relative effects of two approaches to address the learning crisis: a school-only (SO) approach and a lifewide learning (LWL) approach that combined school-based activities with home and community involvement to support primary students’ early-grade reading development.

The Learning Crisis in Least Developed Countries

The 369 million children younger than 15 years in LDCs face considerable obstacles in securing a quality primary education. Basic necessities, taken for granted in developed countries, are often luxuries for these children. Approximately 22% of the population are undernourished, and only one out of every three households has access to electricity (World Bank, 2017). In 2010, only three out of every five adults in LDCs could read a short sentence, and with little to read, those literacy skills often go unused (UNESCO, 2015; World Bank, 2017).

To get to primary school, LDC children—particularly the 68.5% who live in rural areas—often must walk long distances over difficult terrain. Primary schools are underresourced, with an average of 40 pupils per teacher and fewer textbooks than children, forcing many children to share one textbook. In sub-Saharan Africa, only half the schools have toilets or other basic sanitation facilities (UNESCO, 2016). Only 77.8% of teachers are trained, and chronic student and teacher absenteeism is common (World Bank, 2017).

Merely enrolling in a school in no way guarantees that children will complete their primary education. In 2013, the cohort survival rate to the final year of primary school was 53.3% in LDCs (World Bank, 2017). A comparison of the number of LDC students enrolled in the first level of primary school, Primary 1 (P.1), in 2008 (31.2 million) and, 5 years later, in P.6 in 2013 (11.5 million) suggests that the 53.3% survival rate may be an optimistic estimate. The persistent “early-primary bulge” in enrollments observed across the developing world over the past decade suggests that millions of children spend several years stuck in early primary, after which their formal education simply ends (Crouch & Merseth, 2017). Reading assessments in dozens of countries indicate that little learning happens, as hundreds of millions of children spend several years in primary school without acquiring basic reading skills (Gove & Cvelich, 2011). For everyone involved—governments, teachers, children, families, and communities—the economic and social costs are enormous, with grim implications for poverty reduction, economic development, and engagement in a world ever more connected through digital literacy. Given that LDC children struggle with even basic reading skills, we focus our study on early reading development.

Factors Influencing Reading Development

Research conducted in the developed world over more than a half century has conclusively demonstrated that children’s LWL experiences—both in school and in homes and communities—affect their reading development (Heath, 1983; National Institute of Child Health & Human Development [NICHD], 2000; Snow et al., 1998). Despite disagreements over the relative importance of each setting, there is no question that both contribute to children’s learning. Both school- and home-based efforts provide plausible paths to improving student achievement. Accordingly, educators in the developed world focus on children’s LWL experiences in order to improve learning and school achievement.

In contrast, educators seeking to improve learning in the developing world focus almost exclusively on improving schooling, without paying attention to home- and community-based learning opportunities. Several reviews and meta-analyses summarizing these interventions in LDCs have found inconsistent effects but do not delve into the heterogeneity of effects (Ganimian & Murnane, 2016; Kim et al., 2016; McEwan, 2015). For example, researchers in Burkina Faso provided schools with better infrastructure and materials (e.g., separate latrines for boys and girls, textbooks) and found improvements in enrollment and learning, yet a similar intervention in Niger did not produce a similar impact (Ganimian & Murnane, 2016). A meta-analysis of 77 developing world studies (McEwan, 2015) found significant but very small mean effect sizes for school-centric interventions such as teacher training, class size reduction, or provision of instructional materials. Despite the best efforts of international educators, no school-based intervention has demonstrated consistent gains that adequately address the learning crisis in the developing world.

A small body of research indicates that even in LDCs, where both books in the home and readers to read those books may be rare, the literacy ecology of the home significantly correlates with children’s reading achievement (Chansa-Kabali, Serpell, & Lyytinen, 2014; Dowd, Wiener, & Mabeti, 2010; Friedlander, 2013, 2015; Wagner, 2018). However, only one intervention involving systematic efforts to improve the quality of learning experiences throughout a child’s day and year (Dowd et al., 2017) has been evaluated in LDC contexts.

Literacy Boost, designed and implemented by the international nongovernmental organization Save the Children, follows an LWL approach to support children’s education (Friedlander, Dowd, Borisova, & Guajardo, 2012). An LWL approach maximizes learning opportunities not only during a child’s school day but also during every waking moment in the child’s life. The intervention, described in greater detail below, includes both in-school and out-of-school components designed to enhance children’s learning opportunities at school, in their homes, and in the wider communities outside school. Impact evaluations of Literacy Boost, written by Save the Children researchers, report generally positive findings (Dowd et al., 2010; Guajardo, Jacob, Boneti, Kumar, & Isaac, 2016). However, the evaluation designs prevent analysis of the separate contributions made by the in-school intervention alone versus the LWL approach. This is a critical gap in our knowledge of how we should address the learning crisis in LDCs and the broader developing world. As the recent Landscape Report on Early Grade Literacy (Kim et al., 2016) states, “Given that school-based approaches alone have not proved sufficient to reach the goal of widespread reading proficiency among early grade children in many developing countries, broadening the scope of literacy interventions beyond the school is a necessary step” (p. 58).

Opportunities for Learning: School-Only Versus Lifewide Learning Approaches

Efforts to improve primary schooling in LDCs have largely been driven by economic rationales relying on production functions that specify tangible and measurable inputs to produce desirable outputs (Friedlander, 2015; Psacharopoulos & Patrinos, 2004). For decades, schools have been the nodes through which inputs such as teachers, desks, and textbooks were delivered in order to produce improvements in outputs such as human capital and economic growth (Heyneman, 2005). This orderly delivery system does not match many contexts in LDCs, where the majority of people live in rural areas, connected by unpaved and at times impassable roads. To some extent, it is understandable why educational interventions in LDCs have focused almost exclusively on school-based inputs. Villages generally lack connections to electricity, water, and telecommunications; many adult villagers are functionally illiterate, having little need to practice their reading and no materials to read. The language spoken at home may not match the language of instruction in primary schools, if it has a standard orthography at all. Perhaps due to these and other challenges, educators have hesitated to intervene in villages, where the impact on children’s learning may be diffuse, unmeasurable, and possibly nonexistent.

Yet children in LDCs continue to enroll in primary school, spend too many years in the early primary grades, and eventually drop out before completing primary school. The hypothesis driving our study is that children’s learning in LDCs will be meaningfully augmented by an LWL approach, which utilizes children’s full waking day, rather than an SO approach, which only supports classroom learning. If students in LDCs attend school for 4 hours per day (Ganimian & Murnane, 2016) and instruction occurs for a maximum of 180 days per year (Schuh Moore, DeStefano, & Adelman, 2011), time in school accounts for less than 15% of children’s waking hours. Figure 1 graphically depicts the stark difference in potential impact between an SO and an LWL approach. In an SO approach, learning opportunities are only possible during the 15% of children’s time when they are in school, which equates to a mere 2 hours per day, on average, over the course of the year. In contrast, an LWL approach assumes that learning opportunities are omnipresent and seeks to improve those opportunities both in the classroom and during the 12 waking hours a day, on average, that children spend outside the classroom.

Figure 1.

Potential learning impact of school-only and lifewide learning approaches.

Despite the learning potential of an LWL approach, it is uncertain whether the implementation of interventions supporting learning directly in homes and communities can surmount the many and complex challenges of operating outside the established schooling system across LDCs. And even if implemented, it is far from certain that a more comprehensive approach will improve student outcomes beyond SO efforts. The purpose of this study, therefore, is to answer the following questions on implementation fidelity and treatment impact:

Research Question 1: Were the lifewide learning and school-only interventions implemented as intended?

Research Question 2: (a) What is the effect of a life-wide learning intervention on early primary students’ literacy development in rural Rwanda? (b) How do these effects compare with the effects of a school-only approach? (c) What is the impact of both treatments compared with a control group?

Study Setting: Rwanda Overall and Study District

Despite remarkable gains in the two decades since the 1994 genocide, Rwanda is still classified as an LDC: In 2012, 74% of the 10 million Rwandans lived in rural areas, and 3 out of 4 Rwandans’ principal employment was agricultural. Although the average years of schooling for individuals 25 years and older was only 3.8 in 2012, approximately 68% of Rwandans had at least rudimentary literacy skills (National Institute of Statistics of Rwanda [NISR] & Ministry of Finance and Economic Planning [MINECOFIN], 2014b). Students spent an average of 8.6 years in school, but only 34.7% of students reached the sixth and final grade of primary school according to 2012 statistics, indicating that many students repeat several years of primary school but never complete their primary education (World Bank, 2017). The drop-out rate from the first 5 years of primary school averaged 16.3% in 2012; another 12.5% of primary school students repeated a year (World Bank, 2017).

A National Focus on Literacy and a Reading Culture

In its policy document Vision 2020, the leaders of Rwanda envisioned a modern, middle-income country with a knowledge-based economy (MINECOFIN, 2000; Williams, 2017). To achieve this by the year 2020, the government acknowledged the necessity to encourage improvement in the culture of reading across the country. Educators in Rwanda have expressed their intention to use the published research on reading and literacy to promote early acquisition of reading skills. For example, the 2013–2017 Education Sector Strategic Plan calls for training teachers in “effective reading classroom practices” and creating “evidence-based reading instructional materials” (Rwanda Ministry of Education [MINEDUC], 2012, p. 53).

The limited existing research on Rwanda’s culture of reading points to both opportunities and challenges in promoting reading achievement. Families and communities highly value reading, yet there is no culture of reading, and few reading materials are present in any language (Holland, 2012; Ruterana, 2012). Lackluster reading scores on nationally representative reading assessments corroborate the findings concerning an absence of a reading culture (DeStefano, Ralaingita, Costello, Sax, & Frank, 2012).

Administrative Organization of Rwanda and the Study District

Rwanda is a small, landlocked country in the Great Lakes region of eastern central Africa. It is known as “the land of a thousand hills,” with villages scattered across an undulating landscape. The country is made up of five provinces. These five provinces are divided into 30 districts, which are further divided into sectors and villages.² It was across one of these districts that our study took place. Figure 2 shows the nested nature of the administrative divisions in Rwanda, and the population size at both the national and study district levels.

Figure 2.

Administrative Schematic & Populations at the National & District Level.

Rwandan Primary Education

The Rwandan school year begins in January or early February and ends in October or November. The school year consists of roughly 3-month-long trimesters, with a few weeks of holidays between trimesters. Children enroll in P.1 (equivalent to Grade 1 in the United States) once they cross their seventh birthday. Primary school consists of six levels (P.1–P.6). Students in P.1 to P.3 receive instruction in Kinyarwanda, with English as a separate subject. Beginning in P.4, teachers use English as the language of instruction. Primary teachers are subject teachers, meaning that they move from class to class in order to teach various subjects. At the end of P.6, students sit for the national primary school–leaving examination. The exam scores decide the next type of school—secondary or vocational—in which P.6 graduates may enroll.

The education system in Rwanda has both centralized and decentralized elements. The centralized policies are set by MINEDUC and its subsidiary, the Rwanda Education Board. The Rwanda Education Board decides on the curriculum, textbooks, and school-leaving exams for the entire country. Direct supervision of primary schools is decentralized, with one district education officer overseeing a team of sector education officers.

On average, there are five villages for every one primary school (Rwanda MINEDUC, 2016). School catchment areas are not defined, and there are no regulations about which primary school children must attend. That is, each family chooses where to enroll its children, regardless of the village, sector, or even district in which the family lives. The right side of Figure 2 depicts how families living in any village may send their children to any school, regardless of where that school is located. The lack of specific school catchment areas means that some children live in one sector but attend school in another sector.

The Literacy Boost Program

The goal of Save the Children’s Literacy Boost program is to improve children’s reading development by enhancing their LWL opportunities—that is, both in school and in their homes and communities. Its innovation is found in its active engagement of family and community members in supporting children’s literacy development, something commonly ignored by other education interventions in LDCs (Kim et al., 2016). To implement Literacy Boost, country staff select and adapt a variety of activities and techniques from the Literacy Boost toolkit (Dowd et al., 2017)

Literacy Boost supports children’s reading development by providing book banks filled with reading materials and through a set of core activities to train teachers and to engage children in literacy in their homes and communities, as seen in the Literacy Boost logic model displayed in Figure 3. Implementation teams use a toolkit that describes the reading materials and a set of core activities to implement the intervention. Each implementation team adapts Literacy Boost to better fit the culture and language of the region in which the program is implemented.

Figure 3.

The Literacy Boost logic model.

Reading Materials

As seen in Figure 3, both the School and the Home and Community intervention domains include providing reading materials through book banks. Book banks contain a collection of local language, child-friendly reading materials. The Literacy Boost toolkit provides implementers different techniques to create content to fill the book banks, since professionally published children’s books written in the local language(s) are rare in many LDCs (UNESCO, 2016).

Core Activity: Teacher Training (Lifewide Learning and School-Only)

The only Literacy Boost activity within the school domain in the Literacy Boost logic model is teacher training. The goal of the teacher-training activity is to improve classroom reading pedagogy. To achieve this, implementers provide in-service training sessions to all early-primary teachers over the course of an academic year. The sessions cover skills that have been shown to be causally related to success in reading in English (NICHD, 2000). The toolkit specifies nine 1-day training sessions, including creating a child-friendly, print-rich classroom; formative assessment; addressing language issues in the literacy classroom; phonemic awareness; letter knowledge, vocabulary; reading fluency; reading comprehension; and a final concluding session. Each session follows roughly the same set of activities: reflection on the previous training, an introduction to the session topic, a model lesson, formative assessment, lesson planning, and reflection on the day’s session. Training sessions occur at regular intervals throughout the year, providing teachers up to 4 weeks between sessions to use their new pedagogical approaches in their classrooms.

The creators of Literacy Boost designed the teacher training to adapt to local needs and context in four ways: first, by helping teachers incorporate new skills to better teach the government-mandated curriculum; second, by meeting the needs of learners whose family may not speak the language of instruction at home; third, by highlighting content that implementers may want to alter for cultural relevance; and fourth, by encouraging implementers to change the order of the sessions, add new sessions, or combine sessions to meet the local needs of teachers.

The teacher-training sessions involve active learning, maintain coherence in format and content from session to session, provide sufficient time for teachers to learn about and practice the skills and concepts covered by the training, and require teachers’ collective participation. All these characteristics are core features of high-quality teacher professional development (Evans & Popova, 2015).

Core Activity: Reading Awareness Workshops (Lifewide Learning)

The topmost activity within the Home and Community domain of the logic model (Figure 3) is reading awareness workshops. In this activity, implementers invite adult family members of early-primary students to participate in a series of seven workshops. During the workshops, family members learn about the children’s language and literacy development and practice simple activities to encourage learning at home. Three of the seven sessions focus on shared reading. This is a tacit recognition of the fact that the common developed world tableau of a parent reading a storybook to a child is not a common practice in places where storybooks are few, electricity nonexistent, and the habit of shared reading a rarity (Heath, 1983). As with the teacher-training activity, implementers adapt the workshops to the local context and culture, adding to or modifying the content of the sessions to better meet the needs of the community.

Each workshop takes place in the local village, where family members can easily attend. Workshops last for approximately 90 minutes and are led by Save the Children or partner staff members or by a cadre of trained volunteers who live in or close to the village. Volunteers receive remuneration if allowed by local norms and customs. All workshops generally occur over 2 to 3 months. Implementers encourage workshop participants to practice what they learn in the workshops and return to the next workshop ready to discuss their successes and challenges with the previous week’s content before new material is presented. The workshops also provide attendees with techniques for supporting children’s oral language development, thereby enabling family and community members who might struggle to read themselves to have an important and recognized role in children’s learning.

Core Activity: Reading Clubs (Lifewide Learning)

Reading clubs are village-based gatherings for children that meet weekly. Led by a trained local volunteer, children who attend reading clubs participate in a variety of activities that support reading development, including listening to read-alouds; playing games that involve letters, words, or oral language; storytelling; and singing. The Literacy Boost toolkit also recommends a make-and-take activity in reading clubs, thereby ensuring that children have some materials at home to maintain their continuous engagement with reading.

Core Activity: Reading Buddies (Lifewide Learning)

The reading buddies activity pairs up competent older readers, usually preteens or teenagers studying in P.4 through P.6, with P.1 to P.3 students. Once paired, reading buddies share books borrowed from the school or village book banks. This activity provides children with a fun and engaging way to have one-on-one exposure to reading and print.

Study Design and Data

Given the expense and complexity of engaging families and communities in children’s learning in LDCs, our randomized controlled trial set out to quantify the benefit of involving families and communities, beyond the traditional school-focused efforts, in children’s reading achievement. The trial tested the impact of two treatments. One treatment aimed to provide all activities in both the School and the Home and Community domain of Literacy Boost. A second treatment aimed to provide solely the school-based book banks and the teacher-training activity that fall within the School domain. To estimate the impact of the two treatments, we compare treatment group outcomes with each other and with a business-as-usual control group.

Two partner organizations implemented all the activities within the study district. Save the Children implemented all Literacy Boost activities in the School domain, while Umuhuza, a Rwandan nongovernmental organization, implemented all activities in the Home and Community domain (see Figure 3).

We chose the study district based on the existing relationships between the district authorities and the implementing partners, the interest shown by local officials, and the representativeness of the district to of Rwanda as a whole. Except for three urban districts in the capital city, Kigali, the study district resembles the other 26 districts of Rwanda across a range of demographic, educational, and socioeconomic indicators, as shown in the supplemental resources (in the online version of the journal) accompanying this article. One exception to this is the foreign resident population. The project district contained a sizeable refugee population and bordered another country, leading to a high proportion of students who speak a language or dialect other than Kinyarwanda at home (NISR & MINECOFIN, 2014c). We accounted for this minority population during random assignment, as described below.

We collected baseline data in August and September 2013, as close to the end of the school year as possible, as the children were completing P.1. We randomly assigned sectors to the treatment groups in October 2013. Finally, we collected outcome data during September 2015.

Random Assignment

To answer our research questions, we randomly assigned the 21 sectors within the study district to one of two treatment conditions or a control condition. We chose sectors as the unit for randomization for two reasons. First, sectors were an existing functional administrative entity. Second, the village or the school were inappropriate to use as the unit of randomization, since one village could send children to many different schools. By randomizing sectors, we significantly limited the contamination of the treatment and control groups.

Sectors were assigned to one of three study groups: an LWL group of sectors, which participated in all components of Literacy Boost; an SO group, which received book banks in school and participated in the teacher training; and a business-as-usual control group. No restrictions were placed on the control group, meaning that the control sectors could participate in any or all of the other interventions but did not receive any support from Save the Children or Umuhuza. Treatment groups are seen within the dotted lines in Figure 3.

We randomly assigned sectors using a cluster stratified randomization process to balance baseline reading skills and the presence of non-Kinyarwanda-speaking students within each treatment group. We used the baseline reading assessment data to identify 4 sectors containing students who reported speaking a language or dialect other than Kinyarwanda; these sectors created the first randomization block. We separated the 17 remaining sectors into low-, medium-, or high-achieving terciles based on the percentage of P.6 students who passed their end-of-year exams in 2012, the most recent results available at the time of random assignment.³

Reading Skills Measurement and Student Outcomes

As no standardized, validated reading assessments existed in Kinyarwanda, the research team led the creation of the reading assessment tools in 2013. The assessment resembled those used by Save the Children (Dowd et al., 2017) and the widely administered Early Grades Reading Assessment (Gove & Cvelich, 2011). Rwanda Education Board and MINEDUC representatives participated in the instrument creation to ensure that it aligned with the then current expectations regarding reading abilities for students in early-primary levels.

Table 1 contains an overview of the baseline and endline reading assessments. The end line assessment differed somewhat from the baseline assessment. The baseline data included moderators of treatment effects: gender, an indicator for whether students had repeated P.1 prior to baseline, an indicator for student baseline ability to decode written language, and a measure of phonological awareness. For the endline assessment, we replaced some subtests, such as phonological awareness, with other subtests such as listening comprehension and vocabulary.

Table 1

Data Collected During the Baseline and Endline Reading Assessments

Section and Subsection	Description, Examples, and/or N of Items	Time
Background/demographic
General	Sex, age, home language/dialect	Base and end
School related	Repetition history, pre-primary attendance	Base and end
Socioeconomic status	Type of home, household size, household amenities and possessions	Base only
Home literacy ecology
Home print materials	Existence of different text types at home	Base and end
Family literacy habits	N of family members who the child saw reading, read to the child, help the child study, and converse with the child	Base and end
Language and reading skills^a
Syllabic awareness	(a) Blending syllables (6 items)	Base only
Syllabic awareness	(b) Segmenting syllables (6 items)	Base only
Phonemic awareness	(a) Similar beginning sounds (10 items)	Base only
	(b) Blending phonemes (12 items)	Base only
	(c) Segmenting phonemes (12 items)	Base only
Productive vocabulary	Orally identifying pictures (22 items)	End only
Listening comprehension	Answering questions orally after listening to a short story (5 items)	End only
Letter knowledge	Identifying randomly ordered, mixed-case letters (24 items)	Base and end
Decoding	Reading decodable pseudo words (15 items)	Base and end
Common word dictation	Writing high-frequency words (15 items)	Base and end
Reading comprehension	(a) reading one-sentence statements and orally answering simple questions (10 items)	Base and end
Reading comprehension	(b) orally completing written cloze sentences (10 items)	Base and end
Reading fluency	N of words in a connected text read correctly in a minute—includes
	(a) P.1 passage (23 words)	Base and end
	(b) P.3 passage (74 words)	Base and end
	(c) P.4 passage (105 words)	End only

Note. Base = baseline assessment administered in September 2013; End = endline assessment administered in September 2015; P.1 = Primary 1; P.3 = Primary 3; P.4 = Primary 4.

All the subtests were in the Kinyarwanda language. Assessors used the Rukiga language/dialect to ask background questions and provide instructions to students who speak that language/dialect at home. All the data were collected from student reports.

To compare groups parsimoniously, we created the five outcome variables seen in the top row of Table 2: (1) child reached P.3 by 2015, (2) child met the basic literacy threshold, (3) oral comprehension, (4) reading fluency, and (5) reading comprehension.

Table 2

Project Outcome Description, Descriptives, and Reliability Estimates

Outcome	Child Reached P.3 by 2015	Child Met the Basic Literacy Threshold	Oral Comprehension	Reading Comprehension	Reading Fluency
Outcome creation/ scoring	Student coded as “0” if enrolled in P.1 or P.2 in 2015 “1” if enrolled in P.3 in 2015	Student scored “1” if he/she met 3 criteria: (a) scored ≥18 in letter knowledge (b) scored ≥1 in decoding (c) scored ≥1 in dictation	Sum of z scores of (a) listeningcomprehension (b) productive vocabulary	Sum of z scores of (a) reading comprehension (b) cloze	Sum of z scores of correct words per minute on (a) P.1 passage (b) P.3 passage (c) P.4 passage
Mean	0.37	0.69	−0.00	1.00	1.16
Standard deviation	NA	NA	1.61	1.52	2.37
Minimum value	0	0	−5.24	−2.22	−2.65
Maximum value	1	1	4.08	3.01	14.93
Variable type	Binary	Binary	Continuous	Continuous	Continuous
Coefficient alpha	NA	.918	.787	.876	.942
Interrater agreement (%)	NA	79.2	78.6	88.3	78.6
Interrater kappa	NA	.787	.772	.862	.761
Interrater ICC	NA	.962	.943	.944	.932

Note. Means, standard deviations, and minimum and maximum values are presented for the analytic sample at endline. NA = not applicable; ICC = intraclass correlation coefficient.

Child Reached P.3 by 2015

To see whether treatment assignment impacted students’ primary-level promotion, we used the assessors’ recordings of the students’ primary level. That is, we asked whether the students had repeated any primary levels or whether they had reached P.3 as should be expected 2 years following enrollment in P.1. This binary outcome is critical to measure given the persistently high rates of repetition in the early-primary levels in Rwanda and across LDCs in the past decade (Crouch & Merseth, 2017).

Child Met the Basic Literacy Threshold

A bimodal distribution of scores on the reading skills subtests at both baseline and endline suggested that the students broke down into two distinct groups: students who had at least basic literacy skills and students who did not. Thus, we created another binary outcome, the basic literacy threshold. Students who identified 18 or more of the 24 letters of the Kinyarwanda alphabet, wrote at least one dictation item correctly, and read at least one decoding item met the basic literacy threshold.

Oral Comprehension, Reading Comprehension, and Reading Fluency

We created three standardized composite scores for oral comprehension, reading comprehension, and reading fluency. The oral comprehension score, created by adding the z scores of the productive vocabulary and listening comprehension subtests, exists for every student in the endline sample, as the assessors administered these subtests to all the children. Students who passed the basic literacy threshold also have a reading comprehension score (the sum of text comprehension and cloze z scores) and a reading fluency score (the sum of reading fluency z scores from three passages).

We created a composite score, phonological awareness (not shown in Table 2, as this was a control variable, not an outcome of interest), by summing the z scores on five baseline subtests: beginning sound matching, blending phonemes, blending syllables, segmenting phonemes, and segmenting syllables. We created this composite to test for baseline equivalence among the groups and to use as a control in the final analysis.

Student Sampling and Data Collection

At baseline, our sampling frame included all P.1 students across the 102 primary schools in the study district. We randomly sampled 85 of the district’s 102 schools to visit for data collection. We drew a sample of 25 P.1 students from one randomly selected P.1 classroom. Our sample came from only one classroom at each school for two reasons. First, sampling from one classroom caused less overall disruption to instruction. Second, we did not expect any classroom-level effects since all primary teachers move from classroom to classroom during the school day and children therefore receive instruction from multiple teachers each day.

Independent assessors collected all the data in accordance with the protocols approved by the Rwanda National Ethics Committee. Each assessor spoke Kinyarwanda fluently and held at least a bachelor’s degree. A subset of assessors also spoke Rukiga, the language/dialect spoken by a minority portion of students. These Rukiga speakers assessed children in the Rukiga-speaking sectors and used Rukiga to provide instructions and collect background data.⁴

As no data on student learning disabilities existed, all students enrolled in P.1 were eligible to be included in the sample; any students with learning disabilities should be evenly dispersed throughout the treatment conditions and therefore should not bias the impact estimates.

To provide an estimate of interrater agreement and reliability, the team leaders randomly selected 10% of endline students to be assessed by two assessors simultaneously. One assessor administered the assessment as normal, while another assessor listened in and independently marked a second assessment form based on the student’s responses. We then calculated the agreement, kappa coefficient, and reliability between the two raters. Table 2 presents the internal and interrater reliability estimates, both of which suggest good reliability.

At endline, the assessors sought out the same students from baseline to longitudinally assess their reading development following 2 years of treatment. The assessors successfully located and assessed 1,668 of the 2,041 students assessed at baseline. These 1,668 students form our analytic sample. We present the total number of students, schools, and sectors in Table 3.

Table 3

Sample Size and Attrition, Compliance, and Baseline Equivalence

Statistics by Group	Total	Control	SO	LWL	Significant Difference^a
Sample size and attrition
N of sectors	21	7	7	7	NA
N of primary schools	85	28	28	29	NA
Baseline: N assessed	2,041	634	694	713	NA
Endline: N assessed	1,668	521	545	602	NA
Endline: % of baseline assessed	82	82	78	84	NA
Endline: N attrited	373	113	149	111	NA
Endline: % attrited	18	18	23	16	NA
Compliance
N of children who do not live in a control sector but who attend school in a control sector	34	NA	19	4	NA
N of children who do not live in an SO sector but who attend school in an SO sector	56	18	NA	37	NA
N of children who do not live in an LWL sector but who attend school in an LWL sector	31	16	15	NA	NA
N opting out of the assigned treatment	109	34	34	41	NA
% opting out of the assigned treatment	7	7	5	6	NA
Baseline equivalence
Average % female	1668	45	50	49	None
Average % repeated P.1	1,659^b	58	51	55	None
Average socioeconomic status^c	1,668	0.07	−0.09	0.01	None
Average home reading materials index	1,668	−0.08	0.02	0.03	None
Average family reading habits index	1,668	−0.04	0.04	0.01	None
Average phonological awareness score	1,668	−0.09	−0.03	0.09	None
Average % meeting the basic literacy threshold	1,668	23	29	39	None

Note. NA = not applicable; SO = school only; LWL = lifewide learning; P.1 = Primary 1.

Significant differences were only analyzed for baseline equivalence items. No statistically significant difference was observed at a p < .05 level.

Differences in sample sizes are due to missing data.

Socioeconomic status was measured using an index of home assets (e.g., radio, bicycle, computer), created using student-reported data. More conventional measures of socioeconomic status were not possible due to the context in which the study was implemented.

In addition to data on sample size at baseline and attrition at endline, we also report compliance data in Table 3. Students did not comply with the treatment if they lived in one sector but attended school in another sector that was randomly assigned to a different group than their home sector. The decision to randomize at the sector level to minimize compliance issues appears to have worked for the most part. Nevertheless, we observe that 7%, or 109 out of 1,668 students, did not comply with their random assignment. That is, these students attended schools in sectors different from their home sector. The likely explanation for this noncompliance is that these students lived near the sector boundaries and the schools they attended were closer than those located within their home sector. It is unlikely that these 109 students and their families moved homes to opt into or out of a specific treatment, or were in noncompliant schools (i.e., the treatment group did not match the sector in which the child resided), for any reasons having to do with the treatments. However, we could not locate maps with enough detail to discern which villages make up the sector borders, and hence we cannot quantify the level of diffusion of treatment. The absence of any clear pattern suggests that noncompliance was likely unrelated to the intervention.

To establish internal validity of the findings, we tested the analytic sample for differences between the groups at baseline; the results are presented toward the bottom of Table 3. None of the groups varied significantly from each other at baseline.

Although not statistically significant, we still found a sizeable difference (0.18 of a standard deviation) between the control and LWL groups on their baseline phonological awareness scores. Because phonological awareness has been shown to be highly predictive of later reading achievement (NICHD, 2000), we included phonological awareness as a covariate in our models to provide a clearer estimate of the treatment impact.

Implementation Fidelity Data

Save the Children and Umuhuza, the implementing partners, continuously collected data on implementation fidelity throughout the 2-year implementation time frame and provided those data to the research team. Save the Children staff recorded the number of training sessions that each treatment teacher attended. In each village, Umuhuza facilitators first recorded every parent or caretaker who had a child in the early-primary level and then tracked their attendance at the reading awareness workshops. They also trained the reading club leaders to list out all the children in the village who were enrolled in early-primary levels and to record their attendance. No monitoring data were collected for reading buddies.

Methods to Estimate Impact

We estimate the impact of SO and LWL treatments on the student outcomes described above using multilevel models to account for clustering of observations within sectors. The predicted outcome for student i in sector j of randomization block k is given as

Outcom e_{ijk} = β_{0} + β_{1} ((LW L_{j})) + β_{2} ((S O_{j})) + β_{3} ((P A_{i})) + ω_{k} + T_{j} + ε_{ik} .

Random effects $T_{j}$ and $ε_{ik}$ account for error at the sector and student levels, respectively. $ω k$ , a vector of randomization block fixed effects, accounts for the blocked randomization. $P A_{i}$ adjusts for the baseline phonological awareness score of student i. $LW L_{j}$ indicates whether sector j is assigned to the LWL group, and $S O_{j}$ indicates whether sector j is assigned to the SO group. The coefficients of interest that determine treatment impact are $β_{1}$ and $β_{2}$ ; all other covariates (including indicator variables for the randomization blocks) are grand mean centered within the analytic sample for that outcome.

We fit all the models using Stata version 14.1’s mixed and melogit commands. We used restricted maximum likelihood estimation and the Kenward-Roger method to compute degrees of freedom for the models and calculate p values with adjustments for sample sizes at the sector and block levels (Kenward & Roger, 1997; Schaalje, McBride, & Fellingham, 2002). We standardize continuous outcome variables within the analytic sample and report them using effect sizes. For ease of interpretation, binary outcome variables are translated into percentage points to describe the predicted probability of the outcome for an average student in each group, while holding the grand mean–centred covariates constant at 0. As such, predicted probabilities provide the expected outcome for the average student in the sample.

In addition to the main impact models, we run additional models to understand the potential moderating role that gender and baseline achievement might play in each of these treatments. To estimate these models, we run one new estimate for each moderator, adapting the main impact estimates by including a control for the moderator of interest and the interaction between this moderator and each treatment indicator.

Following the comparison of the control and treatment groups, we also compare the effect of LWL directly with that of SO. To estimate the differences between the continuous outcomes, we estimate $\hat{β_{1}} - \hat{β_{2}}$ using Stata’s lincom command. For the binary variables, we estimate the difference in predicted percentage point estimates for the LWL and SO groups (i.e., converting the treatment estimates plus the constant term) using the nlcom command. All comparisons of treatment effects use 15.5 degrees of freedom, which is the number provided by the Kenwood-Roger corrections in the main impact models.

We use the models above to conduct both intent-to-treat (ITT) and treatment-on-treated analyses. To run the ITT analysis, we use the entire 1,668 students assessed at both baseline and endline. As described above, crossover appeared unrelated to any baseline student characteristics for which we had data. Thus, we censored the 109 crossover students; such an analysis will not reduce statistical power significantly as the number of units used for randomization (21 sectors) remains the same, but it may be subject to bias if crossover is driven by unobserved student characteristics (Jönsson et al., 2014).⁵

Findings

Research Question 1: Was Literacy Boost Implemented as Intended?

Implementation Timeline

Implementation began in January 2014 and continued through December 2015. We show the timing of implementation by quarter for each activity within both the School and the Home and Community domains in Table 4.

Table 4

Implementation Timeline and Number of Sectors, Schools, and Villages Reached, by Quarter

Domain	Materials and Activities	Target Groups		2014				2015			Cumulative Total
Domain	Materials and Activities	Target Groups	N and % of Total	Q1	Q2	Q3 and Q4	Annual Total	Q1	Q2	Q3 and Q4	Cumulative Total
School	School-based book banks	SO and LWL	N of sectors		14		14	14			14
			N of schools reached		73		73	73			73
			% of schools reached		100		100	100			100
	Teacher training^a	SO and LWL	N of sectors	14			14	14			14
			N of schools reached	73			73	73			73
			% of schools reached	100			100	100			100
Home and Community	Village-based book banks and reading clubs^b	LWL only	N of sectors reached		7	7	7	6	6	3	7
			N of villages reached		90	44	134	44	48	11	237
			% of villages reached		38	19	57	19	20	5	100
			Min/max % of villages covered across sectors		32–74	11–25	39–95	0–25	0–33	0–12	100
	Reading awareness workshops and home visits	LWL only	N of sectors reached	7	7	7	7	6	6		7
			N of villages reached	48	42	44	134	44	48	11	237
			% of villages reached	20	18	19	57	19	20	5	100
			Min/max % of villages covered across sectors	13–42	11–32	11–25	39–95	0–25	0–33	0–12	100
	Reading buddies	LWL only	N of sectors reached				0	7	6	3	7
			N of villages reached				0	178	48	11	237
			% of villages reached				0	76	20	5	100
			Min/max % of villages covered across sectors				0	61–95	0–33	0–12	100

Note. Q1 = Quarter 1; Q2 = Quarter 2; Q3 = Quarter 3; Q4 = Quarter 4; LWL = lifewide learning; SO = school only; min/max = minimum/maximum.

Teacher data disaggregated by year were not available. Teacher training began in the Q1 and continued throughout the year.

Village-based book banks and reading clubs are presented together as they both were delivered or began concurrently in each village.

Umuhuza, the partner responsible for the Home and Community domain of activities, implemented the village-based activities by randomly dividing the 237 villages assigned to the LWL group into six sets and implementing all activities in one set of villages over the course of approximately 3 months before moving on to a new set of villages. Table 4 provides an overview of these village groupings by sector and year. In 2014, across the seven LWL sectors, Umuhuza reached 134 villages, or 57% of all treatment villages. The percentage of villages reached within specific sectors ranged from 47 to 95. By the end of 2015, Umuhuza reported having implemented LWL activities in all 237 villages assigned to the LWL treatment.

Implementation Fidelity

We present data on implementation fidelity, provided by the implementing partners, in Table 5. All schools within the LWL and SO sectors (N = 73) and all villages within the LWL sectors (N = 237) received book banks. Save the Children provided all 73 treatment schools with book banks in early 2014 and replenished the book banks in early 2015. In larger villages, Umuhuza placed two book banks to meet the demands of the larger populations. The implementing partners filled the book banks with Kinyarwanda-language children’s books published recently by the Rwanda Children’s Book Initiative (Malik et al., 2015).

Table 5

Implementation Fidelity Data

Domain	Activity or Input	Eligible Group(s)	Unit	Total Assigned to Treatment	Total Reached	% Reached of Total Assigned	Notes
School	Teacher training	LWL and SO	Teachers	1,001	877^a	88	All LWL and SO primary teachers were invited to attend all sessions in both 2014 and 2015The “reached” teachers attended >60% of sessions during the 2-year implementation
	School-based book banks	SO	Schools	34	34	100	Held approximately 200 titlesDelivered to all schools in 2014150 books added in 2015
	School-based book banks	LWL	Schools	39	39	100
Home and Community	Village-based book banks	LWL	Villages	237	237	100	Held approximately 120 titles
	Reading awareness workshops	LWL	Villages	237	237	100	All 10 workshop sessions were implemented weekly for 10 weeksWorkshops were conducted only once per village
			Households	9,750	8,352	86
			Parents	19,141	14,285	75
			Home visits	9,750	4,893	50
	Reading clubs	LWL	Villages	237	237	100	Reading clubs meet weeklyDesigned to continue indefinitely
	Reading clubs	LWL	Students	16,019^b	11,683	73	Reading clubs meet weeklyDesigned to continue indefinitely
	Reading buddies	LWL	Villages	237	237	100	Was begun in January 2015Designed to continue indefinitely
	Reading buddies	LWL	Students	16,019^b	Data not available		Was begun in January 2015Designed to continue indefinitely

Note. SO = school only; LWL = lifewide learning. Unless otherwise noted, all data were provided by the implementing partner. In some cases, villages received two sets of activities within the Home and Community domain due to the large population sizes.

Data are not disaggregated by treatment group.

Estimate of total N of students assigned to treatment from Rwanda MINEDUC (2016).

For the core activity of teacher training, the goal was to train all primary teachers in the 73 schools assigned to either treatment group. Eight teacher trainers, hired by Save the Children, invited the teachers to attend the teacher-training activity. Save the Children did not pay teachers to attend the training sessions but provided those who attended with travel stipends of RWF 4,000 (around US$5.00) per training. To facilitate training, the trainers grouped schools into local clusters, and one school hosted each training within each cluster.

The trainers designated local resource persons within each cluster to help support the pedagogical improvements. To do so, during the first training session in early 2014, teachers within each sector voted for an experienced and respected peer to be a model teacher of Kinyarwanda. These model teachers participated in the extra training for trainers, quarterly or semiannual refresher trainings, and received additional transport allowance to assist in training sessions outside their own cluster.

Save the Children trainers also visited schools during weekdays between training sessions to support teachers in applying the new pedagogical approaches they had learned in the training sessions. The trainers conducted lesson observations using a standard lesson observation form, and then they met with teachers following the lesson to offer feedback. The observation results were shared only with the observed teacher and not used as an accountability measure.

The teacher trainers led a full cycle of teacher training sessions on a regular basis across 2014 and then repeated the training sessions again for all the teachers in 2015.⁶ As can be seen in the upper portion of Table 5, 877 (88%) of the 1,001 eligible teachers who taught in the treatment schools attended 60% or more of the teacher training sessions. Unfortunately, teacher attendance data disaggregated by treatment group were unavailable, as were estimates of the number of teachers trained each year or the rate of teacher turnover.

For the core activity of reading awareness workshops, Umuhuza planned to provide one complete set of reading awareness workshops to each of the 237 villages assigned to the LWL treatment. Prior to the start of the workshops, Umuhuza adapted the 7 existing workshops and added three new sessions for a total of 10 reading awareness workshops. The topics of the three additional sessions were (1) Brain Development, (2) Emotional Well-Being/Social Competence/Positive Images, and (3) Cultural and Spiritual Roots. Umuhuza included these to both engage participants with culturally relevant knowledge and educate families about how to better support their children’s growth.

Thirteen community facilitators, hired by Umuhuza, led the implementation of reading awareness workshops. The facilitators were all Rwandan citizens and had completed secondary school at a minimum. During a preliminary meeting in each village, the community facilitators recorded the number of families with children in early-primary grades in order to accurately target beneficiaries. The villagers then agreed on a weekly meeting time that best fit their schedules. The facilitators held the workshops once weekly for 10 consecutive weeks. Umuhuza encouraged families to send two parents or caregivers (hereafter, “parents”) to attend the workshops, but this was not a requirement. When not preparing for or leading the workshops, the community facilitators visited the homes of the workshop participants to monitor whether the villagers were successfully applying the knowledge and skills that they had learned and practiced during the workshops. Umuhuza reported that reading awareness workshops took place in all 237 villages assigned to the LWL treatment. Approximately 86% of eligible households participated, and approximately 75% of parents attended the workshops. However, we cannot determine from the data provided what percentage of parents attended a few, most, or all of the workshops.

Another core activity, reading clubs, began a few weeks following the first reading awareness workshop in a village, meeting weekly. In each village, reading club leaders were volunteers from among the reading awareness workshop participants, or respected, literate villagers who were recruited by the workshop participants to lead the clubs. Umuhuza trained the reading club leaders in how to lead the reading clubs. To incentivize volunteers, Umuhuza paid national health insurance fees for the volunteer and three relatives, a total cost of RWF 12,000 per leader per year (about US$15). Umuhuza reported that volunteers held reading clubs in all 237 LWL-assigned villages and reached approximately three quarters of all eligible children.

Finally, Umuhuza planned to pair every child in P.1 through P.3 with a reading buddy, with reading club leaders organizing the activity in each village and pairing the students. It is important to note that reading buddies did not begin until early 2015, halfway through the intervention.⁷ No implementation data on reading buddies were available.

Research Question 2: Impact of the Lifewide Learning Treatment and School-Only Treatments

Our main results are presented in Table 6, which shows the beta coefficients and the predicted percentage (for binary outcomes) or z scores (for continuous outcomes) for each outcome by group. This table compares the outcomes for the control group against the outcomes for each of the two treatment groups, with statistically significant differences indicated by asterisks. It further compares the outcomes for the SO treatment group directly with the outcomes for the LWL treatment group, with statistically significant differences indicated by a superscript a.

Table 6

Intention-to-Treat Results: Comparison of Group Outcomes

Group	Metric	Reached Primary 3 by 2015	Met the Basic Literacy Threshold	Oral Comprehension	Reading Comprehension	Reading Fluency
LWL	Coefficient	.49*	.22	.35^*,a	.33**	.29^**,a
	Standard error	(.21)	(.41)	(.13)	(.09)	(.09)
	% or z score	40%	76%	.20	.13	.14
SO	Coefficient	.55**	.05	.07	.21*	.13
	Standard error	(.20)	(.39)	(.13)	(.09)	(.08)
	% or z score	41%	73%	−.08	.02	−.02
Control	Constant	−.92	.94	−.15	−.19	−.15
	Standard error	(.15)	(.28)	(.09)	(.07)	(.06)
	% or z score	29%	72%	−.15	−.19	−.15
Sample size^b	N of sectors	21	21	21	21	21
Sample size^b	N of students	1,668	1,650	1,650	1,135	1,136

Note. Asterisks denote statistically significant differences between the treatment group and the control group. LWL = lifewide learning; SO = school only.

LWL > SO (p < .05).

Differences in sample sizes are due to missing data.

p < .05. **p < .01.

Both LWL and SO treatments had similar positive impacts for promotion to P.3 and reading comprehension. An average LWL student was 11 percentage points more likely than an average control student to be promoted to P.3 by endline (p < .05), and an average SO student was 13 percentage points more likely (p < .01). Effects sizes (es) for impacts on reading comprehension were .33 for LWL (p < .01) and .21 for SO (p < .05). Only LWL had a positive and statistically significant effect on students’ oral comprehension (es = .35, p < .05) and reading fluency (es = .29, p < .01). Furthermore, the differences between the effects of LWL and SO on both oral comprehension (es = .28, p < .05) and reading fluency (es = .16, p < .05) were large enough that the differences were unlikely to have arisen by chance. Although both treatments had some impact on literacy outcomes, an average control student had a 72% predicted probability of passing the basic literacy threshold, and neither treatment impacted students’ probability of exceeding this threshold. Results from the treatment-on-treated analysis are similar in both directions and, as such, are not reported here but are available on request.

Of the four moderating variables tested, only gender had a consistently statistically significant and positive mediating role, and only for the more advanced literacy skills of reading fluency (LWL interaction es = .38, p < .01; SO interaction es = .31, p < .05) and reading comprehension (LWL interaction es = .32, p < .05; SO interaction es = .35, p < .05). Results for the impacts of either treatment for boys were not statistically different from 0, indicating that the literacy impacts of both programs accrued disproportionately to girls. No other moderators provided clear and consistent patterns, and we omit the full results for want of space; the results are available from the authors on request.

Discussion and Conclusion

This study’s purpose was to test and compare two approaches to improving children’s early reading development in an LDC—an SO and an LWL approach. Both treatments worked to improve student outcomes when compared with the control group. The SO approach significantly increased students’ promotion rate from P.1 to P.3 and improved reading comprehension skills among students who met the basic literacy threshold, when compared with the control. The LWL approach also increased student promotion rate when compared with the control, but in contrast to the SO approach, it significantly improved additional reading skills—oral comprehension, reading comprehension, and reading fluency. On two of the five outcomes—oral comprehension and reading fluency—the LWL group of children outperformed the SO group as well. Our principal conclusion, therefore, is that supporting children’s LWL opportunities has greater potential to impact learning positively than supporting children through SO approaches.

The small effect sizes, even for the more robust LWL intervention, are somewhat larger than one commonly accepted definition of a small effect size, d = .20. The effect sizes are statistically reliable and exceed those of meta-analytic studies of LDC interventions that estimate effect sizes between .03 and .23 (Conn, 2017; McEwan, 2015). The overall impact might be underwhelming to Western audiences since developed world research demonstrates that both schools and homes are settings that contribute to children’s literacy, or more generally academic, development. Developed world educators already assume that interventions targeted at improving the literacy context in both settings are more likely to improve student learning than interventions targeting just the school. But settings in LDCs often do not resemble those in developed countries, where infrastructure such as roads and utilities are taken for granted. Little can be taken for granted in LDCs, which is probably why educational interventions have focused almost exclusively on schools. If nothing else, this study demonstrates that an SO focus is needlessly narrow.

In one sense, therefore, the implications of the study are very straightforward. If we are serious about substantially improving school achievement in LDCs, we must continue exploring, developing, and evaluating programs that extend learning opportunities beyond the school day. School-based interventions are necessary but not sufficient, as they do not seek to improve learning during the 85% of children’s waking hours that are spent outside school.

Challenges and Issues Going Forward

The data and findings point to several challenges that face educators across LDCs, as we discussed in the opening paragraphs of this article. First, nearly one third of children could not recognize three quarters of their alphabet, write at least one word correctly, and read at least one simple decodable word correctly despite having attended primary school for 3 or more years. These findings suggest that LWL’s impact is not on basic literacy skills but rather the higher-order skills of reading comprehension and reading fluency.

Second, our moderation analysis suggests that benefits in these higher-order skills accrued mostly or entirely to girls. These findings mirror other impact evaluations conducted by Save the Children researchers in diverse LDC contexts such as Zimbabwe, Sri Lanka, and Indonesia (Brown, 2014; Jonason & Okfriani, 2017; Leer & Navaratnam, 2016). The findings also echo a recent meta-analysis that found a statistically significantly higher aggregate effect on girls’ cognitive outcomes in early-childhood education programs when compared with boys’ outcomes, although the authors of the meta-analysis discount the .03 to .04 standard deviation difference between genders as lacking substantive meaning (Magnuson et al., 2016). Given that neither the SO nor the LWL treatment specifically addressed any gender-related issues, and that boys and girls had equal access to all the treatment activities, there may be more systemic issues within the context or culture that enabled girls who mastered their basic literacy skills to benefit more from the interventions.

Third, even among students who met the basic literacy threshold, we found modest intervention effects. LWL produced significant effects in oral comprehension, reading comprehension, and reading fluency, but the effect sizes were modest. To state this more meaningfully, compared with the control students, the LWL students had on average 1 more vocabulary item correct out of 22, answered 0.7 more reading comprehension questions out of 12, and read four more words correctly per minute from a passage levelled for P.3 students.

Fourth, only 40% to 41% of children in the intervention conditions who started in P.1 were predicted to reach P.3 2 years later, controlling for baseline covariates. This implies that 60% of treatment children repeated a primary level at least once. Though significantly fewer than in the control condition— where 72% repeated one or more grades—the number of children repeating in the treatment groups is hardly a desirable statistic.

What explains these troubling findings? We can only speculate.

The 31% of the sample who did not reach the basic literacy threshold after 3 or more years of schooling likely have obstacles to learning that the Literacy Boost intervention could not adequately address. Prior to the commencement of the study, we assumed that some children would likely struggle with learning difficulties, even to the point of being learning disabled (Snow et al., 1998). However, we were unable to identify these learners due to the lack of psychometrically validated diagnostic tests in Kinyarwanda, the limited time available for assessment, and budgetary constraints. Therefore, some of the 31% of students who did not respond to the intervention might require additional time and/or more intensive instruction and opportunities to learn basic literacy skills. Large class sizes and lack of specialized training for teachers in how to teach children facing learning difficulties might also create issues in providing these students with sufficient intensive instruction, whether individualized or in small groups, to help them acquire necessary and very basic skills. As shown elsewhere, other students may also have physical impairments that interfere with their learning (UNESCO, 2015). Still others may suffer from hunger and hence may not have the daily nutrition needed for learning.

These factors might also explain the relatively modest effects we saw in reading and oral comprehension and reading fluency. These effect sizes might also be explained by the continued lack of adequate support at home. This is suggested by the fact that the reading awareness workshops did not achieve the targeted number of participating parents. There are several possible reasons for the absence of these parents. They might not have had the time to attend the workshops or did not see a role for themselves in supporting children’s learning beyond sending their children to school. Future analyses might shed light on these possibilities by exploring whether participation in treatment activities is associated with changes in parental attitudes and improvements in children’s basic and more advanced literacy skills.

Future efforts must develop and evaluate targeted interventions for those with specific challenges. For example, children with learning difficulties will require more intense and skill-specific instruction that lies outside what program implementers currently provide. Students with physical impairments (e.g., poor eyesight, poor hearing) require diagnoses and remedies (e.g., eyeglasses, cochlear implants). Students from families that see little value in education or provide little support for their children’s learning might be helped by more intensive and systematic efforts by local and national authorities and fellow community members. These and many other recommendations must be explored if we wish to extend the reach and impact of interventions that address the learning crisis in the developing world.

Our data and findings do not cover a pressing concern for educators and economists alike, namely the cost effectiveness of the treatment groups’ gains when compared with each other and the control group. It may be that the logistically challenging LWL activities are prohibitively expensive in certain contexts, something that future research should consider. Future research must also examine both the short-term gains-per-unit cost in reading skills, as has been done for other literacy interventions in LDC contexts (e.g., see Piper & Zuilkowski, 2015), and the longer-term impact of efforts designed to change the LWL opportunities for all children. Accounting for such longer-term impact will better account for the way families have applied their new practices to other children within the family, which may yield greater differences in gains-per-unit cost when compared with the simpler short-term impact.

Limits to the Generalizability of the Findings

Certain contextual factors in Rwanda might have facilitated the clear findings that emerged from this study, suggesting limits to the findings’ generalizability to other LDCs. Rwanda is very organized politically, with clear boundaries between districts, sectors, and villages. Rwanda also has a highly organized and efficient communication system. One noteworthy example of this organization and communication system is the monthly Umuganda meeting, led by village leaders, which all Rwandans are required to attend. This highly organized administrative system has two implications for generalizability. First, it was relatively simple to understand which children were assigned to which treatment condition. Second, the communication system facilitated the implementation of treatment activities. It is not clear whether a randomized controlled trial in a place that is less organized could have been conducted with the same degree of rigor and treatment fidelity as this one.

A second challenge to the study’s generalizability to other LDCs is that, concurrent with our study, Rwanda’s MINEDUC, the Rwanda Education Board, and several nongovernmental organizations implemented a nationwide intervention called the Language, Literacy, and Learning (L3) initiative (Education Development Center, 2017). Implementation of L3 began in January 2015, halfway through our randomized controlled trial. L3 provided all primary students with an early-grades literacy textbook and a few supplemental readers, and it explicitly laid out a scoped and sequenced reading curriculum to improve reading pedagogy. Also accompanying the new curriculum were prerecorded audio lessons that teachers played for their students several times a week.

All the schools in our study, regardless of their assignment to treatment or control, received L3 materials and training. It is impossible to know whether this initiative enhanced or attenuated the effects of the SO or LWL intervention in relation to the control condition. On the one hand, L3 activities might have provided a foundation for the gains seen in the LWL and SO groups. If, in fact, the L3 initiative did provide such a foundation, it might explain why no impact was seen on children’s probability of meeting the basic literacy threshold: L3 activities assisted all students equally to surpass the basic literacy threshold. On the other hand, the audio and scripted lessons may have reduced teachers’ opportunities to use the pedagogical techniques they learned in the Literacy Boost teacher-training sessions, thereby diminishing the impact of the teacher training. In any case, our conclusions about SO and LWL effects acknowledge the presence of L3 materials and activities in all the schools involved in the study.

Other contextual factors might have obscured or moderated the program’s impact. One such factor may be the method used by Umuhuza to implement the LWL treatment. Due to the large geographical area covered, with many hard-to-access villages spread among very hilly terrain, and limited program staff, villages were grouped into waves, with the LWL treatment provided to all villages in a wave before the activities were offered to villages in the next wave. In practice, this meant that by the end of the first year of implementation, 45% of the villages in LWL sectors still had not participated in LWL activities or received village-based book banks, as shown in Table 4. A question for future research would be how to expedite the roll-out of complex programs over challenging geographic regions or, alternatively, whether the improvement in student outcomes justifies the expense of the additional personnel required for a more expedited implementation.

Another factor that may have moderated our findings lay in the timing of the data collection. Baseline data collection occurred toward the end of children’s P.1 academic year. Treatment activities began approximately 4 months later, at the start of the next academic year. Given the high repetition and low promotion rates observed in the control group, it is reasonable to assume that many children in the LWL and SO groups repeated their P.1 year in 2014. During the endline data collection in 2015, we did not ask the children whether they had repeated P.1 in 2014, before the intervention began. By the time we had realized this oversight, the data collection had been completed. Therefore, we cannot know when (before or after 1 year of implementation) the students repeated.

A third factor possibly moderating the findings is that our implementing partner, Umuhuza, targeted only parents for participation in the reading awareness workshops. Although the reasons for doing so were understandable given Umuhuza’s past work in educating new parents and a desire to keep the workshops to a small size, the focus on parents may have limited the impact of the treatment. Targeting only parents does not account for the family structures in rural villages across LDCs. In many rural villages, it is common for the parents, as the principal earners in the family, to be absent from the home or community for long periods, either during the day or for multiple days or weeks, in order to earn a living. Children may interact more with their close relatives—siblings, grandparents, cousins—in their day-to-day routines than with their parents. Including other family members could have helped improve children’s learning as more of their family adopted best practices in supporting children’s learning outside school. This topic would benefit from further study.

A final factor that might have moderated program impact was the language or dialect spoken in the border areas of the district. Approximately 15% of children reported speaking Rukiga, a variant of Kinyarwanda, at home. The sparse linguistic literature on Rukiga does not clarify if Rukiga and Kinyarwanda are dialects of one language or distinct languages in and of themselves, or the degree of mutual intelligibility between the two. In fact, we received inconsistent responses from Rwandans when we inquired whether the two were simply different varieties of one language (e.g., British and American English) or two related yet distinct languages that were not entirely mutually intelligible (e.g., Spanish and Portuguese).

Despite these complexities and limitations, it is clear that children’s lives outside school should be factored into future efforts to address the learning crisis in LDCs. We cannot afford to ignore 85% of children’s waking hours as we work to make continued progress in eradicating the low levels of literacy throughout much of the world.

Supplemental Material

DS_10.3102_0002831218792841 – Supplemental material for Lifewide or School-Only Learning: Approaches to Addressing the Developing World’s Learning Crisis

Supplemental material, DS_10.3102_0002831218792841 for Lifewide or School-Only Learning: Approaches to Addressing the Developing World’s Learning Crisis by Elliott W. Friedlander, Nicole Arshan, Sen Zhou and Claude Goldenberg in American Educational Research Journal

Footnotes

Notes

Elliott W. Friedlander is the research director of Stanford University’s Literacy Boost in Rwanda: A Randomized Control Trial project, 485 Lasuen Mall, Stanford, CA 94305; e-mail: elliottf@stanford.edu . His research focuses on ways to improve children’s reading achievement in the developing world through family and community engagement.

Nicole Arshan is a principal education researcher and evaluation methods lead at SRI International. Dr. Arshan studies the impacts of scaling strategies for interventions designed to improve teaching and learning.

Sen Zhou is a postdoctoral researcher at the China Institute for Educational Finance Research, Peking University. Her research focuses on how to improve educational opportunities and outcomes for economically disadvantaged students, both in K–12 schools and in higher education.

Claude Goldenberg is the Nomellini & Olivier Professor of Education Emeritus at Stanford University. His research focuses on literacy development and academic achievement among language minority students.

References

Brown

(2014). Literacy Boost Zimbabwe: 2nd year endline report. Washington, DC: Save the Children. Retrieved from https://resourcecentre.savethechildren.net/library/literacy-boost-2nd-year-endline-zimbabwe-march-2014

Chansa-Kabali

Serpell

Lyytinen

(2014). Contextual analysis of home environment factors influencing the acquisition of early reading skills in Zambian families. Journal of Psychology in Africa, 24, 410–419.

Conn

K. M.

(2017). Identifying effective education interventions in sub-Saharan Africa: A meta-analysis of impact evaluations. Review of Educational Research, 87, 863–898. Retrieved from doi:10.3102/0034654317712025

Crouch

Merseth

K. A.

(2017). Stumbling at the first step: Efficiency implications of poor performance in the foundational first five years. PROSPECTS, 1–22. Retrieved from https://link-springer-com-443.web.bisu.edu.cn/journal/11125

DeStefano

Ralaingita

Costello

Sax

Frank

(2012). Task Order 7: Early grade reading and mathematics in Rwanda. Research Triangle Park, NC: RTI International. Retrieved from pdf.usaid.gov/pdf_docs/pdact621.pdf

Dowd

A. J.

Friedlander

E. W.

Jonason

Leer

Sorensen

L. Z.

Guajardo

. . . Pisani

(2017). Lifewide learning for early reading development. New Directions for Child and Adolescent Development, 155, 31–49. doi:10.1002/cad.20193

Dowd

A. J.

Wiener

Mabeti

(2010). Malawi Literacy Boost 2009 Year 1 report. Washington DC: Save the Children. Retrieved from https://resourcecentre.savethechildren.net/sites/default/files/documents/6909.pdf

Education Development Center. (2017). Literacy, language, and learning initiative: National fluency and mathematics assessment of Rwandan schools: Endline report 2016. Kigali, Rwanda: Author/Rwanda Education Board. Retrieved from http://l3.edc.org/documents/EDC-L3-Endline-Evaluation.pdf

Evans

D. K.

Popova

(2015). What really works to improve learning in developing countries? An analysis of divergent findings in systematic reviews (Policy Research Working Paper No. 7203). Washington, DC: World Bank. Retrieved from https://openknowledge.worldbank.org/handle/10986/21642

10.

Friedlander

E. W.

(2013). Environmental factors associated with early reading achievement in the developing world: A cross-national study. International Journal of Educational Research, 57, 25–38. doi:10.1016/j.ijer.2012.10.006

11.

Friedlander

E. W.

(2015). Towards learning for all: Understanding the literacy ecology of rural Rwanda (Unpublished doctoral dissertation). Stanford University, Stanford, CA. Retrieved from https://purl.stanford.edu/vx971nt5655

12.

Friedlander

E. W.

Dowd

A. J.

Borisova

Guajardo

(2012). Life-wide learning: Supporting all children to enjoy quality education. In Addressing inequalities: The heart of the post-2015 development agenda and the future we want for all. New York, NY: UN Women/UNICEF. Retrieved from https://files.eric.ed.gov/fulltext/ED567231.pdf

13.

Friedlander

E. W.

Goldenberg

C. N.

(Eds.). (2016). Literacy Boost in Rwanda: Impact evaluation of a two year randomized control trial. Stanford, CA: Stanford University, Graduate School of Education. Retrieved from https://www.researchgate.net/publication/308077436_Literacy_Boost_in_Rwanda_Impact_Evaluation_of_a_2_Year_Randomized_Control_Trial

14.

Ganimian

A. J.

Murnane

R. J.

(2016). Improving education in developing countries: Lessons from rigorous impact evaluations. Review of Educational Research, 86, 719–755.

15.

Gove

Cvelich

(2011). Early reading: Igniting education for all (A report by the Early Grade Learning Community of Practice). Research Triangle Park, NC: Research Triangle Institute. Retrieved from https://www.rti.org/sites/default/files/resources/early-reading-report-revised.pdf

16.

Guajardo

Jacob

L. V.

Boneti

Kumar

Isaac

(2016). World Vision India pro-lit baseline report: Hindi and English reading skills of Grade 2 & 3 students in Agra, Aparajita, Borio, and Dumka ADPs. Washington, DC: Save the Children/World Vision. Retrieved from https://resourcecentre.savethechildren.net/sites/default/files/documents/literacy_boost_world_vision_pro-lit_india_baseline-_december_2014.pdf

17.

Heath

S. B.

(1983). Ways with words: Language, life and work in communities and classrooms. Cambridge, England: Cambridge University Press.

18.

Hess

R. D.

Holloway

S. D.

(1984). Family and school as educational institutions. Review of Child Development Research, 7, 179–222.

19.

Heyneman

S. P.

(2005). The history and problems in the making of education policy at the World Bank 1960–2000. International Journal of Educational Development, 6, 23–58.

20.

Holland

(2012). Community based barriers and opportunities to promote reading attainment among early grade learners in the rural Southern Province of Rwanda (Final report). Kigali, Rwanda: Centre for Global Development Through Education/Rwanda Education Board. Retrieved from http://idd.edc.org/sites/idd.edc.org/files/USAIDL3CommunityLiteracyStudy_0.pdf

21.

Jonason

Okfriani

(2017). Indonesia creating early readers for academic success programme: Endline report. Washington, DC: Save the Children. Retrieved from https://resourcecentre.savethechildren.net/library/indonesia-creating-early-readers-academic-success-programme-endline-report

22.

Jönsson

Sandin

Ekman

Ramsberg

Charbonneau

Huang

. . . Drummond

(2014). Analyzing overall survival in randomized controlled trials with crossover and implications for economic evaluation. Value in Health, 17, 707–713.

23.

Kenward

M. G.

Roger

J. H.

(1997). Small sample inference for fixed effects from restricted maximum likelihood. Biometrics, 53, 983–997.

24.

Kim

Y.-S. G.

Boyle

H. N.

Zulkowski

S. S.

Nakamura

(2016). Landscape report on early grade literacy. Washington, DC: USAID. Retrieved from https://allchildrenreading.org/resources/usaid-landscape-report-early-grade-literacy/

25.

Leer

Navaratnam

(2016). Literacy Boost: Sri Lanka, Endline Report 2016. Washington, DC: Save the Children. Retrieved from https://resourcecentre.savethechildren.net/library/literacy-boost-sri-lanka-endline-report-2016

26.

Lockheed

M. E.

(1990). Primary education (English) (A World Bank policy paper). Washington, DC: World Bank. Retrieved from http://documents.worldbank.org/curated/en/358591468320068554/Primary-education

27.

Magnuson

K. A.

Kelchen

Duncan

G. J.

Schindler

H. S.

Shager

Yoshikawa

(2016). Do the effects of early childhood education programs differ by gender? A meta-analysis. Early Childhood Research Quarterly, 36, 521–536.

28.

Malik

S. S.

Balfour

B. J. V.

Nzabonimpa

J. P.

Cozzolino

Dib

Dowd

A. J.

(2015). Endline evaluation of Rwandan Children’s Book Initiative. Kigali, Rwanda: Save the Children International. Retrieved from https://rwanda.savethechildren.net/sites/rwanda.savethechildren.net/files/library/RCBI Endline Report.pdf

29.

McEwan

P. J.

(2015). Improving learning in primary schools of developing countries: A meta-analysis of randomized experiments. Review of Educational Research, 85, 353–394.

30.

Ministry of Finance and Economic Planning. (2000). Rwanda Vision 2020. Kigali, Rwanda: Author. Retrieved from http://www.minecofin.gov.rw/fileadmin/templates/documents/NDPR/Vision_2020_pdf

31.

National Institute of Child Health & Human Development. (2000). Report of the National Reading Panel. Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction (NIH Publication No. 00-4769). Washington, DC: Government Printing Office.

32.

National Institute of Statistics of Rwanda, & Ministry of Finance and Economic Planning. (2014b). RPHC4 thematic report: Educational characteristics of the population. Kigali, Rwanda: Author. Retrieved from http://www.statistics.gov.rw/publication/rphc4-thematic-report-educational-characteristics-population

33.

National Institute of Statistics of Rwanda, & Ministry of Finance and Economic Planning. (2014c). RPHC4 thematic report: Population size, structure, and distribution. Kigali, Rwanda: Author. Retrieved from http://statistics.gov.rw/publication/rphc4-thematic-report-population-size-structure-and-distribution

34.

Piper

Zuilkowski

S. S.

(2015). Teacher coaching in Kenya: Examining instructional support in public and nonformal schools. Teaching and Teacher Education, 47, 173–183. doi:10.1016/J.TATE.2015.01.001

35.

Psacharopoulos

Patrinos

H. A.

(2004). Returns to investment in education: A further update. Education Economics, 12, 111–134.

36.

Ruterana

P. C.

(2012). The making of a reading society: Developing a culture of reading in Rwanda (Unpublished doctoral dissertation). Linköping University Educational Sciences, Linköping, Sweden. Retrieved from http://liu.diva-portal.org/smash/record.jsf?pid=diva2%3A549886&dswid=-6258

37.

Rwanda Ministry of Education. (2012). Education sector strategic plan. Kigali, Rwanda: Author. Retrieved from http://www.mineduc.gov.rw/shortcut/links/education-sector-strategic-plan/

38.

Rwanda Ministry of Education. (2016). 2015 Education statistical yearbook. Kigali, Rwanda: Author. Retrieved from http://www.mineduc.gov.rw/fileadmin/user_upload/Amatangazo/2015EducationStaiaticalYearbookF.pdf

39.

Schaalje

G. B.

McBride

J. B.

Fellingham

G. W.

(2002). Adequacy of approximations to distributions of test statistics in complex mixed linear models. Journal of Agricultural, Biological, and Environmental Statistics, 7, 512–524.

40.

Schuh Moore

A.-M.

DeStefano

Adelman

. (2011). Time misspent, opportunities lost: Use of time in school and learning. In Hawkins

J. N.

Jacob

W. J.

(Eds.), Policy debates in comparative, international, and development education (pp. 247–264). NewYork, NY: Palgrave Macmillan.

41.

Snow

C. E.

Burns

M. S.

Griffin

(1998). Preventing reading difficulties in young children. Washington, DC: National Academy of Sciences.

42.

UNESCO. (2000). The Dakar framework for action: Education for all: Meeting our collective commitments. Dakar, Senegal: Author. Retrieved from http://unesdoc.unesco.org/images/0012/001211/121147e.pdf

43.

UNESCO. (2013). The global learning crisis: Why every child deserves a quality education. Paris, France: Author. Retrieved from http://unesdoc.unesco.org/images/0022/002238/223826e.pdf

44.

UNESCO. (2015). EFA global monitoring report: Education for all 2000–2015: Achievements and challenges. Paris, France: Author. Retrieved from http://unesdoc.unesco.org/images/0023/002322/232205e.pdf

45.

UNESCO. (2016). Education for people and planet: Creating sustainable futures for all. Paris, France: Author.

46.

United Nations. (2013). The least developed countries report 2013: Growth with employment for inclusive and sustainable development. Paris, France: Author. Retrieved from unctad.org/en/PublicationsLibrary/ldc2013overview_en.pdf

47.

Wagner

D. A.

(2018). Learning as development: Rethinking international education in a changing world. New York, NY: Routledge.

48.

Williams

T. P.

(2017). The political economy of primary education: Lessons from Rwanda. World Development, 96, 550–561. doi:10.1016/j.worlddev.2017.03.037

49.

World Bank. (2017). DataBank: Education statistics—all indicators. Retrieved from http://databank.worldbank.org/data/reports.aspx?source=education-statistics-~-all-indicators&preview=on#

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.57 MB