Insights from investigating early childhood ebooks on literacy,cognitive development,and social and emotional learning outcomes

Abstract

Researchers were invited to investigate the effectiveness of bibliotherapeutic electronic books, or ebooks, social and emotional learning (SEL) digital platform in afterschool sites in a large school district situated in the Rocky Mountain region of the United States. This study was not particular to the platform’s efficacy; instead, we sought to explore how bibliotherapeutic ebooks can boost literacy, cognition, and SEL skills in young children. Researchers used a quasi-experimental design with nonequivalent treatment/control groups. Groups were assigned at the site level with individual students as the unit of analysis. Student data on literacy, cognitive ability, and SEL outcomes were collected pre-treatment and post-treatment. The researchers found no significant change in the studied groups. Upon digging deeper into SEL outcomes, researchers uncovered an invisible problem: the school district’s state-mandated measurement suite did not neatly map or align to the constructs within the afterschool curriculum for SEL. Interpretations and implications, including how districts define and assess SEL, are included in the discussion.

Keywords

Literacy social and emotional learning ebooks,assessment bibliotherapy cognitive ability

Introduction

Teaching children to be emotionally intelligent has been shown to improve behavior and conduct in classrooms, which can lead to healthier peer relationships exhibited schoolwide (Cipriano et al., 2019; Darling-Churchill and Lippman, 2016; Reyes et al., 2012; Webster-Stratton and Reid, 2004). Brackett (2019) suggests that emotional identification should be the starting point for all social and emotional skill acquisition. Being emotionally intelligent is also considered to be a predictor of future academic success (Ashdown and Bernard, 2012). In an early meta-analysis, social and emotional learning (SEL) skill-building was linked to an 11-percentile boost in children’s academic achievements (Durlak et al., 2011). In another meta-analysis, SEL interventions led to improvements in positive attitudes and prosocial behavior, as well as boosts in academic performance (Taylor et al., 2017).

The Collaborative for Academic, Social, and Emotional Learning (CASEL) and its SEL Framework have become increasingly adopted across schools throughout the United States (Brackett, 2019). Implemented in thousands of classrooms, the work of CASEL has reached over 30 million children in the past 25 years (CASEL, 2021). CASEL defines SEL as the process of how children “acquire and apply the knowledge, skills, and attitudes to develop healthy identities, manage emotions and achieve personal and collective goals, feel and show empathy for others, establish and maintain supportive relationships, and make responsible and caring decisions” (What Is SEL, 2021: 1).

Social and emotional learning is a set of teachable competencies or skills considered fundamental to success in school and life (CASEL, 2021; Zins and Elias, 2007). The CASEL SEL Framework focuses on five competency areas: self-awareness, self-management, social management, responsible decision-making, and social awareness (CASEL’s SEL Framework, 2020). In more detail, the five core competencies in CASEL’s SEL Framework are:

• Self-Awareness: the abilities to understand one’s own emotions, thoughts, and values and how they influence behavior across contexts.

• Self-Management: the abilities to manage one’s emotions, thoughts, and behaviors effectively in different situations and to achieve goals and aspirations.

• Social Awareness: the abilities to understand the perspectives of and empathize with others, including those from diverse backgrounds, cultures, & contexts.

• Relationship Skills: the abilities to establish and maintain healthy and supportive relationships and to effectively navigate settings with diverse individuals and groups.

• Responsible decision-making: the abilities to make caring and constructive choices about personal behavior and social interactions across diverse situations. (CASEL’s SEL Framework, 2020: 2)

Surrounding the five core competencies are the environments that children inhabit, including classrooms, schools, homes, and communities (Brackett et al., 2015; CASEL’s SEL Framework, 2020). Through a developmental lens, SEL skills should include “(a) emotional processes, (b) social/interpersonal skills, and (c) cognitive regulation or executive function skills” (Schonert-Reichl, 2020: 79). CASEL refers to this as Systemic SEL, the “nested, interacting settings and processes involved in Systemic SEL at proximal (classrooms, schools, families, communities) and distal (districts, states, national, international) ecological levels,” where children acquire and develop SEL skills (Mahoney et al., 2021: 1).

A Systematic SEL approach means that SEL skills start in subject area classrooms (Mahoney et al., 2021; Singh et al., 2020; Srinivasan, 2019). However, challenges may exist for teachers who seek to embed and infuse SEL lessons across all levels of curricula (Jones and Bouffard (2012). One approach is to integrate SEL in literacy classrooms, where teachers can weave in social and emotional lessons (e.g. empathy, perspective-taking) when analyzing written narratives (Tussey & Haas, 2021). Fitzgerald (2020) found opportunities in early childhood classrooms to teach literacy while simultaneously infusing goals associated with SEL.

Bibliotherapy, literacy, and SEL

Bibliotherapy is an approach to infusing SEL with literacy education (Heath et al., 2017; McCulliss and Chamberlain, 2013). A term first coined by Crothers (1916), bibliotherapy describes reading materials assigned “in order to expand an individual’s level of self-understanding and to expand the understanding of others’ perceptions” (Heath et al., 2017: 550). Examples are stories that embed moral and spiritual lessons, such as parables found in the Holy Bible and Aesop’s fables, tales from ancient Greece shared intergenerationally (Heath, 2017; Heath et al., 2017). Heath et al. (2017) proposed that bibliotherapeutic stories can be harnessed “to build a strong foundation of SEL that will positively influence children’s behavior” (p. 551).

Bibliotherapeutic stories have been found to enrich support for families through times of grief and loss (Bowman, 2021; Briggs and Pehrsson, 2008). In addition to trauma use cases, bibliotherapy can also promote empathy, compassion, and altruism in readers (Wilkinson, 2020). Empathy from reading literature is theorized to relate to an effect known as theory of mind, a concept that describes how readers vicariously experience fictional worlds (Bal and Veltkamp, 2013; Frith and Frith, 2005). Theory of mind can depend on the strength of narrative that mentally transports readers into fiction (Bal and Veltkamp, 2013).

Contemporary bibliotherapy carries on the tradition of using morals from literary narratives to teach and reinforce societal values and cultural norms (Başarı et al., 2018; Heath et al., 2017; Kottler, 2015). SEL-aligned changes observed have included increases in “empathy; positive attitudes; personal and social adjustment; positive self-image; new interests; tolerance, respect, and acceptance of others; realization that it is good in all people; socially accepted behaviors; and an examination of moral values, which can result in character development” (Cornett and Cornett, 1980 as cited in McCulliss and Chamberlain, 2013: 15). Many of these effects align in some way with competencies in CASEL’s SEL Framework. For instance, having empathy and demonstrating tolerance, respect, and acceptance of others aligns with capacities of self-awareness.

Heath et al. (2017) proposed a classroom-based bibliotherapeutic approach for children ages five through 11 that could be aligned with CASEL’s SEL Framework. Each lesson was “based on a carefully selected children’s picturebook that focuses on one of the CASEL competencies” (Heath, 2017: 454). A website was also created to share resources and lesson plan activities for educators: https://education.byu.edu/buildingsocialskills.

It is noted that bibliotherapy tends to be used as an approach to teach social skills, which are not necessarily SEL skills, and is broadly defined as “using books and reading material to find solutions for personal challenges” (Forgan, 2002; Heath, 2017: 453). Bibliotherapy has also been used for cognitive behavioral therapy (CBT) (Heath et al., 2017; McCulliss and Chamberlain, 2013). One example are Social Stories, narratives designed to model social situations for children diagnosed with autism spectrum disorder (ASD) (Gray, 2021). However, Social Stories are not quite bibliotherapy, as social skills are not necessarily SEL skills.

Can bibliotherapy teach children literacy and SEL skills? Heath et al. (2017) was the only case identified in the literature that attempted to align children’s bibliotherapy with CASEL. Although in the literature, no empirical research was shared about its efficacy.

Overview of the ebook platform

The authors of this paper were invited to investigate the effectiveness Peekapak, a digital literacy platform used in afterschool sites in a large school district situated in the Rocky Mountain region of the United States. Peekapak features bibliotherapeutic electronic books, or ebooks, as core to its SEL-aligned curriculum. This research aimed to investigate the impact of its bibliotherapeutic ebooks on related student-level factors, specifically, literacy, cognitive, and SEL outcomes.

Peekapak is a standards-aligned digital literacy platform featuring bibliotherapeutic stories with moral messages for early childhood learners (e.g. kindness, empathy, teamwork). Themed stories are available as leveled readers with varying text complexities that can be differentiated based on proficiency. There are four guided reading levels in English and two in Spanish. Fountas & Pinnel (F&P) Literacy is the service that provides guidance to its English leveled readers. Its F&P Text Level Gradient is a guided reading tool that aligns text levels to grade-level goals (Fountas & Pinnel Literacy, 2021).

The ebooks feature colorful illustrations accompanied by passages of written text and are also available in a set of printed books; our study focused on the digital curriculum. Tools native to ebooks can promote literacy development by providing young learners guidance as they read through texts (Morgan, 2013). For instance, many of these texts, such as those in Peekapak’s collection, include features that allow readers to highlight words and to hear passages read aloud (Morgan, 2013).

The Peekapak Web site explicitly states that its curriculum is “CASEL-aligned” (Peekapak, 2021). All of its lessons are mapped to CASEL’s five competencies model: self-awareness, self-management, social awareness, relationship skills, and responsible decision-making (CASEL, 2021). In total, Peekapak offers 10 bibliotherapeutic lessons (Peekapak’s “Core 10”) per grade level for students from pre-Kindergarten through grade 5, children who range in age from three to 11 in the state where this study took place. Specifically, the 10 lessons in the curriculum cover themes of self-regulation, respect, gratitude, kindness, empathy, honesty, perseverance, teamwork, optimism, and courage (Peekapak, 2021). An 11th lesson on Pandemic SEL was added in 2020, during the COVID-19 pandemic and after our data collection and analysis. This lesson was outside the scope of our study.

According to its cofounder, rather than offer lessons on the five CASEL domains, lessons “draw from different components simultaneously in a simple, dynamic, and digestible manner” (Shah, 2020: 6). In addition to CASEL’s SEL Framework, lessons were “inspired” by the Ontario Early Years Continuum of Development, and the Illinois Social Emotional Curriculum (Peekapak, 2021: 1). The storybooks are populated by a diverse assortment of human children and anthropomorphic characters (e.g. talking hedgehogs). Research shared on the Peekapak Web site points to the work of Maruyama (2010) on perspective-taking and emotional attachment that children may have towards animal characters. When animals are characters in stories, “attitudes toward animals and humans related to their perspective-taking abilities instead of comparing the group differences of these attitudes and abilities” have been observed (Maruyama, 2010: 73). Other research shared by Peekapak includes how emotive illustrations in picturebooks can teach emotional literacy (Nikolajeva, 2013). Kidd and Castano’s (2013) theory of mind study about evoking empathy through reading fiction is also cited. Research Behind Peekapak states, “Original stories are at the heart of the Peekapak program. Every story introduces the learning topic in an interesting and relatable way, and draws students into the lessons, while developing students’ empathy and theory of mind” (2021: 2).

Peekapak is more than a set of 10 leveled storybooks; it is a robust curricular platform of bibliotherapeutic picturebooks that include classroom and at-home SEL-aligned activities. When students log on to the Web site, they check in on a mood board in the online lessons, where they click on a face that matches their feelings to a range of moods: happy, excited, sad, angry, calm, or nervous. Teachers can view student moods on a dashboard while students journal about their feelings.

Each unit of the program is structured with a story and three “Big Ideas” intended to scaffold learning. The first introduces a skill and explains why it is important to cultivate, followed by different ways to demonstrate that skill. The final Big Idea “graduates” students to apply and internalize that new skill in their lives. Each unit also includes posters that can be printed or projected and detailed lesson plans to guide teacher implementation, whole-class discussions, activities, and ways to modify the lessons (Peekapak, 2021).

Each of the units also has a series of student learning games to help reinforce learning from the classroom. Set in the fictional village of myPeekaville, students are given opportunities to practice SEL skills virtually. In myPeekaville, players first are prompted to create avatars that resemble how they look in the real world. The stated purpose is to further immerse players by having them “become” Peekapak characters (Peekapak, 2021). Players then see a map of the myPeekaville village, including a path that winds around a town square and other buildings. Using avatars, players click on the path and buildings and can then engage in the mini-games. The mini-games are where children go on quests to help storybook characters overcome SEL challenges and manage emotions. One of the mini-games is a digital card game where players match emotions to characters’ faces, practicing SEL skills taught in the bibliotherapeutic stories (e.g. empathy for how a character is feeling) (Peekapak, 2021).

The curriculum explicitly recommends that children read the storybooks in class and at home, where lessons should continue. There, parents and caregivers are provided opportunities to reinforce vocabulary from the storybooks. SEL skills can also be practiced at home with play-based activities. This home-school connection aligns with Systemic SEL (Mahoney et al., 2021), where SEL should be taught and reinforced throughout the environments that children inhabit. Our study was limited to data collected at school-based sites, so home-level participation could not be observed or measured.

Method

When the authors adopted this study, it was initially designed to examine the effect of the intervention on factors measured by the school district’s currently existing measures of literacy, SEL, cognitive ability found in Teaching Strategies (TS) GOLD, the measurement suite assigned to this age group. In this type of study, there could be initial concerns about construct alignment between what is proposed to be measured and the mechanisms of impact in the afterschool program. Ideally, the intervention program itself would either come with its own validated measurement device to monitor progress or be pilot tested against a school’s measurement suite. Both of these scenarios require significant research, time, and money investment on behalf of the intervention program. While these are important steps, they are likely outside the budget and ability of many educational startup companies. The potential for this lack of alignment is an important issue to consider as more products are marketed to schools and families.

This study was not particular to the efficacy of the platform; instead, we sought to explore the relationships of how bibliotherapeutic ebooks can boost literacy, cognition, and SEL to young children. In order to investigate the relationships between literacy, cognition, and SEL scores, a quasi-experimental, nonequivalent treatment control group study was designed to answer the following questions:

(1) Does using Peekapak’s ebooks and curriculum impact student literacy, cognitive performance, or SEL scores?

(2) How might the nature of this impact inform future curriculum and educational outcome alignment?

Groups were assigned at the site level with individual students as the unit of analysis. Student data on literacy, cognitive performance, and SEL outcomes were collected pre-treatment and again post-treatment. Causal inference for the effect of an early childhood literacy platform on targeted outcomes is determined through a comparison of pre/post-change scores between the treatment and control groups.

A multiple analysis of variance (MANOVA) was conducted to determine if students who participated in the intervention developed greater literacy, cognitive ability, or SEL outcomes than those who participated in traditional afterschool programming. Change scores were calculated by subtracting fall scores from spring scores to measure growth over the duration of the school year.

Data collection

Data were collected during the 2018-2019 academic year. Eight afterschool sites served as the treatment group and three served as the control group. Classes contain a range of elementary grade levels (see Table 1). Only data from students enrolled in these grades during the study period was used in the sample. Basic demographic characteristics of the sample show a roughly equal distribution between student reported gender (Female = 48%, Male = 51%). Participants were roughly equal across grades as well, with about a quarter of students coming from Kindergarten, first, second, and third grades, children typically age five through eight in the state where this study took place. The majority of students were White (53%), but the sample also had a strong Hispanic representation (32%).

Table 1.

Descriptive statistics of participants (n = 1726).

Characteristic		Frequency (%)
Gender	Female	828 (48)
Gender	Male	880 (51)
Grade level	Kindergarten	483 (28)
	1st grade	414 (24)
	2nd Grade	414 (24)
	3rd grade	414 (24)
Race/Ethnicity	Asian	69 (4)
	Black	86 (5)
	Hispanic	552 (32)
	Native American	10 (<1)
	Pacific Islander	1 (<1)
	Two or more	100 (6)
	White	914 (53)

Measurements

Although Peekapak teaches literacy with its SEL-aligned narratives, it does not assess literacy, cognitive, or SEL scores on its platform. In this study, we relied on data shared by the school district from its state-mandated measurement and assessment programs. GOLD, a suite of measurement tools utilized by the school district, was used to assess children on literacy, cognitive ability, and SEL outcomes. GOLD assessment values were entered by the student’s primary school teacher; their observations were not directly linked to the afterschool environment of the program. Instead, it was hypothesized that student-level changes influenced by the afterschool program could be observed in daily classroom behavior and testing.

GOLD’s literacy measurements feature various educational objectives that play a strong role in factoring in a student’s overall ability to access the written elements of the Peekapak curriculum. Objective 15 assesses phonological awareness, phonics skills, and word recognition. Specifically, teachers check to see if students notice and discriminate rhyme, alliteration, discrete units of sound, and can apply phonics concepts and knowledge of word structure to decode text. Objective 16 pertains to knowledge of the alphabet, including the ability to name and identify letters, and the identification of letter-sound correspondences. Objective 17 pertains mainly to printed text in books. Objective 18 of literacy assesses the extent that the student corresponds to and responds to books and other texts. Here, the student is measured on the interaction during reading experiences, book conversations, and text reflections. Objective 18 also measures emergent reading skills, the ability to retell stories and recount details from informational texts, use context clues to read and comprehend texts, and read fluently. Lastly, Objective 19 assesses writing skills (writing the student’s name, conveying ideas and information, using writing conventions) (“GOLD Teaching Strategies,” 2017).

In addition to GOLD’s literacy measurements, cognitive outcomes were included as well. Cognitive behaviors on GOLD start with Objective 11, measuring the extent that the student attends and engages, persists, solves problems, shows curiosity and motivation, and shows flexibility and inventiveness in thinking. Objective 12, remembers and connects experiences, measures the ability to recognize and recall, and make connections. Objective 13 has one measurement: the ability to use classification skills. Objective 14 assesses the extent that the student uses semiotics, or symbols and images, to represent something not present. Observable behaviors for Objective 14 include the ability to think symbolically, and whether the student engages in sociodramatic play (“GOLD Teaching Strategies,” 2017).

There are three objectives of GOLD that pertain to SEL: (1) Regulates own emotions and behaviors; (2) Establishes and sustains positive relationships; and, (3) Participate cooperatively and constructively in group situations (“GOLD Teaching Strategies,” 2017). Observed behaviors for Objective 1, about regulating emotions and behaviors, include managing feelings, following limits and expectations, and taking care of needs appropriately. For children in the age range in this study, other behaviors in this domain include eating and drinking, toileting, and personal hygiene. Objective 2, which pertains to establishing and sustaining positive relationships, includes relationships formed with adults, responses to emotional cues, interactions with peers, and the ability to make friends. Objective 3, about participating cooperatively and constructively in group situations, has two observable behaviors: balancing the needs and rights of self and others, and the capacity to solve social problems. Each of these three Objectives is scored by the classroom teacher at four intervals throughout the school year (“GOLD Teaching Strategies,” 2017).

Results Matter is an initiative the state where this district is situated has used since 2005. The stated goal is to ensure that state and federal accountability is met with childhood assessment tools (Results Matter Handbook, 2018). GOLD was selected as one of three accepted vendors that met the necessary criteria (Results Matter Handbook, 2018). Cor Advantage, which is aligned to High Scope, and Assessment, Evaluation, and Programming System for Infants and Children (AEPS), are the other measurement tools, each of which are used to assess their respective sets of curricula. GOLD is different from the other two measurement programs because it does not include curriculum; it is strictly an assessment suite.

GOLD was among the three providers selected because it met or partially met certain conditions in the Results Matter rubric. Specific to our study, GOLD factors include: “Assessment tool aligns to research-based preschool curricula” and “Assessment aligns to the Colorado Early Learning and Development Guidelines.” Other factors pertained to who would be collecting and inputting assessment data (typically the teachers), affordances for individualized education plans (IEPs), and cost. The state’s reviewer notes mentioned that GOLD had “no indication in submitted materials on how using the assessment promotes improved teacher practices” (“Results Matter Preferred Criteria and Indicators,” 2018: 1).

Peekapak is optimally a home and school curriculum, which aligns to Systemic SEL, where SEL competencies are best cultivated and reinforced in all of the environments that children inhabit (CASEL, 2021; Mahoney et al., 2021). There is a section in the GOLD platform for teachers, but not afterschool providers, to upload “Family Observation” documentation answering questions, such as “How is your child holding a book?” To get this documentation, the teacher has to proactively contact parents to ask questions. It is also unknown whether parent documentation is accurate, as there may be bias in reporting on one’s own child.

Data analysis

A MANOVA was conducted to determine if any differences in growth existed between afterschool programs across literacy, cognitive abililty, and SEL domains. A MANOVA is used when the independent variable in a study is categorical, containing at least two levels (i.e. Peekapak and traditional afterschool programming) and more than one continuous outcome variable (i.e. literacy, cognitive ability, and SEL outcomes as measured in GOLD). The first step of conducting a MANOVA is to determine if any group differences exist overall and then to interpret values in the analysis to specify where any differences between groups may have occurred. The analysis was conducted using SPSS v.23, and the alpha significance level for the hypothesis tests was set at 0.05.

Results

The following results contain GOLD measurements of literacy, cognitive ability, and SEL outcomes taken before students were exposed to the afterschool conditions (i.e. “Pre”), after the programs had been completed (i.e. “Post”), and the change score between these two time points (i.e. “Change”). The change score is used to indicate growth over the school year. While all groups can be seen to show increases in all the constructs of interest, change scores between the intervention and control groups were later tested using a MANOVA. For literacy, we see fairly similar scores at pretreatment measurement with intervention and control reporting average item means of 593 (SD = 93) and 540 (SD = 124), respectively. This translates into a difference of 53 points between the two groups at the starting point. This difference does not change dramatically at the time of post-measurement (i.e. Intervention, 686 (SD = 80); Control, 630 (SD = 116); Difference of 56 points). Once these group baseline level differences are removed by examining change scores, we see much closer values between the two conditions (i.e. Intervention, 113 (SD = 47); Control, 124 (SD = 60); Difference of 11 points). A similar pattern can be seen when observing pre, post, and change scores across the remaining constructs of cognitive and SEL ability. Table 2 presents the values described above.

Table 2.

Descriptive statistics of TS GOLD literacy, cognitive, and SEL, pre, post, and change values (n = 1726).

Characteristics		Pre (mean SD)	Post (mean SD)	Change
Literacy	Intervention	593 (93)	686 (80)	113 (47)
	Control	540 (124)	630 (116)	124 (60)
	Total	546 (122)	637 (113)	123 (58)
Cognitive	Intervention	493 (85)	571 (99)	127 (61)
	Control	453 (112)	521 (117)	132 (65)
	Total	459 (110)	527 (116)	131 (65)
SEL	Intervention	394 (58)	458 (80)	89 (42)
	Control	374 (73)	427 (87)	87 (47)
	Total	377 (71)	431 (86)	87 (48)

In order to test if the differences in change scores between treatment and control groups was statistically significant, a MANOVA was conducted. Prior to performing the MANOVA, statistical assumptions were tested to ensure that the results of the analysis would be free of statistical bias, all of which were met. A MANOVA generates a F statistic to examine if the differences between treatments across outcome variables are significantly different from each other. This F statistic is converted into a p value which is then used to determine the statistical significance of the tests. Initial results of the MANOVA suggested that there were not any significant difference between groups and that further investigation was not warranted (F (2, 1726) = 203.58, p > 0.05). More specifically, even though there appeared to be some differences in change scores between the two groups on the measure of Social and Emotional ability (i.e. Intervention, 89 (SD = 42); Control, 87 (SD = 47; Difference of 2 points), this difference was not greater than the random variance or “noise” within the data.

A discussion exploring curriculum and educational outcome alignment is presented in the following section to address the second research question and the lack of observed impact in the intervention.

Discussion

Although Peekapak has produced promising research in the past, most results relied on teacher observations of student behavior and perceptions of student change. One such study sought to investigate teacher perceptions of the change in student behavior and social and emotional skill development over the 2017-2018 academic year (“Research Behind Peekapak,” 2020). One hundred and 17 teachers were surveyed based on their observations of students and the classroom. Reports showed pretest and posttest change scores increasing significantly over several dimensions of prosocial behavior, such as the number of students who shared with others. While this suggests growth in SEL components, it should be noted that they are based on teacher perception surveys, with teachers being the unit of analysis. Our study used measurements that were administered at the district level to measure SEL for all students. SEL scores were entered by teachers similarly to how math and reading scores are, but in this case, students were the unit of analysis. The difference in outcomes may be due to a discrepancy in how the teachers rate their overall perception of their class’ SEL levels versus those observed on an individual basis.

Another study referenced by “Research Behind Peekapak” found that students who used Peekapak five or more times scored significantly higher on a standardized language arts measure than students who did not use it at all (“Research Behind Peekapak,” 2020). It was not clear which age groups were used in this study, but the results from the current analysis did not yield the same outcome. The current sample used students between Kindergarten and third grade. If those who showed growth were older than this range, it may attribute to the greater use of Peekapak stories to increase literacy.

In 2019, a qualitative study examined teacher experiences in using the learning games on the Peekapak platform. Overall, reports were positive, with the myPeekaville game being viewed as a beneficial support for reinforcing concepts and allowing students to practice the skills they were learning (“Research Behind Peekapak,” 2020). Teachers also reported observing a greater use of SEL vocabulary, such as a 35% increase in the statement “My students feel comfortable sharing their thoughts and feelings with others” and a 46% increase in the statement “My students have the vocabulary to discuss how they are feeling” (“Research Behind Peekapak,” 2020). In light of these findings, it would be reasonable to expect some increase in student-level outcomes, but as noted, the majority of these results were based on teacher perception and observation rather than student outcome data, with the exception of the 2017-2018 standardized language arts literacy measure results.

In our study, students’ data were collected using standardized measures of literacy, cognitive ability, and SEL outcomes. It should be noted that all student outcomes did significantly increase between pretest and posttest observations, but the differences between students who used the Peekapak program were not significantly different than those students who attended traditional afterschool programing. This raises an interesting question: If teachers observe changes in students, why did they not appear in this study? The reasoning may be the alignment between Peekapak content areas and those measured by the school district.

The observed behaviors were recorded and inputted by the teacher, not the afterschool provider, to the GOLD platform. Further, measurement data were recorded quarterly, not daily. It is possible that short-term behavior changes may have been demonstrated, but not long-term changes that would be observed by classroom (not afterschool) teachers. Regarding Systemic SEL, the Peekapak program occurred in the afterschool program and possibly continued in the home. However, it was not used with students in other parts of their nested environments that they inhabit, such as classrooms, schoolwide (i.e. in school culture), or in the community. It is unknown if SEL lessons were practiced or cultivated in these other spaces.

Components that assess written language (spellings, pronunciations, phonics, phonemes, grammatical rules) are more clearly defined and aligned with some GOLD assessments. Many elements of Peekapak and the way that its curriculum needed to be conducted did not align with the standardized assessments taking place at the school level.

For new and innovative educational products, this study may have identified an “invisible problem.” If the constructs within an educational program are not explicitly aligned with those being used by a district—even if effects are occurring—they may not be possible to observe. Upon digging deeper, we found something potentially problematic: the school district in the study was tethered to the subscribed measurement suite; in this case, GOLD was pre-arranged (ahead of the researchers taking on this work) to report on literacy, cognitive, and SEL growth.

Interpretations and implications

The alignment of measures and constructs is a topic discussed in a range of disciplines such as organizational psychology, disability, and education (Adair et al., 2018; Hosbein and Barbera, 2020; Sullivan and Ford, 2010). This important concept is essential to accurate research as the measurement represents a value that is observed, and the construct is the topic or theme of interest in the study. Suppose the definition of the content of measurement is not identical to the construct of an intended treatment. In that case, it is not easy to determine if an ineffective treatment causes a null result or because the instrument of measurement was recording a different phenomenon.

The SEL curriculum did not neatly align to measures used by the school district. Only two areas of GOLD mapped to self-regulation and teamwork in Peekapak’s Core 10. Respect, gratitude, kindness, and honesty are traits of cooperative relationship building; they do not clearly align. Further, empathy and theory of mind, which are cited in Peekapak’s research relating to literary fiction, are not assessed by GOLD.

Although the field of SEL dates back decades, there are a sheer number of competing and overlapping frameworks that exist aside from CASEL’s. Berg et al. (2017) identified 136 frameworks in 14 SEL domain areas. Across these frameworks, terminologies are sometimes used interchangeably, even though there are nuanced differences. Parts of Peekapak’s Core 10, such as respect, honesty, and courage, align more closely to character education, not competencies explicit in the CASEL SEL Framework. “Perseverance” in the Core 10 may align with the CASEL SEL competency of “growth mindset” as well as a character education trait. Character education programs tend to focus on aligning students to a school’s set of values (e.g. being honest, demonstrating resilience, having perseverance) compared to social and emotional skills, which more closely aligns with emotional intelligence (Elias et al., 1997; CASEL, 2021).

Finn and Hess (2019) wrote that terminologies around SEL skills are sometimes ambiguous, making them subjective to assess. Nuances in terminologies cloud meaning, particularly when these meanings are assessed. The convolution and fragmentation of the terminologies, differences in competencies, and the sheer number of frameworks may be impeding the science needed to advance the field of SEL (Jones et al., 2016). For instance, self-management is a competency in some frameworks; in others, it may be described as self-control or self-regulation, “an umbrella construct” bridging “concepts and measurements from different disciplines (e.g. impulsivity, conscientiousness, self-regulation, delay of gratification, inattention-hyperactivity, executive function, willpower, intertemporal choice)” (Moffitt et al., 2011: 2693 as cited in Jones et al., 2016: 4). About the imprecision of SEL terminologies, Zhou (2020) wrote, “One school will focus on growth mindsets, another on restorative justice, another on the prevention of bullying, and so on” (p. 13).

We suspect that this is an example of something that may be occurring regularly: not great alignment between SEL interventions—and possibly literacy and cognitive interventions—with school district measures. Education agencies need processes that can be implemented with fidelity across multiple sites with minimal training and oversight.

Limitations

This study relied on a quasi-experimental design and would benefit from a randomized controlled trial design in future research. Other measures were not explored, such as the amount of time spent using the program and parental engagement.

Footnotes

Declaration of Conflicting Interests

This research was funded through a grant provided by Gary Community Investments. The payments were in no way connected to the product being evaluated.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Matthew Farber

Author Biographies

Matthew Farber, Ed.D. is an associate professor of Educational Technology at the University of Northern Colorado, where he co-directs the Gaming SEL Lab.

William Merchant, Ph.D. is an associate professor of Applied Statistics and Research Methods at the University of Northern Colorado, where he co-directs the Gaming SEL Lab.

References

Adair

Ullenhag

Rosenbaum

, et al. (2018) Measures used to quantify participation in childhood disability and their alignment with the family of participation‐related constructs: a systematic review. Developmental Medicine and Child Neurology 60(11): 1101–1116.

Ashdown

Bernard

(2012) Can explicit instruction in social and emotional learning skills benefit the social-emotional development, well-being, and academic achievement of young children? Early Childhood Education Journal 39(6): 397–405. DOI: 10.1007/s10643-011-0481-x

Bal

Veltkamp

(2013) How does fiction reading influence empathy? An experimental investigation on the role of emotional transportation. Plos One 8(1): 1–12. DOI: 10.1371/journal.pone.0055341

Başarı

Latifoğlu

Güneyli

(2018) Influence of bibliotherapy education on the social-emotional skills for sustainable future. Sustainability (Basel, Switzerland) 10(12): 4832. DOI: 10.3390/su10124832

Berg

Osher

Same

, et al. (2017) Identifying, Defining, and Measuring Social and Emotional Competencies: Final Report. Washington, DC: American Institutes for Research.

Bowman

(2021) Lessons from the field. family grief care and bibliotherapy: a call for models and studies. Family Relations 70(2): 402–407. DOI: 10.1111/fare.12539

Brackett

(2019) Permission to Feel: Unlocking the Power of Emotions to Help Our Kids, Ourselves, and Our Society Thrive. New York, NY: Celadon.

Brackett

Elbertson

Rivers

(2015) Applying theory to the development of approaches to SEL. In: Durlak

Domitrovich

Weissberg

, et al. (eds) Handbook of social and emotional learning: Research and practice. New York, NY: Guilford Press, 20–32.

Briggs

Pehrsson

(2008) Use of bibliotherapy in the treatment of grief and loss: a guide to current counseling practices. Adultspan Journal 7(1): 32–42. DOI: 10.1002/j.2161-0029.2008.tb00041.x

10.

CASEL (2021) CASEL. https://casel.org. (accessed 10 March 2021).

11.

CASEL’s SEL Framework (2020) CASEL. https://casel.org/wpcontent/uploads/2020/10/CASEL-SEL-Framework-10.2020-1.pdf. (accessed 10 March 2021).

12.

Cipriano

Barnes

Rivers

, et al. (2019) Exploring changes in student engagement through the RULER approach: an examination of students at risk of academic failure. Journal of Education for Students Placed at Risk 24(1): 1–19. DOI: 10.1080/10824669.2018.1524767

13.

Cornett

(1980) Bibliotherapy: The Right Book at the Right Time. Bloomington, IN: Phi Delta Kappa.

14.

Crothers

(1916) A literary clinic. The Atlantic. https://www.theatlantic.com/magazine/archive/1916/09/literary-clinic/609754/. (accessed 10 March 2021).

15.

Darling-Churchill

Lippman

(2016) Early childhood social and emotional development: advancing the field of measurement. Journal of Applied Developmental Psychology 45: 1–7. DOI: 10.1016/j.appdev.2016.02.002

16.

Durlak

Weissberg

Dymnicki

, et al. (2011) The impact of enhancing students’ social and emotional learning: a meta-analysis of school- based universal interventions. Child Development 82: 405–432.

17.

Elias

Zins

Weissberg

(1997) Promoting Social and Emotional Learning: Guidelines for Educators. Alexandria, VA: ASCD.

18.

Finn

Hess

(2019, April) What Social and Emotional Learning Needs to Succeed and Survive. Cambridge, MA: American Enterprise Institute.

19.

Fitzgerald

(2020) Overlapping opportunities for social-emotional and literacy learning in elementary-grade project-based instruction. American Journal of Education 126(4): 573–601. DOI: 10.1086/709545

20.

Fountas & Pinnel Literacy (2021). Heinemann. https://www.fountasandpinnell.com/. (accessed 10 March 2021).

21.

Forgan

(2002) Using bibliotherapy to teach problem solving. Intervention in School and Clinic 38: 75–82. DOI: 10.1177/10534512020380020201

22.

Frith

(2005) Theory of mind. Current Biology 15(17): R644–R646.

23.

GOLD Teaching Strategies (2017) Teaching strategies. https://teachingstrategies.com/solutions/assess/gold/. (accessed 10 March 2021).

24.

Gray

(2021) Carol gray social stories. https://carolgraysocialstories.com/. (accessed 10 March 2021).

25.

Heath

(2017) Addressing children’s social emotional needs with children’s literature. School Psychology International 38(5): 453–457. DOI: 10.1177/0143034317704478

(2017) Using Children’s literature to strengthen social and emotional learning. School Psychology International 38(5): 541–561. DOI: 10.1177/0143034317710070

27.

Hosbein

Barbera

(2020) Alignment of theoretically grounded constructs for the measurement of science and chemistry identity. Chemistry Education Research and Practice 21(1): 371–386.

, et al. (2016) What is the same and what is different? Making sense of the “non-cognitive” 1 domain: helping educators translate research into practice. Harvard Graduate School of Education. https://easel.gse.harvard.edu/files/gse-easel-lab/files/words_matter_paper.pdf. (accessed 10 March 2021).

29.

Jones

Bouffard

(2012) Social and emotional learning in schools: from programs to strategies. Social Policy Report 26(4): 1–33.

30.

Kidd

Castano

(2013) Reading literary fiction improves theory of mind. Science 342(6156): 377–380. DOI: 10.1126/science.1239918

31.

Kottler

(2015) Stories We’ve Heard, Stories We’ve Told: Life Changing Narratives in Therapy and Everyday Life. Oxford, UK: Oxford University Press.

32.

Mahoney

Weissberg

Greenberg

, et al. (2021) Systemic social and emotional learning: Promoting educational success for all preschool to high school students. The American Psychologist 76(7): 1128–1142. DOI: 10.1037/amp0000701

33.

Maruyama

(2010) The Effects of Animals on Children’s Development of Perspective-Taking Abilities. Dissertations and Theses. Paper 159. DOI: 10.15760/etd.159

34.

McCulliss

Chamberlain

(2013) Bibliotherapy for youth and adolescents-school-based application and research. Journal of Poetry Therapy 26(1): 13–40.

, et al. (2011) A gradient of childhood self-control predicts health, wealth, and public safety. Proceedings of the National Academy of Sciences 108(7): 2693–2698.

36.

Morgan

(2013) Multimodal children’s ebooks help young learners in reading. Early Childhood Education Journal 41(6): 477–483. DOI: 10.1007/s10643-013-0575-8

37.

Nikolajeva

(2013) Picturebooks and emotional literacy. The Reading Teacher 67(4): 249–254. DOI: 10.1002/trtr.1229

38.

Peekapak (2021) Peekapak. https://www.peekapak.com. (accessed 10 March 2021).

39.

Research Behind Peekapak (2020) Peekapak. https://s3-amazonaws-com-443.web.bisu.edu.cn/peekaplatform/media30/resources/PEEKAPAK_-_Research_%26_Design_Document_-_2019.pdf. (accessed 10 March 2021).

40.

Results Matter Handbook (2018, October) Colorado Department of Education. https://docs.google.com/document/d/13FcTDaKXOQpcSghF4VjQcD0q0LSRVd5sY_nHtg0vO7g/edit#. (accessed 10 March 2021).

41.

Results Matter Preferred Criteria and Indicators (2018) Colorado Department of Education. https://www.cde.state.co.us/resultsmatter/preferred-criteria-decisions-and-vendor-responses. (accessed 10 March 2021).

42.

Reyes

Brackett

Rivers

, et al. (2012) Classroom emotional climate, student engagement, and academic achievement. Journal of Educational Psychology 104(3): 700–712. DOI: 10.1037/a0027268

43.

Schonert-Reichl

(2020) Social and emotional learning: definitions, frameworks, and evidence. In: Singh

Duraiappah

Ramaswamy

(eds) Rethinking Learning: A Review of Social and Emotional Learning for Education Systems. New Delhi, India: UNESCO MGIEP, 53–86.

44.

Shah

(2020, October 1) How Peekapak aligns with CASEL. Peekapak http://blog.peekapak.com/how-peekapak-aligns-with-casel/.(accessed 10 March 2021).

45.

Singh

Duraiappah

Ramaswamy

(eds) (2020) Rethinking Learning: A Review of Social and Emotional Learning for Education Systems. New Delhi, India: UNESCO MGIEP.

46.

Srinivasan

(2019) SEL Every Day: Integrating Social and Emotional Learning with Instruction in Secondary Classrooms. New York, NY: W. W. Norton & Company.

47.

Sullivan

Ford

(2010) The alignment of measures and constructs in organizational research: The case of testing measurement models of creativity. Journal of Business and Psychology 25(3): 505–521.

48.

Taylor

Oberle

Durlak

, et al. (2017) Promoting positive youth development through school‐based social and emotional learning interventions: a meta‐ analysis of follow‐up effects. Child Development 88(4): 1156–1171. DOI: 10.1111/cdev.12864

49.

Tussey

Haas

(2021, March 21) Combining Social-Emotional Learning and Literacy in Early Childhood Classrooms. https://medium.com/educate-pub/combining-social-emotional-learning-and-literacy-in-early-childhood-classrooms-cf51f81d9b8e. (accessed 15 March 2022).

50.

Webster-Stratton

Reid

(2004) Strengthening social and emotional competence in young children—The foundation for early school readiness and success. Infants & Young Children 17(2): 96–113. DOI: 10.1097/00001163-200404000-00002

51.

What Is SEL ? (2021) CASEL. https://casel.org/what-is-sel/. (accessed 10 March 2021).

52.

Wilkinson

(2020) John stuart mill: recurrent depressive disorder; bibliotherapy; happiness, altruism, compassion and empathy – psychiatry in philosophy. British Journal of Psychiatry 216(6): 337–337. DOI: 10.1192/bjp.2020.39

53.

Zhou

(2020, April 27) Another education war? The coming debates over social and emotional learning. Phi Beta Kappan. https://kappanonline.org/another-education-war-social-emotional-learning-debates-zhao/. (accessed 10 March 2021).

54.

Zins

Elias

(2007) Social and emotion learning: promoting the development of all students. Journal of Educational and Psychological Consultation 17(2–3): 233–255. DOI: 10.1080/10474410701413152