Abstract
The President’s Committee on the Arts and Humanities report heartily supported arts integration. However, the President’s Committee called for a better understanding of the dimensions of quality and best practices. One promising arts integration method is drama-based pedagogy (DBP). A comprehensive search of the literature revealed 47 quasi-experimental DBP intervention studies conducted since 1985. The literature showed that designs were generally weak for making causal inferences and that outcomes other than achievement were infrequently studied. A meta-analysis of this research suggested that DBP has a positive, significant impact on achievement outcomes in educational settings. Effects were strongest when the intervention (a) was led by a classroom teacher or researcher rather than a teaching artist, (b) included more than five lessons, and (c) was integrated into English language arts or science curriculum compared to other domains. Positive effects across psychological and social outcomes were found. Implications for policy and practice are discussed.
The President’s Committee on the Arts and Humanities (2011) recently released their report on arts and education, which heartily supported arts integration as an effective and cost-efficient way to address teachers’ and students’ needs, referring to arts integration as “the most significant innovation in the field over the last two decades” (p. 40). Additionally, the President’s Committee called for a better understanding of the dimensions of quality and best practices—when, for whom, and what content areas are best served by arts integration methods. One promising arts integration method is drama-based pedagogy (DBP).
The effectiveness of DBP was assessed in one comprehensive meta-analysis almost 30 years ago (Kardash & Wright, 1986) and three more limited meta-analyses 20 years ago (Conrad, 1992) and 14 years ago (Conrad & Asher, 2000; Podlozny, 2000). However, a larger literature on the effects of drama-based strategies on students’ academic achievement and related academic outcomes has grown since these initial meta-analyses. The literature suggests that we do not as of yet have a clear understanding of the overall effectiveness of DBP and, perhaps more important, the conditions under which it is more or less effective. In the present report, we synthesize over 25 years of accumulated research to address the following overarching questions:
What does the cumulative research suggest regarding the impact of DBP on student outcomes, including academic outcomes, psychosocial functioning, and 21st-century skills?
Do characteristics of the intervention, students, or outcome influence the magnitude or direction of the effect of drama-based pedagogy?
Defining Drama-Based Pedagogy
Drama-based pedagogy describes a collection of drama-based teaching and learning strategies to engage students in learning. Teachers can use DBP to facilitate learning in both nondrama content (e.g., geometry, social/emotional skills, etc.) and drama-specific content. DBP focuses on an embodied process-oriented approach to learning. Specifically, the major defining features of DBP are the following: (a) it is facilitated and directed by a classroom teacher, teaching artist, or other facilitator trained in DBP; (b) it works toward academic and/or psychosocial outcomes for the students involved; (c) it focuses on a process-oriented and reflective experience; and (d) it draws from a broad range of applied theatre strategies (see Table S1, available online at http://RER.sagepub.com/supplemental). For a detailed description of all the DBP strategies discussed in this document, see Dawson and Lee (2014), and for an in-depth review of the definition of DBP used in this meta-analysis, see Lee (2013).
Researchers and practitioners have diversely referred to DBP as creative drama (McCaslin, 1996), story dramatization (Ward, 1986), process drama (Heathcote & Bolton, 1995), drama-in-education (Bolton, Davis, & Lawrence, 1987), theatre-in-education (Jackson, 1993), theatre of the oppressed (Boal, 1974), applied theatre techniques (Cawthon & Dawson, 2009), theatre games (Spolin, 1986), dramatic inquiry (Edmiston, 2013), enactment strategies (Willhelm, 2002), improvisation, and role-playing. Many of these labels have developed in different contexts and have a specific intention. For example, theatre games were used to develop young actors, whereas enactment strategies began as a way to work with English language arts (ELA) teachers. However, there is also an assortment of terms that describe drama activities that would not be considered DBP as we have defined it for this review, though they are likely to be confused with it. These include dramatic play, theatre for young audiences, and drama therapy. These concepts and related themes are outside the scope of this meta-analysis.
Theoretical Perspectives on Drama-Based Pedagogy
Broadly, DBP has been theorized to be an effective instructional approach likely to enhance achievement and other adaptive student outcomes compared to traditional instruction because it aligns with social constructivist ideas (Edmiston, 2013). In particular, facilitators rely on social and cultural understanding of the learners, scaffold the learning, and co-construct meaning through dialectical interactions with others and the environment (Vygotsky, 1978). Additionally, we suggest that DBP may be effective because it reflects an environment in which basic psychological needs for autonomy, competence, and relatedness are supported, which in turn facilitates students’ adaptive engagement and persistence (Deci & Ryan, 1985, 2008a, 2008b).
DBP explicitly attempts to align whole lessons and individual strategies with social constructivist theory. For example, during Image Work, the leader invites participants to create a frozen image of a concept with their bodies (e.g., freedom or justice). Next, the class interprets the frozen image by describing the details of the image that are based on concrete evidence (e.g., “Her hand is placed over her heart”). Then, the leader asks the students to analyze what that image might mean (e.g., “She is saying the pledge or she loves her country”). Finally, the leader asks the students to relate the image to an example of freedom that they may or may not see in their lives. Throughout this process, the leader invites the social and cultural understanding of the learner to influence their interpretations of the image. In addition, the leader is scaffolding their learning through building on familiar ideas to relate to larger concepts. In this way, the leader encourages multiple interpretations of the images such that the students’ individual and collective understandings of the concept may be broadened, challenged, and/or confirmed.
The somewhat limited research on social constructivist teaching suggests that student learning through social constructivist approaches leads to greater social and academic gains (W. S. Barnett et al., 2008; Terwel, Oers, Dijk, & Eeden, 2009) and deeper connections (Webb & Mastergeorge, 2003), and students make more unique contributions in class (Yager & Akcay, 2008). In sum, to the extent that DBP is in line with tenets of social constructivist theory, it is expected that students in DBP classrooms will have a deeper understanding of content material compared to students in traditional classrooms.
Additionally, we might also expect DBP to lead to benefits in learning and learning-related outcomes as it aligns with the motivation theory self-determination theory (SDT). Specifically, SDT posits that environments that support basic psychological needs for autonomy, competence, and relatedness will be more facilitative of students’ engagement and persistence (Deci & Ryan, 1985, 2008a, 2008b). The need for autonomy reflects an individual’s desire for her or his actions to emanate from the self (DeCharms, 1968; Ryan & Deci, 2000). The second psychological need is competence, which is the desire of individuals to be effective in interactions with the environment (Harter, 1982; Ryan & Deci, 2000). The final psychological need is for relatedness (or belongingness), which reflects most individuals’ desire to establish a close emotional bond and attachment with other people (Baumeister & Leary, 1995; Ryan & Deci, 2000). In line with theory and research from SDT, many practices of DBP would also seem supportive of psychological needs. More specifically, it is standard practice in DBP for the leader to set up a structure for learning (e.g., provide instructions or introduce the material to be learned) and then allow students to direct the action and make choices about learning throughout the process. For example, the leader may ask the students to reflect on their work during a theatre game, rate their performance, decide on strategies to make it better, and then try the theatre game again.
Likewise, the structure of DBP supports a student’s need for perceived competence. For example, within a role drama, students may create characters based on their prior knowledge and experiences. If they have little prior knowledge of the specific topic, they are able to participate to the extent that they are comfortable. Leaders offer support to students by discussing potential characters as a whole as well as the motivations and opinions of these characters. Thus, students who have less prior knowledge may choose to adopt one of these thoroughly discussed characters or students who have extensive prior knowledge may choose to create an entirely different character.
Finally, DBP practices may be supportive of relatedness by design. Many of the DBP approaches include theatre games that are intended to facilitate community development and team building among the participants. For example, in Jump, the teacher and students are in a standing circle. The group challenge is for everyone in the circle to jump simultaneously. There cannot be a leader, but rather, the group must work together and attune to the movement within the circle such that every student jumps together. When this is accomplished, students tend to attribute their success to the cohesiveness of the group.
Students’ psychological need satisfaction in the classroom as theorized by SDT has been found to positively relate with their interest in the academic material (Minnaert, Boekaerts, & De Brabander, 2007), academic engagement (Park, Holloway, Arendtsz, Bempechat, & Li, 2012) and flow or total involvement in a task (Kowal & Fortier, 1999). In sum, the design of DBP may readily support the psychological needs of students (autonomy, competence, and relatedness). Given this enhanced support for psychological needs, it is expected that DBP will better support students’ engagement in tasks, persistence toward goals, and academic interest compared to the traditional classroom.
The Effects of Drama-Based Pedagogy
When looking at research over the past 25 years, the outcomes measured in DBP research can be categorized into three categories: (a) immediate achievement and learning outcomes in nondrama curriculum, (b) immediate achievement and learning outcomes in drama curriculum, and (c) other psychological and social outcomes related to learning, including engagement, attitudes toward school and academics, academic self-concept, and prosocial attitudes and behavior.
Achievement and Learning Outcomes in Nondrama Curriculum
A great deal of research has examined the effects of DBP on immediate achievement across a variety of subject areas. However, despite the many hypothesized learning benefits of DBP, findings have been mixed. For example, although some studies focused on language arts curriculum have suggested that DBP leads to positive effects on oral and written language outcomes (Hendrickson & Gallegos, 1972; Moore & Caldwell, 1990; Niedermeyer & Oliver, 1973; Wagner, 1986, 1990), other studies have found no effects of DBP on these outcomes (Harris & Rosenberg, 1983; Ingersoll & Kase, 1970; Lawton & Brandon, 2005; Stewig & Vail, 1985). For example, Moore and Caldwell (1990) found that students who participated for 15 weeks in 45-minute drama sessions that focused on developing characters had significantly greater gains in their writing skills than students who received traditional language arts during the same period. Alternatively, Lawton and Brandon (2005) found no significant differences between schools who participated in a yearlong DBP intervention and control schools on reading achievement as measured by a national standardized test, controlling for preintervention assessment of the outcomes.
In contrast to the mixed results in language arts, the effects of DBP on achievement outcomes in other domains have been predominantly positive. For example, many studies have found positive DBP effects on academic achievement in science (Braund, 1999; Dorion, 2009; Francis, 2007; Kase-Polisini & Spector, 1994; Sloman & Thompson, 2010; Warner & Andersen, 2004), foreign language acquisition (Bournot-Trites, Belliveau, Spiliotopoulos, & Seror, 2007; Erdman, 1991; Shacker, Juliebö, & Parker, 1993), and math achievement (Fleming, Merrell, & Tymms, 2004; Kayhan, 2009; Walker, Tabone, & Weltsek, 2011), among others. Though not as prevalent as other achievement areas, studies measuring 21st-century skills—such as creativity, collaboration, and communication—as designated by Partnership for 21st Century Skills (2009) also present a mixed picture. For example, in one 18-week DBP intervention study in a theatre class, treatment students actually displayed negative effects in communication skills per an observer rating scale (Ballou, 2000). In contrast, another study found significant gains in creativity after a 20-week DBP intervention in a language arts classroom (Fischer, 1989).
Achievement and Learning Outcomes in Drama Curriculum
Very few studies have measured outcomes in drama learning (e.g., role-taking ability). However, although many of these studies have found positive results (Huntsman, 1982; Karakelle, 2009; Rosenberg, Pinciotti, & Smith, 1983; Wright, 2006; Yeh & Li, 2008), some have found no effect for DBP (Harris & Rosenberg, 1983). For example, Wright (2006) found that upper elementary students who participated in 10 weeks of creative drama sessions experienced significant growth in role-taking abilities as compared to control students. In contrast, Harris and Rosenberg (1983) found that a 15-week creative drama intervention focused on role taking and affective responses had no effect on sophomore English students’ ability to display and discuss an affective response to literature from a character’s point of view.
Psychological and Social Outcomes
In addition to studying academic gains, some studies have focused on the effects of DBP on psychological and social outcomes potentially related to learning, including attitudes toward academics and school, self-perceptions of competence (i.e., self-concept, self-discrepancy, and self-efficacy), and prosocial attitudes and behavior. No clear picture has emerged regarding the effects of DBP on school and academic attitudes—that is, although some studies have found DBP to enhance attitudes (Bournot-Trites et al., 2007; Fleming et al., 2004; Francis, 2007; Gourgey, Bosseau, & Delgado, 1984; Walsh-Bowers & Basso, 1999) or self-perceptions of competence (e.g., Fleming et al., 2004), other studies have found no difference in the effects of DBP on attitudes toward academics (Freeman, Sullivan, & Fulton, 2003; Kariuki & Humphrey, 2006) or self-perceptions of competence (Huntsman, 1982).
For example, Kariuki and Humphrey (2006) found that students in a classroom where DBP was used to teach math had significant gains in achievement but no change in attitude toward math when compared to a control group. In a similar study, Fleming et al. (2004) found that students in the control group actually had significantly more positive attitudes toward math than those in the matched experimental group, though the experimental group reported a more positive math self-concept. However, there appears to be relatively high consistency among studies that have examined the effect of DBP on prosocial attitudes and behavior, with most revealing positive effects on outcomes including attitudes toward marginalized groups such as older adults (Bramwell, 1990), LGBT (lesbian, gay, bisexual, and transgender) youth (Aldredge, 2010; Hanley & Gay, 2002), victims of bullying (Burton, 2010; Merrell, 2005), and students with disabilities (Miller & Rynders, 1993).
In sum, a range of outcomes have been studied in an attempt to understand the effects of DBP. In all, research findings have largely revealed inconsistent findings across most categories of outcomes, and the debate that started in the 1960s regarding the overall effectiveness of DBP has continued to the present day.
Previous Reviews of DBP Intervention Research
As suggested earlier, DBP may have a critical, positive impact on instruction and learning in the classroom. To that end, researchers and practitioners need to have a nuanced understanding of how DBP may be most profitably used for educational outcomes. Four meta-analytic studies have been conducted to synthesize the effects of DBP on academic and social/emotional outcomes including Kardash and Wright (1986), Conrad (1992), Conrad and Asher (2000), and Podlozny (2000).
Results from early research syntheses have presented a mixed picture of the effects of DBP on language arts outcomes (Kardash & Wright, 1986; Conrad, 1992). Results from the earliest meta-analysis conducted by Kardash and Wright (1986) suggested that drama-based strategies had a significant positive effect on oral language skills (d = 0.46), but not on reading (d = −0.05) among elementary school–age students. Other meta-analyses (e.g., Conrad, 1992; Podlozny, 2000) suggested that the effects of DBP (0.06 < d < 0.77) were positive among elementary students across various language arts outcomes (e.g., reading, reading readiness, vocabulary, oral language, and writing).
Likewise, two of the four meta-analyses have summarized effects on self-perceptions of competence and have come to different conclusions. Although Kardash and Wright (1986) found that drama-based strategies had a positive effect on self-esteem among elementary students (d = 0.42), Conrad and Asher (2000) found no effect on self-concept or self-esteem among elementary students (d = 0.01), despite including studies from a similar timeframe. Prior meta-analyses have given only limited attention to achievement in domains outside of language arts. However, those that have examined the effects of DBP on achievement in other domains have generally suggested positive effects in math achievement (d = 0.29; Conrad, 1992) and drama skills (0.24 < d < 1.53; Kardash & Wright, 1986).
Despite the undeniable contributions of these early syntheses (Conrad, 1992; Conrad & Asher, 2000; Kardash & Wright, 1986; Podlozny, 2000), they also have a number of limitations: namely, previous reviews are at this point outdated and limited in scope—they focused primarily on achievement and self-concept in elementary language arts classrooms. A preliminary review of the most recent research articles suggests that a significantly broader set of outcomes have been examined in the past 25 years, including creativity, attitudes toward school, prosocial attitudes and behavior, as well as achievement in science and math. Likewise, although three of the earlier meta-analyses focused on elementary school–age students and one of the meta-analyses included participants up to eighth grade, more recent research on effects of DBP has also been conducted with middle school, high school, and college students. Given the growth in DPB research across the past 25 years, as well as the broadening of focus in both content areas and student demographics, an updated synthesis of the research is warranted.
Factors That May Influence the Effects of DBP
Mixed findings suggest that the effects of DBP may be very complex and not unequivocally beneficial. Previous meta-analyses found multiple significant moderators. Two meta-analyses examining the effect of DBP on academic outcomes with elementary students suggested that the effect size was inversely related to the age of the student through elementary school (Conrad, 1992; Kardash & Wright, 1986). In line with this finding, it may be that as students advance to middle and upper elementary, they may begin to focus less on play and more on productive work (Erikson, 1959). Thus, DBP may be seen as simple play and not as beneficial for older students. In contrast, Podlozny (2000) found in her meta-analysis that DBP had a more positive effect among older compared to younger elementary school students. Thus, alternatively, it may be the case that DBP is more effective for older elementary students because they become less egocentric (Piaget, 1952). That is, the early focus on the self among younger elementary students could supplant many of the key beneficial attributes of DBP, namely, relatedness and role-playing. In sum, we can expect that the effects of DBP may be moderated by the developmental maturation of the participants, though the exact pattern of this effect is difficult to predict.
DBP may also be differentially effective for female compared to male students, but it is unclear why this difference may exist. The existing research has been inconsistent regarding the role of gender. One meta-analysis found a slight advantage of DBP on language arts academic outcomes for groups that had more participating females than males in the research study (Kardash & Wright, 1986), although another meta-analysis found no gender difference (Conrad, 1992). In general, language development occurs earlier and language performance is better among females compared to males, even among children as young as 2 or 3 years (Burman, Bitan, & Booth, 2008), which could explain why DBP may have a stronger effect for females in language-based outcomes. However, the role of gender remains uncertain in the effects of DBP on academic outcomes in other domains.
One factor that may contribute to the effectiveness of DBP is the duration of the intervention. If students are familiar with learning in a more traditional classroom setting, then it may take time to adjust to the more active strategies used in DBP. Even students who are readily active in learning may need practice in how to participate in theatre games or role-playing in an educational setting. Based on past meta-analyses, there seems to be little difference between interventions that last a few days and those that last a few weeks, but there was a significantly stronger effect for more intensive DBP interventions that spanned 12 weeks to a year or more (Conrad, 1992; Deasy, 2002). In contrast, one meta-analysis found no significant difference in effects by the length of the intervention (Conrad & Asher, 2000). Complicating matters more, Kardash and Wright (1986) found that the effect size was inversely related to the minutes per session, was positively related to sessions per week, and had no relationship to the span of the DBP treatmen. In sum, although we might expect that interventions that include brief individual sessions frequently over a long period of time may yield stronger findings, mixed findings in past DBP research suggest that this moderator deserves further investigation.
Using Chi’s (2009) framework for instructional strategies, theory suggests that the type of DBP strategy may differently affect learning outcomes. According to Chi’s theory, interactive strategies are expected to have significantly better effects on learning outcomes compared to constructive strategies and constructive strategies are theorized to have a more positive effects compared to active strategies. According to this framework, in constructive instructional strategies, learners make inferences that go beyond the given information by explaining their reasoning or making new connections between ideas; however, in interactive strategies, students dialogue extensively about the topic and incorporate one another’s ideas, building on a partner’s ideas, revising personal misconceptions, or challenging and/or defending ideas, among other activities.
In DBP, strategies may vary along this dimension and thus, may vary in their effects. For example, improvisation strategies, which typically require students to build on prior knowledge through dialogue with another person (i.e., interactive), may yield more positive effects than story dramatization, which usually involves retelling a story through movement and repetition of a story (i.e., constructive). In Kardash and Wright’s (1986) meta-analysis, the researchers found that the type of creative drama treatment was a significant moderator. Although they did not classify the strategies using the same instructional strategy framework (Chi, 2009), it seems that they allude to two similar categories. Treatments that incorporated improvisation strategies effected student outcomes by half of a standard deviation greater than treatments that only incorporated story dramatization (Kardash & Wright, 1986).
The effects of DBP may also vary depending on the experience level of the facilitator and how the lesson is developed and/or delivered. With only a basic understanding of DBP, facilitators may lead a series of theatre games that will encourage students to be active (Chi, 2009) but may not invite students to thoughtfully consider applications to the curriculum. In other words, these facilitators may see DBP as a fun activity rather than a learning opportunity. Thus, we might expect to find DBP to have stronger positive effects when delivered by more experienced facilitators. Prior research has provided support for this hypothesis. One meta-analysis found that the largest effect sizes were associated with studies led by the most experienced facilitators (Kardash & Wright, 1986).
Two characteristics of the outcome may also contribute to the effect of DBP including the alignment between the outcome and the treatment and the targeted domain for the DBP intervention. The alignment between the measured outcome and the DBP treatment may influence the observed effect. For example, a study that measures standardized math achievement as an outcome of DBP use in the language arts curriculum is poorly aligned (Walker, Tabone, et al., 2011). Alternatively, a study may use DBP to teach role-taking ability and then measure a student’s ability to empathize and assume alternative perspectives (Wagner, 1986). This circumstance would be considered a close or proximal alignment between the treatment and the outcomes measured. It may be expected that more proximal or closely aligned treatment and outcomes would show a stronger effect size than poorly aligned treatment and outcomes.
Thus far, no meta-analyses have tested the outcome domain of interest (e.g., language arts, science, etc.) as a potential moderator for the effects of DBP. It may be that DBP is more closely related to language arts concepts and skills compared to other domains such as mathematics, and thus, DBP may facilitate a stronger, more positive effect in domains such as language arts, where narrative is part of the subject matter. Alternatively, domains that are typically taught using noninteractive strategies may particularly benefit from the additional ways of teaching the same material. To contribute to a fuller understanding of the impact of DBP, this potential moderator needs to be explored.
The Present Meta-Analysis
Twenty-five years of research has accumulated since the most comprehensive meta-analysis of DBP (Kardash & Wright, 1986). Furthermore, the current research on DPB seems to have become broader in both the samples and outcomes examined. Finally, a number of factors seem likely to moderate the effects of DPB, some of which prior meta-analysis have been able to examine and some of which can be examined with a new synthesis of the research. This project will summarize research examining the effects of DBP among preschool through college students and attempt to address the following research questions: (a) What does the cumulative research suggest regarding the impact of DBP on student outcomes including academic outcomes, psychosocial outcomes, and 21st-century skills? (b) Do characteristics of the intervention, students, or outcomes influence the magnitude or direction of the effect of DBP?
We predict that overall, the effect of DBP will be positive across all students. However, we also expect to find stronger, positive effects for (a) elementary school–age students compared to older students and for (b) female compared to male students. We also expect that more positive effects will result when (c) students experienced interventions that included frequent, brief sessions that occurred over a long period compared sessions are infrequent or the intervention as a whole is brief; (d) strategies used were interactive compared to constructive or active; (e) the intervention is delivered by a more experienced facilitator, and (f) outcomes are well aligned with the intervention goals. Although we do not have a hypothesis about the role of subject domain, we believe that this moderator is also important to examine in this meta-analysis in order to provide theoretical and practical guidance.
Method
Research syntheses primarily focus on empirical studies and seek to summarize past research by drawing overall conclusions from multiple, separate investigations that address related or identical topics. This project employed state-of-the-art methods to perform the research syntheses (Cooper, Hedges, & Valentine, 2009). These methods involved an approach that views research synthesis as a data gathering exercise and applies criteria similar to those employed to judge the validity of primary research (Cooper, 1998). The approach required (a) precise problem definition; (b) exhaustive and unbiased gathering of the research evidence; (c) careful examination of the strengths and weaknesses of the included research; (d) appropriate methods for data integration, including meta-analysis; (e) cautious interpretation of the cumulative evidence; and (f) complete reporting of the syntheses’ methods and results.
Study Inclusion Criteria and Search Strategies
For a primary study to be included in this research synthesis, several criteria had to be met. Most important, each study had to have assessed in some way the relationship between DBP as defined earlier and a student outcome, including academic achievement in drama or nondrama outcomes, attitudes toward academics, or another measure of student psychosocial functioning, such as self-concept. The studies included in the meta-analysis must all be experiments or quasi experiments with at least one experimental and one control group. This criterion means that participants in the intervention condition received a DBP treatment whereas participants in a control condition did not; most of the latter typically received traditional instruction. Only studies conducted in educational settings during school hours with preschool through college students were included. We included only samples of students who were typically developing as indicated by the researchers of the study. In addition, only studies with samples from the United States, Canada, United Kingdom, and Australia were included due to the similarity in typical schooling and shared language. Finally, enough information had to be provided in order to calculate an effect size.
Using a broad set of search strategies, we attempted to identify and retrieve the entire population of published and unpublished studies that examined the relationship between DBP and student outcomes since the last major review of the literature (Kardash & Wright, 1986). First, we searched PsychInfo, ERIC, Dissertation Abstracts International, Academic Search Complete, the International Bibliography of Theatre and Dance, American Economic Association, and Google Scholar electronic databases for documents catalogued since October 1985 (the end time point that Kardash and Wright used to gather reports in their meta-analysis) using each of the following keywords: Creative Drama* OR Drama Based OR Applied Theatre OR Process Drama OR Arts Integration OR Improvisation.
Then, we conducted a hand search of journals, including Journal for Learning Through the Arts, The International Journal for Learning Through the Arts, Youth Theatre Journal, Research in Drama and Education, and Children’s Theatre Review. In addition, the Social Sciences Citation Index database was used to search for documents catalogued since 1985 that had been cited by the four previous meta-analyses. Then, we searched advocacy and research websites affiliated with arts and education, including Americans for the Arts, the Chicago Arts Partnerships in Education, the Southeast Center for Education and the Arts, the Community Arts Network, and the Arts and Learning Special Interest Group through the American Education Research Association.
Next, we employed five strategies to directly contact researchers who may have studied DBP. First, we contacted the dean, associate dean, or chair of the 98 colleges, schools, or departments of education at doctoral-granting institutions of higher education with high research productivity and requested that they ask their faculty to share with us any research they have conducted that related to DBP. Second, we contacted researchers who have been the first author on two or more articles on DBP during the past 10 years. Third, we contacted the director of research in the regional labs for educational research. Fourth, we contacted recipients of grants from Department of Education Arts and Education Model and Dissemination and the National Endowment for the Arts. Finally, the Reference sections of relevant documents were examined to determine if any cited works might be relevant to our topic. A search using these strategies located a total of 2,892 nonduplicate, potentially relevant documents.
Titles and abstracts for each document were examined by the first author, and any article that mentioned assessing the effects of DBP was retained. Further inspection of full texts of 132 documents retained after this initial screening suggested that there were 47 reports that met inclusion criteria. Topic-relevant reports were excluded for various reasons. Most commonly, studies were excluded because the design did not include a control condition. The second most common reason studies were excluded was due to the country of the sample, most often Turkey and Singapore. Finally, many studies did not provide information to calculate an effect size, and attempts to contact authors to obtain this information were unsuccessful. Only a small number of studies were excluded because the sample was not a kindergarten through college sample or because the sample was from a non–typically developing population.
Information Retrieved From Primary Research
Numerous different characteristics of each study were included in the database. These characteristics encompass six broad distinctions among studies: (a) the research report, (b) the research design, (c) the intervention/DBP variable, (d) the sample of students, (e) the student outcome measure, and (f) the estimate of the relationship between DBP and the student outcome. Table 1 outlines the characteristics of studies that we coded.
Outline summary of coding guide
Note. DBP = drama-based pedagogy.
Variability in Research Study Designs
Generally, the studies did not vary in design such that most studies conducted a quasi-experimental design with a nonequivalent control group. Generally, students were put into groups because their teacher or school principal self-selected the school or classroom to be in the intervention. Additionally, most studies made no effort to equate groups through matching or post hoc statistical methods. Only one study randomly assigned participants to condition at the student level (Warner & Andersen, 2004); however, four studies used random assignment at the school or classroom level (Fizano, 1999; Rose, Parks, Androes, & McMahon, 2000; Walker, McFadden, et al., 2011; Walker, Tabone & Weltsek, 2011). but used the student as the unit of analysis with no attempt to equate participants at the student level.
There was one quasi experiment that matched groups on gender only (Cooney, 1999) and one quasi experiment that matched students on a constellation of premeasures including reading level, an at-risk index, and socioeconomic status (Ballou, 2000). We would have liked to compare the effects of different research designs; however, these more rigorously designed studies measured different types of outcomes (e.g., attitudes toward school or achievement). Therefore, no analyses between differing research designs could be conducted. Occasionally, studies measured outcomes throughout the study (every week); however, so few studies did this that the weekly measures could not be included in the analysis. We did, whenever possible, compute adjusted effects by preintervention scores of the outcome because we had almost exclusively quasi-experimental studies with nonequivalent control group designs.
Variability in Study Samples and Interventions
Generally, the studies reported a relatively even distribution of females to males. This sample of studies did vary in duration of the intervention, grade level of the intervention, and domain of the intervention. The duration of the intervention ranged from 1 lesson to 100 lessons. Studies used various grade levels as their samples, from preschool through college students. Although ELA represented the majority of the studies, many other domains were represented, including science, social studies, math, theatre, and foreign language. We also decided to include one study (Cooney, 1999) that used DBP to teach theatre because this was compared to and varied from the traditional mode of instruction for theatre (e.g., lecture).
Variability in Outcomes
It is important to note the variability of studies within each outcome as well. Generally, achievement outcomes included two types of measures: (a) a test on specific academic content or (b) a standardized test on broad academic content. Outcomes related to 21st-century skills measured things such as critical thinking, creativity, and self-regulatory skills through validated measures, teacher- or researcher-designed measures, observer ratings (e.g., coding a writing sample), or interviews. Outcomes in social skills varied, including problem behavior, ability to make friends, conflict resolution, cooperation, and recognizing emotion in others. These were measured by observer ratings (e.g., teachers coding for friendship making), validated self-report measures, or researcher-developed self-reported surveys.
Attitudes toward academics, attitudes toward others, and attitudes toward schools were always measured through a validated self-report survey. For the outcome of attitudes towards others, studies included measures of attitudes toward elderly, attitudes toward students with disabilities, and attitudes toward bullying and bullies. Self-perceptions of competence were measured through either a research-developed survey or a validated survey and included measures of self-concept, self-esteem, self-efficacy, and self-discrepancy. Although these outcomes each have specific research literatures, many of the terms overlap and are closely related and are categorized as such for this present study. Motivation was measured in various ways, including desire to learn, engagement in the process, time on task, attention, and interest. These outcomes were measured by observer ratings and validated self-report measures.
Coder Reliability
Following coding training in which coders were provided detailed definitions, descriptions of codes, opportunities to help revise the coding guide, and practice coding, two coders extracted information from all reports selected for inclusion. All studies were coded by the first author and a second coder who was either a graduate or an undergraduate student. Discrepancies were noted and discussed by the coders, and if agreement was not reached, a third coder, the second author, was consulted. All discrepancies were resolved. Prior to resolving conflicts, the coders had a 94% rate of agreement for 33,000 codes; however, the code for strategies used in the DBP intervention was particularly problematic due to its subjective nature and accounted for 22.7% of the discrepancies.
Statistical Procedures
Effect Size Estimation
The standardized mean difference or the d index (Cohen, 1988) was used to estimate effects. In this synthesis, we subtracted the control condition postintervention outcome mean from the DBP intervention condition postintervention outcome mean and divided the difference by their pooled standard deviation. Thus, positive effect sizes indicate that students who received DBP had more positive outcomes than students who did not receive DBP. When available, we calculated effect sizes based on the means and standard deviations of the student outcomes. If means and standard deviations were not available, we retrieved the information needed to calculate d indexes indirectly from inferential statistics (see Lipsey & Wilson, 2000). In addition to retrieving or computing unadjusted postintervention effect sizes, effect sizes that adjusted or controlled for the outcome variable prior to intervention were also retrieved or calculated if the information was available.
Methods of Data Integration
First, the distribution of effect sizes and sample sizes was examined to determine if any were statistical outliers. Grubbs’s (1950) test was applied (see also V. Barnett & Lewis, 1994). If outliers were identified, these values were set at the value of their next nearest neighbor. We used inverse variance weighted procedures to calculate average effect sizes across all comparisons (Borenstein, Hedges, Higgins, & Rothstein, 2005). Also, 95% confidence intervals were calculated for average effects. If the confidence interval did not contain zero, then the null hypothesis of no DBP/non-DBP difference was rejected. Possible moderators (e.g., grade level, duration of the treatment) of the DBP to student outcome relationship were tested using homogeneity analyses (Cooper et al., 2009; Hedges & Olkin, 1985). The analyses were carried out to determine whether (a) the variance in a group of individual effect sizes varies more than predicted by sampling error and/or (b) multiple groups of average effect sizes vary more than predicted by sampling error.
We used a shifting unit of analysis (see Cooper et al., 2009, for a description). In this procedure, each effect size associated with one study is first coded as if it were an independent estimate of the relationship between DBP and the outcome. However, when estimating the overall effect of DBP, we averaged these effects prior to analysis so that the one sample only contributed one effect size. In contrast, when conducting moderator analyses, if a single sample provided a test of the effect of DBP for more than one category of a moderator (e.g., one sample provided the effect of DBP on both math and reading achievement), we allowed a single sample to contribute one effect to each moderator category. This method retains as much data as possible from each study while holding to a minimum any violations of the assumption of independent data points.
Rather than opt for a single model of error, we conducted all analyses twice, once employing fixed-error assumptions and once employing random-error assumptions (see Hedges & Vevea, 1998, for a discussion of fixed and random effects). This sensitivity analysis allowed us to examine the effects of the different assumptions (fixed or random) on the findings. Differences in the results may influence interpretation of the results. For example, if a moderator is found to be significant under a fixed-effects assumption but not significant under a random-effects assumption, then this result suggests a limit on the generalizability of the inferences of the moderator.
We tested for possible publication bias and missing reports using Duval and Tweedie’s (2000a, 2000b) trim-and-fill procedure. This test estimates the potential impact of missing reports on the observed average effect by imputing the missing values necessary to achieve a normal distribution. All statistical processes were conducted using the Comprehensive Meta-Analysis software package (Borenstein et al., 2005). Only outcome measures that were reported in two or more separate reports with two or more independent samples were meta-analyzed.
Results
The literature search uncovered 47 studies that tested the effect of DBP on various student-related outcomes. The 47 studies reported 288 separate effect sizes based on 64 independent samples. The authors, sample sizes, effect sizes, and other important study characteristics are listed in Table S2 (available online at http://RER.sagepub.com/supplemental). The 47 studies appeared between January 1985 and December 2012. The sample sizes ranged from 8 to 702.
For each set of outcomes, Grubbs’s (1950) test was used to identify outliers within that set of sample sizes and within that set of effect sizes. After Winsorizing outliers, all studies were retained for further analysis. See Table 2 for descriptive information related to the effects retrieved for each outcome and results of the Grubbs’s test for each outcome. Grubbs’s test was not conducted for the sample of studies assessing the effects of DBP on attitudes toward others, drama skills, or the adjusted effects of DBP on motivation, attitudes toward others, and drama skills, because each of these outcomes had fewer than three contributing to the average weighted effects.
Descriptive effect information and results of Grubbs’s test for outliers among sample and effect sizes by outcome category
Overall Effects of Drama-Based Pedagogy
Overall the effects of DBP on all outcomes were positive (see Table 3). DBP had a significant unadjusted effect on the following outcomes achievement: 21st-century skills, attitudes toward the domain, attitudes toward others, motivation, drama skills, and absenteeism. Some of the results varied depending on the model assumptions (i.e., fixed- or random-effects assumptions). In particular, unadjusted effects were significant under a fixed-effects (FE) models but not significant under random-effects (RE) assumptions for the following outcomes: 21st-century skills, motivation, and attitudes toward others. We found similar patterns under the adjusted-effects estimates with one exception—the adjusted effects in motivation were not significant under either model. Trim-and-fill analyses looking for missing effects on the left side of the distribution (see Table 4) suggested that the findings were relatively unchanged (though the size of effects decreased some) even when potentially missing values were imputed.
Overall effects of DBP on academic related outcomes
Note. CI = confidence interval; DBP = drama-based pedagogy.
p < .10. *p < .05. **p < .01. ***p < .001.
Results from trim-and-fill analyses for unadjusted and adjusted effect size estimates by outcome category
Note. U = unadjusted effect estimates; A = adjusted effect estimates; FE = fixed-effects model; RE = random-effects model; CI = 95% confidence interval; NA = not applicable, because we could not conduct the analyses due to few studies.
Moderator Analyses
Next, moderators of the effect of DBP were assessed for achievement, self-perceptions of competence, 21st-century skills, attitudes toward domain, attitudes toward school, social skills, and motivation. Because attitude toward others and drama skills were not statistically heterogeneous, we did not conduct moderator analyses on these outcomes. Moderator testing was only conducted if variability was present among the moderators (more than two studies populating two or more levels of the analysis). Due to limited reporting in the sample of studies, we were unable to conduct an analysis for type of strategy as the moderator. In addition, we could not conduct a moderator analysis on the experience level of the facilitator but did use the type of leader (e.g., classroom teacher or teaching artist) as a way to account for variability in the leader.
In addition to the theoretical moderator outlined in the introduction, we added the following methodological moderator analyses to check for publication bias and/or measurement bias including publication status of the report and type of measure used for the outcome. Additionally, when possible, we conducted a moderator analyses on the type of outcome within the broader category to see if there was any variability in effects across the related outcomes we had collapsed. As previously mentioned, we could not conduct any analyses comparing various designs (i.e., quasi experiment with equivalent control group, quasi experiment without equivalent control group, randomized experiment) due to limited variability.
Achievement Moderator Analyses
We conducted moderator analyses of the unadjusted and adjusted effects of DBP on achievement using eight of the moderators of theoretical and methodological interest (see Table 5). Six of the eight moderators revealed a statistically significant effect, including grade level, number of lessons, leader, proximity between DBP and outcome, domain of outcome, and publication status, though there was variability regarding the set of assumptions under which each was statistically significant. We found a similar pattern of results among the adjusted effects; however, proximity between DBP intervention and the outcome was no longer a statistically significant moderator. With the exception of proximity between the intervention and outcome, all moderator analyses contained more than two groups being compared. As such, next, we describe the pairwise contrasts we conducted when overall moderator analyses indicated that there was significant heterogeneity across all groups. The proximity of the outcome to intervention moderator analysis indicated that DBP interventions that are more closely aligned with the measured outcome produced higher effect size estimates compared to less closely aligned interventions and outcomes.
Moderators of the effects of DBP on achievement outcomes
Note. CI = confidence interval; DBP = drama-based pedagogy. The k for each moderator may not add up to the overall k from Table 3. Studies were excluded for moderator analyses due to insufficient reporting or some studies contributed more than one effect when the study had multiple levels of the moderator.
p < .10. *p < .05. **p < .01. ***p < .001.
Grade level
Initially, we compared all grade levels; however, if two or more adjacent grade levels were not statistically different, then we collapsed the category. Under the fixed-effects model, the weighted average unadjusted effects estimate statistically varied by grade level. We conducted pairwise comparisons to test the difference between levels. The largest average unadjusted effect was for prekindergarten/lower elementary, which was statistically different from upper elementary (FE: Q = 8.86, p < .001), middle school (FE: Q = 6.42, p < .01), and high school/college (FE: Q = 3.97, p < .05). None of the other grade levels significantly varied from one another.
The weighted average adjusted effects estimates statistically varied by grade level under the fixed-effects model but not under the random-effects model. We conducted pairwise comparisons to test the difference between levels. The largest average adjusted effect was for lower elementary, which was statistically different from prekindergarten (FE: Q = 5.98, p < .01) and upper elementary (FE: Q = 5.04, p < .01) but not statistically different from middle school participants. None of the other grade levels statistically varied from one another.
Number of lessons
The initial categories for number of lessons were 1 to 5 lessons (k = 11), 6 to 10 lessons (k = 7), 11 to 20 lessons (k = 11), and more than 20 lessons (k = 10). Three studies were excluded from this analysis because the number of lessons was not reported (Francis, 2007; Ranger, 1995; Saab, 1987). Under both fixed- and random-error assumptions, the average weighted effect (both unadjusted and adjusted) of DBP on achievement when 11 to 20 lessons were implemented was not significantly different from the average weighted effect of DBP on achievement when participants received 21 or more lessons; therefore, these categories were collapsed for analyses.
Samples of participants who had 6 to 10 lessons had the largest average unadjusted weighted effect and this effect was statistically different from those who had 1 to 5 lessons (FE: Q = 23.38, p < .001) and 11 or more lessons (FE: Q = 20.10, p < .001) under fixed-effects assumptions. However, 1 to 5 lessons and 11 or more lessons were not statistically different. The average adjusted weighted effect of DBP on achievement presented a similar pattern in effects. The largest average adjusted effect was 6 to 10 lessons, which was statistically different from 1 to 5 lessons (FE: Q = 7.18, p < .01) and 11 or more lessons (FE: Q = 21.67, p < .001) under fixed-effects assumptions. Adjusted effect estimates for 1 to 5 lessons and 11 or more lessons were not statistically different.
Leader characteristics
Four studies were excluded from the analysis due to insufficiently reporting the type of leader (Byerly, 1994; Kariuki & Humphrey, 2006; Ranger, 1995; Wright, 2006). Due to a significant variability among studies under a fixed-effects model, we conducted pairwise comparisons for type of leader. The largest average unadjusted effect was for DBP interventions led by the classroom teacher, which was significantly different from interventions led by a teaching artist, FE: Q = 8.11, p < .01. Likewise, the average effect for researcher-led interventions was larger and significantly different from the average effect for interventions led by a teaching artist, Q(1) = 5.68, p < .05. There was no difference in the effect of classroom teacher–led interventions compared to researcher-led interventions.
The adjusted average weighted effect of DBP on academics significantly varied among type of leader under the fixed-effects and random-effects assumptions; therefore, pairwise comparisons were conducted under both models. The largest average adjusted effect was for DBP interventions led by the classroom teacher, which were significantly different from interventions led by a teaching artist under fixed- but not under random-effects models, FE: Q(1) = 22.10, p < .001; RE: Q(1) = 3.22, p > .05. Classroom teacher–led interventions did not significantly differ from interventions led by a researcher. The average adjusted effect for interventions led by the researcher was larger and significantly different from the average effect for interventions led by a teaching artist under both models, FE: Q(1) = 14.62, p < .001; RE: Q(1) = 5.63, p < .05.
Domain of DBP intervention
Studies were coded for the domain of the DBP intervention including science, language arts, reading, social studies, and mathematics. The weighted average unadjusted effect of DBP on achievement significantly varied for different domains under both fixed-error and random-error assumptions. Pairwise comparisons under both models yielded similar patterns. The largest average effect was for science, which was significantly different from language arts, FE: Q(1) = 11.85, p < .001; RE: Q(1) = 9.57, p < .01; reading, FE: Q(1) = 4.54, p < .05; social studies, FE: Q(1) = 11.60, p < .001; RE: Q(1) = 5.17, p < .05; and mathematics, FE: Q(1) = 21.10, p < .001; RE: Q(1) = 14.90, p < .001, under fixed-effects and random-effects assumptions. In addition, weighted average unadjusted effects for DBP on achievement in language arts were significantly different from reading, FE: Q(1) = 5.59, p < .05; social studies, FE: Q(1) = 35.91, p < .001; and math, FE: Q(1) = 5.78, p < .05, under fixed-effects assumptions only. Reading also differed from mathematics under both models, FE: Q(1) = 17.64, p < .001; RE: Q(1) = 5.95, p < .01, and from social studies, FE: Q(1) = 5.14, p < .01, under fixed-effects assumptions. No significant differences were found in average unadjusted weighted effects between social studies and math outcomes.
The adjusted average weighted effect of DBP on achievement significantly varied by domain of the outcome under the fixed-effects assumptions. Pairwise comparisons suggested that the effect of DBP on science was significantly different from language arts, FE: Q(1) = 18.42, p < .001; mathematics, FE: Q(1) = 11.95, p < .001; and reading, FE: Q(1) = 16.62, p < .001. No other domains significantly differed from one another.
Self-Perceptions of Competence Moderator Analyses
Studies that measured self-perceptions of competence included outcomes such as self-concept, self-efficacy, and a lack of discrepancy between actual and ideal selves (see Ballou, 2000; Danner, 2003; Wright, 2006). We conducted moderator analyses (see Table 6) on the effect of DBP on self-perceptions of competence using six of the moderators of theoretical and methodological interest. Again, we collapsed categorical divisions when it was theoretically relevant and appropriate to maintain moderator categories with a reasonable number of samples. Two of the six moderators revealed a statistically significant effect (publication status and type of outcome), though there was variability regarding the set of assumptions under which each was statistically significant.
Moderators of the effects of DBP on self-perceptions of competence outcomes
Note. CI = confidence interval; DBP = drama-based pedagogy. The k for each moderator may not add up to the overall k from Table 3. Studies were excluded for moderator analyses due to insufficient reporting or some studies contributed more than one effect when the study had multiple levels of the moderator.
In these studies, self-discrepancy was a lack of consistency between ideal and actual self.
p < .10. *p < .05. **p < .01. ***p < .001.
Publication status
Although this moderator presented significant heterogeneity, it was not in the expected direction to suggest publication bias; therefore, no additional analyses were conducted.
Type of outcome
In this sample of studies, four categories for type of outcome were present, including self-discrepancy (k = 2), self-efficacy (k = 4), self-image (k = 2), and self-concept (k = 10). For each outcome, higher scores indicate a more positive view of the self as competent. This moderator accounted for a significant amount of variability in the unadjusted effects under fixed-error assumptions. The largest average effect of DBP was on measures of self-discrepancy, which was significantly larger than the average effect of DBP on self-efficacy, FE: Q(1) = 6.37, p < .01; as well as significantly greater than the average effect of DBP on self-image, FE: Q(1) = 5.64, p < .05. Alternatively, the average effect of DBP on self-discrepancy was not significantly greater than the average effect of DBP on self-concept. Average effects of DBP on self-efficacy and self-image were not significantly different, though they were different from self-concept, FE: Q(1) = 5.17, p < .05. Finally, the average effect of DBP on self-image and self-concept were not significantly different.
21st-Century Skills Moderator Analyses
Studies that measured 21st-century skills included creativity, critical thinking, and communication outcomes. We conducted moderator analyses (see Table 7) on the effect of DBP on 21st-century skills using eight moderators of theoretical and methodological interest. Again, we collapsed categorical divisions when it was theoretically relevant and appropriate to maintain acceptable sample sizes across categories. Three of the eight moderators revealed a statistically significant effect (grade level, leader, outcome measure), though there was variability regarding the set of assumptions under which each was statistically significant.
Moderators of the effects of DBP on 21st-century outcomes
Note. CI = confidence interval; DBP = drama-based pedagogy. The k for each moderator may not add up to the overall k from Table 3. Studies were excluded for moderator analyses due to insufficient reporting or some studies contributed more than one effect when the study had multiple levels of the moderator.
p < .10. *p < .05. **p < .01. ***p < .001.
Grade level
All studies were included in this analysis. The average weighted unadjusted effect of DBP on 21st-century skills significantly varied for different grade levels under FE assumptions but not under RE assumptions. Pairwise comparisons were conducted under fixed-error models only. The largest average effect was for high school/college samples and was significantly different under fixed-error assumptions from the weighted average effect for elementary students, Q(1) = 4.65, p < .05, and middle school students, Q(1) = 4.94, p < .05. However, the average unadjusted effect of DBP for middle school students was not significantly different from that for elementary school student effects.
Leader
Four studies were excluded from the analysis (Ballou, 2000; McGregor, 2001; Minnesota Department of Human Services, 2008; Smith, 2010) due to insufficiently reporting the type of leader. The analysis of the average weighted unadjusted effect of DBP on attitudes toward 21st-century skills revealed significant heterogeneity depending on whether the intervention was facilitated by the classroom teacher, researcher, or teaching artist under a fixed-error model but not under a random-error model. The largest average effect was for DBP interventions led by the researcher, which were significantly different from interventions led by a teaching artist, Q(1) = 6.93, p < .01, but not significantly different from interventions led by a classroom teacher.
The adjusted average weighted effect of DBP on 21st-century skills also significantly varied among type of leader under fixed-error assumptions but not under random-error assumptions. The largest adjusted average effect estimate was for interventions led by the researcher, which was significantly different from the average effect of interventions led by the classroom teacher under fixed-error assumptions, Q(1) = 5.84, p < .01, as well as from interventions led by teaching artists, Q(1) = 7.59, p < .01. However, interventions led by classroom teachers and teaching artists did not vary significantly.
Measure of outcome
For unadjusted effects, a significant amount of variability was attributed to the type of outcome measure under the fixed-error assumptions but not under random-error assumptions; therefore, pairwise comparisons were conducted under a fixed-error model only. The largest average weighted unadjusted effect estimate of DBP on 21st-century skills was found for a researcher- or teacher-developed measure. It was not significantly different from outcomes measured by validated measures but was significantly different from outcomes measured by surveys, FE: Q(1) = 5.90, p < .01. Weighted average effect estimates based on validated surveys and researcher-developed surveys did not significantly differ from one another.
Similar results were produced for adjusted weighted effect estimates of DBP on achievement outcomes. The moderator significantly accounted for variability under both fixed- and random-error assumptions. The largest adjusted average weighted effect estimate of DBP on 21st-century skills was from a researcher-developed test and was significantly different from outcomes measured by validated measures, Q(1) = 14.59, p < .001; RE: Q(1) = 14.59, p < .001, and significantly different from outcomes measured by surveys, FE: Q(1) = 28.41, p < .001; RE: Q(1) = 28.41, p < .001.
Attitudes Toward the Domain Moderator Analyses
We conducted moderator analyses (see Table 8) on the effect of DBP on attitudes toward the domain using six moderators of theoretical and methodological interest. The moderators for proximity between DBP and the outcome and the publication status were significantly heterogeneous. Results of these analyses suggested that DBP interventions that were aligned closely with the measured outcome had larger effects compared to poorly aligned studies and published reports had larger effects as compared to unpublished reports.
Moderators of the effects of DBP on attitudes toward the domain outcomes
Note. CI = confidence interval; DBP = drama-based pedagogy. The k for each moderator may not add up to the overall k from Table 3. Studies were excluded for moderator analyses due to insufficient reporting or some studies contributed more than one effect when the study had multiple levels of the moderator.
p < .10. *p < .05. **p < .01. ***p < .001.
Attitudes Toward School Moderator Analyses
We conducted moderator analyses (see Table 9) on the effect of DBP on student attitudes toward academics using seven moderators of theoretical and methodological interest. Overall, fewer studies reported outcomes measuring attitude toward academics. Therefore, we collapsed categorical divisions when it was theoretically relevant and appropriate. The moderators for number of lessons, proximity between DBP and the outcome, and the publication status were all significantly heterogeneous. Results of these analyses suggested that interventions that had 11 to 20 lessons had a larger average effect compared to interventions with more than 20 lessons, interventions more closely aligned with the outcome produced larger effects, and unpublished reports had larger effects as compared to published reports.
Moderators of the effects of DBP on attitudinal outcomes toward school
Note. CI = confidence interval; DBP = drama-based pedagogy. The k for each moderator may not add up to the overall k from Table 3. Studies were excluded for moderator analyses due to insufficient reporting or some studies contributed more than one effect when the study had multiple levels of the moderator.
p < .10. *p < .05. **p < .01. ***p < .001.
Social Skills Moderator Analyses
Studies that measured social skills included conflict resolution with peers (Walsh-Bowers & Basso, 1999) and recognizing emotion in others (Smith, 2010). We conducted moderator analyses (see Table 10) on the effect of DBP on social skills using only three of the moderators of theoretical and methodological interest. Overall, there was not enough variability among the other moderators to conduct further analyses. None of the moderators accounted for significant heterogeneity; therefore, no further analyses were conducted.
Moderators of the effects of DBP on social skills outcomes
Note. CI = confidence interval; DBP = drama-based pedagogy. The k for each moderator may not add up to the overall k from Table 3. Studies were excluded for moderator analyses due to insufficient reporting or some studies contributed more than one effect when the study had multiple levels of the moderator.
For analysis, problem behavior was the lack of problem behavior. In other words, a positive d would mean less problem behavior and a negative d would mean more problem behavior.
p < .10. *p < .05. **p < .01. ***p < .001.
Motivation Outcomes Moderator Analyses
Studies that measured motivation included outcomes such as engagement in the process as measured by a survey (Warner & Andersen, 2004) and interest in the topic as measured by time on task (Laurin, 2010). We conducted moderator analyses (Table 11) on the effect of DBP on motivation using six moderators of theoretical and methodological interest. Only grade level significantly moderated the effect of DBP on motivation. Results suggested that studies with participants in upper elementary had more positive results than studies with participants in lower elementary.
Moderators of the effects of DBP on motivational outcomes
Note. CI = confidence interval; DBP = drama-based pedagogy. The k for each moderator may not add up to the overall k from Table 3. Studies were excluded for moderator analyses due to insufficient reporting or some studies contributed more than one effect when the study had multiple levels of the moderator.
p < .10. *p < .05. **p < .01. ***p < .001.
Relationship Among Moderators
Because each moderator was tested individually, the possibility exists that moderators were confounded with one another (see Cooper et al., 2009, for a discussion). Thus, we tested for the possibility that there was a relationship among significant moderators of DBP effects on achievement and attitudes toward academics in pairwise analyses using Pearson’s chi-square test for five significant moderators, including type of leader, grade level, subject of the outcome, proximity between intervention and outcome, and number of lessons (see Tables S3 and S4, available online at http://RER.sagepub.com/supplemental). Other outcomes did not have multiple significant outcomes of theoretical interest.
The only significant relationship between moderators of achievement outcomes was between the number of lessons and the leader of the intervention. Results indicated that studies in which classroom teachers had been the leader were also more likely to have an intervention lasting 11 or more lessons. Next, we analyzed the potential relationship for moderators of DBP effects on attitudes toward academics. No significant relationships between moderators for the effects of DBP on attitudes toward academics were found.
Discussion
The results of this research synthesis and those of previous syntheses seem to warrant the broad tentative conclusion that DBP has a positive effect on achievement and a variety of related psychological and social outcomes. That said, these positive effects were not evident under all sets of statistical assumptions, effect sizes types, samples, or circumstances. We outline nuances in the findings in the following discussion. Furthermore, there were obvious limitations in the research base. Namely, the vast majority of studies used a quasi-experimental, nonequivalent control group design, which provides a weak basis for drawing causal conclusions. Furthermore, far fewer research attempts have been made to examine the effect of DBP on psychological and social outcomes, compared to achievement outcomes, though it seems reasonable to suggest that some of the effect on achievement may occur via psychological and social mechanisms. As such, we know far less about how DBP influences these psychological and social indicators of student functioning compared to achievement.
In the following, we focus on illustrative findings in the hopes of developing a more nuanced understanding of the effects of DBP. Due to the weak design of many of the reports in this sample, the unadjusted effects should be considered more cautiously than the adjusted effects. That is, the adjusted effects may account for preexisting differences between the control and intervention groups; therefore, this potentially allows for a less biased effect estimate. When relevant, we highlight when the unadjusted and adjusted effects agree or disagree.
Fit of Data to Predictions and Consistency With Previous Syntheses
We predicted that there would be a stronger positive effect for students who received DBP as compared to students who did not receive DBP (control conditions) for each of the outcomes. Overall, achievement outcomes are positive and significant under fixed- and random-effects assumptions for both unadjusted and adjusted effect size estimates. To translate this finding into a potentially more meaningful metric, the average student who receives DBP as part of the curriculum would score better than 67% to 70% of the students who did not receive DBP as part of the curriculum (see Cooper et al., 2009, for an explanation of U3 metric translation of effect size estimates). Based on past meta-analyses and the current finding, DBP should be considered a viable pedagogical method for teachers to raise achievement outcomes alongside other research-based instructional methods.
In addition, attitudes toward domain are significant under all the assumptions and models. Translating the findings to percentages suggests that the average student who is a part of the DBP intervention would report more positive attitudes toward academics than 55% to 59% of students who were not part of the DBP intervention. Thus, the meaningful use of DBP in the classroom not only raises student academic achievement but also improves students’ attitudes toward that academic discipline. In other words, students do learn and enjoy learning when teachers use DBP to teach the curriculum.
Based on the findings of this meta-analysis, many areas of DBP warrant further research. The effects of DBP on motivation outcomes need to be investigated. Only six studies reported effects on motivation; however, much of the research and advocacy literature in arts integration suggests that motivation is one of the prevailing reasons to use DBP strategies in the classroom. In particular, we are not well informed of how various moderators may influence the DBP to motivation effect. With so few studies, we were only able to conduct a moderator analysis using grade level in which effects significantly favored the upper elementary grades. It might be that as students get older, DBP may have a stronger impact on their motivation. Unfortunately, we did not have enough studies at the middle, high school, or college levels to allow for analysis.
Even though all of the overall effects were positive, self-perception of competencies and social skills were not significant. The previous meta-analysis that focused on self-perception of competencies (Conrad & Asher, 2000) also found no significant effect for these types of outcomes. Previously, there has not been a meta-analysis on social skills; however, DBP has certainly been touted as an effective means to promote prosocial behaviors, which this study suggests may be unfounded (Deasy, 2002). We also predicted and found that variability in DBP effects on various outcomes could be explained by a variety of theoretically and methodologically relevant moderators as described in the following section.
Grade Level
We predicted that there would be a stronger, positive effect for lower elementary school–age students as compared to older students. For achievement outcomes, the unadjusted and adjusted effects suggested that DBP had the strongest effect on outcomes for preschool/lower elementary and lower elementary students. However, in the 21st-century skills and motivational outcomes, DBP had the strongest effects for upper elementary and middle school samples. In the previous meta-analysis (Kardash & Wright, 1986; Podlozny, 2000), contradictory findings suggested that this moderator may not have a consistent impact on the effects of DBP. These differences may be in part due to the type of outcome. DBP may help students’ achievement outcomes in younger students and motivational/attitude outcomes in older students. Of the 47 studies in this meta-analysis, 15 studies were conducted with middle/high school students and 3 studies were conducted with college students. Because these studies were spread across different outcomes, we need to interpret these findings with caution. More research needs to be done with middle/high school and college students in order to clarify grade level as a potential moderator.
Duration of Intervention
We predicted that there would be a stronger positive effect for interventions that included frequent, brief sessions as compared to interventions in which the sessions were infrequent or the intervention as a whole was brief. We coded for duration of the intervention in various ways (i.e., number of lessons, number of weeks, minutes of the intervention); however, we were only able to conduct moderator analyses on number of lessons due to insufficient reporting. Based on past meta-analyses, interventions that last between a few days to a few weeks did not significantly differ, but DBP interventions that span 12 weeks to a year or more did have stronger effects (Conrad, 1992; Deasy, 2002). However, Kardash and Wright (1986) found that the effect size was inversely related to the minutes per session, was positively related to sessions per week, and had no relationship to the span of the DBP treatment.
In the present meta-analysis, the number of lessons accounted for a significant amount of the variance in achievement outcomes, as well as attitude toward achievement. This pattern was most evident when comparing effects for the achievement outcomes. For achievement, DBP interventions that consisted of 6 or more lessons (to as many as 100 lessons) were significantly more positive than DBP interventions that consisted of 5 or fewer lessons. For attitudes toward domain, interventions that consisted of 11 to 20 lessons were significantly more positive than interventions that consisted of 21 or more lessons. Taking both of these findings together, the data suggest that DBP interventions need to be longer than 5 lessons, but after 20 lessons, DBP may not have any additional benefits on academic and attitude-related outcomes.
Leader
We predicted that there would be a stronger positive effect for interventions delivered by a more experienced facilitator. As noted earlier, reports did not consistently document the leader’s years of experience in DBP or the leader’s exposure to professional development in DBP (the two possible ways that we attempted to capture experience in the coding guide). As a small step toward understanding the potential moderating effects of the facilitator, we did conduct moderator analyses on the type of leader (i.e., researcher, classroom teacher, or teaching artist).
The only past meta-analysis to test this moderator found that the drama specialist (similar to the teaching artist in this study) had a significantly less positive impact on effect estimates (Conrad, 1992). The results of this synthesis revealed some interesting similar patterns. For achievement and 21st-century skills, interventions directed by the researcher or classroom teacher had significantly more positive effects than interventions led by a teaching artist. However, for attitudes toward school, the interventions led by a teaching artist had more positive effects. Interventions led by a teaching artist also had the most positive effect, albeit not significant, for self-perceptions of competence. This pattern may suggest that when classroom teachers use DBP to teach curriculum, they have a positive effect on the students, no matter the outcomes. Alternatively, interventions led by teaching artists may have a positive effect on students when the outcomes address attitudes and self-perception outcomes. Ideally, teachers understand the content and their students, but teaching artists may be able to pique student interest and motivation toward the curriculum. When coupled with the classroom teacher’s expertise in content, the teaching artist may help improve the attitude and motivation of students. This also suggests that teaching artists may need more extensive training in pedagogical content knowledge and content knowledge to be more effective in the classroom.
Further research needs to be conducted on the differing effects of the DBP leader on student outcomes. In this study, we included studies documenting student responses to DBP (e.g., Minnesota Department of Human Services, 2008); however, we did not find studies that documented the types of interactions and student responses to classroom teachers and teaching artists. A mixed-methods study could be invaluable to understanding how classroom teachers and teaching artists might partner for stronger positive effects for students. Additionally, researchers may want to explore the type of training that might serve training teaching artists and classroom teachers to use DBP in the classroom. To begin this process, researchers need to document the experience level and type of professional development leaders have when they implement DBP in the classroom.
Alignment Between DBP and Outcome
We predicted for this exploratory analysis that there would be stronger positive effect of DBP when the outcome was well-aligned with and more proximal to the intervention. For most of the outcomes captured in this meta-analysis (achievement, attitude toward academics, self-perceptions of competence, social skills, and 21st-century skills), studies with directly aligned outcomes with the DBP intervention had more positive effects. In other words, if the DBP intervention focused on writing skills and the outcome was a measure of writing skills (directly aligned), then the average effect was stronger and more positive than if the DBP intervention focused on writing skills and the outcome was a measure of problem solving (indirectly aligned).
Domain of Outcome
For achievement outcomes, DBP interventions used in ELA and science curriculum seem to have the largest impact. The domain of the intervention accounted for a significant amount of variance under fixed- and random-error models for achievement. For achievement, language arts and science were the domains that were significantly different from the other domains. However, in 21st-century outcomes, a similar pattern (although not significant) emerged; interventions in ELA were more effective than those in other domains. These findings may be attributable to various possibilities. Much of DBP focuses on stories, role-playing, and inquiry, which closely aligns with ELA and science curricula. DBP is most prevalent in ELA classrooms; thus, the most effective ways to use DBP may be most well developed in these classrooms. Typically, handbooks for DBP focus on implementation in the ELA classroom (see Heinig, 1992; McCaslin, 1996; Willhelm, 2002), suggesting that other domains may benefit from materials on effectively using DBP.
Limitations
Despite the comprehensive nature of this research synthesis and our careful consideration of the nuances across studies, there are still a number of limitations. It is important to note that some of these findings were based on a small number of effect sizes, so it is difficult to place a great deal of confidence in the specific magnitude of the estimated effects. In particular, we suggest caution when interpreting moderator effects for motivation and social skills as they are based on very few studies. Furthermore, many of the studies were excluded from specific moderator analyses because of insufficient reporting of information. If researchers, policymakers, and practitioners hope to improve the implementation and effectiveness of DBP, then future research must report specific information vital for dissemination and replication.
As previously mentioned, most of the studies included in this research synthesis and meta-analysis were quasi experimental, meaning that the students were not randomly assigned to the DBP and non-DBP conditions. This methodological limitation can result in biased findings for various reasons. For example, teachers who are already effective may be self-selecting to do the intervention, or their students may have already been exposed to active learning strategies similar to DBP. Unfortunately, this type of design is often the most feasible option when examining interventions intended for applied settings. Thus, we should focus on the adjusted effect estimates that take into account previous achievement. Additionally, only results that are significant under fixed- and random-error models may be generalized to the population.
In this meta-analysis, we performed an exhaustive literature search and conducted trim-and-fill analyses, but undoubtedly some studies published or unpublished were excluded unintentionally. For studies that were included, we did not use an indication of quality research inclusion criteria (see Cooper et al., 2009). Our approach results in a more comprehensive view of the research literature; however, it may also lead to including reports based on poorly designed research and interventions.
Another limitation that influenced our ability to conduct meaningful analyses was the lack of thorough reporting in the research literature on drama-based pedagogy. This concern has been discussed at length by other researchers (Eisner, 1998; Fleming et al., 2004; Mages, 2006; Wagner, 1998; Winner & Cooper, 2000); however, it is worth mentioning again. The field of drama and education needs to make a practice of reporting standard statistical information (i.e., sample sizes, means, and standard deviations for each sample) as well as intervention information (i.e., DBP strategies used, length of the intervention, leader of the intervention). In particular, providing lessons plans with DBP strategies used in the intervention can help provide models of effective practice. Rather than suggesting that all DBP is effective, we need to find ways to measure and understand the impact of specific approaches.
Implications for Policy and Practice
As supported by the robust finding for the effects of DBP on achievement and attitudes toward achievement, DBP may be considered an effective instructional approach for teachers in the prekindergarten through college classrooms. These findings have implications for policymakers, professional development providers, college professors, and educators. However, it should be noted that we are not offering recommendations for specific arts programming in the schools (e.g., music, visual arts, drama), as that is beyond the scope of this meta-analysis. As the Common Core Standards are developed for nationwide implementation (National Governors Association Center for Best Practices, 2010), including language that supports the use of DBP to meet these standards would be recommended given the generally positive average effects and infrequent significant negative effects found in this synthesis. Furthermore, we believe that professional development providers and college professors of educators would be well advised to consider ways to train in-service and preservice educators to use DBP in the classroom. In particular, DBP is likely to be particularly beneficial for educators in areas of ELA and science who have facility with using DBP.
To that end, educators who use DBP need to do so in meaningful ways in order to positively influence student outcomes. It may be beneficial for educators to partner with teaching artists to address a constellation of student outcomes. Unfortunately, at this point it is difficult to assess the specific approaches that have an impact. We suspect that offering a variety of strategies designed to help students embody concepts and access material in multiple ways is important. It is our hope that future research will be more descriptive of specific DBP approaches and compare the impact of various DBP approaches on student outcomes. Specifically, we need better reporting about the intervention (e.g., how many lessons, how many weeks, who led the intervention), as well as detailed descriptions of sample lessons or strategies that were used. This would allow for researchers to build on one another’s work and deepen lines of research to try to understand the similarities among interventions that have stronger positive effects on students’ outcomes. In addition, this would aid practitioners in understanding specific ways to implement DBP in the classroom.
Conclusion
DBP appears to have a positive impact on a variety of outcomes. In particular, DBP has a significant effect on achievement in various domains, attitudes toward achievement, 21st-century skills, and drama skills. These effects are even greater when certain characteristics are present, for example, when a classroom teacher uses DBP for achievement outcomes or a teaching artist uses DBP for attitudinal outcomes. However, the effects of DBP should not be overgeneralized, and relevant parties need to be aware of the limitations in the current research for drawing unambiguous causal conclusions—especially when pursuing funding streams or policy support. As the definitions and measurements stand today, researchers, policymakers, or practitioners cannot substantiate the claim that DBP will have a significant impact on a child’s self-concept, nor can they substantiate the claim that more than 20 lessons of DBP will result in stronger outcomes. Rather than suggesting that DBP is a panacea for a wide range of educational concerns, we believe our synthesis suggests that DBP is one option that can result in adaptive outcomes for students. We invite researchers, policymakers, and practitioners to build on these findings and pursue research, policy, and practice that facilitate the meaningful, intentional, and effective use of drama-based pedagogy.
Footnotes
Notes
Authors
BRIDGET KIGER LEE is a postdoctoral researcher at The Ohio State University, 333 Arps Hall, 1945 N. High Street, Columbus, OH, 43210-1172, USA; e-mail:
ERIKA A. PATALL is an assistant professor of educational psychology at The University of Texas at Austin, 1 University Station, D5800, Austin, TX 78712-0383, USA; e-mail:
STEPHANIE W. CAWTHON is an associate professor of educational psychology at The University of Texas at Austin, 1 University Station, D5800, Austin, TX 78712-0383, USA; e-mail:
REBECCA R. STEINGUT is a doctoral student at The University of Texas at Austin, mailing address: c/o Erika A. Patall, The University of Texas at Austin, 1 University Station, D5800, Austin, TX 78712-0383, USA; e-mail:
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
