Explicit Writing Instruction in Synthesis: Combining In-Class Discussion and an Online Tutorial

Where Do Students Struggle?

In our experience, students struggle with aspects of academic writing involving abstract and critical thinking: integrating ideas across multiple sources, organizing information in terms of themes rather than articles, and developing an original argument logically derived from the reviewed literature. These aspects of writing are often referred to as synthesis (Boscolo et al., 2007; Rosenblatt, 2010; Spivey & King, 1989; Torraco, 2005). Scholarship about the teaching of writing supports our observation that synthesis writing is a common obstacle for students.

First, researchers have noted that students often write about multiple sources by drafting separate, sequential summaries of scholarly articles but generally fail to integrate information across them (Bosolo et al., 2007; Froese et al., 1998; Rosenblatt, 2010). Some have labeled this as passive participation in the writing process, where students see their role as “knowledge-telling” or merely reporting about existing research, rather than “story-telling” (Kellogg, 2008; Maher et al., 2014; Segev-Miller, 2004). Somewhere between 50% (Rosenblatt, 2010) and 85% (Young & Leinhardt, 1998) of students’ attempts at synthesis writing show signs of serial summarizing.

Second, students struggle to identify relevant information from source articles and to organize it into a coherent narrative. In one intervention study, instructors spent one semester modeling critical thinking and instructing students about finding quality sources and later evaluated students’ oral poster presentations and written literature reviews (Hayes-Bohanan & Spievak, 2008). Although students’ oral explanations of their literature reviews were promising, the written posters and papers lacked critical analysis. In another intervention study, Segev-Miller (2004) found that the majority of students were initially unable to identify a central organizing argument that applied to the body of literature being reviewed. Similarly, Goddard’s (2003) students struggled to pick out important points from an article and found themselves lost in the details. These struggles manifest as failed attempts to frame the importance of an issue in a way that is clear and convincing to the audience (Maher et al., 2014).

Third, students often fail to find their own voice when writing; instead they mimic the writing they have read in their sources (Boscolo et al., 2007; Goddard, 2003). This is characterized by attempts to achieve synthesis by merely “borrowing sentences and connecting information from each source” (Boscolo et al., 2007, p. 422) and can be attributed to students lacking the expertise required to generate a novel argument that moves beyond information contained in the original sources (Segev-Miller, 2004). The ability to ground one’s thinking in existing scholarship while extending what is known about a particular issue remains a challenge to most undergraduate students (Maher et al., 2014).

How Best to Improve Writing?

Much of the scholarship about improving student writing takes a case-study approach (Cumming et al., 2016), which is of limited value to the scholar or teacher looking to improve student writing. Often, case studies are either insufficiently described (e.g., Chandler & Dedman, 2012; Cook & Murowchick, 2014) or are so course-specific that they offer little assistance to instructors seeking to implement a simple, generic intervention in their class (e.g., Adams, 2011; Borg & Deane, 2011). Studies that take an experimental approach are likely to appeal to and benefit teachers looking for evidence-based instruction practices, but there are relatively few of these available. Recent literature reviews about teaching synthesis writing found only 16–21 quasi-experimental peer-reviewed studies in this area (Barzilai et al., 2018; van Ockenburg et al., 2019) that targeted students from middle school to higher education in content areas including history, social science, and natural sciences (Cumming et al., 2016). The limited number of studies in combination with the variety of settings and approaches tested makes it difficult for psychology instructors to figure out how best to apply the findings to their courses.

An additional consideration when implementing an intervention in a course is the amount of time and effort it requires. The reality is that many psychology instructors are only able to teach writing “on the side.” Time spent teaching writing is time taken away from teaching the core psychology content of the course. Yet of the 19 studies reviewed by van Ockenburg et al. (2019), only two could be implemented in under 45 min, and most of the successful interventions required upward of 100 min of class time and involved instructors providing extensive feedback on multiple iterations of the same assignment (Barzilai et al., 2018). For example, Hayes-Bohanan and Spievak (2008) focused on critical thinking across a whole semester, and Boscolo et al. (2007) conducted a synthesis writing intervention for 12 weeks. Although there may be a connection between time spent teaching writing and the effect size of the intervention (van Ockenburg et al., 2019), in our experience many instructors simply cannot or will not devote this much time and effort to writing instruction. Thus, it seems important to investigate whether a brief intervention with relatively low instructor involvement can be successful.

Successful interventions appear to emphasize one or more of the following aspects of synthesis writing: analyzing sources to identify key information, providing explicit instruction about organizing and integrating information from sources, and modeling the process of constructing arguments from or about sources (Cumming et al., 2016; Spivey & King, 1989; van Ockenburg et al., 2019). In particular, interventions that help students organize information from multiple sources, often through the use of graphic organizers, seem to be successful (Barzilai et al., 2018). For example, Segev-Miller (2004) reported improvement in writing following a mapping activity, and Darowski et al. (2016) found that when students were taught to use color-coded sticky notes to represent key evidence from multiple sources, the level of source integration increased.

The concrete nature of graphic organizers may be the key to their success. Considering that the complex nature of writing places a high mental load on limited capacity functions, such as attention and working memory (Kellogg, 2008), it makes sense to teach students a concrete way of organizing their ideas. Using graphic organizers may improve student writing because they eliminate the need for ideas to solely reside in working memory. Removing key ideas from working memory arguably frees up space to generate ways of connecting those ideas. Indeed, working memory capacity is correlated with various aspects of writing (Vanderberg & Swanson, 2007), and graphic organizers may serve as an external extension of working memory. An additional benefit of using graphic organizers relates to time. The concrete nature of these techniques makes them particularly well-suited for video tutorials that can be assigned outside of class (e.g., Darowski et al., 2016).

A second feature common to many successful interventions is combining explicit instruction of source integration with opportunities for collaborative discussions and practice (Barzilai et al., 2018). This approach involves the instructor explaining to students what makes good synthesis writing and giving students the opportunity to critique writing samples (e.g., Boscolo et al., 2007; Cargill & Smernik, 2016). The effectiveness of this approach may also stem from its concrete nature. Critiquing a writing sample gives students a concrete process to follow, which helps transform what might at first be an abstract ideal—synthesis writing—into more tangible techniques and processes that underlie quality synthesis writing. To this end, Farrell (2012), applying Dreyfus’ model of skill acquisition to information literacy, argued that students ideally learn first through concrete applications and gradually mature to more situated, embodied, and intuitive applications.

Not only are students likely to benefit from the opportunity to critique writing samples, instructors are likely to benefit too. In many cases, an instructor’s understanding of how to write well in their field is implicit, making it difficult for them to clearly instruct students in writing (Hayes-Bohanan & Spievak, 2008; Tapp, 2015; van Ockenburg et al., 2019). Guiding students through a critique of writing samples may help the instructor recognize his or her own implicit assumptions about what constitutes high quality synthesis writing.

Building a Better Intervention

Based on our review of the literature, effective synthesis writing instruction should help students identify, organize, and link information from their sources. Effective interventions likely make use of graphic organizers and collaborative discussions about what makes quality synthesis writing. Offering clear definitions and examples of synthesis writing should help make implicit understanding explicit. And finally, requiring relatively little class time or instructor effort to implement should maximize the number of instructors who are able to use the intervention.

A recently developed tutorial addresses many, but not all, of these factors. Darowski et al. (2016) created an online writing tutorial that guides students in selecting key ideas from sources, organizing the information around themes, and linking or integrating the themes into a full paper. The tutorial uses concrete techniques to reduce the intrinsic mental load of learning synthesis writing. Students offload key ideas from working memory onto sticky notes and are instructed to use those notes to organize and link information into themes. To help both students and instructors better articulate what synthesis writing is, the tutorial explicitly describes and shows examples of synthesis. Finally, the tutorial is flexible in format and is broken into short segments, with a total run time of under 12 min. It can be used in class or assigned as homework and is designed for any course in psychology.

Darowski et al. (2016) implemented the tutorial in a research methods course, assigning the tutorial as out-of-class homework. Instead of using a pre–posttest design, they used a between-subjects design that compared students’ writing across semesters and were able to blind scorers to condition. Although overall ratings of synthesis were not higher for students using the tutorial, these students did cite more sources per paragraph than students who did not use the tutorial. This suggests that students were benefitting from the concrete approach to organizing key information from their sources but were still struggling to understand and recognize the characteristics of quality synthesis writing.

We propose that this tutorial can be combined with a single class period intervention aimed at helping students to develop a more concrete understanding of the characteristics of synthesis writing. In particular, incorporating additional practice with synthesis through instructor modeling and collaborative critiques of writing samples should help students to develop the missing synthesis skills. In the current study, students in an introductory psychology course viewed the tutorial outside of class and then attended a class period devoted to discussing and practicing synthesis skills.

Method

Participants

Participants were students in two honors sections of introductory psychology, offered during fall semesters of consecutive years and taught by the same instructor (the third author). The no-tutorial group was comprised of the students who took this course the first fall (n = 19, 11 men), and the tutorial group was comprised of students who took this course the second fall (n = 16, 11 men). Each semester, students had the option to complete the project individually or with a partner. In the no-tutorial group, 12 students worked in pairs, and seven students worked individually, generating a total of 13 posters. In the tutorial group, six students worked in pairs, and 10 students worked individually, generating a total of 13 posters. Very few of the students in either group were psychology majors, and all but two of the students in each group were incoming freshmen.

Materials & Procedure

Project

Both groups of participants completed the same project, which accounted for approximately 20% of their final course grade. This project involved crafting a literature review poster that explored a common myth about psychology (Lilienfeld et al., 2011). The project was broken down into several scaffolded assignments encompassing the majority (9 weeks) of the semester. The first assignment asked students to think about the qualities of their audience. In the next assignment, students conducted an initial search of the literature and received feedback about the appropriateness of their sources. After those assignments, students received instruction about writing (described below) during one class period. After the writing instruction, students produced a rough draft of their poster. Students brought their rough drafts to class and engaged in a peer-review exercise. During the peer-review session, students worked in groups of three to four students and reviewed each other’s posters. They were given specific criteria to use for their review (e.g., identify the argument, are there enough details for each study?). The final poster contained an introduction that identified the common myth, a section that presented research that could be interpreted as evidence for or against the myth, and a conclusion that took a position on whether the myth was likely to be true or false. Students were required to include at least five sources. These posters were more like condensed literature reviews than traditional conference posters presenting an introduction, methods, results, and conclusion. On average, posters had 589 words. This condensed writing exercise was appropriate for introductory psychology students given their limited experience with the discipline’s content and writing genre. Finally, students presented their work in a poster session on the final day of class.

Tutorial group writing instruction

Following the method of Darowski et al. (2016), students in the tutorial group viewed the online video tutorial titled “Using Synthesis in Your Writing” outside of class. The tutorial introduced students to the concept of synthesis, provided concrete and step-by-step instructions for a color-coding method designed to increase synthesis in writing, and concluded with examples of this method being used in writing. ¹ In addition, these students also spent a subsequent class period receiving direct instruction about synthesis. First, students were asked to spend 10 min explaining in writing what is meant by “support, not report” in order to evaluate whether they had watched the tutorial. All students received credit. Next, the instructor spent approximately 10 min reviewing the information presented in the tutorial in a short lecture to ensure that all of the students understood the concept described in the tutorial. During this lecture, the instructor defined key terms, walked through an example of a paragraph that used synthesis successfully, and identified useful phrases for setting up synthesis in writing. After the lecture, students spent 15–20 min reading examples of student writing taken from an upper level psychology course and worked in small groups (three to four people) to discuss the extent to which the writing sample used synthesis and how the use of synthesis in the sample could be improved.

No-tutorial group writing instruction

The no-tutorial group completed all of the same scaffolded assignments with the exception of the synthesis tutorial. During one class period, the no-tutorial group received instruction about writing, which included a definition of synthesis and a list of useful phrases for setting up synthesis in writing. They then worked in small groups to review and evaluate the effectiveness of examples of student writing taken from an upper level psychology course. Therefore, both groups spent one class period learning about synthesis and working in small groups to critique writing samples. However, the tutorial group was given a concrete method for using synthesis in their writing and was asked to use what they had learned in the tutorial as the basis for their writing sample critiques.

Results

To establish that the groups were comparable in terms of general academic ability, we compared final course grades for the tutorial (M = 81.42%, SD = 13.48%) and no-tutorial groups (M = 76.66%, SD = 19.28%). Although the tutorial group had higher final course grades, this difference was not significant, t(33) = 0.83, p = .412, d = 0.29, suggesting that any difference in synthesis writing cannot be explained by initial differences in academic abilities between groups.

To measure the degree to which students used synthesis in their writing, the first two authors, who were blind to condition, independently scored the final version of each student’s or pair’s poster using a previously published rubric (Darowski et al., 2016). Although this rubric evaluated synthesis along with two additional writing qualities, we focused only on the synthesis component (see Appendix A). Rubric scores ranged from 1 (well below average) to 5 (well above average). Thus, two independent synthesis ratings were generated for each poster; these scores were within one point of each other 85% of the time. We used the average of the synthesis rubric ratings in our analysis. As predicted, the tutorial group (M = 3.27, SD = 0.88, 95% CI [2.73, 3.81]) showed more synthesis in their writing than the no-tutorial group (M = 2.35, SD = 1.01, 95% CI [1.80, 2.89]), t(24) = 2.49, p = .020, d = 1.01.

While scoring the posters using the synthesis rubric, we noted that at times it was difficult to use the rubric to evaluate overall synthesis because students showed different levels of proficiency with the various aspects of synthesis (e.g., use of organization and transitions would be at a fairly high level, whereas developing an original argument would be at a lower level). To examine which aspects of synthesis students were mastering following the tutorial, we conducted a second round of blind scoring that reevaluated use of synthesis in terms of its three key components: level of integration across sources, logical organization and effective use of transitions, and development of an original argument (see Appendix B for the revised synthesis rubric). Ratings were within one point of each other at least 77% of the time (integration: 77%; organization and transitions: 77%; argument: 88%). Again, we used the average of the two independent ratings in the analysis (see Table 1). To adjust for increased probability of a Type I error in these post hoc tests, we applied a Bonferroni correction and adopted α = .0167 as the adjusted critical value. Students in the tutorial group showed significantly higher integration, t(24) = 3.05, p = .006, d = 1.24, and use of organization and transitions, t(24) = 3.63, p = .001, d = 1.48. There was no significant difference between the groups in terms of their ability to develop an original argument, t(24) = 0.84, p = .41, d = 0.34.

OPEN IN VIEWER

Table 1.

Means, (Standard Deviations), and 95% Confidence Intervals for Rubric Scores by Synthesis Component and Tutorial Condition.

Condition	Source Integration	Organization/Transition	Argument Development
No tutorial	2.12 (0.85)	2.46 (0.99)	2.73 (1.24)
	[1.61, 2.63]	[1.86, 3.06]	[1.99, 3.47]
Tutorial	3.31 (1.13)	3.81 (0.90)	3.11 (1.10)
	[2.63, 3.99]	[3.27, 4.35]	[2.45, 3.77]

Note. Rubric scores ranged from 1 (well below average) to 5 (well above average).

Discussion

A common writing difficulty for students occurs when they synthesize information from multiple sources (Cumming et al., 2016). Synthesis involves selecting, organizing, and connecting relevant information from source texts to support a new argument. Although scholars and instructors agree that this type of writing is difficult, the scholarship about teaching writing does not yet offer clear, easy-to-implement guidelines for psychology teachers interested in improving their students’ synthesis writing. Although there is no one-size-fits-all solution to teaching synthesis writing (Barzilai et al., 2018; Cumming et al., 2016; van Ockenburg et al., 2019), successful interventions appear to have some combination of explicit description of synthesis writing, strong use of examples, concrete methods for applying synthesis, and ample guided practice opportunities (Boscolo et al., 2007; Goddard, 2003; Hayes-Bohanan & Spievak, 2008; Kellogg, 2008; Segev-Miller, 2004).

Because the complexity of synthesis writing carries a high working memory load (Kellogg, 2008), interventions seem particularly likely to be successful when they employ concrete approaches to teaching advanced skills (Farrell, 2012). Unfortunately, most of the examples of concrete approaches to writing instruction involve lengthy time-consuming interventions that cannot readily be adapted into already content-heavy courses (Boscolo et al., 2007; Hayes-Bohanon & Spievak, 2008; van Ockenburg et al., 2019). Our study explored whether higher synthesis scores could be realized through a brief intervention that presented students with concrete techniques to support synthesis writing. This intervention consisted of viewing a tutorial outside of class that introduced a color-coding method of organizing key information from sources and spending one class period on instructor-guided practice in evaluating synthesis writing samples. This brief intervention resulted in significant differences in introductory psychology students’ synthesis writing quality; compared to students in the no tutorial group, students who completed the tutorial and the in-class practice were more likely to integrate information across sources and to use better organization and transitions.

Although synthesis overall was stronger following the intervention, students in the tutorial group did not generate better original arguments. The tutorial provided explicit instruction about how to integrate and to organize information from sources using the color-coding technique, but it did not provide a concrete guide for how students should develop a central argument. To fully instruct students about argument development—in addition to integration and organization—instructors may need to plan additional instruction, prepare high quality examples, or identify another step-by-step method that students could implement to develop this skill.

Classroom interventions provide more ecologically valid findings, but present methodological challenges. We acknowledge several related to our study. First, the assignment of students to conditions was nonrandom. Second, because we used a between subjects design, we can identify writing-quality differences between conditions but we cannot identify synthesis improvement within participants. Third, with nonrandom assignment and a between subjects design, it was important to establish ability equivalency between conditions. For our study, we compared overall student grades. We found no significant difference in grades between conditions, but students’ mean grade in the tutorial group was higher than the no tutorial group by about half a grade. This may represent a confound, such that students in the tutorial group received higher synthesis ratings on the posters than students in the no tutorial group because they happened to be a set of students with greater academic abilities. However, we point out that when comparing two courses, even if taught by the same instructor, it is possible that changes in exams or assignment point values may impact overall course grades in ways unrelated to underlying abilities. Fourth, although we used student-level data to examine ability equivalency, we used poster-level data (because some students worked in pairs) to examine synthesis. This prevented us from using grades as a covariate in our synthesis analysis to control for ability. Additionally, the no tutorial group had twice as many pairs that worked together on the poster assignment as the tutorial group. Working in pairs may make synthesizing more or less difficult, depending on the collaboration dynamic.

Future researchers should design writing intervention studies so they can tease apart the independent contributions of tutorial learning and classroom learning. This will require additional conditions (e.g., control group, in-class instruction group, tutorial group, combined group). Researchers should also consider how to create greater parallelism between the control group and intervention group(s) (e.g., if the intervention group watches a video tutorial, the control group watches a control version of a tutorial).

Despite the challenges inherent to classroom interventions, we argue that combining an out-of-class viewing of the “Using Synthesis in Your Writing” tutorial with a single in-class guided practice session represents a reasonable first step for instructors who are looking to incorporate direct synthesis writing instruction with minimal time investment. As teachers balance presenting content with developing students’ skills, brief interventions like this one provide valuable remediation.