Study Activities That Foster Generative Learning: Notetaking,Graphic Organizer,and Questioning

Abstract

Fourth graders were asked to read a text and either to fill in a compare-and-contrast graphic organizer, answer a set of structured questions, take notes, or simply read the text. Both the graphic organizer and questioning groups outperformed the read-only group on a comprehension test (d = 1.24 and 1.22, respectively) and a memory test (d = 0.54 and 0.53, respectively). No significant differences were found between the notetaking and read-only groups on the comprehension test (d = 0.30) or the memory test (d = 0.20). Results showed more integrative eye movements between paragraphs for the graphic organizer (d = 1.53) and the questioning groups (d = 1.90) than the read-only group, but not for the notetaking group (d = −0.06). On all three measures, the graphic organizer group and the questioning group exhibited a generative learning strategy, whereas the notetaking and read-only groups exhibited a linear learning strategy.

Keywords

learning strategies graphic organizers questioning notetaking eye tracking fourth graders

Objective and Rationale

Research on learning strategies shows that the outcome of reading academic texts depends not only on what information is presented but also on the learner’s cognitive processing of the material during reading (Fiorella & Mayer, 2015; Kiewra, 2005; Miyatsu, Nguyen, & McDaniel, 2018; Ponce, Mayer, Loyola, López, & Méndez, 2018b). In reading expository text, students can take a linear strategy in which they read and store the text sequentially or a generative strategy in which they mentally reorganize the material into a coherent structure as they read (Fiorella & Mayer, 2015, 2016; Ponce & Mayer, 2014a, 2014b). Although the linear strategy may be the default for less skilled readers, generative learning theory (Fiorella & Mayer, 2015, 2016) predicts that the generative strategy will lead to better learning outcomes (e.g., better performance on comprehension and memory tests) and more integrative eye movements during learning (e.g., number of transitions between one paragraph and another). The objective of this study is to determine the cognitive consequences of priming fourth-grade students to use generative learning strategies in a computer-based learning environment—for example, filling in graphic organizers or answering structured questions—or linear strategies—typing in notes into a box or simply reading without any prompts. Fourth-grade students may be unfamiliar with how to read expository texts because the primary grades are focused on building decoding fluency, so prompts may be particularly helpful for them.

Reading comprehension strategies, either implemented on paper or through computer implementations usually include strategies such as notetaking (Boers, Warren, Grimshaw, & Siyanova-Chanturia, 2017; Kiewra, 1987), questioning (O’Brien, 1997), and graphic organizers (Ponce, López, & Mayer, 2012); however, few studies have investigated the cognitive processes underlying the use of these strategies. Therefore, this study analyzes the cognitive processes involved in reading expository text while filling in a graphic organizer, answering a set of structured questions, taking notes, or doing no activity. To achieve this objective, we use an eye tracking methodology to observe eye movements while learners read a text using one of these learning conditions. Comparisons among the eye movement patterns observed for different learning activities allow for inferences to be made about the underlying cognitive processes involved and their relation to learning outcomes (Ponce et al., 2018b; Tsai, Huang, Hou, Hsu, & Chiou, 2016).

Imagine an instructor that asks her fourth-grade students to read on a computer screen expository texts such as the one shown in Figure 1. The rhetorical structure underlying the passage in Figure 1 is compare-and-contrast, because the text compares and contrasts two native Chilean peoples—the Changos and Rapa-nui—along several dimensions. Let us consider two instructional prompts that a computer application may have implemented. One option is to ask students to fill in an interactive graphic organizer to process the information in the text as shown in Figure 2. A second option is to ask students to respond to a series of questions as shown in Figure 3. The important common feature of the graphic organizer and questioning strategies shown in Figures 2 and 3 is that both are intended to prime a generative learning strategy in which the reader mentally organizes the content into a compare-and-contrast structure. Given this scenario, the instructor may ask about the effectiveness of the strategies—graphic organizer or structured questions—compared to each other and to reading without strategy prompts.
Figure 1.
The Changos and Rapa-nui passage in the read-only condition.

Figure 2.
The Changos and Rapa-nui passage and the interactive graphic organizer.

Figure 3.
The Changos and Rapa-nui passage and the process-structured questions.

Imagine now a second scenario in which the instructor may be tempted to ask her fourth graders to use a simpler technique such as taking notes from the text as shown in Figure 4. In this case, the instructor may ask whether a simpler technique such as notetaking can be as effective as the more generative ones (i.e., graphic organizers or questions) or at least more effective than a situation in which students only read the text (i.e., control condition), which is more likely to prime a linear learning strategy.
Figure 4.
The Changos and Rapa-nui passage and the editor for notetaking.

Learning strategies such as notetaking, filling in graphic organizers, and answering specific sets of questions have been studied previously, specifically their effects on learning outcome through tests of memory and comprehension (Dunlosky, Rawson, Marsh, Nathan, & Willingham, 2013; Fiorella & Mayer, 2015; Miyatsu et al., 2018). What is less known, however, is the cognitive processing primed by these computer-supported strategies while reading a text. Thus, using an eye tracking methodology, we focus on the cognitive processes primed by these learning strategies during learning.

Literature Review

Graphic organizers, questioning, and notetaking

Research in cognitive and educational psychology is beginning to lead to evidence-based recommendations for effective study strategies (Dunlosky et al., 2013; Fiorella & Mayer, 2015; Kiewra, 2005; Mayer, 2011; Miyatsu et al., 2018; Roediger, 2013) as well as evidence-based practical guides for students on how to study (Brown, Roediger, & McDaniel, 2014; Mayer, 2019; Weinstein & Sumeracki, 2019). This study focuses on three possible study activities for children learning from an onscreen text lesson: filling in a graphic organizer, typing in answers to questions about relations, and taking unguided notes.

Creating or filling in graphic organizers is a form of mapping (Fiorella & Mayer, 2015), in which learners create a spatial representation of selected words from the lesson. Ponce and Mayer (2014a, 2014b) have found that college students can benefit from filling in interactive graphic organizers during learning from an onscreen text passage. Complementary work shows that learning by creating graphic organizers allows students to remember more and make inferences (Beyer, 1997; Jairam & Kiewra, 2010; Ponce, Mayer, & Lopez, 2013), although students may need training and scaffolding to be able to use graphic organizers effectively (Ponce, Mayer, López, & Loyola, 2018a). In a review, Fiorella and Mayer (2015) reported that in eight out of eight experiments involving matrix graphic organizers—such as used in the present study—students who created or completed graphic organizers scored higher on learning outcome posttests than students who did not, with a median effect size of d = 1.07. However, all the studies involved college students (Jairam & Kiewra, 2010; Kauffman, 2004; Kauffman, Zhao, & Yang, 2011; Ponce & Mayer, 2014a), and so the present study expands the focus to include elementary school students.

With respect to the questioning prompts such as shown in Figure 3, this strategy is part of a more general learning strategy known as question answering (or elaboration), in which learners respond to questions posed by an instructor in class, included in textbooks to process its content, or as part of computer-based applications to interact with learners (Fiorella & Mayer, 2015; García-Rodicio, 2015; Weinstein & Sumeracki, 2019). Questions might be used at the beginning of the class to evaluate students’ knowledge about a specific topic or at the end of the lesson to encourage discussion of the content studied (Ramsey, Gabbard, Clawson, Lee, & Henson, 1990). While reading, questions can help in identifying important concepts and making connections between different elements in the text. Questions may also be used to scaffold more complex cognitive skills required to comprehend texts such as making inferences, hypothesizing, and predicting (Dean, 2003; Fordham, 2006). Specifically relevant for this study, questions about key content and structure of the text help students focus on the text’s internal structure (Beyer, 1997). Finally, question answering can be seen as a form of retrieval practice or the testing effect, which has been shown to be an effective learning strategy with prose passages (Dunlosky et al., 2013; Fiorella & Mayer, 2015; Kang, McDermott, & Roediger, 2007; Weinstein, McDermott, & Roediger, 2010). Thus, in the present study, answering targeted questions about relations among key elements is intended to prime similar cognitive processing as filling in a matrix.

Notetaking is one of the most common learning strategies used by students during learning, such as in lectures or when reading a text (Dunlosky et al., 2013; Fiorella & Mayer, 2015; Kiewra, 1987; Miyatsu et al., 2018). In the present study, students engaged in unguided notetaking, in which they received no scaffolding or training about what to include or how to organize it. In a review, Fiorella and Mayer (2015) found that in 26 of 30 experiments, writing summary notes caused improvements on posttest performance, with a median effect size of d = 0.50. However, none of the studies involved elementary school students, and the effects were strongest when students received guidance and training in how to take summary notes. Notetaking is intended to facilitate the cognitive processes of encoding and storage of information in long term memory. Unguided notetaking can foster a linear strategy, which downplays deeper elaboration of the material. Research shows that the notes taken by students are usually incomplete, fail to focus on important aspects of the content, constitute a fragmented list of facts, or they simply represent verbatim phrases taken from the learning material without further elaboration (Kiewra, 1985; Kiewra, DuBois, Christian, & McShane, 1988; Miyatsu, Nguyen, & McDaniel, 2018). For these reasons, we included unguided notetaking—with no scaffolding or training—as a kind of control activity in the present study (see Figure 4).

Eye movement research during reading

Eye fixations and saccades are two basic eye-movement measures employed to infer cognitive processing during reading. Findings in eye movement research show that new information is encoded during fixations, and as texts increase in difficulty, longer fixations, more saccades, and regressions (i.e., backward saccades) are observed (Rayner, 1998; Rayner, Chace, Slattery, & Ashby, 2006).

Studies in eye movement during reading typically examine variables related to the text and the reader such as the effect of passage difficulty level in reading comprehension (Rayner et al., 2006), the influence of text structure on cognitive processing (Ariasi & Mason, 2011), individual differences in strategy use (Hyönä, Lorch, & Kaakinen, 2002), vocabulary learning (Liu, 2014), and the effects of including pictures in texts to aid learning (Johnson & Mayer, 2012; Mason, Pluchino, & Tornatora, 2013). However, in most of these studies, the reader is cognitively active but behaviorally passive; in other words, reading behavior is observed while the reader reads the text, but the reader does not perform any additional learning activity such as taking notes or highlighting sections of the text.

Few studies have used eye-tracking data to make inferences about cognitive processing for different learning strategies. As an example, Ponce and Mayer (2014b) used eye tracking measures to examine how notetaking and filling in a graphic organizer affected cognitive processing during a reading task. During the experiment, college students read a text on a computer that compared the characteristics of eastern steamboats (in the top section of the paragraph) and western steamboats (in the bottom section of the paragraph). One group was instructed to read the text and fill in a compare-and-contrast graphic organizer, while a second group was instructed to read the text and take notes, and a third group read the text twice (control group). Fixations and transitions (i.e., saccades between areas of interests) were recorded while executing the learning task, and a memory test (i.e., cloze-test) and reading comprehension test (i.e., summary) were administered after they finished the task. Eye movement measures showed that participants read the text differently depending on the learning strategy employed, implying that different cognitive processes were primed by them. For example, participants in the graphic organizer group made more transitions between the top and bottom sections of the text than participants in the notetaking and read-only groups. Furthermore, the graphic organizer group scored significantly higher than the read-only group on both memory and comprehension measures, whereas the notetaking group scored significantly higher than the read-only group only on the memory test.

Theory and Predictions

According to generative learning theory, the quality of learning outcomes depends on the type of cognitive processing that learners engage in during learning (Fiorella & Mayer, 2015; Mayer, 2009; Wittrock, 1989). Generative learning theory states that three cognitive processes can operate in working memory during learning—that is, selecting, organizing, and integrating. The process of selecting involves learners paying attention to relevant information in the learning material (e.g., distinguish the elements being compared in a text). The process of organizing involves learners building a coherent structure of the incoming information (e.g., organize the information in a compare-and-contrast matrix). The process of integrating involves leaners connecting the new information with previous knowledge activated from long-term memory (e.g., relating the compare-and-contrast structure for the present text with past experience or similar schemas).

It is useful to distinguish between generative processing—which occurs when learners engage in selecting, organizing, and integrating during learning—and linear processing—which occurs when learners simply represent the material as presented without engaging in additional cognitive processing (Ponce & Mayer, 2014a, 2014b). Concerning learning outcomes, generative processing leads to meaningful learning outcome reflected by good test performance on retention (e.g., a memory test) and transfer (e.g., a comprehension test) whereas linear processing leads to rote or no learning outcome reflected in poor test performance (Mayer, 2009, 2011). Therefore, in the present study, we predict that the graphic organizer group will outperform the read-only group on the comprehension and memory tests (Hypothesis 1), the questioning group will outperform the read-only group on the comprehension and memory tests (Hypothesis 2), and the notetaking group will not differ from the read-only group on the posttests (Hypothesis 3).

Concerning learning processes, in the present study involving the compare-and-contrast passage shown in Figure 1, generative processing is reflected in making more eye movements between the top and bottom paragraphs because this reflects an effort to make connections and build a coherent structure, whereas linear processing is reflected in making fewer eye movements between the first and second paragraph because the learner is more likely to process the words in linear order. Therefore, we predict that the graphic organizer group will outperform the read-only group in up-down transitions indicating generative processing (Hypothesis 4), the questioning group will outperform the read-only group on up-down transitions indicating generative processing (Hypothesis 5), and the notetaking group will not differ from the read-only group in up-down transitions indicating linear processing (Hypothesis 6).

Study 1

The purpose of Study 1 was to determine how three learning strategies for text comprehension—questioning, graphic organizers, and notetaking—affect learning outcomes (as measured by comprehension and memory tests) in fourth graders in school settings.

Method

Participants and design

The participants were 192 fourth-grade students from four elementary schools located in Santiago, Chile. The schools were contacted based on previous relationships with the research team. They consisted of 6 fourth-grade classrooms with one classroom from School 1 (N = 40), one classroom from School 2 (N = 34), two classrooms from School 3 (N = 38), and two classrooms from School 4 (N = 80). School 4 was a school for girls only. We established four groups based on a between-subject design. Participants for the questioning and graphic organizer groups (which required pretraining) were randomly selected from Schools 1 and 2, whereas participants for the notetaking and read-only groups (which did not require pre-training) were randomly selected from Schools 3 and 4. Students in the questioning and graphic organizer groups were pretrained on these strategies during a 90-minute session because fourth graders were not familiar with these specific techniques. Given that it was not possible to separate the students for the pretraining session, we decided that only Schools 1 and 2 would receive such training. On the other hand, language arts teachers from each school indicated to us that students had been learning how to take notes based on identifying main ideas from texts, so pretraining on notetaking was deemed unnecessary. All students were native Spanish speakers.

Materials

The materials consisted of a standardized reading comprehension pretest, slideshow training lessons for questioning and graphic organizers, a paper-based text lesson, three study strategy sheets, and two paper-based posttests. The pretest was the reading comprehension section of a test called (in Spanish) Prueba de Comprensión Lectora y Producción de Textos (CL-PT) (Medina, Gajardo, & Fundaciœn-Educacional-Arauco, 2009), a standardized test commonly used in elementary schools in Chile. All materials were in Spanish.

The training lessons for graphic organizers and questioning consisted of slideshows implemented in PowerPoint that contained an explanation of the techniques, two one-paragraph texts to show the operation of each strategy in which the instructor could type in directly information on either the graphic organizer or answer the questions in the questioning strategy, and one additional text to practice. The first text called “Brothers” compared two brothers and their main characteristics such as age, personality, and interests. The second text titled “Homes” compared a house and an apartment focusing on the number of bedrooms, which floor they were in, and if they had a garden or a balcony. A third text titled “Telephones” was handed out to students as an in-class exercise to practice the use of these two techniques. This text compared two types of smartphones, indicating their users, technical characteristics, and usefulness in fulfilling different purposes.

The paper-based text lesson was a 200-word text entitled “Chile’s Indigenous Peoples: Chonos and Mapuches” (which in Spanish is, “Pueblos Originarios de Chile: Chonos y Mapuches”), as shown in Figure 1. It was structured into two paragraphs, each one describing each indigenous group along various categories and their corresponding characteristics: place where they lived, main activity, artistic or cultural activities, social organization, and how they buried their dead members. The first paragraph described the Chonos in terms of the five characteristics, and the second paragraph described the Mapuches along the same five characteristics.

The three study strategy sheets each presented a specific study aid: the graphic organizer, structured questions, or a blank sheet of paper for taking notes. The graphic organizer strategy consisted of an empty matrix with textboxes to fill in information about the elements that were compared in the texts, the attributes of comparison, and the values for each attribute identified. In this case, students were expected to write down in the boxes at the top of the matrix the name of the indigenous people referred to in the text (i.e., Changos and Rapa-nui). In the boxes in the middle of the matrix, students were expected to write down the attributes of comparison, and in the boxes on the right and left sides, the respective values for those attributes. Regarding the structured questions, these were the followings: What are the elements that are compared in the text? What are the attributes of comparison that appear in the text? And, what similarities or differences do you find between the elements compared? These questions are based on the structure of a comparison and can be used to guide the analysis of the text, like the visual structure of the graphic organizer (Beyer, 1997). Blank spaces were left between the questions, so students could answer them according to the information presented in the text. The notetaking sheet contained a box in which students wrote their notes.

The two posttests were a memory test and a comprehension test. The memory test consisted of a sheet of paper containing the passage but with 15 words erased and replaced with blank spaces. The Cloze format was devised to measure memory of the passage. The comprehension test consisted of a blank sheet of paper given to students with the instructions to write a summary of the text. We developed a four criteria rubric to grade the summary to measure the level of understanding of the comparison been made in the original text. The criteria were (a) summary text structure, (b) elements that are compared in the text, (c) attributes of comparison, and (d) degree of similarity or difference mentioned. For each criterion, the marker gave between 1 and 4 points, depending on specific guidelines given in the rubric. The maximum score in the summary test was 16 points. The tests were blindly scored. Each test was scored only once by a member of the research team. To check the reliability of the marking process, another member marked a random sample of tests; we found a high consistency (r = .86) between the two evaluators.

Procedure

The schools’ principals and deputy principals were contacted to ask for their collaboration in this study. We explained to them the purpose of the study and offered as compensation to prepare a report on the results of the reading comprehension pretest (i.e., CL-PT test). After each principal agreed to participate, we sent to them the informed consent to be distributed among parents or guardians. Between 1 and 2 weeks later, the informed consents signed by parents or guardians were received in each school.

The first step was to administer the CL-PT pretest to all fourth graders in each school. Afterwards, we arranged with each language arts teacher for a date to train students in Schools 1 and 2 on the graphic organizer and questioning techniques. The main reason to train students on these two strategies is that fourth graders did not have experience using them in class. However, they had experience learning how to use the notetaking technique as indicated by the language arts teachers.

Subsequently, we visited both schools and trained the students during a 90-minute session. For this, the slideshow presentation as described in the material section, which included the texts titled “Brothers” and “Homes” were used to explain each technique. In addition, the text titled “Telephones” was handed out to students to practice with these techniques under the supervision of the researchers and the language arts teacher. Students were not assessed on how well they understood these study aids.

A week after the training session, two research assistants visited Schools 1 and 2 again to run the experiment. The research assistants were not involved in the training sessions and were not aware of the hypotheses of the study. Students were randomly assigned to the graphic organizer or questioning groups based on their average score on the reading comprehension pretest. All students in each school were listed in a spreadsheet, including their pretest scores, gender, and an additional column that used a function to generate a random number for each student which was used to sort out the list. Means and standard deviations (SDs) on the pretest scores and gender (coded as 0 and 1) for the graphic organizer and questioning groups were computed several times until the means and SDs were very similar among the groups (statistically tested with a t test for pretest scores and a chi-square test for gender).

All students were handed out the same printed material, the first sheet of paper asked to complete personal information, the second sheet of paper included the text title “Pueblos Originarios de Chile: Chonos y Mapuches” with the instruction “read the following text with attention, we recommend reading it more than once.” Students were given 10 minutes to read this text. Subsequently, we handed out a third sheet of paper containing either a graphic organizer or structured questions to the respective students according to their assigned treatment group. Students were given another 10 minutes to complete the task involved. Once the time was up, we recollected the sheet of paper that contained the text and students kept the sheet of paper that contained the graphic organizer or structured questions with the information they filled in. Afterwards, a fourth sheet of paper was handed out to all students containing instructions to write a summary of the original text; students could use the information they completed in either the graphic organizer or structured questions. Students were given 10 minutes to write the summary. Next, we collected the summary and the sheet of paper with the respective strategy and handed out a sheet of paper containing the memory test. Students were given another 10 minutes to complete this task; they did not have access to the original text, nor the information on the graphic organizer or structured questions.

Regarding the notetaking and read-only strategies, we followed the same basic procedure. In each classroom of Schools 3 and 4, students were randomly assigned to the notetaking or read-only groups. Students in School 3 were randomly assigned to either group based on their scores on the reading comprehension pretest and gender, and students in School 4 were randomly assigned by only using their pretest scores (this was an all-girls school). Thus, all students working with the notetaking technique were in one classroom and those that only read the text were in another classroom. Students in the notetaking group followed the same procedure as the one indicated for the graphic organizer and structured questions, except there was no training process. With respect to the read-only group, students were given 10 minutes to read the text, followed by first the comprehension test and second the memory test, both without access to the original text.

Results

Preliminary analysis: Are the groups equivalent on basic characteristics?

Given that this study was conducted in school settings with children in the Chilean school system, logistical requirements did not allow us to maintain the level of random assignment that could be achieved in a laboratory study or even a large-scale field study. Thus, a first step is to determine whether this method of assignment resulted in groups that were equivalent on basic characteristics. Our rationale for analyzing the data at the student level is that each student was given a text lesson to read individually.

The sample size, mean score (and SD), and proportion of girls (PG) on the standardized reading comprehension pretest (CL-PT) for each group were the following: questioning group (N = 35, M = 27.10, SD = 6.74, PG = .49), graphic organizer group (N = 39, M = 28.15, SD = 7.38, PG = .46), notetaking group (N = 58, M = 26.92, SD = 8.09, PG = .81), and read-only group (N = 60, M = 27.24, SD = 5.43, PG = .85). An analysis of variance (ANOVA) showed that the four groups did not differ significantly from each other on the pretest, F(3, 188) = 0.264, p = .851. We conclude that the groups were equivalent regarding their reading comprehension skill. Concerning age, all students were fourth graders. Regarding gender, a chi-square test showed that the groups differed significantly, χ²(3) = 27.58, p < .001. Therefore, we included gender as a covariate in all analysis in Study 1. However, the same significant effects were found when the analysis was conducted without a covariate.

Do learning strategies improve learning outcomes?

According to the generative learning hypothesis, students in the questioning and graphic organizer groups should perform better than the control group on the comprehension test and the memory test (Hypotheses 1 and 2), whereas the notetaking group should not (Hypothesis 3). Table 1 shows the mean score (and SD) on the comprehension test and memory test for the four groups as well as the effect size (based on Cohen's d) in comparison to the read-only group. An analysis of covariance (ANCOVA) with gender as a covariate showed that the groups differed on the comprehension test, F(3, 184) = 19.466, p < .001; posthoc Dunnett's tests (with p < .05) indicated that the questioning group and the graphic organizer group significantly outperformed the read-only group, whereas the notetaking group did not. In other words, this finding indicates that students in the graphic organizer and questioning groups wrote on average better summaries than the notetaking and read-only groups. An ANCOVA with gender as a covariate showed the groups differed significantly on the memory test, F(3, 186) = 3.076, p = .029; posthoc Dunnett’s tests (with p < .05) indicated that the questioning group and graphic organizer group significantly outperformed the read-only group, whereas the notetaking group did not. In other words, this finding demonstrates that students in the graphic organizer and questioning groups remembered on average more words than the notetaking and read-only groups. These results show that study activities that are highly scaffolded and trained (i.e., questioning and graphic organizers) are effective for fourth graders but study activities that are not scaffolded or trained (i.e., notetaking) are not effective for fourth graders. This suggests that fourth graders may need scaffolding and training to assist them in using generative learning strategies effectively. This supports Hypotheses 1, 2, and 3 and is the primary contribution of Study 1.
Table 1.
Means, Standard Deviations, and Effect Sizes on Memory and Comprehension Tests for Each Group.

Group N Memory test
Comprehension test

M SD d N M SD d

Questioning 34 7.94* 2.99 0.53 33 10.09*** 3.28 1.22

Graphic organizer 39 8.03* 3.21 0.54 38 10.16*** 3.41 1.24

Notetaking 58 6.97 2.69 0.20 58 7.00 2.94 0.30

Read-only 60 6.43 2.77 – 60 6.05 3.32 –

Note. Effect sizes (d) computed in relation to the read-only group. Asterisks indicate group scored significantly higher than the read-only group.

p < .05. p < .001.

Study 2

The purpose of Study 2 was to determine how three learning strategies for text comprehension—questioning, graphic organizers, and notetaking—affect learning processes (as measured by eye transitions) in fourth graders in school settings using an eye tracking methodology.

Method

Participants and design

The participants were a subset of the original students from Study 1, with 15 in the questioning group, 18 in the graphic organizer group, 16 in the notetaking group, and 16 in the read-only group. They served in the same group as they had served in Study 1. The data quality from the eye tracker was not of sufficient quality for two students in the questioning condition and one student in the notetaking condition, so these were not included in the data analysis. The groups did not differ significantly on reading comprehension pretest score or proportion of girls and boys (as shown in the Results section).

Materials

The computer-based material consisted of an onscreen version of another 200-word text entitled “Chile’s Indigenous Peoples: Changos and Rapa-nui” (which in Spanish is, “Pueblos Originarios de Chile: Changos y Rapa-nui”). This text is different to one used in Study 1. This was implemented in PowerPoint. This two-paragraph text compares the Changos and Rapa-nui indigenous groups in terms of location, main activities, cultural or artistic expression, social organization, and how they buried their members; like the texts used in Study 1. The first paragraph described the Changos in terms of the five characteristics, and the second paragraph described the Rapa-nui in terms of the five characteristics. Similar to the material used in Study 1, the graphic organizer was embedded into the PowerPoint slide and implemented in Adobe Flash. It had functionalities to add new text boxes so users could type in directly into the PowerPoint slide (as shown in Figure 2). The questioning and notetaking techniques were implemented using the textbox object available in PowerPoint (as shown in Figures 2 and 3, respectively). All materials were in Spanish.

Apparatus

The eye tracker used in this study was the Tobii Pro X2-60 mounted on a personal computer with a 23-in. monitor. Tobii Studio software was installed on this computer and run at 60 Hertz.

Procedure

Two weeks after Study 1 was run in all schools, we revisited the schools to run Study 2 and collect data with the eye tracker. We arrived with the eye tracker and computer and installed it in a suitable office provided by the schools. In each school, we randomly selected students that employed the graphic organizer or structured questions strategies in Study 1 and assigned them to the same strategy in Study 2. We arranged with the language arts teacher in each school to ask a group of students to participate in the eye tracking session. The students went to the office where the equipment was set up.

After arriving, each student was asked to seat in front of the computer where the eye tracker was installed. We explained the purpose of the study, the procedure, the functioning of the eye-tracker, and continued with the calibration process. Subsequently, we explained how to fill in information on either the graphic organizer or structured questions or notetaking sheet, depending on which treatment the student had received during Study 1. Students were asked to read the instructions on the screen and were given 15 minutes to complete the task. They followed the instructions—read the text first and then fill in either the graphic organizer or the structured questions or take notes (depending on their assigned group). To make sure all groups spent 15 minutes on the material, students in the read-only group were asked to read it at least twice. We did not test students on reading comprehension or memory in the computer version of the experiment.

The students in the four groups did not differ significantly in their total number of eye-fixations on the first paragraph, F(3, 58) = .763, p = .519, or the second paragraph, F(3, 58) = .486, p = .694, indicating that they had equivalent exposure to the material.

Results

Preliminary analysis: Are the groups equivalent on basic characteristics?

As in Study 1, a first step is to determine whether the groups were equivalent on basic characteristics. The sample size, mean score (and SD), and proportion of girls (PG) on the standardized reading comprehension pretest (CL-PT) for each group were the following: graphic organizer group (N = 18, M = 29.60, SD = 8.32, PG = .50), questioning group (N = 13, M = 30.49, SD = 6.38, PG = .54), notetaking group (N = 16, M = 28.15, SD = 7.55, PG = .75), and read-only group (N = 16, M = 27.42, SD = 5.98, PG = .75). An ANOVA showed that the four groups did not differ significantly from each other on the pretest, F(3, 58) = 0.549, p = .651. A chi-square test showed that the groups did not differ significantly on gender, χ²(3) = 3.886, p < .274. We conclude that the groups were equivalent in terms of their reading comprehension skill. Concerning age, all students were fourth graders. To be consistent with Study 1, we included gender as a covariate in all analysis in Study 2. However, the same significant effects were found when the analysis was conducted without a covariate.

Do generative learning strategies improve learning processes?

We interpret up-down transitions (i.e., moving one's eyes between the first and second paragraphs during learning) as attempts to organize and integrate the material, which are generative learning processes (Ponce & Mayer, 2014b). According to the generative learning hypothesis, students in the questioning and graphic organizer groups should exhibit more eye transitions between the paragraphs than the read-only group (Hypotheses 4 and 5), whereas the notetaking group should not (Hypothesis 6). Table 2 shows the mean number of up-down transitions (and SD) for each group as well as the effect size (based on Cohen's d) in comparison to the read-only group. An ANCOVA analysis with gender as covariate showed that the groups differed significantly from each other, F(3, 57) = 15.601, p < .001, and posthoc Dunnett’s tests (with p < .05) indicated that questioning group and the graphic organizer group generated significantly more up-down transitions than the read-only group, whereas the notetaking group did not. Consistent with predictions, these results indicate that students in the graphic organizer and questioning groups made more attempts to connect pieces of information presented in the first paragraph with pieces of information presented in the second paragraph. This is an indication that scaffolded study activities such as graphic organizers and questioning caused fourth-grade students to organize and integrate the material during learning as compared to reading without scaffolding (consistent with Hypotheses 4 and 5); in contrast, notetaking which is a study activity without much scaffolding did not cause a change in cognitive processing during learning as compared to the read-only group (consistent with Hypothesis 6). As with Study 1, this suggests that fourth graders may need scaffolding and training to assist them in using generative study activities effectively.
Table 2.
Means, Standard Deviations, and Effect Sizes on the Transitions Between Areas of Interest.

Group N Up-down transitions

M SD d

Questioning 13 18.38* 7.90 1.90

Graphic organizer 18 17.61*** 8.80 1.53

Notetaking 15 6.73 5.20 −0.06

Read-only 16 7.00 3.81 –

Note. Effect sizes (d) computed in relation to the read-only group. Asterisks indicate group scored significantly higher than the read-only group.

***p < .001.

General Discussion

Empirical Contributions

Students who were required to fill in graphic organizers or answer structured questions during learning showed better learning outcomes (as indicated by scores on comprehension and memory tests) and deeper learning processes (as indicated by number of eye transitions between paragraphs) as compared to the read-only group. However, students who were required to take notes did not differ significantly from the read-only group on these measures of learning outcome or learning process. This shows that fourth graders benefit from study activities that involve some guidance—such as answering specific questions or filling in a provided matrix—but they do not benefit from a study activity that does not involve guidance—such as taking notes. The main new empirical finding is that when the learners are elementary school children, generative learning strategies for comprehension of text are helpful when they include scaffolding—that is, direct guidance on what to do—but not when they do not.

Theoretical Implications

These results support the generative learning hypothesis, which holds that asking students to engage in generative learning activities during learning causes students to engage in deeper learning processes and build better learning outcomes (Fiorella & Mayer, 2015). In particular, graphic organizers and questioning are intended to prime three basic cognitive processes leading to meaningful learning outcomes: selecting relevant information, mentally organizing it into a coherent structure, and integrating it with relevant prior knowledge.

Practical Implications

This study helps extend the generative learning hypothesis to elementary school children, by revealing an important boundary condition. Students learn better from a text lesson when they engage in generative learning strategies that involve scaffolding, such as filling in graphic organizers or answered structured questions. Apparently, unguided notetaking is not a productive learning strategy for young learners, although other research shows it can be effective for adults (Fiorella & Mayer, 2015).

Limitations and Future Directions

A limitation is that the present study was not based on random assignment of individual participants to groups. The groups were equivalent on a reading comprehension pretest scores and all were fourth graders. The proportion of girls in each group differed, but the treatment effects remained when the effect of gender was statistically controlled. Also, Study 2 involved a subset of the same participants as in Study 1, so we cannot rule out carryover effects from Study 1 to Study 2. The results should be interpreted considering these limitations, which was caused by the decision to conduct the study in school settings with a limited number of participants. In Study 1, students had access to their notes for the comprehension test (but not for the memory test) so it would useful to determine the same differences would be present if students did not have access to their notes. Generalization of these results to other learning situations is not possible due to the limited number of students involved and the convenience of the sample. Finally, future work is needed to compare the effectiveness of the three learning strategies with learners of varying age groups in the same study, and it would useful to determine whether explicit training would improve the effectiveness of notetaking in fourth graders.

Group	N	Memory test		Comprehension test
Questioning	34	7.94*	2.99	0.53	33	10.09***	3.28	1.22
Graphic organizer	39	8.03*	3.21	0.54	38	10.16***	3.41	1.24
Notetaking	58	6.97	2.69	0.20	58	7.00	2.94	0.30
Read-only	60	6.43	2.77	–	60	6.05	3.32	–

Group	N	Up-down transitions
Questioning	13	18.38***	7.90	1.90
Graphic organizer	18	17.61***	8.80	1.53
Notetaking	15	6.73	5.20	−0.06
Read-only	16	7.00	3.81	–

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This study was partially funded by the Universidad de Santiago de Chile through grant DICYT #061863PA.

Data Accessibility Statement

Research Data abstract for Study Activities That Foster Generative Learning: Notetaking, Graphic Organizer, and Questioning by Héctor R. Ponce, Richard E. Mayer, M. Soledad Loyola, and Mario J. López in Journal of Educational Computing.

Héctor R. Ponce is Professor of Information Systems at the University of Santiago of Chile and scientific director of the Interactive Technology Laboratory (VirtuaLab) at the same university. His research has concentrated on research and development of software components that implement visual and spatial learning strategies, examined how learning strategies affect cognitive processing during learning using an eye-tracking methodology, and experimented with interactive technologies for just-in-time formative assessment during whole-class instruction.

Richard E. Mayer is Distinguished Professor of Psychology at the University of California, Santa Barbara. His research interests are in applying the science of learning to education, with current projects on multimedia learning, computer-supported learning, and computer games for learning. He served as President of Division 15 (Educational Psychology) of the American Psychological Association and Vice President of Division C (Learning and Instruction) of the American Educational Research Association. He is the winner of the Thorndike Award for career achievement in educational psychology and the Distinguished Contribution of Applications of Psychology to Education and Training Award, and is ranked #1 as the most productive educational psychologist in the world in Contemporary Educational Psychology. He serves on the editorial boards of 12 journals mainly in educational psychology. He is the author of more than 500 publications including 30 books, such as Learning as a Generative Activity (with L. Fiorella), Computer Games for Learning, Applying the Science of Learning, e-Learning and the Science of Instruction: Fourth Edition (with R. Clark), Multimedia Learning: Second Edition, Learning and Instruction: Second Edition, Handbook of Research on Learning and Instruction: Second Edition (co-editor with P. Alexander) and the Cambridge Handbook of Multimedia Learning: Second Edition (editor).

M. Soledad Loyola is Senior Researcher at the Interactive Technology Laboratory (VirtuaLab) of the University of Santiago of Chile. She has conducted research on learning with computer technology, particularly on reading and writing.

Mario J. López is Professor of Information Technology at the Industrial Engineering Department, University of Santiago of Chile and director of the Interactive Technology Laboratory (VirtuaLab) at the same university. His research has concentrated on research and development of software components that implement visual and spatial learning strategies. He has directed several projects on educational technology funded by the Chilean Council for Science and Technology.

References

Ariasi

Mason

(2011) Uncovering the effect of text structure in learning from a science text: An eye-tracking study. Instructional Science 39(5): 581–601. doi:10.1007/s11251-010-9142-5.

Beyer

B. K.

(1997) Improving student thinking: A comprehensive approach, Boston, MA: Allyn and Bacon.

Boers, F., Warren, P., Grimshaw, G., & Siyanova-Chanturia, A. (2017). On the benefits of multimodal annotations for vocabulary uptake from reading. Computer Assisted Language Learning, 30(7), 709–725. doi:10.1080/09588221.2017.1356335.

Brown

P. C.

Roediger

H. L.

McDaniel

M. A.

(2014) Make it stick: The science of successful learning, Cambridge, MA: Harvard University Press.

Dean

(2003) Improving children’s learning: Effective teaching in the primary school, London: Routledge.

Dunlosky

Rawson

K. A.

Marsh

E. J.

Nathan

M. J.

Willingham

D. T.

(2013) Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest 14(1): 4–58. doi:10.1177/1529100612453266.

Fiorella

Mayer

R. E.

(2015) Learning as a generative activity, New York, NY: Cambridge University Press.

Fiorella

Mayer

R. E.

(2016) Eight ways to promote generative learning. Educational Psychology Review 28(4): 717–741. doi:10.1007/s10648-015-9348-9.

Fordham

N. W.

(2006) Crafting questions that address comprehension strategies in content reading. Journal of Adolescent & Adult Literacy 49(5): 390–396.

10.

García-Rodicio

(2015) Questioning as an instructional strategy in multimedia environments: Does having to answer make a difference? Journal of Educational Computing Research 52(3): 365–380. doi:10.1177/0735633115571931.

11.

Hyönä

Lorch

R. F.

Kaakinen

J. K.

(2002) Individual differences in reading to summarize expository text: Evidence from eye fixation patterns. Journal of Educational Psychology 94(1): 44–55. doi:10.1037/0022-0663.94.1.44.

12.

Jairam

Kiewra

K. A.

(2010) Helping students soar to success on computers: An investigation of the SOAR study method for computer-based learning. Journal of Educational Psychology 102(3): 601–614. doi:10.1037/a0019137.

13.

Johnson

C. I.

Mayer

R. E.

(2012) An eye movement analysis of the spatial contiguity effect in multimedia learning. Journal of Experimental Psychology: Applied 18(2): 178–191. doi:10.1037/0033-2909.124.3.37210.1037/a0026923.

14.

Kang

S. H. K.

McDermott

K. B.

Roediger

H. L.

(2007) Test format and corrective feedback modify the effect of testing on long-term retention. European Journal of Cognitive Psychology 19(4–5): 528–558. doi:10.1080/09541440601056620.

15.

Kauffman

D. F.

(2004) Self-regulated learning in web-based environments: Instructional tools designed to facilitate cognitive strategy use, metacognitive processing, and motivational beliefs. Journal of Educational Computing Research 30(1–2): 139–161. doi:10.2190/AX2D-Y9VM-V7PX-0TAD.

16.

Kauffman

D. F.

Zhao

Yang

Y.-S.

(2011) Effects of online note taking formats and self-monitoring prompts on learning from online text: Using technology to enhance self-regulated learning. Contemporary Educational Psychology 36(4): 313–322. doi:10.1016/j.cedpsych.2011.04.001.

17.

Kiewra

K. A.

(1985) Investigating notetaking and review: A depth of processing alternative. Educational Psychologist 20(1): 23–32.

18.

Kiewra

K. A.

(1987) Notetaking and review: The research and its implications. Instructional Science 16(3): 233–249. doi:10.1007/bf00120252.

19.

Kiewra

K. A.

(2005) Learn how to study and SOAR to success, Upper Saddle River, NJ: Pearson Prentice Hall.

20.

Kiewra

K. A.

DuBois

N. F.

Christian

McShane

(1988) Providing study notes: Comparison of three types of notes for review. Journal of Educational Psychology 80(4): 595–597. doi:10.1037//0022-0663.80.4.595.

21.

Liu

P.-L.

(2014) Using eye tracking to understand the responses of learners to vocabulary learning strategy instruction and use. Computer Assisted Language Learning 27(4): 330–343. doi:10.1080/09588221.2014.881383.

22.

Mason

Pluchino

Tornatora

M. C.

(2013) Effects of picture labeling on science text processing and learning: Evidence from eye movements. Reading Research Quarterly 48(2): 199–214. doi:10.1002/rrq.41.

23.

Mayer

R. E.

(2009) Multimedia learning, 2nd ed. New York, NY: Cambridge University Press.

24.

Mayer

R. E.

(2011) Applying the science of learning, Boston, MA: Pearson/Allyn & Bacon.

25.

Mayer

R. E.

(2019) How to be a successful student: 20 study habits based on the science of learning, New York, NY: Routledge.

26.

Medina

Gajardo

A. M.

Fundación-Educacional-Arauco (2009) Prueba de comprensión lectora y producción de textos (CL-PT): Kinder a 4to año básico [Reading comprehension and text production test (CL-PT): From kindergarten to 4th grade], Santiago de Chile, Chile: Ediciones Universidad Católica de Chile.

27.

Miyatsu

Nguyen

McDaniel

M. A.

(2018) Five popular study strategies: Their pitfalls and optimal implementations. Perspectives on Psychological Science 13(3): 390–407. doi:10.1177/1745691617710510.

28.

O’Brien

(1997) A computer program to provide practice in questions and answers for learners of English. Computer Assisted Language Learning 10(3): 299–305. doi:10.1080/0958822970100307.

29.

Ponce

H. R.

López

M. J.

Mayer

R. E.

(2012) Instructional effectiveness of a computer-supported program for teaching reading comprehension strategies. Computers & Education 59(4): 1170–1183. doi:10.1016/j.compedu.2012.05.013.

30.

Ponce

H. R.

Mayer

R. E.

(2014a) An eye movement analysis of highlighting and graphic organizer study aids for learning from expository text. Computers in Human Behavior 41. : 21–32. doi:10.1016/j.chb.2014.09.010.

31.

Ponce

H. R.

Mayer

R. E.

(2014b) Qualitatively different cognitive processing during online reading primed by different study activities. Computers in Human Behavior 30: 121–130. doi:10.1016/j.chb.2013.07.054.

32.

Ponce

H. R.

Mayer

R. E.

Lopez

M. J.

(2013) A computer-based spatial learning strategy approach that improves reading comprehension and writing. Educational Technology Research and Development 61(5): 819–840. doi:10.1007/s11423-013-9310-9.

33.

Ponce

H. R.

Mayer

R. E.

López

M. J.

Loyola

M. S.

(2018a) Adding interactive graphic organizers to a whole-class slideshow lesson. Instructional Science 46(6): 973–988. doi:10.1007/s11251-018-9465-1.

34.

Ponce

H. R.

Mayer

R. E.

Loyola

M. S.

López

M. J.

Méndez

E. E.

(2018b) When two computer-supported learning strategies are better than one: An eye-tracking study. Computers and Education 125. doi:10.1016/j.compedu.2018.06.024.

35.

Ramsey

Gabbard

Clawson

Lee

Henson

K. T.

(1990) Questioning: An effective teaching method. The Clearing House 63(9): 420–422.

36.

Rayner

(1998) Eye movements in reading and information processing: 20 years of research. Psychological Bulletin 124(3): 372–422. doi:10.1037/0033-2909.124.3.372.

37.

Rayner

Chace

K. H.

Slattery

T. J.

Ashby

(2006) Eye movements as reflections of comprehension processes in reading. Scientific Studies of Reading 10(3): 241–255. doi:10.1207/s1532799xssr1003_3.

38.

Roediger

H. L.

(2013) Applying cognitive psychology to education: Translational educational science. Psychological Science in the Public Interest 14(1): 1–3. doi:10.1177/1529100612454415.

39.

Tsai

M.-J.

Huang

L.-J.

Hou

H.-T.

Hsu

C.-Y.

Chiou

G.-L.

(2016) Visual behavior, flow and achievement in game-based learning. Computers & Education 98: 115–129. doi:10.1016/j.compedu.2016.03.011.

40.

Weinstein

McDermott

K. B.

Roediger

H. L.

(2010) A comparison of study strategies for passages: Rereading, answering questions, and generating questions. Journal of Experimental Psychology: Applied 16(3): 308–316. doi:10.1037/a0020992.

41.

Weinstein

Sumeracki

(2019) Understanding how we learn, New York, NY: Routledge.

42.

Wittrock

M. C.

(1989) Generative processes of comprehension. Educational Psychologist 24(4): 345–376. doi:10.1207/s15326985ep2404_2.