Integrating Information

Abstract

The case study reported here explores the processes involved in producing a written synthesis of three history texts and their possible relation to the characteristics of the texts produced and the degree of comprehension achieved following the task. The processes carried out by 10 final-year compulsory education students (15 and 16 years old) to produce their syntheses, including the integrations they verbalized while performing the task, were examined in detail with a double-analysis system. The results revealed a tendency for the students who engaged in more elaborative patterns to make more integrations and produce better texts. These students seemed to benefit more from the task in terms of comprehension. Conversely, the students who followed a more reproductive pattern by and large copied ideas from the source texts and achieved low levels of comprehension.

Keywords

written synthesis patterns reading and writing relations integrations learning

The study reported here is part of a line of research into the epistemic character of reading comprehension and written composition. From the socioconstructivist perspective, it is assumed that reading and writing in socially significant activities may promote new ways of understanding, learning about, and representing reality and new cognitive capacities and functions (Kozulin, 2000). We are particularly interested in deepening our knowledge of the processes involved in tackling hybrid reading and writing tasks, such as summaries and syntheses, which require the student to consider information from various documents to elaborate a new product, often with a view to fostering deeper understanding of their contents. Our general hypothesis is that the processes carried out by students in performing such tasks influence the quality of the written products for which they elaborate and, in the final analysis, the learning potential associated with this elaboration.

The Epistemic Potential of Synthesis Tasks

Previous research has concluded that when reading and writing are used together, their epistemic power is greater than when they are employed separately (Tierney & Shanahan, 1996; Tynjälä, 2001). On hybrid tasks, students alternate recurrently between the role of writer and the role of reader (of the source texts [ST], of notes, of their own text, of the sources again, etc.), engaging in a sort of dialectic with themselves that helps to explain the learning potential of these tasks (Tierney, O’Flahavan, & McGinley, 1989).

This potential increases in the case of complex tasks: those involving two or more STs and requiring the student to produce a text of one’s own whose contents and structure must be decided by the student. These tasks face the student with a simultaneous problem of content (“What should I say [taking into account what I have read]?”) and rhetoric (“How should I say it [fitting my voice in between other voices]?”)—the resolution of which may lead the student to transform one’s knowledge (Bereiter & Scardamalia, 1987) as a result of the comprehension, integration, and elaboration processes involved.

Producing a written synthesis requires the student to organize, select, and connect the information from the various texts (Spivey, 1990, 1997; Spivey & King, 1989); the author must find a common thread (Segev-Miller, 2004) and generate a new structure enabling one to integrate and organize the contents in an original text. Following Segev-Miller (2004), performing the task involves reading and rereading the STs and the student’s own text, reexamining the STs on numerous occasions, and correcting and reelaborating one’s own text in a recurrent process enabling an author to transform the information on three levels: conceptual, rhetorical, and linguistic. The integration component acquires an extremely important role, as the possibility of linking up and connecting ideas from the different STs in a new text is a necessary condition for constructing synthesis texts.

Integration of Textual Content

Over the past few years, various studies have looked at integration and its effect on understanding and learning. Wiley and Voss (1999), who presented students with seven STs, found that writing arguments requiring the integration of information enhances the recognition of inferences and knowledge transformation, whereas explaining, narrating, or making summaries of the same texts did not have the same effect. Cerdán and Vidal-Abarca (2008) found that students performing a task involving intertextual integration (writing an essay) obtained better deep learning outcomes than students who had to answer intratextual questions (both tasks referring to the same items of information in the text). However, they did not find any differences attributable to the task in regard to superficial comprehension measurements. Cerdán, Vidal-Abarca, Martínez, Gilabert, and Gil (2009) obtained similar findings in comparing the learning levels of students who, after reading a text, had to (a) answer questions requiring them to interpret and integrate distant information and (b) answer questions that could be located in specific fragments of the texts and required little inference. In a recent study (Miras, Solé, & Castells, 2008), the students who produced the more appropriate texts (i.e., who selected the information from the STs better and integrated it) were also the ones who performed better on a comprehension/learning task related to the contents of the synthesis, the integration component being the most important in explaining this performance. Taken as a whole, these studies seem to show that integration is important, especially in deep learning and in the use of elaborated knowledge.

In spite of the potential importance of the processes leading to the integration and synthesis of texts, few studies have dealt with them. Those that have focused on processes have stressed their linearity or recursivity as explaining the different quality of the texts produced. Working from this perspective, Lenski and Johns (1997) carried out a case study with six secondary education students who were asked to write a research report on a topic based on reading various texts. The students had to find the sources of information, read, and write. Depending on the order in which the students carried out these processes, the investigators distinguished three patterns or profiles: sequential, spiral, and recursive. With the exception of one student, who adopted a recursive pattern and wrote an integrated report, the others adopted sequential or spiral patterns and wrote summaries, paraphrases in their own words of what they had read in the texts. More recently, Rouet, Vidal-Abarca, Bert-Erboul, and Millogo (2001) observed that students examining a text on which they had to answer questions requiring them to connect information from different parts of the text engaged in a review-and-integrate search pattern, whereas students who were given questions requiring them to locate specific items of information displayed a locate-and-memorize search pattern. These patterns were in turn related to differences in the learning achieved. In a previously cited study, Cerdán and Vidal-Abarca (2008) found that a task involving intertextual integration led participants to read relevant information more slowly than irrelevant information, but this did not happen on an intratextual task.

These studies suggest the importance of a recursive use of reading in producing a synthesis. Such a task involves handling and understanding two or more documentary sources from which relevant items of information that can be connected to one another have to be selected. Producing a synthesis requires students to read and reread the texts (N. Nelson, 2008) both to identify the relevant information and to elaborate and integrate it. A single reading generally leads to a superficial understanding of the text (Sánchez, 1998), which makes it difficult to carry out these processes and thus suggests the need for rereading after reading the STs and while writing or after writing their text to compose and/or revise. However, making a synthesis poses not only a reading problem; it is also a demanding writing task requiring the mediation of planning, textualization, and monitoring throughout the whole process (Flower & Hayes, 1981), as well as the use of review strategies ensuring a good fit between the student’s on text and the STs, which in turn leads back to rereadings to ensure not only elaboration of the information but also the monitoring already mentioned.

Previous studies (Mateos & Solé, 2009; Solé, Miras, & Gràcia, 2005) revealed differences in the reading and writing strategies employed by more or less expert students in making summaries and syntheses, as well as a certain correspondence between the strategies employed and the overall quality of the written product. The study reported here is a step further along this path. It aims at examining in detail, with a focus on the integration component, the processes carried out by 10 secondary school students in writing a synthesis. We expected to find

processes of various levels of complexity according to whether they were more linear or more recursive, more direct or more mediated;

a greater number of integrations made by students using recursive patterns in comparison to students using linear ones;

different synthesis products depending on the degree to which the information was selected and integrated into a coherent whole; and

better performance in a comprehension task for the students who had produced syntheses with more recursive patterns and more integrations.

Method

Participants

The participants in this study were 10 fourth-year compulsory secondary education students (5 boys and 5 girls) aged between 15 and 16 years. They were selected at random from an overall sample of 48 pupils attending a state-subsidized private school in Barcelona.¹ In their teacher’s opinion, all the participants were good or average readers. None of them had any specific reading difficulties.

Design

The students were asked to write a synthesis of three history texts, which formed the basis of a qualitative case study.

Materials

Three texts were drawn up containing complementary information on the general topic of the synthesis (see Table 1): the impact of foreign intervention on the development and outcome of the Spanish Civil War. The texts, which the teacher regarded as suitable for the average level of the participants, had an mean word length of 335.

Table 1.

Characteristics of the Texts

Source Text	No. of Words: Origin	Content Items
ST1: Expositive—data and facts, includes a table	331: Adapted from the textbook	Context: elections, coup d’état, war. Countries helping each side.^a Nonintervention pact,^a breach.
		Table: type and origin of military aid.
ST2: Expositive—data and facts	326: Composed on the basis of several source texts	European context: democratic and fascist regimes. Countries helping each side.^a Nonintervention pact.^a
ST3: Argumentative—assessments	348: Composed on the basis of several source texts	Countries helping each side.^a Nonintervention pact.^a Discrepancies regarding the assessment of the foreign aid received by each side (type, time, amount, impact on outcome).

Content items present in more than one text.

Pretask measurements

The students’ prior knowledge of the synthesis topic was assessed by means of a six-item multiple-choice general knowledge questionnaire on the Spanish Civil War (when it occurred, the sides taking part, the outcome, etc.; see example in Appendix A). It was thought advisable to measure prior knowledge of the subject of the texts, as the influence of this variable on comprehension is well established (Boscolo & Mason, 2003; McNamara, Kintsch, Songer & Kintsch, 1996; Stahl, Hynd, Britton, McNish & Bosquet, 1996; Strømsø & Bråten, 2002).

Comprehension measurement

The level of the participants’ understanding upon completion of the task was assessed by means of a test comprising 11 questions of varying complexity (Cerdán et al., 2009): 5 “low level” questions (4 multiple-choice and 1 open-ended), which involved recovering information from the texts; and 6 “high level” questions (4 multiple-choice and 2 open-ended) requiring the students to interpret and integrate information present in one or more of the texts (see examples in Appendix A).

Procedure

Prior to the synthesis task, the participants took the prior knowledge test with their classmates. In this session, they were informed about the synthesis task that they were to perform later on. The task was presented as just another history-learning activity, and the students were told that the syntheses would be given to the teacher. While the majority of the class did the task together in the classroom, the 10 participants in the case study—who had given their consent—did it individually in the presence of a researcher.

In the session, the researchers explained the think-aloud method to the students (Coté & Goldman, 1999; Ericsson & Simon, 1980). Students were told to say what they were thinking (whatever came into their mind) as they read or wrote. They were provided with examples and invited to raise any doubts or queries. They then did a trial run during which, if they failed to verbalize spontaneously, they were asked at certain times, “What are you thinking?”

Following the trial run, the students were given the following written instructions:

The texts you are going to read are about foreign involvement in the Spanish Civil War (1936-1939). It is generally accepted that this involvement had repercussions on the development and outcome of the war.

Read the texts.

Write a text including the most important information and ideas dealt with in the texts you have read. You can read the texts as many times as you want. You can make rough drafts and take notes.

The students were shown the three texts one after the other on the computer screen and were given a Word document in which to write their own text. They were told that they could go from one text to another whenever they liked, open another document for writing notes, use the sheets of paper on the desk, and reread the instructions. When the students considered themselves ready, they began. The investigator confined herself to intervening as described above if necessary to provoke verbalizations.

Information on the processes carried out by the participants in producing their syntheses was obtained via Camtasia Studio 6 software, which enabled to record the sequence of actions performed by the students (the order in which the texts were read and the time devoted to reading each, writing of the students’ own text, returning to the texts during or after writing, etc.). In addition, a video recording was made of the whole sequences.

A 2-hour session was set aside for the task, which proved to be sufficient in every case. The following day, each participant answered the comprehension questions on the contents of the texts. The sequence described took place the week before the class began a teaching unit on the civil war.

Coding and Analyzing Procedures

Products

The students’ written products were analyzed in regard to a set of categories enabling the identification of differences conceptually relevant to the synthesis texts: selection of information, integration, text organization, elaboration of information, and comprehension errors (Flower et al., 1990; Segev-Miller, 2004; Spivey, 1997). Before the written products were analyzed, the essential units of information for performing the task adequately were established by interjudge agreement. We prepared a model synthesis that included nine information units regarded as essential for a successful synthesis—six of which contained information that was in just one of the texts, one of which contained information that was in two of the texts, and two of which contained information to be found in all three texts. A similar procedure was employed to define four core items of information that needed to be integrated to achieve a coherent synthesis: two intratextual integrations (ST1 and ST3) and two intertextual integrations (ST1–ST2 and ST1–ST2–ST3; see Appendix A). The dimensions for each of the categories were agreed. The categories and dimensions of analysis were as shown in Table 2. To ensure reliability, the written products were analyzed by two researchers, who obtained 79% overall agreement and resolved discrepancies collaboratively.

Table 2.

Categories and Dimensions of Analysis

Category	Dimensions
Textual organization and coherence	Juxtaposed summaries of two or more source texts or list of ideas
	Alternating fragments from two or more source texts
	Text given its own structure
Selection of information	Completely insufficient selection of information (fewer than three items)
	Partial and problematic selection of information (four or five items)
	Sufficient selection of information (six to nine correct items)
Integration of information	Number of intratextual integrations
	Number of intertextual integrations
Elaboration of the information	Copy
	Paraphrase
	Elaboration of ideas from the texts
Errors of comprehension	Important, affecting comprehension
	Unimportant, not affecting overall comprehension

Processes

A single protocol was prepared for each participant based on the record of the reading and writing actions provided by the Camtasia software and the audiovisual recording (plus reference to the STs and the text produced by the student). The actions were coded using the category system—the first level of analysis—which makes it possible to characterize the reading and writing actions performed (see Table 3), as well as their duration and sequence. These categories were identified in a previous study using comparable tasks (summary or synthesis of two STs) performed by participants, some of whom were the same age as those in this study and some of whom were a different age. Our previous analysis (Mateos & Solé, 2009; Solé et al., 2005) had shown that the system includes all the pertinent actions students engage in. As well as the information supplied by the analysis, the time taken by each participant to complete the task was recorded.

Table 3.

Action Categories

Read ST1, ST2, ST3

Read part of ST1, ST2, ST3

Reread ST1, ST2, ST3

Read/reread ST1, ST2, ST3, and outlines

Read own notes and/or outlines

Read the text in production

Read the text produced

Read the task instructions

Copy part of ST1, ST2, ST3

Write notes and/or outlines

Write the synthesis text

Modify the text produced

Modify text using the spell checker

Note: ST = source text.

On the second level of analysis, we dealt with the verbalizations uttered by the participants while performing the task. The videotaped comments were directly coded using subtitling software (Subtitle Workshop 2.03, DekSoft). In segmenting the think-aloud protocols, we built on the work of Coté, Goldman, and Saul (1998) and defined our unit of analysis as an “event,” which was “a comment or set of comments on the same core sentence or group of sentences in the context of the reading or writing procedure associated with those comments” (p. 14). The utterances were coded using a category scheme based on previous research findings (Coté et al., 1998; Flower et al., 1990; McGinley, 1992; Palincsar & Magnusson, 2001; Pressley & Afflerbach, 1995). The specificity in our case consisted in the production of a single-category scheme for both reading and writing in accordance with our view of the hybrid nature of the synthesizing tasks involving them. Each event was assigned to one category depending on the predominant character of the comment. The major event categories were task analysis, planning, meaning construction, monitoring, and evaluative and affective reactions. This system has been used with similar tasks given to students at different educational levels (including the level of the participants in this study), and it made it possible to differentiate satisfactorily between the verbalizations uttered during both reading and writing (Mateos, Villalón, De Dios, & Martín, 2007; Solé et al., 2005).

In this study, we looked at the verbalizations that were coded as integration. In our system, these are meaning construction operations. Meaning construction events included restatements of a sentence or a previously generated idea, paraphrases, elaborations relating content to personal knowledge, elaborations consisting in drawing conclusions (synthesizing, generalizing as well as extending beyond the text), and integrations. Integrations involve connecting items of information appearing in two texts—intertextual integration—or fragments at some distance from each other in the same text—intratextual integration. In addition to coding the type of integration (intra/inter), whether they were correct or incorrect was taken into account. The verbalizations were analyzed by two researchers, who obtained 74% agreement.

In addition, an interpretative analysis was made of the protocols, with the aid of the recording, the reference to the STs, the text produced, and the verbalizations uttered by the participants, concerning the linear/recursive and direct/mediated nature of the processes carried out by the participants. To perform this analysis, we took into account three conceptually distinct components having to do with recursivity and mediation in reading and writing processes: rereading, planning and revision (Flower et al., 1990; Lenski & Johns, 1997; McGinley, 1992).

Rereading in hybrid tasks means the students going back and forth between their text and the STs to compare, deepen, or check the information. In a study with similar participants and tasks (Solé, 2009), we were able to identify forms of rereading that appear to pursue different goals. Rereading to select involves going back to the ST to identify the information to be included in the student’s text. Indeed, such rereading may give rise to two quite distinct processes. In the selection/capture process, consisting of extremely rapid switching between the STs and the participant’s text, the student selects ideas from the former and imports them almost literally into the latter. Yet, the selection/elaboration process involves rereading fragments of the former for a longer time, while the information is elaborated to varying degrees and not simply copied as it is.

Deepening rereading aims to enhance understanding of a source (or part of a source), deepen knowledge, clear up doubts, or resolve problems of interpretation. Yet another possible type of rereading in the performance of hybrid tasks is rereading to check, to make sure that the text produced is consistent with the STs. A recursive pattern necessarily involves various rereadings, and these must take place throughout the whole process. The rereading indicators were deduced from the action sequence in which the students engaged and from their verbalizations.

Planning writing implies a clear mediation between what one has read (and possibly elaborated) and what one is going to write in the text. Writing a synthesis is a demanding task that requires the performer to take decisions about what one is going to say and how one is going to say it; the presence of planning indicates that the author is directing the task, whereas the absence of planning means that the author is merely following the STs. The planning indicators were established mainly on the basis of the participants’ verbalizations.

Last, revision constitutes a new mediation, another chance to elaborate and act recursively, without ignoring the STs when checking the accuracy and suitability of what has been written. This recursivity, linked mainly to the student’s rereading of one’s text, may allow the student to detect and correct mistakes of interpretation, whereas the absence of revision means that such errors remain in the student’s text. The revision indicators are mainly deduced from the action sequence in which the participants engage.

The processes employed by participants were analyzed by two researchers, who obtained 79% agreement.

Results

The participants in this study obtained an average score of 4.2 correct answers out of 6 (SD = 1.98) on the prior knowledge test (α = .62, determined in relation to the whole sample).

Only two students (Ahm and Mon) obtained a prior knowledge score a standard deviation below the mean, while three (Mari, Mar, and Gui) achieved the maximum score (see Table 6).

Products

Table 4 summarizes the results of the analysis for each text on the categories described in the Coding and Analyzing Procedures section. (To correctly interpret the scores of Ahm and Mon on certain dimensions, it is necessary to bear in mind that their texts were in large part verbatim reproductions of different parts of the STs.)

Table 4.

Analysis of the Texts Produced by the Students

Name	Organization	Elaboration	Selection of Information	Mistakes Affecting Comprehension	Intertextual Integrations	Intratextual Integrations
Ahm	Juxtaposition STs	Copy	Completely insufficient	Unimportant; a few suggesting lack of understanding ST1	0	0
Mon	Juxtaposition STs	Copy	Completely insufficient	No	0	0
Xav	Juxtaposition STs, list of ideas	Paraphrase / copy	Partial and problematic	Unimportant; a few suggesting lack of understanding ST1	0	0
Mari	List of ideas	Paraphrase / some elaboration	Completely insufficient	Important; confusions: chronology, sides, countries and recipients of aid	0	0
Sil	Alternates paragraphs	Paraphrase / some copying	Completely insufficient	Important; confusions: chronology, relations country / regime / aid, recipients of aid	0	0
Mar	Juxtaposition STs	Elaborations / paraphrase	Partial and problematic	Unimportant; confusion chronology	0	1
Gui	Juxtaposition STs	Elaborations	Partial and problematic	No	0	3
Alb	Alternates paragraphs	Paraphrase / copy	Partial and problematic	Unimportant; confusion chronology	1	0
Elo	Alternates paragraphs	Paraphrase / elaborations	Sufficient	Unimportant	1	1
Bru	Own structure: Schema	Elaborations / paraphrase	Sufficient	Unimportant	1	1

Note: ST = source text.

The texts produced by the participants have common features: They are extremely dependent on the STs, not just in regard to the contents—which is what would be expected—but also in regard to the way and the order in which the information is organized. In the majority of cases, they fail to select sufficient information; most of them contain errors of interpretation of varying degrees of importance. Moreover, 5 of the 10 students did not integrate at all, and 2 made only one integration.

The analyses carried out confirm the difficulty involved in this type of task. Indeed, none of the students’ texts has the necessary features to be considered a fully adequate synthesis, as they all fail to provide a central theme for structuring and integrating the information. However, this does not mean that all the products were the same; the characteristics that we propose, on the basis of the dimension categories along which we have analyzed the texts, make it possible to distinguish three types of product:

Failed texts: texts whose authors follow a strategy of saying linearly (unchanged) the information in the STs. They faithfully copy and/or paraphrase the ideas in the order in which they appear in the STs. They do not adequately select or integrate the information. These texts present problems of coherence. Five of the texts produced were “failed”: copying (Ahm, Mon), list of ideas (Xav), or linear paraphrase with serious mistakes (Sil, Mari).

Juxtaposed summaries: texts whose authors seem to follow a strategy of saying briefly what is most important in each ST. They abide by the task instructions, but as they are not oriented to establishing connections, they may omit core items of information (e.g., the regimes of the countries that supplied aid / the side to which the aid was provided; nonintervention pact / breaches thereof). The writers of summary texts select items of information from the sources one by one and include intratextual integrations. They do not copy; they paraphrase and/or elaborate the ideas in the STs. They make mistakes that do not affect comprehension. Two students produced texts in this category (Mar and Gui).

Attempted syntheses: texts whose authors were able to carry out the task instructions. They seem to implement a strategy of examining the texts to extract relevant information and link it up in a text of their own. They select information from the sources without omitting any substantive items, connect it (by means of intratextual and intertextual integrations), and organize it in their own text. They do not copy; at most, they paraphrase and elaborate. Three texts fit this outline: those of Bru, Elo, and Alb, with the latter’s being the only product whose allocation to one of the categories was problematic; the information in it was partial and some fragments copied, but we gave greater weight to the integration of information from ST1 and ST2.

Processes: Task Performance Patterns

The first level of analysis provided a description of the sequence of actions followed by each participant in performing the task (see Appendix B). An additional, interpretative analysis was made of these sequences taking into account the kind of rereading, planning, and revision activities the participants carried out (see Method). These dimensions make it possible to form a fairly accurate understanding of the processes employed by each student—whether they are more linear or recursive, more direct or mediated—and to appreciate differences among the participants.

In theory, a simpler general strategy might be distinguished from a more complex one. The former would consist of a linear process, with little mediation, in which the writing is not planned; there is no revision during the process of writing, and none at the end (or is confined to the use of a spell-checker). The complex strategy would be characterized by a recursive, mediated process in which the writing is planned and revised during the process and, at the end, checking what has been written by going back to the STs (or at least one of them). There would also be a range of patterns between these two strategies at varying distances from either pole.

An analysis of the sequences of actions performed by the 10 participants in tackling the task made it possible to identify three types of procedures or patterns:

Linear/reproductive pattern: sequential reading of the STs (ST1, ST2, ST3) with no, or very little, mediation (highlighting, copying ideas from the texts as notes). Following the reading (without any planning), there appears a rereading sequence, selecting/capturing ideas from the STs in the same order as they occur in those texts. These are isolated ideas that are copied or slightly paraphrased. The switching between the source and the author’s text is quick, at intervals of barely a few seconds. The student’s units of reference for producing one’s text are ideas in the text taken in isolation. The author neither reads nor revises the text while writing it. If any corrections are made, it is usually by running the spell-checker. This pattern contains no elaboration, deepening, or checking rereadings. In spite of the differences among them, five participants (Ahm, Mari, Mon, Sil, Xav) displayed a linear/reproductive pattern (the above-mentioned general “simple strategy”).

Linear/elaborative pattern: sequential reading of the STs (ST1, ST2, ST3) mediated by occasional deepening rereadings, highlighting, or notes. Following the reading, there is some kind of planning, and there appears a sequence of rereading involving the selection/elaboration of ST ideas, which are paraphrased and elaborated in the student’s text. The switching between STs and one’s own text takes place at somewhat longer intervals (more than 30 seconds at least), allowing the participant to elaborate the information; the units of reference for the production of the synthesis are paragraphs or ideas that can be linked to each other. During textualization, there are frequent rereadings of the text in production, which may be modified or corrected using the spell-checker. There may be a final revision and occasionally rereading for checking purposes. Three students (Alb, Gui, and Mar) displayed patterns similar to this one.

Linear/elaborative pattern with elements of recursivity: This pattern is a link in the chain between the pattern just described and what might be considered a proper recursive pattern. It involves a more mediated and recursive use of reading, with frequent and diverse types of rereadings during the performance of the task; the continuous writing sequence is still linear and dependent on the order in which ideas are presented in the STs. In this pattern, reading of a ST is always followed or accompanied by rereading for deepening and/or checking in which the sequence ST1–ST2–ST3 may be broken. After reading, there is some kind of planning, followed by a sequence of rereading involving the selection/elaboration of ST information and textualization, whose features are similar to those of the elaborative pattern, although in this case they may include rereading of any of the STs for checking purposes. There is a final revision of the text produced, which includes rereading one or more of the STs for the purposes of checking. Notwithstanding their differences, Bru and Elo employed an elaborative pattern with elements of recursivity.

In most cases, the processes ascribed to one or another of the defined patterns fit all their characteristics. The exception, once again, is Alb, whose sequence of actions included elements of all three patterns, although overall it was closest to the linear/elaborative model.

Taken as a whole, the analysis revealed that half the participants used a simple process that was nonrecursive (without elaboration, deepening, or checking rereadings) and unmediated. The other participants engaged in more elaborative processes during reading and rereading, but their writing was still extremely linear. In this connection, it is revealing that six participants did not plan what they were going to write at all; two did so generically (“a final summary of the three texts”), while the other two specified slightly more what they were going to do (“I’ll start by doing a short introduction about the State Spain had before the elections, and then I’ll begin writing about the national [anti-Republican] side who wanted to start making a coup, and then I’ll write”). Only the students whose pattern included elements of recursivity referred to the STs when revising their own text.

Last, the analysis of the time taken by the students to perform the task shows a high degree of variability, ranging from the 26 minutes 11 seconds taken by Gui to the 74 minutes 31 seconds taken by Sil (average: 51 minutes 31 seconds). A comparison between the type of pattern and the time taken to perform the task did not reveal any clear trend.

Processes: Integration

The results of this analysis made it possible to identify the number of inter- and intratextual integrations made by the participants, determine whether they were correct or incorrect, and establish whether they were present throughout the entire task performance sequence or confined to one or more actions. In Table 5, the participants are ordered according to their patterns; then, the total number of integrations made throughout the process (specifying those that were incorrect) and the actions (or type of actions) during which they occurred are indicated.

Table 5.

Integrations: Type, Frequencies, and Location

	Reads STs				Reads STs and Outlines				Rereads ST				Rereads ST + Copies				Rereads ST + Writes Text				Writes Text				Reads Text Produced
	INTRA		INTER		INTRA		INTER		INTRA		INTER		INTRA		INTER		INTRA		INTER		INTRA		INTER		INTRA		INTER
Pattern: Name	C	IN	C	IN	C	IN	C	IN	C	IN	C	IN	C	IN	C	IN	C	IN	C	IN	C	IN	C	IN	C	IN	C	IN
REP
Ahm																1
Mon	1	1														1
Xav															1
Sil	2	2															2		2	1
Mari		1	1														1	2	2	1
ELAB
Alb																	2	1	1
Gui	5								1										1								1
Mar	5	1	5	1							1	2					1				1
ELAB + REC
Elo	4		2	1							2						3	1	4
Bru					1		3		3	1	3						4		3

Note: ST = source text; INTRA = intratextual integration; INTER = intertextual integration; C = correct; IN = incorrect; REP = reproductive; ELAB = elaborative; ELAB + REC = elaborative + recursivity.

The data also suggest that in the more reproductive patterns, there are very few integrations and that they tend to be confined to occasional or specific actions. The cases of Sil and Mari are interesting in this regard: Although these students followed this pattern, they actually verbalized more integrations. The fact that most of the integrations were incorrect and occurred during reading or selection/capture rereading and the writing of the text may be interpreted as a consequence of the difficulties in understanding the texts experienced by them. Yet, the students who followed a more elaborative pattern produced more integrations during the process, especially when the pattern included elements of recursivity. In most cases, the integrations were correct.

Last, it is noteworthy that 69% of the intratextual integrations (31/45) and 58% of the intertextual integrations (24/41) occurred while the participants were reading or rereading the STs. The remainder (14 intratextual, 31%; 17 intertextual, 42%) occurred while they were rereading the STs and writing. Only one instance of intratextual integration occurred in a writing action. These results suggest the importance of deep, elaborated reading and hybrid sequences of elaborative reading and writing in fostering the establishment of connections among the items of information presented in the texts.

Comprehension Results

The day after they performed the synthesis task, the students answered a written comprehension test on the texts they had read. This test comprised open-ended and multiple-choice questions: five requiring the students to recover information (maximum: 6 points) and six requiring them to interpret and integrate information (maximum: 8 points). The reliability for the multiple-choice questions was .61. The correctness criteria for the open-ended questions were specified, and two researchers marked them independently, obtaining 78% overall agreement.

The mean for information recovery was 3.6 (SD = 1.77) and that for integration, 4.3 (SD = 1.88). The scores obtained by each participant are listed in Table 4, which also indicates whether their level of comprehension was above or below the average (± 1 standard deviation). These results, which must be interpreted with caution, make more sense when examined in conjunction with the processes employed and the texts written by the participants.

Processes, Products, and Comprehension Results

In an overview (see Table 6), a certain correspondence was found among the pattern, the texts produced, and the degree of comprehension. The results of the students employing a linear/reproductive pattern, which is associated with failed products, were between low average and below average on both the interpretation/integration questions and the recovery questions. Except for one case, the results of the students employing an elaborative pattern (without any elements of recursivity) who produced summaries or attempted syntheses were between average and above average on both comprehension measurements, the difference in regard to the students who employed a reproductive pattern being most pronounced on the interpretative measurement.

Table 6.

Processes, Products, and Comprehension Results

Name	Pattern	Integrations Along the Process	Integrations in the Product	Product	Comprehension Recovery	Comprehension Integration	Prior Knowledge^c
Ahm	Linear / reproductive	1 incorrect	—	Failed text	1^a	3	2
Mon	Linear / reproductive	2 correct, 1 incorrect	—	Failed text	4	2^a	0
Xav	Linear / reproductive	1 correct	—	Failed text	5	4	4
Mari	Linear / reproductive	6 correct, 3 incorrect	—	Failed text (serious errors)	1^a	2^a	6
Sil	Linear / reproductive	4 correct, 4 incorrect	—	Failed text (serious errors)	4	3	5
Alb	Linear / elaborative	3 correct, 1 incorrect	1	Attempted synthesis	3	4	3
Gui	Linear / elaborative	8 correct	3	Juxtaposed summaries	6^b	7^b	6
Mar	Linear / elaborative	13 correct, 4 incorrect	1	Juxtaposed summaries	5	5	6
Bru	Linear / elaborative / recursive	15 correct, 2 incorrect	2	Attempted synthesis	2	7^b	5
Elo	Linear / elaborative / recursive	18 correct, 1 incorrect	3	Attempted synthesis	5	6	5

Below the average.

Above the average.

Maximum, 6.

Our results concerning the role played by prior knowledge in influencing the students’ performance are not conclusive. It seems to have been important in the case of Gui, who had a high degree of prior knowledge and who scored above average on both comprehension measurements. The results of Mon and Ahm seem to point in the same direction—they both had very little prior knowledge and had poor comprehension scores. But Mari’s case is more surprising. With a prior knowledge score 6 of 6, she performed below average on recovery and integration. The results of Xav and especially Sil are even more remarkable—with good prior knowledge scores, their scores were average on recovery and below average on integration. In the cases analyzed, a high level of prior knowledge on its own does not appear to be associated with better performance on the comprehension tasks. The data suggest that a better performance is achieved when an acceptable level of prior knowledge is coupled with a more elaborative task performance pattern. Testing this hypothesis, which would help to gain a better understanding of the variables involved in comprehension and the relations among them, will require further empirical studies.

An interesting result is the difference between the number of integrations in the product and the integrations uttered during the production process. This discrepancy shows up the fact that many of the connections that the students make while reading and rereading to help them in their writing never find their way into the text that is actually written. A preliminary examination of a possible correspondence between the integrations made during the process (both correct and incorrect) and the answers given to the comprehension test questions referring to them also produced some interesting results: Although the integrated contents do not appear in the text written by the student, they turn up later in the answers to the test.

Discussion

The aim of this qualitative study was to gain a greater understanding of the processes underlying synthesis production. The analysis of the action sequence carried out by the 10 participants in the research and the products they generated has produced results in line with our expectations. The analysis of the texts produced by the students points up some important differences between them; although none of them can be considered a fully successful synthesis, the distance between a failed text and an attempted synthesis was considerable. As we also expected, it was possible to identify production processes of varying degrees of complexity as regarding both recursivity and mediation; the most complex processes were also those in which the participants made the largest number of integrations throughout the sequence. Our results suggest a general trend, according to which more complex processes are associated with texts closest to true syntheses, whereas failed texts are associated with the adoption of a simple, linear, direct process that is manifestly inadequate for tackling a demanding task such as this. The data also indicate a correspondence between the quality of the processes and the products, on one hand, and the level of understanding manifested on a specially drafted comprehension test, on the other. These results acquire a more explanatory dimension if we take a somewhat closer look at the dimensions considered in the analysis of the sequences of actions carried out by the students, particularly their more or less linear/recursive and direct/mediated character.

First, it is surprising that half the students—those who employed a linear/reproductive pattern—seemed to rely on a single reading of the texts as being sufficient to perform the task. Our results indicate that the more successful products are associated with a diversified use of rereading. Conversely, students producing less successful syntheses use rereading only to capture ideas from the STs, which are then reproduced almost literally in their own texts. This is in line with the findings of Cerdán and Vidal-Abarca (2008), Lenski and Johns (1997) and McGinley (1992).

Second, what happens with writing is equally, or more, relevant. Although we did find certain more recursive uses of rereading, even the students in question approached the matter of writing taking as their reference the sequence ST1–ST2–ST3, which predominated over any idea or theme structuring their text (which explains the absence of suitable syntheses). It is significant that the students started to write with hardly any planning. This simplified approach to textualization may help to explain why many of the integrations manifested by students employing more elaborative patterns while reading and rereading/writing did not appear in the final text. Our interpretation is that, faced with the difficulty of constructing their own text, the students borrow the organization and structure of the STs. This strategy drastically reduces the task’s epistemic potential.

Third, half of our participants adopted a linear revision pattern. These students used the spell checker as their sole means of revision, which limited their corrections to spelling and often gave rise to new difficulties in the text they had written. In the linear/elaborative pattern, revision was generally associated with reading the text in production, sometimes with the introduction of modifications other than to spelling (also to contents and structure). The students employing a linear/elaborative pattern with elements of recursivity were the only ones who did not lose sight of the STs (or at least one or two of them) when revising their text. In relation to revision and planning, our study corroborated the difficulties encountered by students in writing to transform knowledge (Bereiter & Scardamalia, 1987; Miras, 2000).

One aspect of this that deserves to be commented on concerns the fact that more than two thirds of the participants tended to conceive the task they were asked to perform as a summary of each text or their most important ideas: In other words, they did not approach the task with a view to bringing out the connections among the texts. It cannot be ruled out that this was due, in part, to the written instructions they were given, although other studies (Flower, 1990; Mateos & Solé, 2009) have suggested that the tendency to confuse composing a text based on more than one ST with summarizing the texts is fairly common. Also Penrose (1992), in a single-source studying or writing task, concluded that the cognitive operations students engaged in were determined more by the students’ interpretations of the tasks than by the tasks themselves. All these studies and our results suggest the need to pay more attention to students’ interpretations of tasks.

Another aspect to be commented on has to do with the complexity of the STs: Despite the fact that the teacher regarded them as suitable for the pupils’ level, it cannot be ruled out that they were actually more difficult than usual, especially ST3, on account of its argumentative structure.

Our study has other features that require discussion. There is controversy regarding the possible repercussions of use of the “think aloud” method on task performance. It is agreed that it causes participants to take longer over the task, but otherwise the findings are not conclusive (see, e.g., Crain-Thorenson, Lippman, & McClendon-Magnuson, 1997; Ericsson & Simon 1980, 1993; Fletcher, 1986) and suggest that it sometimes hinders task performance and other times enhances it.

In view of the qualitative nature of this study, its results must be considered with prudence and any temptation to extrapolate them avoided. Such precaution is even more important in regard to the results concerning the understanding achieved after performing the synthesis task (a judgement corroborated by Cerdán & Vidal-Abarca, 2008; Wiley & Voss, 1999). Although we used a test of acceptable reliability that is similar to the instruments commonly employed in this field of research, equating the degree of understanding with the answers to a number of questions is too restrictive an approach. The trend suggested by the results should be interpreted as just that—a trend.

However, if we take these aspects into account, our study has important implications for teaching. Our results confirm that being able to read and write is not enough to be able to successfully complete tasks, such as producing a synthesis, which require going beyond the text presented, analyzing it in depth, comparing and checking the information in it, and elaborating this information. These results suggest that certain processes are more suitable than others for connecting information from various texts and that they involve strategies that can be taught and learned. Although for reasons of space we cannot fully develop this point here, such strategies would include the following:

teaching students to understand that the task they are expected to perform consists not only in finding but also, and especially, drawing out the connections among certain items of information, depending on the goal being pursued;

guiding students’ reading to help them identify the fragments of information in the texts that are related to this goal and to one another;

helping students to understand the nature of the relationships among ideas—complementarity, opposition, causality, and so on;

teaching students to use writing to support previous activities and to make an outline or rough draft in which to integrate the contents identified;

teaching students to use reading and rereading to support the process of writing their own text;

teaching students to write beyond simply copying the STs by helping them to paraphrase and elaborate information; and

teaching students to revise their texts in the light of the goals set and the information in the STs.

It is particularly important to take into account the specific features of the educational situations in which such strategies can be taught and learned. As pointed out by Nelson (1990), these situations often lead students to focus on the product and look for the easiest ways to it, shying away from getting involved in complex thought processes (such as those underlying the integration and synthesis of information). Knowledge of these processes is a necessary, though not a sufficient, condition for proposing the appropriate means of teaching them.

Our research has also methodological implications. On that score, we were able to verify the double system of analysis (actions and verbalizations) already employed in other research, in particular the viability of the “hybrid” system of categories of verbalizations. A specific contribution of our procedure is the integration of both analyses—patterns and integrations—into a strategy aimed at explaining and understanding the dynamics of these processes and their particular concretion in a written text.

This study’s most important contribution to research on this issue is that it provides new information for understanding reading and writing processes and products involved in the performance of hybrid tasks. Studies in this area often analyze written products without considering the fact that they are the result, not only of writing, but also of reading and frequently omit to look at the processes involved. However, such an omission implies tacit acceptance that the product faithfully reflects the process. Our study of integrations shows that the distance between those that appear in the text produced and those verbalized during production may be quite large and questions the frequent conflation we have referred to while pointing up an interesting matter that deserves to be investigated. Our approach is also an alternative to the most frequently employed type of analysis, which fails to look at the written products globally as “text”; often what is done with these products is to identify in them one or more variables that are then correlated with a measure of comprehension/learning, but the text is not considered in its own right. This implicitly means accepting that writing and the overall quality of what is written have no bearing on understanding or learning, which is at least debatable and contradicts what we know about the epistemic nature of writing. To go in depth in the conditions that promote an epistemic use of reading and writing, in the processes involved and in its consequences for learning, constitutes an exciting challenge that definitely will mark priorities of research in the years ahead.

Footnotes

Appendix A: Sample Questions

Appendix B: Analysis and Results

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This research project was funded by the Spanish Ministry of Science and Innovation (Project EDU2009-14278-C02-02).

Notes

Bios

Isabel Solé is a professor of educational psychology at the University of Barcelona. Her work involves the study of epistemic reading and writing, mainly focusing on learning processes and results using multiple written texts in secondary and higher education. Her current research ranges a variety of themes: epistemological beliefs, reading and writing beliefs, written production and learning. She has published articles on the processes and mental operations involved in writing synthesis from multiples texts in secondary and higher education.

Mariana Miras is a professor of educational psychology at the University of Barcelona. Her initial studies were related to oral language development, sociolinguistic topics, and epistemic writing. Her current interests focus primarily on academic literacy, hybrid tasks, processes, and products, especially written synthesis and its impact on learning.

Núria Castells is an associated professor at the University of Barcelona. Her initial studies were related to early literacy learning and teaching. She is also interested in epistemological beliefs, reading and writing beliefs, hybrid tasks that involve reading and writing, and their impact on learning.

Sandra Espino is an assistant professor at the University of Barcelona. Her interests are related to understanding how the reading and writing processes are used as learning instruments in educational contexts. She is interested in studying the use of note taking and how the relation among the variables involved in its conditions of use are related with the students’ learning approach.

Marta Minguela is a doctoral student at the University of Barcelona. She is studying the metacognitive processes involved in reading comprehension in secondary education.

References

Bereiter

Scardamalia

(1987). The psychology of written composition. Hillsdale, NJ: Erlbaum.

Boscolo

Mason

(2003). Topic knowledge, text coherence, and interest: How they interact in learning from instructional texts. Journal of Experimental Education, 71, 126-148.

Cerdán

Vidal-Abarca

(2008). The effects of tasks on integrating information from multiple documents. Journal of Educational Psychology, 100(1), 209-222.

Cerdán

Vidal-Abarca

Martínez

Gilabert

Gil

(2009). Impact of question-answering tasks on search processes and reading comprehension. Learning and Instruction, 19, 13-27.

Coté

Goldman

S. R.

(1999). Building representations of informational text: Evidence from children’s think-aloud protocols. In van Oostendorp

Goldman

(Eds.), The construction of mental representations during reading (pp. 169-193). Mahwah, NJ: Erlbaum.

Coté

Goldman

Saul

E. U.

(1998). Students making sense of informational text: Relations between processing and representation. Discourse Processes, 25, 1–53.

Crain-Thorenson

Lippman

M. Z.

McClendon-Magnuson

(1997). Windows on comprehension: Reading comprehension processes as revealed by two think-aloud procedures. Journal of Educational Psychology, 89(4), 579-591.

Ericsson

K. A.

Simon

H. A.

(1980). Verbal reports as data. Psychological Review, 87(3), 215-251.

Ericsson

K. A.

Simon

H. A.

(1993). Protocol analysis: Verbal reports as data. Cambridge: MIT Press.

10.

Fletcher

C. R.

(1986). Strategies for the allocation of short-term memory during comprehension. Journal of Memory and Language, 25, 43-58.

11.

Flower

(1990). The role of task representation in reading-to-write. In Flower

Stein

Ackerman

Kantz

M. J.

McCormick

Peck

W. C.

(Eds.), Reading to write (pp. 35-75). New York: Oxford University Press.

12.

Flower

Hayes

(1981). Plans that guide the composing process. In Fredriksen

Dominic

(Eds.), Writing: The nature, development, and teaching of written communication (Vol. 2, pp. 39-58). Hillsdale, NJ: Erlbaum.

13.

Flower

Stein

Ackerman

Kantz

M. J.

McCormick

Peck

W. C.

(1990). Reading to write: Exploring a cognitive and social process. New York: Oxford University Press.

14.

Kozulin

(2000). Instrumentos psicológicos. Barcelona: Paidós.

15.

Lenski

S. D.

Johns

J. L.

(1997). Patterns of reading-to-write. Reading Research and Instruction, 37(1), 15-38.

16.

Mateos

Solé

(2009). Synthesising information from various texts: A study of procedures and products at different educational levels. European Journal of Psychology of Education, 24(4), 435-451.

17.

Mateos

Villalón

De Dios

M. J.

Martín

(2007). Reading-and-writing tasks on different university degree courses: What do the students say they do? Studies in Higher Education, 32, 489-510.

18.

McGinley

(1992). The role of reading and writing while composing from multiple sources. Reading Research Quarterly, 27, 227-248.

19.

McNamara

D. S.

Kintsch

Songer

N. B.

Kintsch

(1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14, 1-43.

20.

Miras

(2000). La escritura reflexiva: Aprender a escribir y aprender acerca de lo que se escribe. Infancia y Aprendizaje, 89, 65-80.

21.

Miras

Solé

Castells

(2008, June). The impact of writing on learning: The effect of students’ conceptions on their written products and learning results. Paper presented at the 11th International Conference of the EARLI Special Interest Group on Writing, Lund, Sweden.

22.

Nelson

(1990). This was an easy assignment: Examining how students interpret academic writing tasks. Research on Teaching in English, 24, 362-396.

23.

Nelson

(2008). The reading-writing nexus in discourse research. In Bazerman

(Ed.), Handbook of research on writing (pp. 435-450). New York: Erlbaum.

24.

Palincsar

A. S.

Magnusson

S. J.

(2001). The interplay of first-hand and text-based investigations to model and support the development of scientific knowledge and reasoning. In Carver

Klahr

(Eds.), Cognition and instruction: Twenty five years of progress (pp. 151-194). Mahwah, NJ: Erlbaum.

25.

Penrose

A. M.

(1992). To write or not to write: Effects of task and task interpretation on learning through writing. Written Communication, 9(4), 465-500.

26.

Pressley

Afflerbach

(1995). Verbal protocols of reading. Mahwah, NJ: Erlbaum.

27.

Rouet

J. F.

Vidal-Abarca

Bert-Erboul

Millogo

(2001). Effects of information search tasks on the comprehension of instructional text. Discourse Processes, 31(2), 163-186.

28.

Sánchez

(1998). Comprensión y redacción de textos. Barcelona: Edebé.

29.

Segev-Miller

(2004). Writing from sources: The effect of explicit instruction on college students’ processes and products. Educational Studies in Language and Literature, 4, 5-33.

30.

Solé

(2009, July). La estrategia de relectura en la elaboración de tareas híbridas de lectura y escritura para aprender. Paper presented at the 16th European Conference on Reading, Braga, Portugal.

31.

Solé

Miras

Gràcia

(2005). Interpersonal appropiation of learning instruments: Summarising to learn. Paper presented at the First ISCAR International Congress, Sevilla, Spain.

32.

Spivey

N. N.

(1990). Transforming texts: Constructive processes in reading and writing. Written Communication, 7(2), 256-287.

33.

Spivey

N. N.

(1997). Reading, writing and the making of meaning: The constructivist metaphor. San Diego, CA: Academic Press.

34.

Spivey

N. N.

King

J. R.

(1989). Readers as writers composing from sources. Reading Research Quarterly, 24, 7-26.

35.

Stahl

S. A.

Hynd

C. R.

Britton

B. K.

McNish

M. M.

Bosquet

(1996). What happens when students read multiple source documents in history? Reading Research Quarterly, 31, 430-456.

36.

Strømsø

H. I.

Bråten

(2002). Norwegian law students’ use of multiple sources while reading expository texts. Reading Research Quarterly, 37, 208-227.

37.

Tierney

R. J.

O’Flahavan

J. F.

McGinley

(1989). The effects of reading and writing upon thinking critically. Reading Research Quarterly, 24, 134-173.

38.

Tierney

R. J.

Shanahan

(1996). Research on the reading-writing relationship: Interactions, transactions, and outcomes. In Barr

Kamil

M. L.

Mosenthal

P. B.

Pearson

P. D.

(Eds.), Handbook of reading research (Vol. 2, pp. 246-280). New York: Longman.

39.

Tynjälä

(2001). Writing, learning and the development of expertise in higher education. In Tynjälä

Mason

Lonka

(Eds.), Writing as a learning tool: Integrating theory and practice (pp. 37-56). Dordrecht, Netherlands: Kluwer Academic Press.

40.

Wiley

Voss

J. F.

(1999). Constructing arguments from multiple sources: Tasks that promote understanding and not just memory for text. Journal of Educational Psychology, 91, 301-311.