Integration across linguistic backgrounds: examining intertextual connections in L1 and L2 academic writing / Integración entre orígenes lingüísticos: análisis de las conexiones intertextuales en la escritura académica L1 y L2

Theoretical background

This study is guided by Spivey’s (1984) model of discourse synthesis, referring to a form of academic writing wherein writers construct a personal understanding of an issue, described across multiple texts, by integrating information across text, and then reflect such personal understanding through a newly formed written product (Spivey, 1984; Spivey & King, 1989). Three main processes are involved in discourse synthesis — selection, organization and connection formation or integration — with each of these processes carried out during both reading and writing. During selection, learners choose information from sources to include in their mental models, or cognitive representations of texts. Learners then select ideas from these mental models to include in the written products that they compose. During organization, learners utilize cues from the source texts (e.g., headings, text structures) to structure or pattern their mental models, with such organizational features serving as a framework that students can use to cognitively represent information from multiple texts. Learners externalize the structure of their mental models through writing by using various discourse markers (e.g., first, further) and adhering to commonly used text structures (e.g., cause/effect, problem/solution). Finally, during connection formation or integration, learners generate links between relevant information within and across texts and represent these intratextual and intertextual connections in their mental models. Learners may also use discourse markers during writing (e.g., moreover, however; Dueñas, 2009; Martínez, 2004) to reflect connections between texts. In this study we focus on this last process, integration. We focus on integration both because this is the culminating cognitive process required for discourse synthesis and because multiple text integration has been found to be a particular challenge for both L1 and L2 writers (Doolan, 2021; List et al., 2019; Plakans & Gebril, 2017).

Multiple text integration

The term ‘integration’ refers to students’ linking or connecting of shared ideas introduced in different sections of the same text (i.e., intratextual integration) or across disparate texts (i.e., intertextual integration; Britt et al., 1999; Primor & Katzir, 2018). Integration is essential for students to develop cohesive mental models or understandings of issues described across texts (Britt et al., 1999; Perfetti et al., 1999). Additionally, both intratextual and intertextual integration have been identified as essential features of students’ successful writing based on multiple texts (List et al., 2019; Wineburg, 1991). In this study, we focused on multiple text integration referring specifically to the distinct process of connecting information from different texts.

Students’ integration has commonly been assessed through writing (e.g., Du & List, 2020; Latini et al., 2019; McCarthy et al., 2022). Three main approaches to assessing integration in writing have been introduced. First, at a holistic level, students’ written responses have been classified according to the extent to which these reflect the Documents Model Framework (DMF; Britt et al., 1999; Perfetti et al., 1999). In particular, the DMF classifies students’ mental representations of multiple texts to the extent to which these feature source use and intertextual linking. That is, students may create mush models, reflecting integrated ideas from multiple texts but failing to track sources of origin. Alternatively, students may fail to create connections across texts, separately representing each text’s content or constructing separate representations models of texts. Finally, students may create documents models, which reflect both the successful integration of content across texts and the tagging of information to sources of origin. The DMF describes the types of cognitive representations of texts that students form; empirical work has found these types of cognitive representations to also be reflected in students’ writing (Barzilai et al., 2021; List, 2019). For instance, List et al. (2019) classified the types of written responses that students composed based on six texts arguing for and against the legalization of sex work, in accordance with the DMF. In List et al.’s sample, 36% of the written responses exhibited a separate representation model of multiple texts; this included students formulating separate representation models with (24%) and without (13%) citations. Twenty-two per cent of written responses reflected mush models, and 20% of responses reflected documents models of multiple texts, with other responses not able to be coded.

A second approach to coding integration has been to analyse the specific types of connections that students formed when writing based on multiple texts (e.g., Cerdán & Vidal-Abarca, 2008; List, 2019; Olsen et al., 2018). For instance, List et al., (2019) examined the extent to which students included instances of explicit (i.e., using connective terms) and implicit (i.e., using a unifying statement) integration in their writing, finding implicit integration to be more common. Additionally, List et al. (2019) classified each instance of implicit or explicit integration that students generated as reflecting an evidentiary (i.e., focused on supporting details), thematic (i.e., focused on main ideas) or contextual (i.e., reflecting author- or source-level relations) connection. Across these levels of connection formation, evidentiary and thematic connections were most commonly formed. Here, rather than examining the specificity of students’ intertextual connections, we examine their structure. In particular, here, we examine students’ abilities to form comparisons and contrasts within and across texts.

As a third approach to scoring integration in students’ writing, lexical indicators have been analysed as evidence of connection formation (e.g., Du & List, 2020; Sonia et al., 2022). For example, Wiley and Voss (1999) compared integration across English L1 participants who were asked to complete a summary, explanation, argument or a narrative writing task based on texts presented as a single textbook chapter or as multiple documents. Wiley and Voss coded the number of connective terms used in students’ essays (e.g., at the same time, then), finding that students who were presented with multiple documents included more connectives, overall, and more causal connectives (e.g., because, due to) in their writing as compared to students reading a textbook chapter. In this study, we draw on this prior work to assess students’ connection formation within and across texts. We use holistic, categorical and lexical approaches to integration scoring.

Reading and writing in English as a second language

Studies examining students’ integration in writing have primarily focused on first language (L1) writers (i.e., Anmarkrud et al., 2014; Britt & Aglinskas, 2002; Cerdán & Vidal-Abarca, 2008; Wiley & Voss, 1999) and, to a more limited extent, on L2 writers (e.g., Cheong et al., 2019, Karimi & Mousavi-Moghadam, 2022; Plakans, 2009). When L2 students’ integration in writing has been examined, investigations have most commonly taken place within the context of high-stakes testing (e.g, Test of English as a Foreign Language, TOEFL; Uludag et al., 2019). For example, Plakans and Gebril (2017) examined discourse synthesis in a sample of L2 individuals, from 73 different countries speaking 47 different languages, completing a TOEFL writing assessment, with a specific focus on students’ organization and cohesion in writing. Organization was captured by examining the summarization patterns that students used in writing. Summaries were categorized according to whether these preferred one text over another (i.e., a written or an oral text) or summarized texts in a balanced fashion, and according to whether or not these included additions or commentary from the students writing. Importantly, students’ integration of the written and oral texts was not explicitly coded for.

Cohesion was assessed using the linguistic analysis tool Coh-Metrix to capture the number of connectives and logical operators (e.g., and, or, if) in students’ responses and the degree of overlap or ‘cohesion through repetition’ in students’ writing. In terms of organization, Plakans and Gebril (2017) found 60% of individuals’ responses to reflect balanced summaries of the two texts, while only 6% of responses included any additional commentary by writers. In terms of cohesion, students’ connectives use did not seem to differ across overall levels of writing performance (i.e., scored from 1–5). More recently, Karimi (2024) studied how L2 students’ written representations of controversial issues are influenced by their pre-existing beliefs, with a bias towards information that supports these beliefs. Although descriptively interesting, these results leave unanswered questions regarding L2 students’ integration in writing and the extent to which this may be fostered. In this study, we examine both L1 and L2 students’ intratextual and intertextual integration in writing, in response to texts organized in two different ways.

In particular, in this study, we present students with texts organized either by theme or by region, with content otherwise identical across texts, such that students have to form intratextual (i.e., in the theme condition) or intertextual (i.e., in the region condition) connections across texts when tasked with comparing college education in the US and Latin America. We do this to examine connection formation or integration across L1 and L2 writers. The role of text structure in fostering integration has been suggested in some prior work (Kurby et al., 2005). For example, Nash et al. (1993) presented participants with two passages about eagles and racoons. Passages were organized either by topic (i.e., eagle or racoon) or chronologically by life stage, and participants received either consistently (i.e., two texts by topic) or inconsistently (i.e., one by topic and one chronologically) organized texts. Nash et al. found that writers most frequently used the organizational structure of the first passage they read as a basis for organizing their essays. Moreover, students produced higher-quality essays when presented with texts following a consistent organizational structure. However, Nash et al. (1993) did not examine the extent to which students integrated information about eagles and racoons across the two texts. Here, we build on this prior work to determine whether varying texts, such that students have to form intratextual or intertextual connections within or across texts, influence integration in writing.

Present study

In this study, we adopt the design of Nash et al. (1993) to enable us to examine intratextual and intertextual integration across L1 and L2 English writers. English L1 and L2 college students were asked to write a research report comparing and contrasting college education in the United States and Latin America. Students were asked to do so based on two texts, organized either by region (i.e., the United States and Latin America) or by topic (e.g., cost of college education in the United States and Latin America; enrollment rates in the United States and Latin America). Texts were created such that those organized by region required students to form intertextual connections; in contrast, texts organized by topic only required intratextual integration for comparisons to be formed. The following research questions guide this study:

Given the design of the task, asking students to compare and contrast features of college education across two regions (i.e., Latin America and the United States), and the design of our texts, intratextual connections were always formed when students read topically structured texts; intertextual connections were always formed when students read texts structured by region. Thus, research questions only focus on students’ intratextual or intertextual connection formation, rather than comparing connection formation within regions but across topics.

Method

Participants

Participants were 141 college students (age: M = 22.09, SD = 3.81) recruited through Prolific, an online recruitment platform. Online data collection is valid (Casler et al., 2013) and offers benefits over in-person data collection (Follmer et al., 2017). Casler et al. (2013) found online samples to be more ethnically and socioeconomically diverse as compared to in-person samples. Moreover, Hauser and Schwarz (2016) demonstrated that online samples tended to submit more valid data than college students.

In the sample, 49.64% (n = 70) of participants identified Spanish as their first language and English as their second language, and 50.35% (n = 71) identified English as their first language. Participants in the Spanish group completed a four-question screener for Spanish language proficiency (M = 3.21, SD = .77). This screener was added to validate individuals’ self-reported language background. More descriptions of students’ language and educational backgrounds are in Table 1. The sample included 36.17% (n = 51) males, 48.93% (n = 69) females and 2.12% (n = 3) non-binary participants; 18 participants opted not to report their gender identity. Participants’ ethnicities included Latinx (44.68%), White (25.53%), Asian (6.38%) and Black (3.54%). Demographics do not include students who preferred not to report gender or race/ethnicity.

OPEN IN VIEWER

Table 1.

Examples of educational and language backgrounds of English L2 participants.

Spanish background	Example
Raised in the US, Spanish spoken at home	‘I come from a Peruvian household and my parents only talked Spanish with me, so it was my native language. Once I started school, I started getting familiar with English to the point where I am proficient now.’
Bilingual household in the US	‘Born in the US to bilingual parents — mother was always bilingual, father was Spanish only but taught himself English in college.’
Raised in the US and a Spanish-speaking area	‘I was born and raised in the United States, but from the age of 10–13, I lived in Puerto Rico, which is a territory of the US. There I went to school in Spanish and returned to the US after.’
Bilingual household and education in a Spanish-speaking area	‘I grew up in Costa Rica, but my parents started teaching me little bits and pieces of English from a young age. The school I went to from pre-K to second grade taught an English class for one period multiple times a week, but that was it. During second grade I was transferred to an American school in Costa Rica, where all classes except Spanish and social studies were taught in English, and a lot of the teachers at the school are American expats, so I learned English very quickly through daily exposure. [. . .]’

Measures

Prior knowledge

Participants answered 14 multiple-choice questions based on information in the two texts. Cronbach’s alpha reliability was .41 for the measure. The prior knowledge measure was researcher-created, based on key concepts and main ideas drawn from the study texts. Sample items are provided in Appendix 1. Due to the low reliability of this measure, scores are presented only descriptively (see Table 4).

Writing fluency

Writing fluency was assessed by asking participants to describe their favourite movie or TV show for two minutes. Specifically, participants were instructed to type as much as possible about their favourite TV show for two full minutes. This was done so that we could capture individuals’ writing fluency, or composition speed based on memory, with no reading materials provided. Participants generated between 17 and 75 words per minute.

English vocabulary

Ten pairs of analogies were taken from an old edition of the vocabulary section of the SAT. Cronbach’s alpha reliability was .52, and this increased to .56 when one item was deleted. Due to the low reliability of this measure, English vocabulary scores are only presented descriptively (see Table 4).

We compared participants’ scores on measures of prior knowledge, writing fluency and vocabulary across experimental conditions and language groups; however, no significant differences were found. These are, therefore, presented descriptively.

Multiple text task

The multiple texts task consisted of a reading phase and a writing phase.

Reading phase

Participants were randomly assigned to two different reading conditions (i.e., requiring intratextual or intertextual connection formation). Students in the intertextual condition were presented with two texts organized by region (i.e., one text on Latin America and the other text on the United States), such that comparing college education across these regions required students to form intertextual connections across texts. Students in the intratextual condition were presented with two texts organized by theme (e.g., graduation rates in Latin America and the United States, college cost in Latin America and the United States), such that comparing college education across these regions required students to form intratextual connections within texts. Texts were counterbalanced. Table 2 overviews topics in the texts.

OPEN IN VIEWER

Table 2.

Summary of reading materials: title, source, length and readability.

	Title	Topics	Source	Length	Readability (Flesch-Kincaid)
Intertextual condition
Cost and enrollment in the US	College education in the US	Cost in the US Debt in the US Enrollment in the US Dropout rates in the US	Dr Kyle Redmond, Professor of Higher Education, UCLA	492	15.5
Cost and enrollment in LA	Higher education in Latin America	Cost in LA Debt in LA Enrollment in LA Dropout rates in LA	Dr Jonathan Smith, Professor of Educational Policy, SUNY	481	20.6
Intratextual condition
Cost in LA and the US	College is unaffordable	Cost in the US Debt in the US Cost in LA Debt in LA	Dr Kyle Redmond, Professor of Higher Education, UCLA	481	17
Enrollment in LA and US	The college dropout crisis	Enrollment in the US Dropout rates in the US Enrollment in LA Dropout rates in LA	Dr Jonathan Smith, Professor of Educational Policy, SUNY	496	16.4

Note: The order of presentation of reading materials was counterbalanced.

All four texts were designed to be trustworthy, attributing information to reputable academic authors. Crucially, students received identical information across intertextual and intratextual text conditions, only organized in different ways. To capture students’ recall and abilities to translate text-based content from the mental models that they formed into writing, students did not have the study texts available during writing.

Participants were asked to rate three items after reading each text: (a) ‘How interesting was this text?’ (b) ‘How well did you understand this text?’ and (c) ‘How well did you comprehend the text?’ All items were rated on a seven-point scale, and self-reported comprehension scores were computed by averaging the latter two items.

Writing phase

Students were then asked to compose a report comparing and contrasting education in the US and Latin America. On average, the reports produced were 249 words in length (SD = 133.70). Coding of essays was informed by prior work (i.e., Du & List, 2020).

Comparison quantity

We determined the total number of comparisons and contrasts those students identified and the overall number of comparisons generated. Although students could have generated any number of relations across topics and texts, there were four main types of relations we expected students to draw: comparing and contrasting the cost, rates of student debt, rates of enrollment and dropout rates across the United States and Latin America.

Comparison quality

Second, each comparison generated was scored for quality using a four-point rating scale. A score of one was assigned to general statements of comparison (1). A score of two or three was assigned when specific evidence about one (i.e., 2) or both (i.e., 3) regions was included in the comparison. A score of four was assigned when comparisons were both specific and elaborated (4). Students’ scores for each comparison were totalled and divided by the total number of comparisons generated, to create an average comparison quality score. Please see Table 3 for sample comparisons. Cohen’s Kappa inter-rater reliability, for all responses scored, was .75 (exact agreement: 81%).

OPEN IN VIEWER

Table 3.

Examples of participants’ comparisons by type and quality level.

	Comparisons	Contrasts
Level 1: General comparison	‘Both Latin America and the United States have higher enrollment rates among people with financial means and higher dropout rates among the poorer populations.’	‘Latin America’s private colleges are cheaper than America’s public ones.’
Level 2: General comparison and specific evidence of one region	‘However, just like the United States, competitiveness rises and it makes it difficult to have a spot; according to the source, approximately 10% of the Latin students from low income backgrounds are chosen to attend.’	‘In Latin American universities, the tuition is cheaper than the United States, and tuition can vary from being free to paying “2,000 to over 15,000”.’
Level 3: General comparison and specific evidence of two regions	‘In the US, 40% of college students don’t graduate, compare to Latin America, where 50% of students drop out.’	‘In the United States, public and private universities are expensive, ranging from 10,000 a year in a public university to 40,000 a year in a private university. In contrast, Latin American universities only charge 200 a year for public and 2,000 a year for private, making them much more affordable.’
Level 4: General comparison, specific evidence of two regions, and source information provided	‘In Latin America, the economic disparity that comes to many, many students is a catalyst that leads to the region’s poor graduation rate, at 46% of all students (Organization for Economic Cooperation and Development). [. . .] Similarly, the United States had a graduation rate of only 42% for four-year programmes and 56% for six years (US Department of Education).’	‘One difference between colleges and universities in the United States and Latin America is the job security associated with college degrees. In Latin America, 40% of college graduates are unemployed (Smith). However, in the United States, only 2% of college graduates are unemployed.’

Note: Evidence supporting comparison is underlined and sources provided are bolded.

Connectives use

We use the Tool for the Automated Analysis of Cohesion (TAACO; Crossley et al., 2016) to calculate the number of connectives included in each response. We included all of the classes of connectives analysed by TAACO, including conjunctions (e.g., and), disjunctions (e.g., or), additions (e.g., further) and contrasts (e.g., on the contrary), among others (see Crossley et al., 2016 for a full list). We considered connectives use to reflect integration (List et al., 2019; Wiley & Voss, 1999), in addition to being an indicator of cohesion (Crossley et al., 2019).

Results

Quantity and quality of relations, connectives use and textual overlap across language groups and experimental conditions

We performed a two-way Multivariate Analysis of Variance (MANOVA) with reading condition (i.e., intertextual or intratextual) and first language (i.e., English or Spanish) to examine each of the four outcome measures of interest (i.e., number of comparisons generated, average relation quality, connectives use and text overlap). Normality and homogeneity assumptions were met. Descriptive information is included in Table 4.

OPEN IN VIEWER

Table 4.

Means and standard deviations of measures.

Measure	First language		Reading condition		Total
	EnglishM (SD)	SpanishM (SD)	Intertextual (i.e., by region)M (SD)	Intratextual (i.e., by topic)M (SD)
Prior knowledge	4.84 (2.12)	5.40 (1.69)	5.15 (1.81)	5.08 (2.06)	5.12 (1.93)
Vocabulary	7.82 (2.05)	7.58 (2.26)	7.58 (2.22)	7.82 (2.10)	7.70 (2.16)
Words per minute	44.70 (13.38)	41.47 (11.27)	41.98 (11.63)	44.30 (13.24)	43.09 (12.44)
Total relations	2.64 (1.40)	2.81 (1.59)	2.88 (1.54)	2.57 (1.45)	2.73 (1.49)
Average relation quality	2.62 (1.06)	2.80 (1.04)	2.59 (1.05)	2.84 (1.04)	2.72 (1.05)
Text overlap	.97 (.09)	.90 (.08)	.89 (.09)	.89 (.07)	.89 (.08)

No significant differences were identified. That is, neither students’ language status nor the organizational structure of texts, nor their interaction, were associated with differences in the number comparisons (ps > .32) and quality (ps > .18) of relations generated, with students’ connectives use (p > .17) nor overlap with texts (ps > .20). See Table 5 for correlations.

OPEN IN VIEWER

Table 5.

Correlation among measures.

Variable	1	2	3	4	5	6	7
1. Prior knowledge	—
2. Vocabulary	−.03	—
3. Writing fluency	−.03	.19*	—
4. Number of relations	−.04	.25*	.16	—
5. Average quality	.04	.09	.10	.11	—
6. Connective use	−.03	−.05	−.07	.03	−.18	—
7. Text overlap	.25**	.24*	.21*	.32**	.31*	−.22*	—

Note: *p < .05, **p < .01

Discussion

We sought to examine integration among L1 and L2 college-level English writers, composing written responses based on multiple texts. This study contributes to prior research in two primary ways. First, this study was designed to both look at students’ writing based on multiple sources of information and to compare writing between participants from two different language backgrounds (i.e., English and Spanish). Previous studies looking at multiple text integration and organization have mostly examined students from one language group (e.g., Gil et al., 2010; Plakans & Gebril, 2017). However, we did not find differences across language status in the variety of writing-related measures examined. This stands in contrast to our hypotheses and substantial prior work with elementary and secondary students finding differences in writing and reading performance across English L1 and L2 students (Hyland & Milton, 1997; Lee, 2003; Lesaux et al., 2010).

Second, we examined how organizational structure can play a role in writing, when students were asked to compose a response based on multiple texts. We did not find differences in writing performance across reading conditions. We discuss results first, in terms of (the lack of) findings related to language status and then in terms of texts’ organizational structure.

Limitations

We must acknowledge a number of limitations. First, the prior knowledge measured had low reliability, suggesting that our questions did not fully capture the extent of students’ familiarity with the topic of college education in the US and Latin America. However, recent studies have shown low reliability to be a common problem for prior knowledge measures (Kulikowich et al., 2023); thus, prior knowledge needs to be assessed multidimensionally in future work (McCarthy et al., 2023). Likewise, the reliability of our vocabulary measure was low, likely due to the limited number of items included. Thus, the reliable assessment of individual difference factors constitutes an important direction for future work. Second, students were not able to return to the two texts provided after completing the reading phase of the study, and were not required to take notes. This was because the writing task was intended to capture students’ translations of their mental models of multiple texts, formed during reading, into writing. Nevertheless, in more naturalistic settings, students often have texts available as they write and are able to take notes. Note-taking has been shown to promote different cognitive processing when learning from texts (Gil et al., 2008). Two additional limitations may be drawn in relation to the multiple text synthesis writing task used in this study. First, it is possible that the reading materials used were too easy for our university sample, such that students were able to draw intratextual and intertextual connections, regardless of language status or text structure condition. Evidence for this comes from students’ average relation scores, which were 2.73 out of a possible 4. Moreover, while we assumed that university students would be familiar with the comparison and contrast genre, students’ task conceptions were not assessed in this study. Likewise, it is possible that differences in intratextual and intertextual connection formation would have emerged had students been asked to complete a more challenging writing task, such as composing an argument which involves greater difficulty when selecting information (González-Lamas et al., 2016).

Conclusion

This study was conducted at the intersection of research on students’ synthesis writing based on multiple texts and efforts to examine the experiences of L1 and L2 writers. In particular, this study explored three indicators of integration in students’ writing (i.e., the types of document models reflected, the types of connections formed and the lexical markers used) to identify discernable patterns across L1 and L2 English writers. No differences were identified, suggesting either that, at the college level, students’ L1 and L2 English writing is highly similar, contrary to what has been found in work with younger samples, or that additional markers of higher-level writing should be investigated to identify discernable patters in these two samples. Exploratory analyses did find that language background interacted with texts’ organizational structure in impacting students’ interest and comprehensibility ratings of texts. In particular, L2 speakers rated texts focused exclusively on Latin America (i.e., organized by region) to be more interesting and easier to comprehend than texts organized by topic. Although preliminary, this is a potentially important finding, linking students’ language or cultural backgrounds to their interactions with texts during synthesis writing. We are hopeful that this finding, as well as additional higher-level writing processes, will be explored in future work.