Assessing L2 secondary student writing feedback literacy and its predictive effect on their L2 writing performance

Abstract

While student feedback literacy has garnered increasing attention in second language education, there is a paucity of research on the relationship between second language (L2) student writing feedback literacy and L2 student writing performance, especially in secondary school contexts. Based on two independent samples of 600 (54.3% female) and 727 (46.2% female) secondary students, the present study validated the L2-Student Writing Feedback Literacy Scale (L2-SWFLS) for secondary students in the Chinese context of English as a foreign language (EFL) and probed into the association between L2 secondary student writing feedback literacy and L2 writing performance. Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were carried out to examine the factorial structure of the L2-SWFLS for secondary students. Structural equation modeling (SEM) was performed to examine the relationship between L2 secondary student writing feedback literacy and L2 writing performance. The findings revealed that the L2 secondary student writing feedback literacy comprised two factors, i.e. Using Feedback (10 items) and Evaluating Feedback (4 items), which had acceptable reliability. However, the L2 secondary students’ writing feedback literacy was not associated with their writing performance, which might be due to the existence of some mediators between the two and the students’ limited level of writing feedback literacy. This study advances the understanding of L2 student writing feedback literacy and provides notable insights for L2 secondary teachers to foster students’ capabilities of using feedback and evaluating feedback.

Keywords

feedback literacy L2 writing instruction L2 writing performance secondary students validation

I Introduction

Although feedback serves as a vital instrument for learning and teaching in second language education, the impact of feedback on the literacy development of second language (L2) students is highly variable due to the complexity of maximizing the learning potential of feedback (Hattie, 2009; Hyland & Hyland, 2019). With a shift in higher education from a transmissive model of teacher-centered feedback to a learner-oriented feedback conceptualization (Carless & Boud, 2018; Hyland & Hyland, 2019; Lee, 2017; Yu & Liu, 2021), L2 writing researchers and practitioners have paid increasing attention to L2 learners’ knowledge, beliefs, practices, abilities, and skills regarding how to appreciate, evaluate, and use L2 writing feedback and manage their emotions in this process, i.e. student feedback literacy in L2 writing (Yu, Zhang, & Liu, 2022).

The small body of research on L2 student feedback literacy tends to focus on conceptualizing student feedback literacy in the L2 writing context (Carless & Boud, 2018; Molloy et al., 2020; Yu & Liu, 2021; Zhan, 2022, 2023) or taking a case study approach to examine student engagement with feedback from multiple sources in university settings (Han & Xu, 2019, 2021; Yu & Liu, 2021). Although Yu, Zhang, and Liu (2022) developed a scale (i.e. the L2-SWFLS) to measure L2 university student writing feedback literacy and validated it in Chinese L2 university student samples, the psychometric properties of the scale might vary across contexts and participants, underscoring the need to take caution in generalizing results to diverse populations, such as L2 secondary students. In addition, while prior studies have pointed out that students of differing L2 writing proficiency exhibit different features in feedback practices or feedback literacy (e.g. Bai & Wang, 2021; Baker & Boonkit, 2004; Chien, 2012; Han & Xu, 2019; Zhou et al., 2022), the specific relationship between writing feedback literacy and writing performance remains unclear. To fill the void, the present study aims to validate the L2-SWFLS for secondary students and then explore how the L2 secondary students’ writing feedback literacy predicts their writing performance. This study contributes to the understanding of L2 student writing feedback literacy in school contexts and could provide useful insights for L2 secondary teachers in developing students’ abilities to use and evaluate feedback.

II Literature review

1 L2 student writing feedback literacy

In higher education, feedback literacy is defined as ‘the ability to read, interpret and use written feedback’ (Sutton, 2012, p. 31). Building on this definition, student feedback literacy is defined as ‘the understandings, capacities and dispositions needed to make sense of information and use it to enhance work or learning strategies’ (Carless & Boud, 2018, p. 1316), which is pivotal to maximizing the learning potential of feedback processes in L2 education (Carless, 2019; Chong, 2021; Han & Xu, 2021; Yu, Zhang, & Liu, 2022; Zhou et al., 2022). Carless and Boud (2018) postulated that student feedback literacy encompasses four components: (1) appreciating feedback; (2) making judgments; (3) managing affect; and (4) taking action. Building on this model, Molloy et al. (2020) and Zhan (2022, 2023) conceptualized student feedback literacy from a learner-centered perspective and an ecological perspective, respectively.

Specifically, ‘appreciating feedback’ refers to students’ acknowledgment of the value of feedback and their active role in feedback processes. As Tian and Zhou (2020) reported, there was a positive association between students’ perceptions and the uptake of feedback. ‘Making judgments’ represents students’ evaluation and decision-making on the quality of works written by themselves or peers. It involves students’ capacity for decoding and providing writing feedback (Lundstrom & Baker, 2009; Yu & Lee, 2016). Yu and Liu (2021) and Ghaffar et al. (2020) recognized the importance of student use of assessment criteria in making feedback judgments. ‘Managing affect’ concerns students’ management of attitudes, feelings, or emotions in feedback processes. As Mahfoodh (2017) stated, L2 writers’ emotional responses could influence their uptake of feedback. Emotional resilience in feedback processes is indispensable for students to make the most of feedback despite emotional fluctuations (To, 2016). Finally, ‘taking action’ involves students’ attempts to store, organize, generalize, and uptake the feedback information to improve their writing and modify their learning goals (Boud & Molloy, 2013; Hattie & Timperley, 2007; Yu, 2019). These four important dimensions capture the construct of student feedback literacy and guide the conceptualization of student feedback literacy in L2 writing.

In L2 writing, Yu and Liu (2021) presented an evidence-based framework to address ways of improving students’ feedback literacy in academic writing. Their framework consists of knowledge repertoire, individual dispositions, and capabilities, and highlights the interaction between feedback providers, feedback receivers, and the technical environment. Emphasizing the incremental nature of feedback, Han and Xu (2019) construed student feedback literacy as cognitive readiness and socio-affective readiness, and Zhou et al. (2022) depicted student feedback literacy as students’ cognitive, affective, and behavioral preparedness emerging from feedback experiences. Cognitive preparedness echoes appreciating feedback and making judgments; affective preparedness resonates with managing affect, and behavioral preparedness aligns with taking action.

Extending the process-oriented conceptualization of feedback literacy, Yu, Zhang, and Liu (2022) went further to define it as ‘students’ knowledge, beliefs, practices, abilities, and skills regarding how to appreciate, evaluate, and use L2 writing feedback and manage their emotions in this process’ (p. 3). Based on the L2 student self-report data in their study, they refined the model of Carless and Boud (2018) in L2 writing by adding ‘acknowledging different feedback sources’, which pertains to students’ ability to identify multi-sources of feedback (e.g. teachers, peers, self, and automated systems) and the respective value of different feedback sources in L2 writing. Specifically, their study showed that teacher feedback was associated with students’ high uptake of feedback (e.g. Cho & Sohn, 2007; Ruegg, 2015; Tsui & Ng, 2000; Zhao, 2010), and peer feedback focused on high-order issues (Yu, 2019) and was comprehensible to L2 learners (Zhao, 2010). They also found that self-feedback enhanced students’ agency in learning writing (Lam, 2013), and automated feedback offered students extended language points to guide students’ learning beyond the current writing tasks (Z. Zhang, 2020).

Of note is that earlier studies of student feedback literacy were unanimously conducted in the higher education context, and few have focused on students’ writing feedback literacy in the secondary school context. Therefore, more work needs to be done to explore L2 secondary student writing feedback literacy. Based on these earlier studies, the L2 student writing feedback literacy can be defined as a five-dimension construct, i.e. (1) appreciating feedback; (2) making judgments; (3) managing affect; and (4) taking action; (5) acknowledging different feedback sources (see Table 1), which informs the validation of the L2-SWFLS for secondary students in this study.

Table 1.

The conceptualization of L2 student writing feedback literacy.

Dimensions	Descriptions
Appreciating feedback	Students’ acknowledgment of the value of feedback and their active role in feedback processes.
Making judgments	Students’ evaluation and decision making on the quality of works written by themselves or peers.
Managing affect	Students’ management of attitudes, feelings, or emotions in feedback processes.
Taking action	Students’ attempts to store, organize, generalize, and uptake the feedback information to improve their writing and modify their learning goals.
Acknowledging different feedback sources	Students’ ability to identify multi-sources of feedback (e.g. teachers, peers, self, and automated systems) and the respective value of different feedback sources.

2 L2 secondary student writing feedback experiences

Student writing feedback practices and literacy are likely to differ due to the varied contexts of L2 secondary and university students and, as a result, the instruments such as L2-SWFLS that were developed based on L2 university students’ data could not be directly generalized to L2 secondary students. When compared to L2 writing instruction and learning in university settings, L2 secondary writing instruction and learning demonstrated distinct characteristics: it was found to be geared towards preparing students for public examinations (Davison, 2007; Hamp-Lyons, 2007; Tsui & Ng, 2000), with the product-oriented approach being dominant (Geng et al., 2022; Lo & Hyland, 2007; Ortega, 2009; Yu, Jiang, et al., 2022). In many L2 contexts such as Hong Kong and mainland China, teachers relied heavily on textbook materials in the teaching of writing, which were disconnected from students’ interests and experiences (Lee et al., 2018). Many teachers viewed themselves as language teachers rather than writing teachers (Ferris, 2003; Zamel, 1985), and thereby they focused predominantly on the language form with accuracy as the sole evaluation criterion, gave feedback to single drafts due to time constraints (Curtis, 2001; Lee et al., 2018), emphasized error- and score-focused summative feedback (Lee & Coniam, 2013; Lee et al., 2018), rarely conducted self-/peer-evaluation and conferencing activities (Ketonen et al., 2020; Lee, 2008; Tsivitanidou et al., 2011), and exposed students to the pre-given three-paragraph essays (Yu, Jiang, et al., 2022).

While students appreciated direct teacher feedback, they wanted teachers not only to pinpoint errors but also to provide corrections and explicate error types. They were rendered passive recipients of feedback and became reliant on the teachers, who were regarded as the sole authority in the feedback processes, relinquishing power to teachers rather than taking the initiative to seek and provide feedback (Belcher & Liu, 2004; Lee, 2004, 2008). They also illustrated that they preferred reading the teachers’ comments to revising their own drafts, and they scarcely made judgments on teacher feedback and tended to copy the answers as corrections without independent and critical thinking (Ketonen et al., 2020; Lee, 2008; O’Donovan, 2017). They tended to have reservations about peer feedback due to their doubts about the accuracy of peer comments, and had an inadequate understanding of assessment criteria or drew on their own criteria in providing peer feedback, expecting teachers to explicitly explain the writing evaluation criteria with examples or assessment rubrics (Ketonen et al., 2020; Lee, 2007; Tsivitanidou et al., 2011). In brief, there is limited information about secondary students’ feedback practices and literacy in L2 writing, though their feedback experiences are unique.

3 Student writing feedback literacy and L2 writing performance

A couple of previous studies have revealed that students of differing L2 writing proficiency displayed different features in their feedback processes (e.g. Han & Xu, 2019; Zhou et al., 2022). Research has also documented that low- and high-proficiency writers differed in their writing processes: the low-proficiency writers employed more unproductive strategies, such as ignoring feedback, and fewer used the ‘revising’ strategy than their high-proficiency counterparts, who made revisions and acted on feedback throughout the entire writing process (Bai & Wang, 2021; Baker & Boonkit, 2004; Chien, 2012).

As regards feedback literacy and L2 proficiency, Han and Xu’s (2019) case study shows that students of differing L2 proficiency showed different characteristics in their development of feedback literacy. In their study, the low-proficiency student had a limited development of feedback literacy as she struggled to comprehend English texts and teachers’ instructions and feedback. The middle-proficiency student’s cognitive readiness grew fast because he firmly believed in the teacher’s authority and expertise and actively engaged in feedback processes. His social-affective readiness grew moderately as he became more confident in providing feedback and showed a positive attitude towards peer feedback. However, his strong belief in teacher feedback might impede his development in making judgments. As for the high-proficiency student, his cognitive and socio-affective readiness both developed: he developed a more positive view of peer feedback and improved his feedback based on teachers’ suggestions. He also fostered empathy by taking into consideration his peers’ feelings when providing and receiving feedback.

Focusing on the L2 writing context, Zhou et al. (2022) adopted a case study approach to probe into the role of writing proficiency in students’ feedback literacy development and revealed that both low- and high-proficiency L2 writers were cognitively prepared to appreciate feedback and emotionally ready to manage their affect in processing feedback. However, the low-proficiency L2 writers were found to be less ready to have learning gains from their feedback practices due to their passive perspective of learner roles than their high-proficiency counterparts, who took on proactive learner roles in feedback processes. This line of studies has indicated the possible relationship between L2 writing proficiency and student feedback literacy, yet they were qualitative in nature and did not measure these two constructs and then analyse their relationship.

To sum up, while current research has focused on disentangling the theoretical complexity of student feedback literacy in the higher education context, little has been done to examine L2 students’ writing feedback literacy in the secondary school context. Yu, Zhang, and Liu’s (2022) L2-SWFLS based on an L2 university student sample cannot be directly generalized to L2 secondary students due to the significant differences between the two contexts. Moreover, although some earlier qualitative research findings have indicated the association between L2 writing feedback literacy and L2 writing performance, there is a paucity of quantitative research on such relationship. To fill such a void, this study aims to address the following two research questions:

• Research question 1: What is the internal factor structure of the L2-SWFLS among Chinese secondary students?

• Research question 2: What is the association between Chinese secondary students’ L2 writing feedback literacy and writing performance?

III Method

1 Context and participants

Learning to write in L2 is one of the most challenging endeavors in L2 learning (Casanave, 2004; Hyland, 2003; Kroll, 1990). In mainland China, writing in L2 has drawn much attention from teachers and researchers, as English writing is a crucial component in the curriculum (e.g. Jiang et al., 2023; Lei et al., 2023; Xu, 2022). In Chinese secondary settings, teachers and students are under high pressure to prepare for high-stakes examinations in which writing is an important task, and thus L2 writing instruction is exam-oriented and language form focused (Yu, Jiang, et al., 2022).

Convenience sampling was adopted. We contacted secondary school principals and teachers from 31 provinces and municipalities in mainland China and asked them to help distribute the online surveys in Wenjuanxing (https://www.wjx.cn) to their students through WeChat groups. A conservative approach was adopted in cleaning the data, removing any incomplete samples in which participants missed some items or samples with a response time of less than 60 seconds. Finally, we obtained 1,327 samples in total. At the scale development stage, Sample 1 comprised 600 Chinese secondary students. At the scale validation stage, Sample 2 comprised 727 secondary students. The demographic characteristics of the two samples are presented in Table 2.

Table 2.

Demographic characteristics of Sample 1 and Sample 2.

	Sample 1 (n = 600)		Sample 2 (n = 727)
	n	Percentage	n	Percentage
Provinces and municipalities:
Anhui	23	3.8	36	5.0
Beijing	3	0.5	8	1.1
Chongqing	166	27.7	140	19.3
Fujian	14	2.3	16	2.2
Gansu	8	1.3	23	3.2
Guangdong	19	3.2	26	3.6
Guangxi	19	3.2	22	3.0
Guizhou	30	5.0	27	3.7
Hainan	3	0.5	5	0.7
Hebei	19	3.2	21	2.9
Henan	29	4.8	37	5.1
Heilongjiang	7	1.2	6	0.8
Hubei	16	2.7	27	3.7
Hunan	22	3.7	32	4.4
Jilin	5	0.8	10	1.4
Jiangsu	10	1.7	21	2.9
Jiangxi	12	2.0	21	2.9
Liaoning	17	2.8	10	1.4
Inner Mongolia	5	0.8	6	0.8
Ningxia	5	0.8	3	0.4
Qinghai	7	1.2	4	0.6
Shandong	28	4.7	17	2.3
Shanxi	17	2.8	16	2.2
Shaanxi	15	2.5	21	2.8
Shanghai	3	0.5	5	0.7
Sichuan	54	9.0	93	12.8
Tianjin	4	0.7	10	1.4
Tibet	2	0.3	1	0.1
Xinjiang	13	2.2	16	2.2
Yunnan	14	2.3	22	3.0
Zhejiang	11	1.8	25	3.4
Gender:
Male	274	45.7	391	53.8
Female	326	54.3	336	46.2
Age (years):
15–18	413	68.8	514	70.7
18–23	187	31.2	213	29.3

2 Instrument

a L2 secondary student writing feedback literacy

L2 secondary student writing feedback literacy was assessed using the 28-item L2-SWFLS for university students (Yu, Zhang, & Liu, 2022). This scale included five dimensions, including:

• appreciating feedback, e.g. ‘I appreciate the role of feedback in continually improving work’;

• making judgments, e.g. ‘I think that writing feedback can build my capacity to develop evaluative judgment’;

• acknowledging different feedback sources, e.g. ‘I recognize that feedback from multiple sources – e.g. teachers, students, school administers – have different perspectives and attitudes’;

• managing affect, e.g. ‘I can manage the emotional challenges of receiving and sifting information which may be unwelcome’; and

• taking action, e.g. ‘I can use feedback to adjust the setting of new learning goals in English writing.’

The participants were asked to self-report their agreement with the statements on a five-point Likert scale from 1 = strongly disagree to 5 = strongly agree.

b L2 writing test

An L2 writing test was conducted to document students’ writing performance. The task was adapted from IELTS writing tests, and the topic was ‘change or not’. The independent writing task in IELTS is a common writing task with topics coming from students’ daily lives, that are suitable and appropriate for the participants. Extensive research has documented that the IELTS writing task is a valid instrument to measure students’ writing proficiency (Mayor et al., 2007; Moore & Morton, 1999; Nguyen et al., 2022; Révész et al., 2017). They were given 30 minutes to write about 120 to 150 words. The evaluation criteria included vocabulary, sentences, structure and organization, and content relevance. All participants (n = 1,372) were asked to write an essay to respond to the writing task. A total of 150 essays were rated by both human raters and Pigai, an automated writing evaluation system launched by Beijing Cikuu Science and Technology Co Ltd in 2011. Its scoring model is calibrated with a large corpus of standard English, English textbook materials, and students’ English essays. Pigai generates a holistic score for an essay (the total score is 100 points) based on four criteria, i.e. vocabulary, sentences, structure and organization, and content relevance. The two human raters were trained by the second author based on the four evaluation criteria, and they were asked to provide a holistic score (the total score is 100 points) on all 150 essays. The inter-rater reliability was .745. Afterward, the two human raters’ scores were averaged. The correlation coefficient between human and automated scoring was .82, indicating the acceptable reliability of the automated scoring. Therefore, the rest of the student essays were independently rated by Pigai.

3 Procedures

Students and their guardians were briefed on the aim and content of this research and provided their consent before the research commenced. With consent provided, they were asked to fill out the Wenjuanxing survey that included the L2-SWFLS and demographic information, i.e. gender and age. They were informed that they had the freedom to withdraw from the research whenever they wanted.

4 Data analytic approach

Students’ self-reported scores on the items in acknowledging different feedback sources and items relating to providing feedback were extremely low (the mean was around 2 out of 5). Based on informal communication with high school students and teachers, teacher feedback tends to be the only feedback source for most high schools in mainland China. Therefore, we eliminated the seven items relating to different feedback sources and providing feedback, such as the item. ‘I recognize that feedback from multiple sources – e.g. teachers, students, school administers – has different perspectives and attitudes’ and the item. ‘I recognize that feedback from multiple sources – e.g. teachers, peers, computers – provides different opportunities for learning’.

Exploratory factor analysis (EFA) was performed with Sample 1 (n = 600) via SPSS 21.0 to examine the dimensionality of the items in the scale. The assumptions for factor analysis were checked, such as Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy and Bartlett’s test of sphericity. Confirmatory factor analysis was conducted with Sample 2 (n = 727) using Mplus 8.4 to examine the psychometric properties of the L2-SWFLS for secondary students. Structural equation modeling (SEM) analysis was performed with Mplus 8.4 to model the relationship between L2 secondary student writing feedback literacy and L2 writing performance by controlling for the demographic variables, i.e. gender and age. Multiple indices were used in evaluating the model fit, including Chi-square difference tests (χ²), Comparative Fit Index (CFI), Tucker–Lewis Index (TLI), Root Mean Square Error of Approximation (RMSEA), and Standardized Root Mean Square Residual (SRMR) (Anderson & Gerbing, 1984; Bentler, 1990; Byrne, 2010). The Chi-square difference tests refer to differences between the observed covariance matrix and the predicted covariance matrix. Values approaching zero indicate a good fit. The CFI compares the amount of departure from close fit for the researcher’s model against that of the null model. Values that are larger than .95 indicate a good fit. RMSEA represents an index of the difference between the observed covariance matrix per degree of freedom and the hypothesized covariance matrix, which denotes the model. RMSEA index smaller than .06 is recommended (Hu & Bentler, 1995). SRMR pertains to the differences between corresponding elements of the observed and predicted covariance matrix, with a value smaller than .08 indicating a good fit.

IV Results

1 Exploratory factor analysis (EFA) results

The assumptions of the factor analysis were checked. The KMO measure of sampling adequacy was .965, close to 1, suggesting that factor analysis can yield reliable and distinct factors. The value is meritorious based on Hutcheson and Sofroniou’s (1999) criterion. Bartlett’s test of sphericity was significant (χ² = 11334.099, df = 210, p < .001), indicating that the correlations between variables are significantly different from zero. To extract the most appropriate number of factors, we followed Kaiser’s (1960) criterion of using eigenvalues exceeding 1. All loadings less than 0.5 were suppressed (Field, 2009). EFA with maximum likelihood estimation and varimax rotation was run with SPSS Statistics 21.0.

The first run of EFA suggested a two-factor solution. Items were explored that had low factor loadings and cross-loadings on more than one factor. Items with loadings less than 0.5 were removed. Interpretability was the criterion to decide upon which factor(s) cross-loading items cluster. The outcome of this item analytic process was the removal of seven items, finally leaving a collection of 14 items. Every removed item was double-checked, and it turned out that all removals had clear justification due to a lack of interpretability. For example, Item 1 ‘I think it important to analyse information in appropriate forms for the purposes of acting on it subsequently’ loaded on Using Feedback, but was more concerned with Evaluating Feedback, and thereby it was dropped. Item 11 ‘I recognize that effective learners are active in identifying their own learning needs’ and Item 12 ‘I am willing to communicate to others my learning needs of English writing’ both relate more to evaluating their own English writing learning needs, and are not relevant to Using or Evaluating Feedback. Therefore, they were eliminated as well.

The final 14 items loaded on two factors, which explained 67.725% of the total variance. Based on the items, they were named Using Feedback and Evaluating Feedback. Using Feedback was the most powerful factor in explaining the variance of student feedback literacy (60.460%). Evaluating Feedback explained another 7.265% of the total variance. The factor loadings are shown in Table 3.

Table 3.

Factor loadings and variances of the 14-item L2-Student Writing Feedback Literacy Scale (L2-SWFLS).

Items	Using feedback	Evaluating feedback
3. I think it important to record information in appropriate forms for the purposes of acting on it subsequently.	.791
6. I can demonstrate openness to receiving comments from others without displaying defensiveness.	.856
14. I can maintain emotional equilibrium and avoid defensiveness when receiving critical feedback.	.839
15. I can manage the emotional challenges of receiving and sifting information which may be unwelcome.	.927
16. I can honestly exchange meaningful information with others about my English compositions.	.814
17. I can respond to feedback information from others through goal-setting and planning how it might be utilized in future work.	.754
18. I can draw inferences from a range of English writing feedback experiences for the purpose of continuous improvement.	.882
19. I can use feedback to adjust the setting of new learning goals in English writing.	.868
20. I can use feedback to promote my English writing learning.	.822
21. I can develop a repertoire of strategies for acting on feedback.	.862
4. I think that communicating with feedback givers can improve my understanding of English writing standards and criteria.		.770
5. I can seek out exemplars as a way to make sense of standards of English writing.		.715
9. I think that standards and criteria are needed in evaluating my English composition.		.545
13. I think that writing feedback can build my capacity to develop evaluative judgment.		.770

Cronbach’s alphas were calculated, which were .923 for the Using Feedback sub-scale and .841 for the Evaluating Feedback sub-scale, respectively. The composite reliability was evaluated as well, and the values were .948 for Using Feedback and .843 for Evaluating Feedback, respectively. Hence, the reliability of the two factors was acceptable.

2 Confirmatory factor analysis (CFA) results

CFA was conducted to first test the model with two first-order factors, that is, Model 1. The results showed that Model 1 had a good fit to the data (χ² = 396.367, df = 74, TLI = 0.954, CFI = 0.963, RMSEA = 0.077 with 90% CI [0.070, 0.085], SRMR = 0.030). To inspect whether there is a second-order factor accounting for the two first-order factors, Model 2 with two first-order factors underpinning a second-order factor was tested. The results showed that Model 2 also had a good fit to the data (χ² = 396.367, df = 73, TLI = 0.954, CFI = 0.963, RMSEA = 0.078 with 90% CI [0.071, 0.086], SRMR = 0.030). The Chi-square difference tests suggested that Model 1 was not significantly better than Model 2 (Δχ² = 0, Δdf = 1, p > 0.05). We also checked CFI and RMSEA values and found that Model 1 and Model 2 did not have significant differences, as the differences were no more than 0.01 (Cheung & Rensvold, 2002). Considering parsimony, Model 1 was more favorable than Model 2. Model 3, the unidimensional model with 14 items underpinning one first-order factor, was also compared with Model 1. The model fit indices displayed an acceptable fit to the data (χ² = 570.222, df = 75, TLI = 0.931, CFI = 0.943, RMSEA = 0.095 with 90% CI [0.088, 0.103], SRMR = 0.038). The model comparison showed that Model 1 was significantly better than Model 3 (Δχ² = 173.855, Δdf = 1, p < 0.01). In addition, the differences between CFI and RMSEA values were greater than 0.01 (Cheung & Rensvold, 2002). Therefore, Model 1 was selected as the final model that best represented the data (see Figure 1).

Figure 1.

The first-order two-factor 14-item model for L2-Student Writing Feedback Literacy Scale (L2-SWFLS).

A closer look at the first-order two-factor 14-item model revealed that, as for Using Feedback, the factor loadings ranged from .716 to .877. In terms of Evaluating Feedback, the factor loadings ranged from .673 to .835. Overall, all factor loadings were above .50, suggesting that the items had high explanatory power for the two factors. The correlation between the two factors was around .80, indicating that they are measuring correlated yet distinguishable aspects of the same construct, namely, L2 secondary student writing feedback literacy.

3 SEM results

The model fit indices showed that the SEM model had an acceptable fit to the data (see Figure 2) (χ² = 452.974, df = 110, TLI = 0.941, CFI = 0.952, RMSEA = 0.065 with 90% CI [0.059, 0.072], SRMR = 0.029). The results also showed that Using Feedback (p = .640) and Evaluating Feedback (p = .810) were not related to L2 writing performance after controlling for the two covariates, i.e. gender and age.

Figure 2.

The structural equation modeling (SEM) model for L2-Student Writing Feedback Literacy Scale (L2-SWFLS) and L2 writing performance.

V Discussion and conclusions

This study makes a novel contribution to L2 writing research due to the scarcity of studies on secondary students’ feedback practices and literacy in the literature and illuminates the uniqueness of secondary student writing feedback literacy, providing valuable information to inform and enhance the development of student feedback literacy in similar contexts. As an additional contribution to the literature on L2 writing feedback and student feedback literacy, we address the relationship between writing feedback literacy and writing performance and reveal the complexity of the relationship between the two, adding to the existing research base about the role of feedback literacy in L2 writing.

The present study found that the L2-SWFLS for secondary students consisted of two factors, i.e. Using Feedback and Evaluating Feedback. Using Feedback included items on appreciating feedback, managing affect, and taking action, lending empirical support for the feedback literacy frameworks (Carless & Boud, 2018; Molloy et al., 2020; Yu, Zhang, & Liu, 2022; Zhan, 2022, 2023). Evaluating Feedback resonated with making judgments in earlier studies (Carless & Boud, 2018; Molloy et al., 2020; Zhan, 2022, 2023; Yu, Zhang, & Liu, 2022). The factorial structure of the L2-SWFLS for university students differed from that for secondary students: appreciating feedback, managing affect, and taking action in the former underpin a higher-order construct named Using Feedback in the latter. The reason might be linked to the differences in L2 writing instruction and assessment practices in the two contexts as reported by earlier studies, such as Geng et al. (2022), Hamp-Lyons (2007), Lee (2007, 2008), Lee et al. (2018), Lee and Coniam (2013), Tsivitanidou et al. (2011), Tsui and Ng (2000), and Yu et al. (2022). The L2 secondary school writing instruction in many places in mainland China is still exam-driven and textbook-based (Geng et al., 2022; Lee et al., 2018; Yu et al., 2022), and the dominant pedagogy is a product-oriented approach in which multiple drafts and multi-round feedback are rare (Curtis, 2001; Geng et al., 2022; Lee & Coniam, 2013; Lo & Hyland, 2007; Ortega, 2009). Students in such contexts are more inclined to be passive recipients of feedback in the feedback processes and depend on teacher feedback without critical evaluation (Ketonen et al., 2020; Lee, 2008; O’Donovan, 2017), so their feedback literacy might be limited in scope.

By contrast, L2 university writing instruction is less test-oriented compared with that in the L2 secondary school context, and the process-oriented instruction is popular in L2 university classes (Jiang et al., 2023), where students have access to feedback from different sources, such as teacher-, peer-, and automated feedback, and they are provided with ample opportunities to write and revise their drafts for multiple rounds based on the feedback they received (e.g. Tian & Zhou, 2020; Z. Zhang, 2020). What’s more, secondary students usually have limited peer feedback experiences, during which they learn how to make judgments on others’ writing based on the evaluation criteria (e.g. Diab, 2011; Zhao, 2010). Probably because of the two reasons above, L2 secondary students’ feedback literacy did not have a high level of granularity. This finding highlighted the necessity of adapting and validating the L2 university student writing feedback literacy scale among secondary students, and it is cautious to generalize the scale developed based on university students’ data to secondary students.

Additionally, compared with the factor loadings for Using Feedback, those for Evaluating Feedback were relatively lower. This might be because secondary students tended to view teachers as authorities and trust teacher feedback as the only source of writing feedback in the secondary school context, thwarting their capabilities of evaluating and making judgments on the feedback. The finding corroborated those from Belcher and Liu (2004), Han and Xu (2019), Ketonen et al. (2020), Lee (2004, 2008), and O’Donovan (2017). This might also be due to the lack of teacher explanation of evaluation criteria, as reported in Ketonen et al. (2020), Lee (2007), and Tsivitanidou et al. (2011), in which students encountered difficulties in making independent and critical judgments in feedback and pronounced their needs of making sense of the writing evaluation criteria by teachers’ exemplar-based explanations or assessment rubrics.

The finding foregrounds the mismatch between research and practice in L2 writing feedback. Despite the efforts and achievements in L2 writing feedback and assessment, the actual classroom practice has slightly lagged behind. Therefore, measures need to be taken in class to improve L2 secondary students’ writing feedback literacy and maximize the learning potential of writing feedback. For example, as for Using Feedback, the process-oriented pedagogy needs to be applied more in L2 secondary schools to allow students to use feedback from different sources other than teacher feedback, such as peers, self, and automated systems, to revise their drafts, considering the reported benefits of peer feedback (Yu & Lee, 2016; Zhao, 2010), self-feedback (Lam, 2013), and automated feedback (Z. Zhang, 2020). As regards Evaluating Feedback, students are encouraged to evaluate the feedback as well as the evaluation criteria to make their own decisions about acting on feedback (e.g. Carless & Boud, 2018; Ghaffar et al., 2020; Yu & Liu, 2021).

In terms of the relationship between writing feedback literacy and writing performance, although a few previous studies reported that students of differing L2 writing proficiency displayed different characteristics in feedback practices and feedback literacy (e.g. Bai & Wang, 2021; Baker & Boonkit, 2004; Chien, 2012; Han & Xu, 2019; Zhou et al., 2022), the results of the present study showed that L2 secondary students’ writing feedback literacy was not related to their L2 writing performance. One possible reason might be related to secondary school students’ limited feedback experiences in L2 writing: they had only teacher feedback and received insufficient feedback training, and therefore, their writing feedback literacy might not be strong enough to have significant predictive power on their L2 writing performance. It is also likely that L2 secondary students’ writing feedback literacy may not relate to their L2 writing performance directly, but they were associated with each other indirectly via some linking mechanisms, such as L2 writing motivation, self-efficacy, self-regulation, writing attitude, and/or writing frequency (e.g. Busse, 2013; Camacho et al., 2021; Graham et al., 2007, 2012; Y. Zhang & Guo, 2012), which warrant future examinations.

We have to take note of several limitations in this study. First, we selected a Chinese secondary student sample, and future researchers are encouraged to recruit samples from other L2 contexts, such as Japan, Korea, Vietnam, and Thailand, to see how L2 student writing feedback literacy functions across diverse socio-cultural milieus. All L2 participants in this study were learners of English as a second language (ESL), and additional research could consider participants who learn other languages as second languages, such as Chinese, Japanese, and Vietnamese, to examine how the L2 student writing feedback literacy scale generalizes across language contexts. Second, only senior secondary students were recruited as participants, and future studies could include both junior and senior secondary students to make the scale more generalizable to secondary students. Lastly, student-reported data were the major source of data, which might suffer from self-report biases. Future studies might also employ interviews, observations, and other data sources to enrich the texture of their findings.

Despite the limitations, this study has theoretical, empirical, and pedagogical contributions. Theoretically, our research provided interesting cross-context and cross-sample insights into the structure of L2 student writing feedback literacy. The conceptualization of L2 student writing feedback literacy comprised of appreciating feedback, making judgment, managing affect, identifying different feedback sources, and taking action may not be applicable in L2 secondary school contexts. Appreciating feedback, managing affect, and taking action might underpin a high-order construct, i.e. Using Feedback. Future studies could be motivated to refine the L2 student writing feedback literacy models. Empirically, this study provides insights into measuring L2 student writing feedback literacy and into understanding the relationship between L2 student writing feedback literacy and L2 writing performance, inspiring future research to examine the internal mechanisms of how L2 student writing feedback literacy affects L2 student writing performance with potential mediators, such as L2 writing self-efficacy, self-regulation, writing attitude, and/or writing frequency.

Pedagogically, secondary teachers could use the scale to identify students’ strengths and weaknesses in writing feedback literacy and provide timely and personalized scaffolding to zero in on the knowledge and skills they need to acquire to be feedback literate. They could also guide students to conduct self-assessment of writing feedback literacy and help them develop the metalanguage of feedback literacy and raise their metacognitive awareness of developing feedback literacy to foster their writing skills. The scale descriptors could potentially inform teachers about training students to enhance their writing feedback literacy. For instance, they could make it possible for students to receive feedback from multiple sources, such as teachers, peers, self, and automated systems, to make them use feedback as a learning tool in developing their L2 writing competence. Teachers may also encourage students to take on the role of proactive learners to seek and provide feedback on their own initiative and process feedback with critical and independent thinking. Additionally, teachers could share evaluation criteria and example essays with students to help them make sense of the evaluation criteria and train them on how to make judgments about their peers’ compositions and provide feedback to peers based on the evaluation criteria.

Footnotes

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Emily Di Zhang

Shulin Yu

References

Anderson

J.C.

Gerbing

D.W.

(1984). The effect of sampling error on convergence, improper solutions and goodness-of-fit indices for maximum likelihood confirmatory factor analysis. Psychometrika, 49, 155–173.

Bai

Wang

(2021). Hong Kong secondary students’ self-regulated learning strategy use and English writing: Influences of motivational beliefs. System, 96, 102404.

Baker

Boonkit

(2004). Learning strategies in reading and writing: EAP contexts. RELC Journal, 35, 299–328.

Belcher

Liu

(2004). Conceptualizing discourse/responding to text. Journal of Second Language Writing, 13, 3–6.

Bentler

P.M.

(1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238–246.

Boud

Molloy

(2013). Rethinking models of feedback for learning: The challenge of design. Assessment and Evaluation in Higher Education, 38, 698–712.

Busse

(2013). How do students of German perceive feedback practices at university? A motivational exploration. Journal of Second Language Writing, 22, 406–424.

Byrne

B.M.

(2010). Structural equation modeling with AMOS: Basic concepts, applications and programming. Taylor and Francis.

Camacho

Alves

R.A.

De Smedt

Van Keer

Boscolo

(2021). Relations among motivation, behaviour, and performance in writing: A multiple-group structural equation modeling study. British Journal of Educational Psychology, 91, 1456–1480.

10.

Carless

(2019). Feedback loops and the longer-term: Towards feedback spirals. Assessment & Evaluation in Higher Education, 44, 705–714.

11.

Carless

Boud

(2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment and Evaluation in Higher Education, 43, 1315–1325.

12.

Casanave

C.P.

(2004). Controversies in second language writing: Dilemmas and decisions in research and instruction. University of Michigan Press.

13.

Cheung

G.W.

Rensvold

R.B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233–255.

14.

Chien

S.-C.

(2012). Students’ use of writing strategies and their English writing achievements in Taiwan. Asia Pacific Journal of Education, 32, 93–112.

15.

Cho

Sohn

(2007). Complementarity of peer and teacher feedback in Korean high school English classes. English Teaching, 62, 307–329.

16.

Chong

S.W.

(2021). Reconsidering student feedback literacy from an ecological perspective. Assessment and Evaluation in Higher Education, 46, 92–104.

17.

Curtis

(2001). Hong Kong student teachers’ responses to peer group process writing. Asian Journal of English Language Teaching, 11, 129–143.

18.

Davison

(2007). Views from the chalkface: English language school-based assessment in Hong Kong. Language Assessment Quarterly, 4, 37–68.

19.

Diab

N.M.

(2011). Assessing the relationship between different types of student feedback and the quality of revised writing. Assessing Writing, 16, 274–292.

20.

Ferris

D.R.

(2003). Response to student writing: Implications for second language students. Lawrence Erlbaum.

21.

Field

A.P.

(2009). Discovering statistics using SPSS. 3rd edition. Sage.

22.

Geng

Liu

(2022). Teaching and learning writing in English as a Foreign Language (EFL) school education contexts: A thematic review. Scandinavian Journal of Educational Research, 66, 491–504.

23.

Ghaffar

M.A.

Khairallah

Salloum

(2020). Co-constructed rubrics and assessment for learning: The impact on middle school students’ attitudes and writing skills. Assessing Writing, 45, 100468.

24.

Graham

Berninger

Abbott

(2012). Are attitudes toward writing and reading separable constructs? A study with primary grade children. Reading and Writing Quarterly, 28, 51–69.

25.

Graham

Berninger

Fan

(2007). The structural relationship between writing attitude and writing achievement in first and third grade students. Contemporary Educational Psychology, 32, 516–536.

26.

Hamp-Lyons

(2007). The impact of testing practices on teaching: Ideologies and alternatives. In Cummins

Davison

(Eds.), International handbook of English language teaching: Volume 1 (pp. 487–504). Springer.

27.

Han

(2019). The development of student feedback literacy: The influences of teacher feedback on peer feedback. Assessment and Evaluation in Higher Education, 45, 680–696.

28.

Han

(2021). Student feedback literacy and engagement with feedback: A case study of Chinese undergraduate students. Teaching in Higher Education, 26, 181–196.

29.

Hattie

(2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge.

30.

Hattie

Timperley

(2007). The power of feedback. Review of Educational Research, 77, 81–112.

31.

L.T.

Bentler

P.M.

(1995). Evaluating model fit. In Hoyle

R.H.

(Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 76–99). Sage.

32.

Hutcheson

Sofroniou

(1999). The multivariate social scientist. Sage.

33.

Hyland

(2003). Second language writing. Cambridge University Press.

34.

Hyland

(Eds.). (2019). Feedback in second language writing: Contexts and issues. Cambridge University Press.

35.

Jiang

Zhou

. (2023). English writing instruction in Chinese students’ experience: A survey study. RELC Journal, 54, 37–54.

36.

Kaiser

H.F.

(1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141–151.

37.

Ketonen

Nieminen

Hähkiöniemi

(2020). The development of secondary students’ feedback literacy: Peer assessment as an intervention. The Journal of Educational Research, 113, 407–417.

38.

Kroll

(1990). Second language writing: Research insights for the classroom. Cambridge University Press.

39.

Lam

(2013). The relationship between assessment types and text revision. ELT Journal, 67, 446–458.

40.

Lee

(2004). Error correction in L2 secondary writing classrooms: The case of Hong Kong. Journal of Second Language Writing, 13, 285–312.

41.

Lee

(2007). Feedback in Hong Kong secondary writing classrooms: Assessment for learning or assessment of learning? Assessing Writing, 12, 180–198.

42.

Lee

(2008). Understanding teachers’ written feedback practices in Hong Kong secondary classrooms. Journal of Second Language Writing, 17, 69–85.

43.

Lee

(2017). Classroom writing assessment and feedback in L2 school contexts. Springer.

44.

Lee

Coniam

(2013). Introducing assessment for learning for EFL writing in an assessment of learning examination-driven system in Hong Kong. Journal of Second Language Writing, 22, 34–50.

45.

Lee

Liu

(2018). Hong Kong secondary students’ motivation in EFL writing: A survey study. TESOL Quarterly, 52, 176–187.

46.

Lei

Wen

Yang

(2023). A large-scale longitudinal study of syntactic complexity development in EFL writing: A mixed-effects model approach. Journal of Second Language Writing, 59, 100962.

47.

Hyland

(2007). Enhancing students’ engagement and motivation in writing: The case of primary students in Hong Kong. Journal of Second Language Writing, 16, 219–237.

48.

Lundstrom

Baker

(2009). To give is better than to receive: The benefits of peer review to the reviewer’s own writing. Journal of Second Language Writing, 18, 30–43.

49.

Mahfoodh

O.H.A.

(2017). ‘I feel disappointed’: EFL university students’ emotional responses towards teacher written feedback. Assessing Writing, 31, 53–72.

50.

Mayor

Hewings

North

Swann

Coffin

(2007). A linguistic analysis of Chinese and Greek L1 scripts for IELTS Academic Writing Task 2. In Linda

Peter

(Eds.), IELTS collected papers: Research in speaking and writing assessment (pp. 250–315). Cambridge University Press.

51.

Molloy

Boud

Henderson

(2020). Developing a learning-centred framework for feedback literacy. Assessment and Evaluation in Higher Education, 45, 527–540.

52.

Moore

T.J.

Morton

(1999). Authenticity in the IELTS academic module writing test: A comparative study of task 2 items and university assignments. IELTS Research Report Volume 1999-2. IELTS Australia.

53.

Nguyen

B.T.T.

Nguyen

T.T.B.

Phan

Q.N.

(2022). Vietnamese IELTS learners’ perceptions of IELTS Writing Task 2 and their performance-based lexical resource. IELTS Research Reports Online Series, No. 5/22. British Council and IELTS Australia.

54.

O’Donovan

(2017). How student beliefs about knowledge and knowing influence their satisfaction with assessment and feedback. Higher Education, 74, 617–633.

55.

Ortega

(2009). Studying writing across EFL contexts: Looking back and moving forward. In Manchón

R.M.

(Ed.), Writing in foreign language contexts: Learning, teaching, and research (pp. 232–255). Multilingual Matter.

56.

Révész

Michel

Lee

(2017). Investigating IELTS Academic Writing Task 2: Relationships between cognitive writing processes, text quality, and working memory. IELTS Research Volume 2017-3. Cambridge: Cambridge University Press.

57.

Ruegg

(2015). Differences in the uptake of peer and teacher feedback. RELC Journal, 46, 131–145.

58.

Sutton

(2012). Conceptualizing feedback literacy: Knowing, being, and acting. Innovations in Education and Teaching International, 49(1), 31–40.

59.

Tian

Zhou

(2020). Learner engagement with automated feedback, peer feedback and teacher feedback in an online EFL writing context. System, 91, 102247.

60.

(2016). ‘This is not what I need’: Conflicting assessment feedback beliefs in a post-secondary institution in Hong Kong. Research in Post-Compulsory Education, 21, 447–467.

61.

Tsivitanidou

O.E.

Zacharia

Z.C.

Hovardas

(2011). Investigating secondary school students’ unmediated peer assessment skills. Learning and Instruction, 21, 506–519.

62.

Tsui

A.B.M.

M.M.Y.

(2000). Do secondary L2 writers benefit from peer comments? Journal of Second Language Writing, 9, 147–170.

63.

(2022). Incremental intelligence matters: How L2 writing mindsets impact feedback orientation and self-regulated learning writing strategies. Assessing Writing, 51, 100593.

64.

(2019). Learning from giving peer feedback on postgraduate theses: Voices from Master’s students in the Macau EFL context. Assessing Writing, 40, 42–52.

65.

Jiang

Liu

Zhou

(2022). Profiles of Chinese secondary students’ L2 writing motivation and engagement. Educational Psychology, 42, 972–990.

66.

Lee

(2016). Peer feedback in second language writing (2005–2014). Language Teaching, 49, 461–493.

67.

Liu

(2021). Improving student feedback literacy in academic writing: An evidence-based framework. Assessing Writing, 48, 100525.

68.

Zhang

E.D.

Liu

(2022). Assessing L2 student writing feedback literacy: A scale development and validation study. Assessing Writing, 53, 100643.

69.

Zamel

(1985). Responding to students’ writing. TESOL Quarterly, 19, 79–101.

70.

Zhan

(2022). Developing and validating a student feedback literacy scale. Assessment & Evaluation in Higher Education, 47, 1087–1100.

71.

Zhan

(2023). What do college students think of feedback literacy? An ecological interpretation of Hong Kong students’ perspectives. Assessment and Evaluation in Higher Education, 48, 686–700.

72.

Zhang

Guo

(2012). A study of English writing and domain-specific motivation and self-efficacy of Chinese EFL learners. Journal of Pan-Pacific Association of Applied Linguistics, 16, 101–121.

73.

Zhang

(2020). Engaging with automated writing evaluation (AWE) feedback on L2 writing: Student perceptions and revisions. Assessing Writing, 43, 100439.

74.

Zhao

(2010). Investigating learners’ use and understanding of peer and teacher feedback on writing: A comparative study in a Chinese English writing classroom. Assessing Writing, 15, 3–17.

75.

Zhou

Liu

Jiang

(2022). Examining the role of writing proficiency in students’ feedback literacy development. Applied Linguistics Review. Advance online publication. https://doi.org/10.1515/applirev-2021-0133