Psychometric Properties of Cognitive and Metacognitive Learning Strategies Scale Among Chinese Senior Secondary School Students

Abstract

This study aimed to investigate the validity of the Cognitive and Metacognitive Learning Strategies Scale (CMLSC) for mathematics learning in Chinese context. Exploratory and confirmatory factor analysis, reliability analysis, measurement invariance across gender groups, and criterion-related validity were conducted on 698 Chinese senior secondary school students. Results supported that the adapted CMLSC was reliable and valid. A bifactor model with one general and four specific factors (i.e., rehearsal, elaboration and organization, critical thinking, and metacognitive self-regulation) was found. Residual invariance across gender groups was also achieved. This adapted CMLSC is expected better to understand students’ mathematics learning strategies in Chinese culture.

Keywords

Cognitive and Metacognitive Learning Strategies Scale psychometric properties Chinese culture bifactor model mathematics learning

Introduction

Cognitive and metacognitive learning strategies are two highly correlated but distinctive constructs in self-regulated learning theory (Panadero, 2017). Cognitive strategies help students retain and understand academic materials, while metacognitive processes ensure that they achieve these goals. Previous studies indicated that adept strategies are associated with higher achievement (Dent & Koenka, 2016).

The Cognitive and Metacognitive Learning Strategies Scale (CMLSC), derived from the learning strategy section of the Motivated Strategies for Learning Questionnaire (MSLQ) (Pintrich et al., 1991), is one of the most widely used instruments. CMLSC contains 31 items regarding students’ cognitive (i.e., rehearsal, elaboration, organization, and critical thinking) and metacognitive strategies. Although CMLSC has been adapted and validated across different countries, subjects, and samples (Duncan & McKeachie, 2005), we adapted this scale for Chinese senior secondary school students in mathematics learning for the following reasons:

First, most previous studies approached samples from WEIRD (Western, educated, industrialized, rich, and democratic) societies (Zimmerman, 2001). However, strategy-use patterns vary in different cultures (Purdie & Hattie, 1996). Many Chinese students may rely heavily on teacher instructions, yet have limited opportunities to develop their own critical thinking and self-regulation skills (Nguyen et al., 2006). Second, learning strategies vary by grade, given the differences in the complexity of tasks and students’ learning experiences (Dent & Koenka, 2016). Third, learning strategies are domain-specific (Credé & Phillips, 2011), and mathematics is a core subject that needs adept learning strategies to learn better. Fourth, there is a gender difference in strategy use. Some suggested that women tend to use more adept learning strategies than men (Bidjerano, 2005), while others indicated no significant difference across gender groups (Metallidou & Vlachou, 2007).

In summary, there is a need to investigate the psychometric properties of the CMLSC among Chinese senior secondary school students in mathematics learning. These psychometric properties include the factor structure, internal consistency reliability, measurement invariance across gender groups, and criterion-validity.

Method

Participants

Our sample includes valid responses from 698 students, who participated voluntarily via the approval of their schools. Consent forms were collected before data collection. There was 46% female. The average age of the sample was 17.1 years, and SD was .49.

Measures

All variables were adapted from MSLQ. Items were translated into Chinese and revised for mathematics learning and answered on an 11-point Likert scale with only endpoints labeled as 0 (completely disagree) and 10 (completely agree).

Learning strategies scale consisted of rehearsal, elaboration, organization, critical thinking, and metacognitive self-regulation. In addition, intrinsic value and task value were used for testing criterion-validity. Appendix A presented the number of items, Cronbach’s alpha, means, and SDs for all variables.

Statistical Analyses

First, two reversed-coded items showed a low item–total correlation. We removed them first as recommended by previous research (Rao & Sachs, 1999).

Second, we split the sample into two random halves to conduct exploratory and confirmatory factor analysis (EFA and CFA) to locate factors and confirm their structure. For the first half, EFA with Principal Axis Factoring and Promax rotation was run with SPSS 26.0. Item loading greater than .32 was used to select items (Tabachnick & Fidell, 2001). We also deleted cross-loading items unless the differences on two factors were greater than .20 (Bedford, 1997).

Third, the other half was used in CFA with the “Lavaan” package (Rosseel, 2012). The value of the comparative fit index (CFI) and Tucker–Lewis index (TLI) greater than .90 and root mean squared error of approximation (RMSEA) less than .08 were interpreted as acceptable fit (Marsh et al., 2004). Meanwhile, the total scale and subscales’ internal consistency reliability were assessed using omega (ω) and omega subscale (ω_s). Hierarchical subscale (ω_hs) was used to assess the reliability of general factor and specific factors were assessed by omega hierarchical and omega hierarchical subscale coefficients. The cutoff value of .50 was used to support scores use (Reise et al., 2013).

Fourth, measurement invariance with the total sample across gender groups was analyzed. Configural, metric, scalar, and residual invariance were tested across gender groups. We used ΔCFI ≤ .010, ΔRMSEA ≤ .015, and ΔSRMR ≤ .030 as the criteria for configural invariance and ΔCFI ≤ .010, ΔRMSEA ≤ .015, and ΔSRMR ≤ .010 for metric, scalar, and residual invariance (Chen, 2007). Meanwhile, the latent mean difference between men and women was tested.

Last, criterion-validity was evaluated by calculating the correlations between CMLSC and subjective task values (i.e., intrinsic value and task value).

Results and Discussion

EFA

Based on the deletion criteria mentioned earlier, 17 items were retained (see Appendix F) after two rounds of EFA. Appendix B presents details of these deleted items. For the final EFA model with retained 17 items, the value of Kaiser–Meyer–Olkin was .909, and Bartlett’s test of sphericity was significant (χ² (136) = 2946.14, p < .001). The obtained four factors explained 58% of the total variance. Four obtained factors were labeled as rehearsal, elaboration and organization, critical thinking, and metacognitive self-regulation, and their corresponding loadings are in Appendix C. The correlations among factors ranged from .236 to .613.

CFA

Next, we run CFA with first-order correlated factor model, second-order model, and bifactor model (Figures 1–3). Appendix D compared the fit indices of the three competing models.

Figure 1.

Factor loading of the first-order correlated factor model. Note. RE = rehearsal; EO = elaboration and organization; CT = critical thinking; ME = metacognitive self-regulation.

Figure 2.

Factor loading of the second-order model. Note. RE = rehearsal; EO = elaboration and organization; CT = critical thinking; ME = metacognitive self-regulation; HS = high-order strategy use.

Figure 3.

Factor loadings of the bifactor model. Note. RE = rehearsal; EO = elaboration and organization; CT = critical thinking; ME = metacognitive self-regulation; GS = general strategy use.

Among the three competing models, the bifactor model showed the lowest Bayesian information criterion value and best fit with RMSEA = .068 (90% CI [.058, .078]), CFI = .951, and TLI = .935. The reliabilities for general strategy use, rehearsal, elaboration and organization, critical thinking, and metacognitive self-regulation were 0.82, 0.60, 0.24, 0.28, and 0.27 respectively. For the reliability of specific factors, only the value of rehearsal was greater than the cutoff value. We would return to this in the discussion section.

Gender Invariance

Results for gender invariance were reported in Appendix E. Configural (M1), metric (M2), scalar (M3), and residual invariance (M4) were fully supported.

Testing for the Latent Mean Difference Between Men and Women

The results of latent mean difference indicated that women had significantly lower mean on the general strategy use (z = −4.837, p < .001, d = −.681) and specific factor for critical thinking (z = −4.835, p < .001, d = −.564), while higher mean on rehearsal (z = 3.219, p < .005, d = .591). No significant differences were found for other factors.

Criterion Validity

The bifactor model correlated with intrinsic value and task value displayed an acceptable goodness of fit with χ² (296) = 1085 (p < .001), CFI = .924, TLI = .910, and RMSEA = .065 (90% CI [.060, .069]). Results revealed that both intrinsic value (r = .122, p < .001) and task value (r = .722, p < .001) were positively associated with general strategy use. Intrinsic value was positively associated with critical thinking (r = .164, p < .01) and metacognitive self-regulation (r = .126, p < .05), while task value was positively associated with metacognitive self-regulation (r = .093, p < .05).

Discussion and Limitations

This study extends previous research in several ways. First, the Chinese version of CMLSC extends the application of MSLQ into mathematics. Considering the role of mathematics learning strategies (Rao et al., 2000), our results would contribute to students’ success in university entrance examinations.

Second, the factor structure of the CMLSC may be different due to the diversity of cultures, grades, and subjects (Duncan & McKeachie, 2005), which may, in turn, affect its applications. We found that the bifactor structure with one general and four specific factors was the best in the Chinese context. The general factor indicates that students tend to coordinate different learning strategies together while studying mathematics. For specific factors, only rehearsal provides meaningful and reliable information independent of the general factor, which is consistent with previous evidence that Confucian-heritage and Asian cultures stress memorization in teaching and learning (Rao & Sachs, 1999). Meanwhile, the elaboration and organization dimensions could not be distinguished from each other.

Third, our findings revealed residual invariance across gender groups, indicating the substantive equivalence of the cognitive and metacognitive strategies for women and men.

Fourth, we found that men generally exhibited higher means in complex learning strategies, while women displayed higher in surface learning strategies. In Confucian culture, mathematics is usually male-stereotyped. Men tend to overestimate, while women underestimate their abilities in mathematics learning (Liu, 2018). Hence, men may have more confidence in mathematics and use more complex strategies.

Last, we further support that intrinsic value and task value are positively correlated with adept strategy use (Berger & Karabenick, 2011). According to expectancy-value theory, students who value the tasks would spend more time using different strategies to achieve academic success (Eccles, 2005).

Some limitations are as follows. First, our sample was selected from Guangdong province, an economically advanced province in China. More research should be investigated with the data from other regions in China. Second, we used cross-sectional data. Future research needs to test the stability of learning strategies over time via longitudinal data. Third, we used self-reported data, and future research needs to triangulate our findings with other sources of data, such as classroom observation and teachers’ reports.

Conclusions

This study supported the reliability and validity of the adapted CMLSC in the Chinese context in mathematics learning. A bifactor structure with one general factor and four specific factors was found to represent students’ strategy use. This adapted CMLSC can help researchers and teachers better understand students’ learning strategies in mathematics learning.