Cross-Cultural Validity of Preschool Learning Behavior Scale in Chinese Cultural Context

Abstract

This study investigated the cross-cultural validity of the Preschool Learning Behavior Scale (PLBS) in the Chinese cultural context. Multiple approaches were used for this purpose, including exploratory factor analysis, confirmatory factor analysis, criterion-related validity evidence, and internal consistency reliability estimates. The findings generally supported the PLBS’ three-factor structure (Competence Motivation, Learning Strategy, and Attention/Persistence) as used in the Chinese cultural context, and with minor adaptations, PLBS could be a psychometrically sound measure for assessing the learning behaviors of Chinese children.

Keywords

approaches to learning learning behavior Chinese preschool children

In recent years, approaches to learning have been identified as one important early predictor of future success (McDermott, Rikoon, Waterman, & Fantuzzo, 2012), because research has shown that positive learning behaviors correlated with children’s engagement in learning activities and social interaction (Coolahan, Fantuzzo, Mendez, & McDermott, 2000), and that it counter-acted against various academic (Domínguez Escalón & Greenfield, 2009) and social (McWayne & Cheung, 2009) risk factors.

McDermott, Leigh, and Perry (2002) developed the Preschool Learning Behavior Scale (PLBS), and validation research (McDermott et al., 2012) revealed three distinct dimensions of PLBS: Competence Motivation (CM, 16 items), Learning Strategy (LS, six items), and Attention/Persistence (AP, four items). However, McDermott et al. (2012) discussed that future researchers should examine the dimensionality of the PLBS by using a wide range of samples, especially within demographically diverse samples.

In China, “approaches to learning” is listed as one of the six critical domains of assessment of school readiness (Ministry of Education of the People’s Republic of China, 2012), and professionals in early childhood education in China have called for effective measures of Chinese children’s approaches to learning (Li & Feng, 2013), such as the use of PLBS in the Chinese cultural context. As stated in the Standards for Educational and Psychological Testing (American Educational Research Association [AERA], American Psychological Association [APA], & National Council on Measurement in Education [NCME], 2002), when a measure is used in a new cultural and linguistic environment, its psychometric qualities cannot be assumed, and evidence for validity and reliability needs to be obtained for such intended use of the measure. This study was designed to examine the applicability of the PLBS in the Chinese cultural context from multiple perspectives: measurement reliability analysis, exploratory factor analysis (EFA), confirmatory factor analysis (CFA), and criterion-related validity evidence.

Method

Participants

Participants were 5,328 children from K-1 to K-3 classrooms (2,745 boys, 51.5%) from Guangdong province in southern China. The parents filled out a questionnaire about children’s personal attributes, and teachers filled out a questionnaire about approaches to learning, social skills, academic competence, and executive functioning for each child.

Measures

As recommended in the Standards (AERA, NCME, & APA, 2002), an iterative process involving translation and back-translation was used to translate the English measures into Chinese by several researchers who were fluent in both English and Chinese, and had rich professional experience in the fields of early childhood education and/or developmental psychology.

Approaches to learning

The PLBS (McDermott et al., 2012) was used to measure the learning-related behaviors of the sampled Chinese children. The teachers used the 29 items of PLBS to rate the behaviors of the sampled children in their respective classrooms.

In the process of translating PLBS items, Item 13 was considered problematic by the researchers. To verify this, the researchers consulted 10 experienced Chinese preschool teachers. Some regarded the behavior described by the item as positive and proactive, whereas others considered it morally problematic in leading children to depend on others to complete a task. Because of this uncertainty, this item was removed.

Social skills and academic competence

Two subscales (46-item Social Skills subscale, and seven-item Academic Competence subscale) from the teacher-reported Social Skills Improvement System Rating Scale (SSIS-RS; Gresham, & Elliott, 2008) were used to assess children’s social skills and academic competence. Details about these subscales were available from Gresham and Elliott (2008). The Cronbach’s α values were .96 for academic competence and .98 for social skills, respectively, in the present study.

Executive functioning

The Behavior Rating Inventory of Executive Function–Preschool Version (BRIEF-P; Gioia, Espy, & Isquith, 2003) was used to assess the children’s executive functioning, with a high global score being interpreted as indicating poor executive functioning. The Cronbach’s α value was .87 for this scale in the present study.

Results

Descriptive Item Analysis

The descriptive statistical analysis revealed that the skewness (−1.14-0.07) and kurtosis (−1.02-0.29) values indicated fairly normal distributions in research practice (Kline, 2005). The item-total correlations were also adequate, except two items (Item 9 and Item 23) with item-total correlations below .20. These two items were considered as poor-performing items, and thus not used in the subsequent analyses (Nunnally & Bernstein, 1994).

EFA

The sample data were randomly split into two equal subsamples (n₁ = n₂ = 2,664) first, and the two independent subsamples were used for EFA and CFA, respectively (Stevens, 2009). For addressing the issue that the response data were categorical, in both EFA and CFA (described below), we used the estimation method of weighted least squares means and variance (WLSMV) adjusted for categorical data (Muthén, du Toit, & Spisic, 1997; Muthén & Muthén, 1998-2010).

We examined three sources of information for factor retention: Kaiser’s rule of eigenvalue > 1, “elbow” effect from the scree plot, and the results based on parallel analysis. Although five factors had eigenvalues greater than 1, both the scree plot (Cattell, 1966) and parallel analysis (O’Conner, 2000) suggested three. Our final decision to retain the three factors was also based on substantive considerations of conceptual clarity, interpretability, and simple structure of the rotated factors.

In the EFA analysis, four items (Items 17, 21, 22, and 24) misloaded on Factor 1, instead of loading on Factor 3 (AP) as expected (McDermott et al., 2002). As there was no obvious reason for this misloading, the decision was made to omit these items in the subsequent analyses. In addition, we removed one item (Item 18) that had much lower loading than the threshold of 0.40 suggested by Gorsuch (1983).

Based on substantive considerations (e.g., content validity and representativeness; McDermott et al., 2012), we labeled the three factors as (a) CM (11 items); (b) LS (six items); and (c) AP (four items). Table 1 shows the factor loadings and communalities of the rotated solution of the 21 items distributed across the three factors.

Table 1.

PLBS Factor Structure (EFA and CFA): Pattern Coefficients (P) and Communalities (h²).

Factors and behavior indicators	EFA (N₁ = 2,664^a)		CFA (N₂ = 2,664)
Factors and behavior indicators	P	h ²	P	h ²
Competence Motivation
Item 1	0.69	0.50	0.75	0.56
Item 2	0.61	0.46	0.74	0.55
Item 3	0.69	0.54	0.77	0.59
Item 4	0.69	0.50	0.70	0.49
Item 5	0.80	0.58	0.75	0.57
Item 6	0.75	0.59	0.81	0.66
Item 7	0.78	0.63	0.83	0.68
Item 8	0.63	0.34	0.58	0.33
Item 10	0.70	0.47	0.65	0.42
Item 11	0.83	0.66	0.80	0.64
Item 12	0.65	0.43	0.61	0.37
Learning Strategy
Item 20	0.43	0.41	0.70	0.49
Item 25	0.77	0.59	0.56	0.32
Item 26	0.89	0.75	0.52	0.28
Item 27	0.85	0.74	0.70	0.49
Item 28	0.59	0.55	0.83	0.69
Item 29	0.45	0.42	0.74	0.54
Attention/Persistence
Item 14	0.81	0.72	0.85	0.72
Item 15	0.79	0.69	0.87	0.75
Item 16	0.71	0.55	0.77	0.60
Item 19	0.60	0.36	0.47	0.22

Note. Communality (h²) is the proportion of item variance explained by the final factor structure. PLBS = Preschool Learning Behavior Scale; EFA = exploratory factor analysis; CFA = confirmatory factor analysis.

Two randomly split equal and independent samples used for EFA and CFA, respectively.

CFA

CFA was conducted by fitting this EFA-generated model to the second random sample reserved for CFA. We examined three widely used descriptive fit indices for model fit: the comparative fit index (CFI), the Tucker–Lewis index (TLI), and the root mean square error of approximation (RMSEA). The initial three-factor CFA model yielded fit indices that suggested reasonable model fit, CFI = 0.928, TLI = 0.919, and RMSEA = 0.076 (0.074, 0.078). Modification indices indicated that the model fit could be improved by correlating three residual variances among Items 25, 26, and 27. The model fit showed some improvement with such minor revision, and the model fit index values, CFI = 0.958, TLI = 0.951, and RMSEA = 0.06 (0.057, 0.062), suggested a viable three-factor model (Hu & Bentler, 1995). Model revision involving correlated residuals is usually considered minor revision, and it could happen that the reason(s) for correlated residuals may not be obvious (Loehlin, 2004). In our case, unfortunately, the substantive reason for the correlated residuals was not clear. The three factors correlated positively with each other, ranging from r = .15 to .50, and all being statistically significant (p < .01). The factor loadings for this final CFA model were all positive and significant, as shown in Table 1. In general, the CFA results supported the three-factor structure revealed by the EFA analysis.

Internal Consistency Reliability Estimates

The total score and all three subscales had adequate to very good internal consistency reliability estimates (especially considering the small number of items for a subscale like AP), with Cronbach’s α values of .88, .80, and .75 for CM, LS, and AP, respectively. The reliability estimates for subgroups were examined for gender, grade level (K-1, K-2, and K-3), and school socio-economic region (urban, suburban, and rural), and the estimates were in the range of upper 0.70 to 0.90. Except some cases involving the AP subscale with only four items, all reliability estimates for the subscales across the subgroups exceeded the acceptable level of .70 as recommended by Hair, Black, Babin, Anderson, and Tatham (2006).

Criterion-Related Validity Evidence

The concurrent validity of the PLBS was investigated by examining the correlations between the PLBS scores (total and subscales) and some relevant child development outcomes. As shown in Table 2, other than a couple of exceptions for K-1 preschoolers, the magnitudes of these correlations could be characterized as ranging from being adequate to strong for the majority of the cases, especially at the total score level of PLBS (|0.39| to |−0.82|; McDermott et al., 2002), providing strong criterion-related validity evidence for PLBS when applied in the Chinese cultural context.

Table 2.

Correlations of PLBS Scores with Academic Competence, Social Skills, and Executive Functioning.

		M	SD	PLBS
		M	SD	Competence Motivation	Learning Strategy	Attention/Persistence	Total score
K-1	Academic Competence	23.83	5.44	.30*	.10*	.41*	.37*
K-1	Social Skills	70.45	22.14	.26**	.04	.41**	.31*
K-2	Executive Functioning	35.17	24.06	−.76*	−.58*	−.37*	−.81*
K-3	Academic Competence	24.62	5.89	.52*	.27*	.44*	.55*
K-3	Social Skills	76.47	25.11	.46*	.28*	.48*	.52*

p < .05. **p < .01.

Discussion and Conclusion

In China, there is a lack of instrument to assess the children’s learning behaviors. This study examined the cross-cultural validity of the PLBS in the Chinese cultural contexts from multiple perspectives. The findings of both EFA and CFA generally confirmed the original three-factor structure (CM, LS, and AP). Very good estimates of internal consistency reliability of the PLBS and its subscales were obtained. Furthermore, the PLBS total score and the subscale scores all showed adequate relationships with relevant child outcomes. All findings from these different perspectives provided solid evidence that the PLBS should be suitable for use in the Chinese sociocultural setting.

A few original items of the PLBS were found to be problematic for different reasons: poor content validity, low item-total correlation, low factor loadings, and factor misloading. Cultural factors could lead to some of these issues. As in other cross-cultural validation studies (e.g., Zhang & Norvilitis, 2002), it is very common that some original items may turn out to be inappropriate in the cross-cultural validation process.

This study has several limitations that could be addressed in future research. First, assessment ratings could be supplemented by either naturalistic or laboratory observations for a better understanding about children’s learning behaviors. Second, we removed a number of items for different reasons, and such removal should be considered tentative, and future research could examine if the removal of these few items can be justified by evidence obtained from other samples.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study and the preparation of the manuscript were supported by the project “A Longitudianl Study on the Effects of Preschool Program Quality on Children’s Learning and Development Outcomes.” (University of Macau Multi-Year Research Grant; MYRG2015-00156-FED). The authors of this paper deeply appreciate the support.

References

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2002). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Cattell

R. B.

(1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245-276.

Coolahan

Fantuzzo

Mendez

McDermott

(2000). Preschool peer interactions and readiness to learn: Relationships between classroom peer play and learning behaviors and conduct. Journal of Educational Psychology, 92, 458-465.

Domínguez Escalón

Greenfield

D. B

. (2009). Learning behaviors mediating the relationship between behavior problems and academic outcomes. NHSA Dialog, 12, 1-17.

Gioia

G. A.

Espy

K. A.

Isquith

P. K.

(2003). The Behavior Rating Inventory of Executive Function—Preschool version. Odessa, FL: Psychological Assessment Resources.

Gorsuch

R. L.

(1983). Factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

Gresham

Elliott

S. N.

(2008). Social Skills Improvement System (SSIS) rating scales. Bloomington, MN: Pearson Assessments.

Hair

J. F.

Jr. Black

W. C.

Babin

B. J.

Anderson

R. E.

Tatham

R. L.

(2006). Multivariate data analysis (6th ed.). Upper Saddle River, NJ: Prentice-Hall.

L. T.

Bentler

P. M.

(1995). Evaluating model fit. In Hoyle

(Ed.), Structural equation modeling: Issues, concepts, and applications (pp. 76-99). Newbury Park, CA: Sage.

10.

Kline

R. B.

(2005). Principles and practices of structural equation modeling (2nd ed.). New York, NY: Guilford Press.

11.

Feng

(2013). 《3-6岁儿童学习与发展指南》解读 [How to interpret the national guidelines for 3-6 years old children’s learning and development]. Beijing: China Beijing People Press.

12.

Loehlin

J. C.

(2004). Latent variable models: An introduction to factor, path, and structural equation analysis (4th ed.). Mahwah, NJ: Lawrence Erlbaum.

13.

McDermott

P. A.

Leigh

N. M.

Perry

M. A.

(2002). Development and validation of the Preschool Learning Behaviors Scale. Psychology in the Schools, 39, 353-365.

14.

McDermott

P. A.

Rikoon

S. H.

Waterman

Fantuzzo

J. W.

(2012). The Preschool Learning Behaviors Scale: Dimensionality and external validity in Head Start. School Psychology Review, 41, 66-81.

15.

McWayne

Cheung

(2009). A picture of strength: Preschool competencies mediate the effects of early behavior problems on later academic and social adjustment for Head Start children. Journal of Applied Developmental Psychology, 30, 273-285.

16.

Ministry of Education of the People’s Republic of China. (2012). 教育部关于印发《3-6岁儿童学习与发展指南》的通知 [Guidelines for 3-6 years old children’s learning and development]. Retrieved from http://www.moe.edu.cn/publicfiles/business/htmlfiles/moe/s3327/201210/xxgk_143254.html

17.

Muthén

B. O.

du Toit

S. H. C.

Spisic

(1997). Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling with categorical and continuous outcomes. Retrieved from http://pages.gseis.ucla.edu/faculty/muthen/articles/Article_075.pdf

18.

Muthén

L. K.

Muthén

B. O.

(1998-2010). Mplus user’s guide (6th ed.). Los Angeles, CA: Author.

19.

Nunnally

J. C.

Bernstein

I. H.

(1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.

20.

O’Conner

B. P.

(2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, & Computers, 32, 396-402.

21.

Stevens

J. P.

(2009). Applied multivariate statistics for the social sciences (5th ed.). New York. NY: Taylor Francis.

22.

Zhang

Norvilitis

J. M.

(2002). Measuring Chinese psychological well-being with Western developed instruments. Journal of Personality Assessment, 79, 492-511.