Study of the Factor Structure,Profiles,and Concurrent Validity of the Mindset Assessment Profile Tool for Elementary Students

Abstract

This study explored the underlying latent structure of items on the Mindset Assessment Profile (MAP) tool, explored whether subgroups of students exist based on the latent structure of MAP items, and tested whether subgroups were differentiated on standardized measures of reading comprehension, vocabulary, and word reading. Participants included 431 fourth-grade students. Confirmatory factor analysis revealed that a three-factor model provided the most parsimonious fit to the data. Results of exploratory finite mixture model analysis with auxiliary regression suggested five classes of students, with the students categorized as growth mindset—high effort profile having the highest observed reading comprehension (M = 451.98 and SD = 38.88) and vocabulary (M = 454.37 and SD = 34.74) scores. By contrast, students categorized as fixed mindset—higher effort had the lowest observed reading comprehension and vocabulary scores. Limitations and directions for future research, and implications for using MAP assessment to inform intervention are discussed.

Keywords

growth mindset effort reading elementary students

Introduction

Self-theories, such as Dweck’s theory of intelligence, are focused on the extent to which individuals believe that aspects of who they are can be changed. Dweck and Leggett (1988) postulated a social cognitive model that suggests that implicit beliefs may hold different assumptions about the malleability and nature of traits or abilities. In the context of Dweck’s theory of intelligence (2006), individuals who believe that an ability or trait is malleable and can be changed conform to an incremental theory of intelligence known as a growth mindset. Conversely, individuals who believe that a given ability or trait is fixed and cannot be changed are characterized as having a fixed mindset (i.e., an entity theory of intelligence (Chiu, Hong, & Dweck, 1997; Dweck, Chiu, & Hong, 1995). There is increasing interest in understanding students’ mindset or their belief that their intelligence and academic skills can be grown or are fixed (e.g., Dweck, 2006; Dweck & Leggett, 1988). Dweck and her colleagues have hypothesized that when students believe that these skills are fixed, they have less motivation for effort and practice. When they make errors or struggle, for example, in the domain of learning to read, they attribute this to a static condition of being a poor reader or having limited intelligence. By contrast, Dweck and colleagues posit that students with a growth mindset of intelligence (hereafter referred to as growth or fixed mindset) believe intelligence and academic skills can develop through effort and can be improved by incorporating feedback (e.g., Hong, Chiu, Dweck, Lin and Wan (1999) and Yeager and Dweck (2012)).

Current Status of Growth Mindset of Intelligence Research

A recently published set of meta-analyses by Sisk, Burgoyne, Sun, Butler, and Macnamara (2018) of growth mindsets explored the correlations between mindsets and academic achievement to estimate the average correlation and potential moderators (e.g., developmental stage and academic risk status) to shed light on individual differences. The average correlation between growth mindset and academic achievement was weak (r = .10), reflecting findings from a prior meta-analysis (Burnette et al., 2013). The effect was not significantly moderated by the academic risk status or socioeconomic status, but the developmental stage was a moderator (i.e., r = .19 for children). Sisk et al. noted the high degree of heterogeneity of effects. Beyond the importance of the findings presented in terms of effect sizes, there are at least two aspects of the meta-analyses that have bearing on the state of growth mindset research. First, of the 273 effect sizes that were used for the first meta-analysis focused on the study of individual differences, only 18% of the sample stemmed from studies focused on elementary-aged students. Petscher, Al Otaiba, Wanzek, Rivas, and Jones (2017) summarized the state of growth mindset research for elementary students as minimal and noted that the corpus of existing studies for this population frequently lacks (a) normative, standardized outcomes; (b) the formation of latent factors of mindset and outcomes; and (c) outcomes such as reading and language that are key to developmental skill acquisition for children at this age level. Second, Sisk et al. (2018) did not evaluate the type of mindset measures used in the reviewed studies, and specifically noted that they found few studies reporting reliability estimates, instead assuming a reliability of .80, citing Dweck et al. (1995) as a resource for this level. They did note that most studies reviewed used Likert scales to ask the extent to which participants agreed or disagreed with statements about the degree to which intelligence is malleable. However, it is notable that for some studies, mindset was measured only by asking one to three items; in contrast, one study (Blackwell, Trzesniewski & Dweck, 2007) used eight items that comprise the Mindset Assessment Profile (MAP).

In addition to underscoring the lack of consistent measurement for mindset broadly, it is critically important to note the lack of psychometric data for the MAP, which is needed to inform work to understand the relations between mindset and important academic outcomes, and specifically to understand these relations for children. For example, at the fourth grade, when children are expected to read to learn and apply reading to a variety of genres in content area reading, there might be a relatively higher association between mindset and reading. An understanding of the relation of mindset to reading could inform reading intervention and could also inform how to group students for instruction. In one preliminary study of the MAP in a sample of fourth-grade students, Petscher et al. (2017) used the MAP (described in the measures section) and a researcher-developed tool of domain-specific growth mindset related to reading. A bifactor model that included a general factor called global mindset (i.e., a factor indicated by the MAP items and the reading mindset items) as well as specific factors of general mindset (MAP items only) and reading mindset (reading mindset items only) provided the best fit to the data compared to a single-factor model or a correlated trait model. The confluence of MAP factors was significantly related to end-of-year reading comprehension above and beyond students’ initial word reading, and explained 15% unique variance in reading comprehension above the word reading ability.

Why additional research is needed on the MAP is four-fold. First, the MAP tool is a key component of the Mindset Works® program that purports to “lay the foundation with growing early mindsets and SEL competencies” and provides a “blended learning approach [that] allows time for classroom discussions, reflection, and application of mindset principles” in schools (Mindset Works, 2019). Second, robust estimates of reliability and validity are not available on scores from the assessment. Moreover, total scores on the eight-item MAP result in assignment to one of ten profiles, five profiles that are associated with fixed mindset (called F1-F5) and five profiles that are associated with growth mindset (called G1-G5) with nuanced heterogeneity across each. See Appendix A for a description of the profile groupings from Mindset Works (2019).

To date, there appears to be no empirical support for the separation of profiles and whether individuals in the profiles are meaningfully distinguished on important, educationally relevant outcomes. Third, as districts around the United States adopt the Mindset Works® program or advocate for growth mindset measurement (e.g., Chartiers Valley School District, Pennsylvania; Jefferson County School District, OR; Manhattan Beach Unified School District, CA; Mineola Union Free School District, NY; and San Juan Unified District, CA), psychometric integrity is critical to ensure that scores are reliable and valid, and demonstrates an empirical capability of classifying students into mutually orthogonal groups. Fourth, given the potential links among mindset, emotional well-being, anxiety, and educational outcomes, it is critical to understand how scores from a widely used assessment can and should be used for understanding individual differences as well as in thinking about clinical applications of scores. Therefore, we proposed three research aims in this study to extend and contribute uniquely to the literature. First, we described the underlying latent structure of scores from the MAP tool. Second, we explored whether subgroups of students exist based on the latent structure of MAP scores. Third, we explored whether the subgroups were differentiated on standardized measures of reading comprehension, vocabulary, and word reading.

Method

Participants

Participants were 431 fourth-grade students from the south and southeast regions of the United States. Female students made up 51% of the sample. With regard to ethnicity, 26% of the sample were identified as Hispanic. The racial composition of the sample was 51% black, 33% white, 7% Native American, 4% Asian, and 5% not reported. The vast majority of the sample were considered low income (82%), 42% were English learners, and 7% were identified as having a disability.

Measures

MAP (Mindset Works, 2019). The MAP is an eight-item assessment from the student mindset survey that measures students’ level of fixed or growth mindset. Students respond to items on a scale of 1–6 based on their self-beliefs of intelligence including “No matter how much intelligence you have, you can always change it a good amount (Item 1),” “You can learn new things, but you cannot really change your basic intelligence (Item 2),” “I like school work best when it makes me work hard (Item 3),” “I like school work best when I can do it really well without too much trouble (Item 4),” “I like school work that I will learn from even if I make a lot of mistakes (Item 5),” “I like school work best when I can do it perfectly without any mistakes (Item 6),” “When something is hard, it just makes me want to work more on it, not less (Item 7),” and “To tell the truth, when I work hard at my schoolwork, it makes me feel like I am not very smart (Item 8).” Total scores from the MAP are then used to classify students into 1 of 10 MAPs. No reliability or validity is reported for this assessment.

Woodcock–Johnson III Tests of Achievement (WJ-III; Woodcock, Mcgrew, & Mather, 2001). Students’ word reading ability was assessed via the individually administered letter-word identification (LWID) subtest. LWID requires students to name individual letters and read real words presented. Test–retest reliabilities for the subtest are .81 for the fourth grade.

Gates–MacGinitie Reading Test (GMRT; MacGinitie, MacGinitie, Maria, Dreyer, & Hughes, 2006). The GMRT is a group-administered, norm-referenced test. The reading comprehension and vocabulary subtests were administered. During the reading comprehension subtest, students are presented with multiple paragraph-length reading passages and related multiple-choice questions. Passages include both narrative and expository texts. Test–retest reliabilities are above .85; alternate-form reliability is .86 for the fourth-grade level. The vocabulary subtest presents words in context, and the student chooses the correct meaning of the target word. Test–retest reliabilities are above .85. Construct validity estimates range from .79 to .81.

Procedure

Fourth-grade students at the 10 schools were administered the reading comprehension subtest of the GMRT during the fourth or fifth week of school. Over three and half weeks, trained research staff administered the reading assessments to students in a quiet area of their schools. Members of the research team read standardized instructions aloud to the students, then read each item aloud as the students read along silently. The first set of items consisted of practice prompts designed to ensure that participants understood the survey format and response patterns. In case any students had struggled with the online format or use of the mouse or keyboard, teams were provided an identical paper version of the survey. Participants took approximately 10 minutes to complete the survey, and no students elected to or were identified as needing the paper version. Participants were administered assessments in a counter-balanced order at the end of September and beginning of October during a class period.

Data Analysis

Confirmatory factor analysis (CFA) initially tested the viability of the MAP items as a unidimensional model. The current version of the MAP does not delineate among the items as subscales; thus, a one-factor model (Model 1) served as a baseline configuration for estimation. A review of the items suggested that question 1 from the assessment was worded positively for growth mindset (i.e., “No matter how much intelligence you have, you can always change it a good deal”), question 2 was worded for fixed mindset (i.e., “You can learn new things, but you cannot really change your basic level of intelligence”), and the remaining six items were worded in such a way to reflect on effort perceptions (e.g., Item 3 = “I like my work best when it makes me think hard”). One alternative factor model included a two-factor model with questions 1 and 2 loading on a factor reflective of mindset, and the other items loading on a perceived effort factor (Model 2). A third model was a three-factor model where each of questions 1 and 2 were set as single-item indicators of separate growth and fixed mindset constructs, respectively, and questions 3–8 loading on the perceived effort construct (Model 3). Last, a bifactor model was specified, where the questions loaded on the specific constructs of Model 3 as well as a global factor that was indicated by all questions (Model 4). The comparative fit index (CFI) and Tucker–Lewis index (TLI) were used to evaluate incremental fit with values of at least .95 commensurate with acceptable fit. The root mean square error of approximation (RMSEA) was also used for judging acceptable model fit with values <.05 frequently used as an optimal target.

Exploratory factor mixture models (E-FMMs; Muthen, 2008) estimated the number of subgroups that best reflected categorical individual differences on the latent constructs. E-FMM can be used to model distinctions in latent variables as opposed to continua (Borsboom, Rhemtulla, Cramer, van der Maas, Scheffer, & Dolan, 2016) and is useful in the psychometric process to evaluate the presence of types in developed measures. Clark, Muthén, Kaprio, D'Onofrio, Viken, & Rose (2013) offered a sequence of five steps to build E-FMM with the broad goal as selecting the best factor mixture model in light of the best factor analytic model. Our modeling process began with identifying the best factor model through the CFA. We then estimated 2–6 classes assuming measurement invariance to facilitate a consistent interpretation of factors across classes (Lubke & Muthen, 2005). Akaike information criterion (AIC), Bayesian information criterion (BIC), the Vuong–Lo–Mendell–Rubin (VLMR), and the adjusted Lo–Mendell–Rubin (LMR) ratio tests were used to judge whether model c or model c-1 provides better fit. A nonsignificant p-value for the VLMR and LMR is indicative that model c does not significantly improve upon the fit of model c-1, and lower AIC and BIC models reflect better fitting solutions. Entropy was used as a measure of model usefulness with values closer to 1.0 representing greater model usefulness. Auxiliary regression analysis for the final E-FMM tested group differences among the c profiles on the distal outcomes of GMRT reading comprehension and vocabulary as well as the WJ-III LWID task. Hedges’ g was used to compute the standardized effect size difference between pairwise c profiles for each of the three distal outcomes. Items were treated as categorical in the estimation of the latent constructs using the weighted least squares multivariate estimator. Mplus 8 software was used for all latent variable models.

Results

Descriptive Statistics

One data point (.2%) was missing on the WJ-III LWID task and was handled through the estimators used in the latent variable analyses. Descriptive statistics and correlations are reported in Table 1. The literacy measures were moderately correlated with each other, but the MAP total score correlated weakly with each of the GMRT reading comprehension (r = .20) and vocabulary (r = .21) scores as well as the WJ-III LWID task (r = .17). A quantile regression (Koenker & Bassett, 1978; Petscher, 2016) of z-scored versions of the MAP and each of the three literacy measures are shown in Figure 1 to illustrate the extent to which the relations of MAP with each of the literacy measures were heteroscedastic according to varying conditional levels of the MAP total score. All three figures show that the relation between the MAP total score and the measures maintained a stronger association with MAP when the conditional MAP total score was low. For example, the plot of MAP and GMRT reading comprehension (upper left) shows that along the x-axis of the plot (i.e., conditional quantile of MAP total score), as the values move from .20 to .80, the y-axis values get systematically smaller. This finding means that at low conditional levels of MAP total score (e.g., .20 quantile), poor reading comprehension was related to low MAP. Low scores on the MAP are indicative of greater fixed mindset; thus, the three quantile correlation plots suggest that at lower total scores of the MAP and therefore higher levels of fixed mindset, low literacy skills were related to one’s fixed mindset. Conversely, at higher levels of MAP total scores, and thus more growth mindset, literacy skills were less related to one’s growth mindset.

Table 1.

Descriptive Statistics and Correlations for Observed Measures.

	MAP total	Reading comp.	Vocabulary	LWID
MAP total	1.00
Reading comp	.20	1.00
Vocabulary	.21	.68	1.00
LWID	.17	.54	.66	1.00
N	431	431	431	430
Mean	30.57	440.14	442.03	479.60
SD	6.07	35.14	32.54	24.94

Note. MAP = Mindset Assessment Profile tool; reading comp = GMRT—reading comprehension; vocab = GMRT—vocabulary; LWID = WJ-III letter-word identification; GMRT = Gates–MacGinitie Reading Test.

Figure 1.

Quantile estimates of the conditional relation between GMRT reading comprehension (upper left), GMRT vocabulary (upper right), and WJ-III letter-word identification (lower center) with MAP total scores.

Confirmatory Factor Analysis

The one-factor model (Model 1; Table 2) did not provide good fit to the data: χ²(14) = 49.56, RMSEA = .077, CFI = .89, and TLI = .83. Model 2 did not converge to a solution, and upon further review of the indicators, it was clear that using Items 1 and 2 on a specific mindset factor was problematic due to both items measuring something the opposite of the other. Thus, the loading values for one item were extremely large and the other very close to 0. Model 3 separated out Items 1 and 2 to be single-item indicators on respective factors of growth and fixed mindset. To do so, the loading for each item was set to 1.0, and the factor variance for these items was fixed to (1 − α) × σ², where α is the reliability of the assessment and σ² is the sample variance. The estimated reliability for the sample on the eight MAP items was α = .35, and respective sample variances for Items 1 and 2 were 2.89 and 3.92. Thus, the factor variances for the single-item indicator factors were set to 1.87 and 2.55, respectively, for the growth and fixed mindset constructs. The fit for three-factor model in Model 3 was improved compared to what was observed for Model 1 but still did not confirm to acceptable incremental fit: χ²(15) = 35.65, RMSEA = .057, CFI = .93, and TLI = .90. The bifactor model (Model 4) did not converge to a solution. Upon review of the model revision considerations in Model 3 and the theta matrix considerations for Model 4, a residual covariance between Items 2 and 8 was suggested. The revised solution with the residual covariance converged (Model 5): χ²(10) = 13.61, RMSEA = .029, CFI = .99, and TLI = .98. Models 1–3 were reestimated adding in the residual covariance, and the best fitting models were the three-factor model with the residual covariance: χ²(14) = 18.63, RMSEA = .028, CFI = .98, TLI = .98, and the bifactor model with the residual covariance. Due to the theoretical simplicity and measurement parsimony for a three-factor solution with single-item indicators, we opted to use the three-factor solution (Figure 2, top). The latent correlation between growth and fixed mindset was effectively 0 due to nonsignificance (ψ = .29, p = .354). The correlation between latent perceived effort and fixed mindset was negative (ψ = −.22, p < .05), and the correlation between latent perceived effort and growth mindset was positive (ψ = .42, p < .001).

Table 2.

Confirmatory Factor Analysis Model Fit Indices.

Model	RMSEA
Model	χ²	df	Estimate	LB	UB	p-value	CFI	TLI
Model 1	49.56	14	.077	.054	.100	.027	.89	.83
Model 2	—	—	—	—	—	—	—	—
Model 3	35.65	15	.057	.033	.081	.297	.93	.90
Model 4	—	—	—	—	—	—	—	—
Model 5	13.61	10	.029	.000	.064	.814	.99	.98
Model 6	24.65	13	.046	.016	.073	.566	.96	.94
Model 7	—	—	—	—	—	—	—	—
Model 8	18.63	14	.028	.000	.058	.877	.98	.98

Note. Model 1 = one-factor model; Model 2= two-factor model; Model 3 = three-factor model; Model 4 = three-factor, bifactor model; Model 5 = Model 4 + residual covariance; Model 6 = Model 1 + residual covariance; Model 7 = Model 2 + residual covariance; Model 8 = Model 3 + residual covariance. Models 2, 4, and 7 did not converge to a solution. RMSEA = root mean square error of approximation; LB = lower bound estimate for the 90% confidence interval of the RMSEA; UB = upper bound estimate for the 90% confidence interval of the RMSEA; CFI = comparative fit index; TLI = Tucker–Lewis index.

Figure 2.

Mindset Assessment Profile three-factor model result (top) and factor mixture model with auxiliary regression (bottom).

Exploratory Factor Mixture Models

A representation of the E-FMM (Figure 2; bottom) shows the two-stage approach used in both estimating the number of appropriate classes and then the auxiliary predictions to the distal outcomes. The c variable represents a categorical factor that is indicated by the three latent factors estimated from the CFA. By testing a series of E-FMM, in our case 2–6 class models, and comparing the indices described in the method’s section, means of the first-order CFA factors across levels of c can be estimated and compared. E-FMMs estimate solutions whereby the last cluster of subjects is a referent condition. The estimated means for the classes in the final E-FMM are deviations from the reference class in that solution where the factor means are set at 0. Fit indices for the various solutions are reported in Table 3. The five-class solution provided the optimal balance among the AIC and nBIC compared to other solutions, and the entropy statistic (.93) suggested good model usefulness. The VLMR (p = .123) and the LMR (p = .129) for the six-class solution were both nonsignificant pointing to a c-1 model (i.e., five class) as fitting statistically better. Factor means for the five-class result are shown in Figure 3, where Class 5 is the referent condition. In order to provide an approximation of context for Class 5 as a referent, we include the raw score means for growth and fixed mindset, and effort, in Table 5.

Table 3.

Factor Mixture Model Fit Results for 2- to 6-Class Solutions.

Class model	Log likelihood	AIC	nBIC	Entropy	VLMR	LMR
2-Class	−4758.00	9601.67	9639.16	.95	<.001	<.001
3-Class	−4716.59	9525.17	9556.23	.95	<.001	<.001
4-Class	−4627.33	9354.65	9399.29	.95	<.001	<.001
5-Class	−4596.31	9298.63	9345.94	.93	<.001	<.001
6-Class	−4590.47	9294.95	9345.83	.92	.123	.129

Note. AIC = Akaike information criterion; nBIC = sample-adjusted Bayes information criterion; VLMR = Vuong–Lo–Mendell–Rubin likelihood ratio test; LMR = Lo–Mendell–Rubin adjusted likelihood ratio test.

Figure 3.

Factor mixture model Mindset Assessment Profile means for Class 1 (fixed mindset, high effort; n = 44), Class 2 (growth mindset with fixed ideas, high effort; n = 152), Class 3 (growth mindset, high effort; n =148), Class 4 (low mindset, average effort; n = 33), and Class 5 (referent, n = 54).

Class 1 (fixed mindset, high effort; n = 44) is similar to the F4-F5 MAP profile (Appendix A) and was characterized by relatively higher levels of fixed mindset, very low growth mindset, and above average levels of perceived effort compared to Class 5. Class 2 (growth mindset with fixed ideas, high effort; n = 152) was similar to the F1-G1 MAP profile and had relatively higher levels of both growth and fixed mindset and effort compared to Class 5. Class 3 (growth mindset, high effort; n =148) was similar to the G4-G5 MAP profile and had high levels of growth mindset, very low levels of fixed mindset, and above average effort. Class 4 (low mindset, average effort; n = 33) was not aligned with any MAP profile and presented with very low levels of both growth and fixed mindset but average levels of effort, and Class 5 (referent; n = 54) also appeared to not be aligned to any MAP profile based on the total scores from Table 4. This class maintained equal effort as Class 4 but lower than Classes 1, 2, and 3; as well, they had higher levels of growth mindset than Classes 1 and 4 but lower than Class 3, and lower levels of fixed mindset compared to Class 1 and 2 but higher than Classes 3 and 4.

Table 4.

Latent MAP Assessment Profile Means of Auxiliary Variables and MAP Total Scores.

		GMRT		GMRT		WJIII		Growth mindset		Fixed mindset		Effort
		Reading comp.		Vocabulary		LWID		Raw total		Raw total		Raw total
Class	N	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD
Class 1	44	437.29	33.39	440.70	31.08	483.05	26.18	3.68	1.57	5.56	1.65	29.86	3.95
Class 2	152	431.51	30.53	433.72	28.42	476.08	20.68	3.17	1.71	5.24	1.76	23.20	3.19
Class 3	148	451.98	38.88	454.37	34.74	483.66	28.24	3.51	1.89	5.61	1.74	21.45	3.93
Class 4	33	444.24	26.96	445.80	26.67	483.92	23.77	2.91	1.70	5.15	1.82	25.64	3.21
Class 5	54	431.59	40.20	430.00	36.20	472.62	30.55	4.06	1.34	5.78	1.66	25.67	3.68

Note. Reading Comp. = GMRT—reading comprehension; Vocabulary = GMRT—Vocabulary; LWID = WJ-III letter-word identification; MAP = Mindset Assessment Profile tool; GMRT = Gates–MacGinitie Reading Test.

Mean performances of the classes on the distal measures are reported in Table 4. Class 3, the growth mindset—high effort profile, had the highest observed reading comprehension (M = 451.98 and SD = 38.88) and vocabulary (M = 454.37 and SD = 34.74) scores, whereas Classes 2 (fixed mindset—higher effort) and 5 (referent) had the lowest observed reading comprehension and vocabulary scores. Classes 1, 3, and 4 were approximately equivalent in their word reading scores and were higher than Classes 2 and 5 in their mean performances. Auxiliary regression results of the E-FMM (Table 5) show that small to moderate standardized effects were observed. Class 3 (growth mindset—high effort) was systematically higher than all other groups in reading comprehension (absolute Hedges’ g range = 0.24–0.59) and vocabulary (absolute Hedges’ g range = 0.28–0.69), with minor differences observed on word reading absolute Hedges’ g range = 0.01–0.38). Class 2 (fixed mindset—high effort) did not outperform any class in terms of standardized differences on reading comprehension, vocabulary, or word reading (g < .20). Class 1 (fixed mindset—lower effort) outperformed Class 2 in both vocabulary (absolute g = 0.23) and word reading (absolute g = 0.30), as well as Class 5 in vocabulary (absolute g = 0.32) and word reading (absolute g = 0.37). Class 4 (inconsistent mindset—average effort) outperformed Class 2 in reading comprehension, vocabulary, and word reading (g = .44 for reading comprehension and vocabulary and g = .35 for word reading) as well as Class 5 (g = .38, .50, .42 in reading comprehension, vocabulary, and word reading, respectively). Class 5 maintained no standardized effect size advantage in the distal outcomes over any group.

Table 5.

Auxiliary Mixture Model Contrasts among Latent MAP Profiles on Reading Comprehension, Vocabulary, and Word Reading Outcomes.

GMRT–Reading Comp.				GMRT–Vocabulary				WJIII–LWID
Contrast	χ²	p-value	Hedges’ g	Contrast	χ²	p-value	Hedges’ g	Contrast	χ²	p-value	Hedges’ g
Class 1 vs. 2	1.07	.302	−.18	Class 1 vs. 2	1.79	.181	−0.23	Class 1 vs. 2	2.64	.104	−.30
Class 1 vs. 3	6.06	.014	.41	Class 1 vs. 3	6.19	.013	.42	Class 1 vs. 3	.02	.894	.02
Class 1 vs. 4	1.01	.316	.23	Class 1 vs. 4	.59	.442	.18	Class 1 vs. 4	.02	.884	0.03
Class 1 vs. 5	0.57	.449	−.16	Class 1 vs. 5	2.44	.118	−0.32	Class 1 vs. 5	3.23	.072	−.37
Class 2 vs. 3	25.56	.000	.59	Class 2 vs. 3	31.45	.000	.65	Class 2 vs. 3	7.02	.008	.31
Class 2 vs. 4	5.70	.017	.44	Class 2 vs. 4	5.39	.020	.44	Class 2 vs. 4	3.06	.080	.35
Class 2 vs. 5	0.00	.989	.00	Class 2 vs. 5	0.45	.501	−.12	Class 2 vs. 5	0.57	.452	−.14
Class 3 vs. 4	1.85	.174	−.24	Class 3 vs. 4	2.46	.117	−.28	Class 3 vs. 4	0.00	.956	.01
Class 3 vs. 5	10.20	.001	−.52	Class 3 vs. 5	18.03	.000	−.69	Class 3 vs. 5	5.22	.022	−.38
Class 4 vs. 5	3.07	.080	−.38	Class 4 vs. 5	5.36	.021	−.50	Class 4 vs. 5	3.75	.053	−.42

Note. Small standardized effect sizes are in bold; moderate standardized effect sizes are bold and italicized. The first class listed in each contrast is the referent such that a positive effect size is in favor of the second class, and a negative effect size reflects mean differences in favor of the first class. p-values in bold are statistically significant after applying Benjamini–Hochberg correction for the false-discovery rate. MAP = Mindset Assessment Profile tool; GMRT = Gates–MacGinitie Reading Test; WJIII-LWID = Woodcock–Johnson III letter-word identification.

Discussion

The results of the present study provide initial evidence of the reliability, factor structure, and estimated profiles for the MAP in elementary school and the relations between empirically estimated profiles and key reading outcomes. Contrary to the current recommendation that the 8-item assessment conforms to a unidimensional construct, findings suggested that a multidimensional structure provided better fit via a three-factor model of growth mindset (MAP Item 1), fixed mindset (MAP Item 2), and effort (MAP Items 3–8) where the mindset factors were reflective of single-item indicators. The growth and fixed mindset factors were not statistically correlated with each other (ψ = .29, p = .354), whereas growth mindset was moderately related to effort (ψ = .42, p < .001), and fixed mindset was weakly related to effort (ψ = −.22, p < .05). That our sample was powered to detect a smaller correlation as significant for fixed mindset effort (−.22) and not a larger correlation (.29) suggests that the point estimate of .29 is more reflective of error (due in part to single-item indicators) than merely a power issue for detecting the effect as significant. The findings here for elementary students largely corroborate what has been found in research on theories of intelligence for adults, namely, that individuals with a fixed mindset have negative perceptions of effort and that individuals with a growth mindset believe intelligence may increase and may do so through higher effort (Blackwell et al., 2007).

This study also reported the first empirical estimation of how well individuals might be classified into groups, with five clusters of individuals extracted. Although the MAP suggests that 10 profiles may be used to classify students, these profiles have not been tested empirically and the relative orthogonality of profile membership from the current version of the assessment is unknown. Findings in this study not only provided strong statistical evidence for the usefulness of a five-profile solution (i.e., entropy = .93), and the salience of choosing five profiles over six profiles (i.e., VLMR p = .123; LMR p = .129) but also the average latent class probabilities for likely class membership were very strong for each of the five profiles (i.e., Profile 1 = .99, Profile 2 = .91, Profile 3 = .94, Profile 4 = .97, and Profile 5 = .97). The resulting profiles were differentiated on standardized word reading, vocabulary, and reading comprehension measures. Individuals who were predicted to be in Class 3 (approximately 33% of the sample), which was characterized by high growth mindset and above average effort, consistently outperformed their peers in all areas of literacy skills, compared to individuals in Classes 1 and 2 that both had higher fixed mindset and lower overall reading comprehension and vocabulary scores. Thus, a relation between the belief that academic skills can develop through effort and be improved over time and reading achievement in the upper elementary grades was noted, consistent with the incremental theory of intelligence (Yeager & Dweck, 2012). Alternatively, profiles with more fixed mindsets were related to lower reading achievement scores.

Together, the findings from the factor analysis and finite mixture model suggest that the MAP should be used differently in elementary school in both how scores are created and how individuals are classified into groups. Because the MAP is used as part of a broader tool set in terms of class-based curriculum on teaching growth mindset, as well as for interventions, it is critical that decisions made about students are driven by sound measurement. In this way, total scores from the MAP should be multidimensional and not unidimensional and the classification of students should reflect the compensatory aspects of fixed and growth mindset along with effort so that effective recommendations and strategies may be provided to teachers and clinicians who use the MAP to guide instruction and interventions. The psychometric and individual difference results set a foundation of better measurement. Improved measurement is critical and foundational to studies of individual differences and causality; thus, findings here may be useful to researchers looking at the role of growth mindset related to key educational outcomes.

This study is limited by factors such as the data being self-reported and importantly that the resulting three-factor structure was inclusive of two factors indicated by single items. Although such representations are statistically allowable and practical (Petrescu, 2013), the findings may speak to the need to expand the item pool on the MAP to more robustly test construct relevant or irrelevant variance. Further, that the correlated three-factor and bifactor solutions provided acceptable incremental fit suggest that the structure may be more complex than what was estimated here, and an increased item pool may bear out whether findings here can be replicated. Future research may wish to not only replicate the findings of this study but also to evaluate differential item functioning and prediction to a wider range of standardized outcomes. Further, it will be important to engage in rigorous testing of the measurement invariance in finite mixture models. Regardless of these limitations and suggestions for future directions, the current research provides new evidence of how the MAP may be used in practice to support individuals’ understanding of growth mindset as part of more robust social–emotional learning in elementary education.

Footnotes

Appendix A

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Institute of Education Sciences (R324A130262). The opinions expressed are those of the authors and do not represent views of the funding agencies or the universities.

ORCID iD

Yaacov Petscher

References

Blackwell

L. S.

Trzesniewski

K. H.

Dweck

C. S.

(2007). Implicit theories of intelligence predict achievement across an adolescent transition: A longitudinal study and an intervention. Child Development, 78(1), 246-263.

Borsboom

Rhemtulla

Cramer

A. O. J.

van der Maas

H. L.

Scheffer

Dolan

C. V.

(2016). Kinds versus continua: A review of psychometric approaches to uncover the structure of psychiatric constructs. Psychological Medicine, 46(8), 1567-1579.

Burnette

J. L.

O'Boyle

E. H.

VanEpps

E. M.

Pollack

J. M.

Finkel

E. J.

(2013). Mind-sets matter: A meta-analytic review of implicit theories and self-regulation. Psychological Bulletin, 139, 655-701.

Chiu

C. Y.

Hong

Y. Y.

Dweck

C. S.

(1997). Lay dispositionism and implicit theories of personality. Journal of Personality and Social Psychology, 73(1), 19.

Clark

S. L.

Muthén

Kaprio

D’Onofrio

B. M.

Viken

Rose

R. J.

(2013). Models and strategies for factor mixture analysis: An example concerning the structure underlying psychological disorders. Structural Equation Modeling: A Multidisciplinary Journal, 20(4), 681-703.

Dweck

C. S.

(2006). Mindset: The new psychology of success. New York: Random House.

Dweck

C. S.

Chiu

C. Y.

Hong

Y. Y.

(1995). Implicit theories and their role in judgments and reactions: A word from two perspectives. Psychological Inquiry, 6(4), 267-285.

Dweck

C. S.

Leggett

E. L.

(1988). A social-cognitive approach to motivation and personality. Psychological Review, 95(2), 256.

Hong

Y. Y.

Chiu

C. Y.

Dweck

C. S.

Lin

D. M. S.

Wan

(1999). Implicit theories, attributions, and coping: A meaning system approach. Journal of Personality and Social Psychology, 77(3), 588.

10.

Koenker

Bassett

Jr. (1978). Regression quantiles. Econometrica: Journal of the Econometric Society, 46, 33-50.

11.

Lubke

G. H.

Muthén

(2005). Investigating population heterogeneity with factor mixture models. Psychological Methods, 10(1), 21.

12.

MacGinitie

W. H.

MacGinitie

R. K.

Maria

Dreyer

L. G.

Hughes

K. E.

(2006). Gates–Macginitie reading tests (4th ed.). Rolling Meadows, IL: Riverside Publishing.

13.

Mindset Works. (2019). The journey to a growth mindset school. Retrieved from https://www.mindsetworks.com/schools/culture

14.

Muthén

(2008). Latent variable hybrids: Overview of old and new models. In Hancock

G. R.

Samuelsen

K. M.

(Eds.), Latent variable mixture models (pp. 1-24). Charlotte, NC: Information Age Publishing, Inc.

15.

Petscher

(2016). Do our means of inquiry match our intentions? Frontiers in Psychology, 7, 1048.

16.

Petscher

Al Otaiba

Wanzek

Rivas

Jones

(2017). The relation between global and specific mindset with reading outcomes for elementary school students. Scientific Studies of Reading, 21(5), 376-391.

17.

Petrescu

(2013). Marketing research using single-item indicators in structural equation models. Journal of Marketing Analytics, 1(2), 99-117.

18.

Sisk

V. F.

Burgoyne

A. P.

Sun

Butler

J. L.

Macnamara

B. N.

(2018). To what extent and under which circumstances are growth mind-sets important to academic achievement? Two meta-analyses. Psychological Science, 29(4), 549-571.

19.

Yeager

D. S.

Dweck

C. S.

(2012). Mindsets that promote resilience: When students believe that personal characteristics can be developed. Educational Psychologist, 47(4), 302-314.

20.

Woodcock

R. W.

McGrew

K. S.

Mather

(2001). Woodcock-Johnson III tests of achievement. Itasca, IL: Riverside Publishing.