Validation and Psychometric Properties of the Original Grit Scale (Grit-O) Among Cambodian Adolescents

Sample and Procedure

The sample consists of 580 Cambodian adolescents (M_age = 15.85 years, SD_age = 2.05, Range_age = 10–22, 63.7% female) living in Siem Reap. Thirty-seven participants (6.8%) did not complete any of the grit items. These participants were retained for the analyses not directly related to the grit items (such as calculating descriptives of other study variables, and sensitivity analyses comparing those with missing grit items to those without) but could not be included in the main analyses; the main analytical sample ultimately consisted of 543 participants.

All of the participants were attending school and were in grades 7–12. Data were collected in 2018 from four schools supported by a local non-governmental organization and two schools supported by the Cambodian government. Several research team members traveled to Cambodia to conduct the survey. The teachers did not play a role in distributing the questionnaires, rather they assisted the team in identifying potential student participants, aiming to maintain balanced proportions of students within each grade and sex. Before distributing the surveys, a Co-PI of the study, a Cambodian native, explained the purpose of the study to the students, reviewed the consent form, and addressed any concerns they may have had. Subsequently, other research team members distributed the questionnaires while the Co-PI ensured that all participants understood the instructions, and answered any questions they had. The students were given the opportunity to accept or decline participation after learning about study procedures and having the opportunity to ask questions. Students took the paper surveys home from school and returned them to school completed the next day. This paper survey consisted of a battery of instruments regarding the students’ personal experiences, families, education, goals, and feelings.

Prior to the study team’s travel to Cambodia, the entire battery was translated into Khmer, the Cambodian language, by a private translation service, and then assessed for accuracy by a co-investigator, a Cambodian native fluent in both English and Khmer.

Measures

Grit

Grit was assessed using the full Grit Scale (Grit-O; Duckworth et al., 2007). Adolescents rated how well 12 items (see Table 1) described themselves using a scale of 1 (not like me at all) to 5 (very much like me). The full scale will herein be referred to as “Grit-O.” Descriptives of all individual items and interitem correlations are detailed in Table 2. The Consistency of Interest subscale included 6 items such as “I often set a goal but later choose to pursue a different one.” All the items in this subscale were reverse-scored such that higher scores reflected greater consistency of interests. This subscale will be referred to as “Grit-C.” The Perseverance of Effort subscale included 6 items such as “I have achieved a goal that took years of work.” Higher scores reflected greater perseverance of effort. This subscale will be referred to as “Grit-P.” Mean values were calculated for Grit-O and both subscales, respectively (see Table 3). Validity and reliability will be assessed for Grit-O, Grit-P, and Grit-C, separately from one another. This is because there is some uncertainty about the factor structure of the scale and because prior work has been inconsistent in whether to employ only the full scale or the subscales separately from each other (see examples listed in a meta-analysis by Credé et al., 2017).

OPEN IN VIEWER

Table 1.

Items in the Grit-O Scale.

Grit-C: Consistency of Interests (Passion) Subscale
1. I often set a goal but later choose to pursue a different one†
2. I have been obsessed with a certain idea or project for a short time but later lost interest†
3. I have difficulty maintaining my focus on projects that take more than a few months to complete†
4. New ideas and projects sometimes distract me from previous ones†
5. My interests change from year to year†
6. I become interested in new pursuits every few months†
Grit-P: Perseverance of Effort Subscale
7. I finish whatever I begin
8. Setbacks don’t discourage me
9. I am diligent
10. I am a hard worker
11. I have achieved goals that took years of work
12. I have overcome setbacks to conquer an important challenge

Note. † Denotes items which were reverse-scored.

OPEN IN VIEWER

Table 2.

Descriptives and Interitem Correlations for Grit Items.

	Descriptives				Interitem Correlations
	Mean	SD	Range	Missing N (%)	1	2	3	4	5	6	7	8	9	10	11
1.	3.26	1.07	1–5	44 (7.6%)	1.00
2.	3.22	1.02	1–5	40 (6.9%)	.58**
3.	3.11	1.05	1–5	42 (7.2%)	.47**	.49**
4.	2.83	1.03	1–5	49 (8.5%)	.39**	.39**	.39**
5.	2.84	1.19	1–5	52 (9.0%)	.30**	.26**	.24**	.37**
6.	2.78	1.09	1–5	47 (8.1%)	.12*	.20**	.18**	.26**	.33**
7.	3.09	1.23	1–5	55 (9.5%)	−.16**	−.23**	−.25**	−.13**	−.24**	−.17**
8.	4.00	1.02	1–5	46 (7.9%)	.01	−.02	−.03	−.06	−.03	−.16**	.13**
9.	3.57	0.9	1–5	48 (8.3%)	.05	−.01	.00	−.06	−.09*	−.15**	.10*	.19**
10.	3.74	0.86	1–5	42 (7.2%)	.00	.00	−.01	−.06	−.01	−.15**	.08	.31**	.61**
11.	3.66	1.01	1–5	47 (8.1%)	.04	−.03	−.02	−.11*	−.02	−.14**	.17**	.25**	.34**	.43**
12.	3.80	0.98	1–5	42 (7.2%)	.05	−.06	−.04	−.11*	.01	−.20**	.14**	.32**	.37**	.47**	.54**

Note. *p < .05, **p < .01; item numbers match the item numbers detailed in Table 1.

OPEN IN VIEWER

Table 3.

Descriptives of All Study Variables.

	Mean	SD	Range (min–max)	Cronbach’s α
Age	15.85	2.05	10–22	--
Gender	0.64	--	--	--
Grit-O	3.33	0.44	2.08–5	0.56
Grit-C: Consistency of Interests Subscale	3.01	0.72	1–5	0.74
Grit-P: Perseverance of Effort Subscale	3.65	0.63	1.5–5	0.69
Intrinsic academic motivation	5.63	0.91	2–7	0.74
EPOCH Perseverance	3.78	0.59	2–5	0.55
Grit-10 (excluding items 6 and 7)	3.40	0.50	2.1–5	0.65
Subscales identified during exploratory factor analysis
Reduced Grit-C (excluding item 6)	3.05	0.77	1–5	0.75
Reduced Grit-P (excluding item 7)	3.75	0.68	1.6–5	0.75

Analysis Plan

All statistical analyses will be conducted using the statistical software R. To assess the randomness of missing data, a sensitivity analysis will be run comparing participants with and without any missing data across the 12 Grit-O items. T-tests will compare mean differences of age, Grit-O, the Grit-C subscale, Grit-P subscale, intrinsic academic motivation, and EPOCH perseverance. A chi-squared test will compare differences in missingness by proportions of gender.

To assess the reliability of Grit-O and its subscales (Research Question 1), Cronbach’s alphas will be calculated. An alpha of 0.7 will be treated as the threshold of acceptable reliability. Pairwise inter-item correlations will also be calculated for the Grit-O items; reliability will be exhibited if the items in the scale are generally significantly and positively correlated with one another.

To assess the validity of Grit-O and its subscales (Research Question 2), Pearson’s correlations will be calculated to assess how academic motivation and EPOCH perseverance are associated with the scale and subscales. A positive and significant association with these scales will support concurrent validity. It is important to note that, traditionally, concurrent validity is assessed by comparing an unvalidated scale to a separate scale that has been previously validated for that particular population. However, because social science research has been quite limited among Cambodian adolescents, the majority of psychometric scales—including the Academic Motivation and EPOCH scales—have not previously been employed among this population. Assessing validity in this study relies on the assumption that Grit-O will operate similarly for Cambodian adolescents as it has among other adolescent samples, as described in the introduction. Findings with regard to concurrent validity should be interpreted with caution.

The third research question is concerned with whether the factor loadings of Grit-O will support the two intended subscales of Grit-C (Consistency of Interests) and Grit-P (Perseverance of Effort), or whether there will be different factors for a Cambodian sample. First, a confirmatory factory analysis will assess the two hypothesized subscales. Model fit will be assessed using the comparative fit index (CFI), the Tucker–Lewis Index (TLI), and root mean square error of approximation (RMSEA). Appropriate model fit will be indicated by large values of CFI and TLI (approximately equal to or greater than .95) and close-fitting RMSEA (approximately equal to or less than .06), as suggested by Hu and Bentler (1999). Given the relatively large sample size of over 500, chi-square is overly sensitive and will be included only for between-model comparisons via chi-square difference tests (Meade et al., 2008).

In the event that the confirmatory factor analysis exhibits insufficient model fit, the sample will be randomly divided into two equal halves, in preparation for an exploratory factor analysis, and subsequent confirmatory factor analysis. An exploratory factor analysis will then be performed on the 12 items in the Grit-O scale, using only half of the available sample. A parallel analysis scree plot will be used to identify the ideal number of factors to extract. Following an identification of the number of factors, a promax oblique factor rotation will be used to assign items to the factors onto which they load most strongly. If any items load insufficiently (<.30) on all factors, the exploratory factor analysis will be repeated while excluding those particular items to re-determine the ideal number of factors. This particular threshold (<.30) is recommended by Costello and Osborne (2005) and is indicative of a less-than-moderate factor loading for that particular item (Tavakol & Wetzel, 2020).

After this exploratory factor analysis, a confirmatory factor analysis will be run to assess the efficacy of the suggested factors, using the other half of the sample. In the event that any items loaded strongly onto multiple factors during the initial exploratory factor analysis, these items will be assigned only to the factor onto which loaded most strongly. Suggestions from modification indices—including allowing cross-loadings across factors—will then be followed until a model with appropriate fit is reached. If the factors in the final model feature item loadings which deviate from the two intended subscales, the reliability and validity of the new factors will be assessed. Both the confirmatory and exploratory factor analyses will use listwise deletion to account for missing data.

Descriptives and Missing Data

The means of all 12 Grit-O items fell between 2.78 and 4, and there was relatively little missing data (see Table 2). Descriptives of all study variables are detailed in Table 3. Correlations between all the study variables are also detailed in Table 2.

Sensitivity Analysis Regarding Missing Data

T-tests and a chi-squared test were conducted to assess differences between those participants without any missing data across the 12 items in the grit scale (N = 472, 81.4%) and those with any missing data across the 12 items (N = 108, 18.6%). Differences were assessed for the variables of age (t (129.63) = 1.44, p = 0.15), gender (χ ² (1) = 0.24, p = 0.63), intrinsic academic motivation (t (108.02) = 0.72, p = 0.48), EPOCH perseverance (t (109.21) = 0.25, p = 0.80), Grit-O (t (88.37) = 0.98, p = 0.33), Grit-C (t (86.96) = 0.39, p = 0.70), and Grit-P (t (87.78) = 1.12, p = 0.27). Inherently, these latter three comparisons excluded participants with complete missing data across all Grit-O items. No study variables had statistically significant group differences, suggesting missingness in the grit items was random.

Reliability of Grit-O and Subscales

The Cronbach’s alpha for Grit-O (0.56) fell short of the generally accepted cutoff of 0.70 as an indicator of acceptable reliability, but the Grit-C (0.74) and Grit-P subscales (0.69) approached or surpassed this cutoff. These findings indicate that while the full Grit-O may not be reliable for a sample of Cambodian youth, its subscales may be more reliable when considered separately from one another.

Pairwise correlations among the 12 items in the full scale are detailed in Table 2. The first six items, belonging to Grit-C, were all significantly positively correlated with one another, and the latter six items, belonging to Grit-P, were nearly all significantly positively correlated. Unexpectedly, there were multiple significant negative pairwise correlations between items in the first half of the scale (belonging to Grit-C) and items in the second half of the scale (belonging to Grit-P). Specifically, items 6 and 7 consistently negatively correlated with items belonging to the opposing subscales. The patterns in the pairwise correlations indicate that while the full scale may not be particularly reliable, the two subscales are more reliable when treated separately from one another.

Validity

Correlations among intrinsic academic motivation, EPOCH perseverance, and the grit variables are detailed in Table 4. Findings of note include that Grit-O was moderately, positively, and significantly correlated with both covariates. Unexpectedly, the Grit-C subscale was negatively correlated with both intrinsic academic motivation and the EPOCH perseverance scale. The Grit-P subscale was significantly and positively related to both intrinsic academic motivation and the EPOCH perseverance scale. All of the correlations with the subscales were weak to moderate in strength. These findings cast doubt on the validity of the Grit-C (Consistency of Interests) subscale.

OPEN IN VIEWER

Table 4.

Pearson’s Correlations Between Covariates and Grit Variables.

	Intrinsic academic motivation	EPOCH Perseverance
Grit-O	.13**	.29***
Grit-C Subscale	−.15***	−.14***
Grit-P Subscale	.35***	.56***
Grit-10 (excluding items 6 and 7)	.15***	.29***
Subscales identified during exploratory factor analysis
Reduced Grit-C (excluding item 6)	−.12**	−.12*
Reduced Grit-P (excluding item 7)	.35***	.55***

Reliability and Validity When Excluding Items 6 and 7

Because the pairwise correlations indicated that items 6 and 7 did not correlate positively with items in the converse subscale, an exploratory analysis was performed assessing the validity and reliability of Grit-O when these two items were excluded. This reduced scale will be referred to “Grit-10.” Cronbach’s alpha was 0.65 for Grit-10, which is a slight improvement from the Cronbach’s alpha for Grit-O (0.56; see Table 3). Regarding validity, Grit-10 was moderately, positively, and significantly correlated with both covariates and was very similar to the findings for the full scale (see Table 4). These findings suggest that Grit-10, excluding items 6 and 7, is more reliable than the full version of Grit-O.

Confirmatory Factor Analysis

The confirmatory and exploratory factor analyses were conducted using a subsample (N = 472; 81.4%) of participants who responded to all 12 items in Grit-O. A confirmatory factor analysis was run to assess whether the two hypothesized factors were supported by the data. The first variable of each factor was treated as the marker variable, all item and factor variances were freely estimated, and the two factors were allowed to covary with one another. Because this initial model did not exhibit a good fit to the data (CFI = 0.855, TLI = .820, RMSEA = .087), we opted to move forward with an exploratory analysis to assess other possible factors within the scale.

Exploratory Factor Analysis for the Full Grit Scale

The exploratory analysis and subsequent confirmatory analysis were each conducted on a randomly generated half (N = 236) of the subsample with no missing data. A parallel analysis scree plot (Figure 1A) was created to visualize the optimal number of factors for the items in the scale, as determined by a parallel analysis of a principal components analysis. This parallel analysis suggested that three factors should be extracted from the items. A maximum likelihood analysis was run, extracting three factors, and employing a promax oblique rotation. The item factor loadings are detailed in Table 5. Importantly, item 7 did not load strongly onto any of the three factors (<.30). For this reason, the principal component analysis and parallel analysis was repeated while excluding this item. Figure 1B illustrates this additional, reduced parallel factor analysis.OPEN IN VIEWER

OPEN IN VIEWER

Table 5.

Item Factor Loadings for the Grit Scale.

	Initial EFA, including all items			Second EFA, excluding item 7			Third EFA, excluding items 6 and 7
Item numbers	F1	F2	F3	F1	F2	F3	F1	F2
1	0.10	0.67	−0.03	0.67	0.19	0.00	0.66	0.16
2	0.02	0.73	0.03	0.75	0.03	0.08	0.75	0.08
3	−0.13	0.70	−0.01	0.71	0.17	−0.18	0.68	−0.06
4	−0.06	0.34	0.25	0.46	−0.19	0.07	0.46	−0.10
5	0.10	0.20	0.56	0.45	−0.07	0.05	0.44	−0.02
6	−0.11	−0.10	0.60	0.16	−0.24	−0.04	-	-
7	0.11	−0.18	−0.20	-	-	-	-	-
8	0.38	−0.01	−0.16	−0.03	0.41	0.08	−0.07	0.42
9	0.67	−0.04	0.03	0.01	0.16	0.51	0.01	0.66
10	0.87	−0.02	0.04	0.05	0.08	0.94	0.04	0.84
11	0.48	0.05	−0.13	0.09	0.68	−0.04	0.01	0.54
12	0.64	0.05	−0.07	0.14	0.77	0.03	0.05	0.67

Note. F: factor; item numbers correspond with numbers in Table 2.

This second parallel analysis again suggested that three factors should be extracted from the items. A second maximum likelihood analysis was run, assessing the efficacy of three factors, and employing a promax oblique rotation. The item factor loadings are detailed in Table 5. In this instance, item 6 did not load strongly onto any of the three factors (<.30), and the component analysis and parallel analysis were again repeated while excluding this additional item.

A third and final parallel analysis, excluding both items 6 and 7, suggested that two factors should be extracted from the items. Figure 1C illustrates this second, reduced parallel factor analysis. A third maximum likelihood analysis was run, assessing the efficacy of two factors, and employing a promax oblique rotation. The item factor loadings are detailed in Table 5. Items 1–5 loaded most strongly onto Factor 1, supporting the intended subscale of Consistency of Interests (Grit-C). Items 8–12 loaded most strongly onto Factor 2, supporting the intended subscale of Perseverance of Effort (Grit-P). These two factors were not significantly correlated with one another (r = −0.1, p = 0.12).

A confirmatory factor analysis was run on the other half of the data to assess whether these two factors, excluding items 6 and 7, were an appropriate fit to the data. The first model assigned items 1–5 to Factor 1 (Consistency of Interests) and items 8–12 to Factor 2 (Perseverance of Effort); the first variable of each factor was treated as the marker variable, and all item and factor variances were freely estimated. This initial model was not a good fit to the data (CFI = 0.913, TLI = .884, RMSEA = .083). Modification indices indicated that allowing items 11 and 12 to covary would improve model fit (MI = 31.50).

A second, final CFA model was run allowing items 11 and 12 to covary. This model exhibited appropriate model fit for the data (CFI = .957, TLI = .942, RMSEA = .059). This model was a significant improvement over the first confirmatory model (χ ² _diff = 29.5, p < .001). The two factors in this final model were not significantly correlated with one another (r = −0.01, p = .93). Final factor loadings reflected the two expected subscales and are illustrated in Figure 2.OPEN IN VIEWER

Newly Identified Subscales: Reliability and Validity

Descriptives of the newly identified subscales (excluding items 6 and 7, respectively) are detailed in Table 3 and are comparable to those of the original subscales.

The Cronbach’s alpha of the reduced Grit-C subscale (excluding item 6; 0.75) was very similar to that of the original subscale (0.74; see Table 3). Pearson’s correlations with covariates were also very similar to the correlations which were observed for the original subscale (see Table 4).

The Cronbach’s alpha of the reduced Grit-P subscale (excluding item 7; 0.75) was an improvement over the Cronbach’s alpha of the original subscale (0.55; see Table 3). Pearson’s correlations with covariates were also very similar to the correlations which were observed for the original subscale (see Table 4).

Reliability

Cronbach’s alphas (see Table 3) indicated that the only subscale that met the threshold of acceptable reliability was Consistency of Interests (Grit-C). Pairwise interitem correlations illustrated that items belonging to the same subscale are more positively and highly related with one another than with items in the other subscale (see Table 2); the number of significant negative pairwise correlations greatly decreased when items 6 and 7 were not considered.

Taken together, these findings shed doubt on the reliability of Grit-O when used in its entirety and suggest that it may be more appropriate to use one or both subscales, separately from one another. These findings also support recent criticism of the Grit-O scale suggesting that the subfactors are related but independent constructs, rather than two factors belonging to a higher order construct (Credé et al., 2017; Karlen et al., 2019).

Validity

Grit-O was positively and significantly related to both intrinsic academic motivation and the EPOCH scale, and this finding changed very little when items 6 and 7 were removed. The Perseverance of Effort subscale (Grit-P) was positively and significantly related to both covariates while the Grit-C subscale was negatively related to both covariates. These findings provide some evidence for the validity of Grit-O as well as the Grit-P subscale but not for the Grit-C subscale.

Factor Analysis and Efficacy of New Factors

A confirmatory factor analysis was conducted on the 12 Grit-O items, followed by an exploratory factor analysis, to assess whether there were different subscales than the expected two, for a sample of Cambodian youth. The final model somewhat supported the proposed subscales. The new Consistency of Interests (reduced Grit-C) subscale excluded item 6 while the new Perseverance of Effort subscale excluded item 7 (reduced Grit-P). Factor loadings and covariances of this final model are illustrated in Figure 2.

Assessments of both the reliability and the validity of the items in these newly identified subscales were roughly the same as the original subscales, with the exception of pairwise interitem correlations, which suggested increased reliability when items 6 and 7 were removed.

Limitations

Limitations of this study include that the translation of the Grit-O scale into Khmer was very literal and did not take into account the context of the Cambodian school experience. For example, three of the twelve items in Grit-O reference “projects” that the participant may have worked on. This phrasing is likely more accessible to American youth than Cambodian youth as American school activities often include projects as part of the curriculum while school assessments in Cambodia are almost exclusively exam-based. For this reason, the meaning of these items may not have been as readily accessible to a Cambodian sample. However, with that in mind, it should be noted that these “project” items performed well in the analyses, and their removal would likely not lead to other empirical solutions. Nonetheless, future work should consider adapting the wording of specific items to be as relevant and accessible as possible.

An additional limitation of this study is its cross-sectional design. A longitudinal design would allow for a test–retest assessment of reliability, as well as assessment of grit as a predictor of future outcomes. Finally, the sample came entirely from Siem Reap, a fairly large, urban city by Cambodian standards (National Institute of Statistics, 2020). It should be noted that only 39.4% of the Cambodian population lives in areas designated as urban (National Institute of Statistics, 2020). Because there are vast differences in the experiences of those living in rural and urban regions of Cambodia (Richards et al., 2021), future work should assess the efficacy of Grit-O when employed with Cambodian youth from rural areas.

A final limitation of this study is that the two scales which were used as covariates to assess the validity of the Grit-O scale and subscales have not been previously validated for use among Cambodian adolescents. Assessing validity in this study relied on the assumption that Grit-O would operate similarly for Cambodian adolescents as it has among other adolescent samples. Findings with regard to concurrent validity should be interpreted with caution.

Conclusion

Investigators who are interested in assessing grit among Cambodian youth should proceed with caution when considering using Grit-O to measure grit. It may be more reliable to instead use the two subscales of Grit-O on their own. Investigators should also consider excluding items 6 and 7 for two particular reasons: 1) These items do not load well onto either of the latent constructs of Consistency of Interests or Perseverance of Effort, and 2) these items consistently negatively correlate with other items in the scale.