Abstract
The Trait Meta-Mood Scale (TMMS) is an emotional intelligence (EI) assessment originally developed for the U.S. population. This scale measures three EI factors—attention, clarity, and repair—to evaluate how an individual perceives one’s own EI skills. Although the TMMS has been adapted for use in several languages and cultures, the structure of the TMMS requires continuous examination across cultures. Specifically, there is a need for stronger validity evidence using confirmatory analyses. This study evaluates the factor structure of the TMMS-Spanish version, known as the TMMS-24, in a sample of students from northern Mexico. Data from high school and college students were used to examine the factor structure of the scale via confirmatory factor analysis. Results support the basis for future cross-cultural research conducted with Hispanic populations within Mexico with the TMMS-24.
The Trait Meta-Mood Scale (TMMS) is an emotional intelligence (EI) assessment originally developed for the U.S. population with items and instructions appearing in the English language only. The 48-item Likert-type scale is designed to evaluate how an individual perceives one’s own EI skills (Salovey, Mayer, Goldman, Turvey, & Palfai, 1995). By clustering such skills on three processes or factors (attention, clarity, and regulation), the TMMS’ theoretical framework describes EI skills as developing similarly to intellectual skills (Mayer & Salovey, 1995). The TMMS is widely used for clinical (Frewen, Dozois, Neufeld, & Lanius, 2012), employment (Piñar & Fernández-Castro, 2011), and developmental (Alegre, 2012) research, as well as making decisions about individuals. In educational settings, it is often used to examine how students’ EI skills are related to their psychological adjustment (Salguero, Palomera, & Fernandez-Berrocal, 2012), self-esteem (Rey, Extremera, & Pena, 2011), and academic achievement (Buenrostro-Guerrero et al., 2012). In college environments, the TMMS has been used to compare EI skills to anhedonic depression (Berembaun, Bredemeier, Thompson, & Boden, 2012), resilience (Smith, Tooley, Christopher, & Kay, 2010), and transformational leadership (Lopez-Zafra, Garcia-Retamero, & Barrios, 2012).
The TMMS Across Languages and Cultures
The TMMS has become an important tool to evaluate self-perceived EI traits. It has been adapted from English to French (Mayer, 1999), Portuguese (Queirós, Fernández-Berrocal, Extremera, Carral, & Queirós, 2005), and Chinese (Li, Yan, Yin, & Wu, 2002), among other languages. Regardless of the language or culture the tool has been adapted for, alternative editions have shown contrasting psychometric findings. The TMMS’ three-factor model, for example, is supported empirically for North American English-speaking samples (Thayer, Rossy, Ruiz-Padial, & Johnsen, 2003), whereas in other English-speaking countries, there also is support for a four-factor model (e.g., Australia; Palmer, Gignac, Bates, & Stough, 2003), suggesting cultural differences beyond differences in languages.
Likewise, the adaptation of the TMMS to diverse environments presents contrasting recommendations about its structure. A three-factor model, for instance, is supported in Germany (Otto, Döring-Seipel, Grebe, & Lantermann, 2001), but items have been eliminated or modified to maintain statistical fit of the model proposed (e.g., Malaysia; Zawawi & Tang, 2009). In such instances, there is a lack of clarity if the items are problematic or if structure requires modification due to language or culture differences. Contradicting evidence resulting from the TMMS adaptation processes justifies additional validity work focused on clearly defining and supporting the structure of the TMMS in a different language or culture.
TMMS in Hispanic Settings
In Spain, the TMMS has been adapted for Spanish-speaking individuals. The tool has resulted in a shorter form (24 items vs. 48 items) known as the TMMS-24 (Fernandez-Berrocal, Extremera, & Ramos, 2004). Despite support for a three-factor model as originally proposed, its structure remains questionable. For example, research conducted in Spain revealed that this shorter edition may be better with a further reduction of items (e.g., from 24 to 22; Martin-Albo, Nuñez, & Leon, 2010).
The TMMS-24 also is widely used in different Spanish-speaking contexts. It is often considered as an adequate EI tool to be used in regions that may be different compared with Spain. The TMMS-24 has been administered to a variety of Hispanic samples living in Argentina (Salvador & Mayoral, 2011), Colombia (Contreras, Barbosa, & Espinosa, 2010), Mexico (Valadez, Perez, & Beltran, 2010), and the United States (Martines, Fernandez-Berrocal, & Extremera, 2006). Because reliability estimates are commonly mistaken for validity evidence (e.g., Argentina: Gugliani & Arias, 2010; Chile: Yañez-Gallecio, Arevalo, Fernandez, Miranda, & Soto, 2008), stronger validity evidence to support the scores on the TMMS-24 contexts outside of Spain is needed (Ceron, Perez-Olmos, & Ibañez, 2011). This study evaluates the factor structure of the TMMS-24 in a culture where it is being widely used, that is Mexico.
TMMS-24 in Mexico
Although Mexican Spanish language has been influenced by other cultures (e.g., American and Zapotec in the northern and southern Mexico, respectively; Butragueño, 2010), the TMMS-24 is used in Mexico to examine how emotional characteristics are related to self-efficacy (Salvador-Ferrer & Morales-Jimenez, 2009), copping (Preciado, Chavez, & Vazquez, 2010), and burnout (Ortiz-Acosta & Beltran-Jimenez, 2011). Despite no evidence of grammatical problems when the TMMS-24 has been used in remote Mexican cities (e.g., northern Monterrey: Paez, Fernandez, Campos, Zubieta, & Casullo, 2006; central Puebla: Salvador-Ferrer & Morales-Jimenez, 2009), only exploratory analyses (exploratory factor analysis; Sanchez, Rodriguez, & Padilla, 2007) and small samples have been used to gather validity evidence for the scores. This data-driven approach coupled with small sample sizes is necessary but insufficient evidence to develop a strong validity argument for using the TMMS-24 scores in Mexican contexts (Mundfrom, Shaw, & Ke, 2005). Thus, strong psychometric examinations to adapt the TMMS-24 to Mexican samples are needed if the scores are used to make individual decisions.
This study examined the internal structure of the TMMS-24 with a Mexican sample to investigate whether the three-factor structure is supported in this culturally different yet Spanish-speaking sample. The primary question that guided this work was “Is the TMMS-24’s three-factor model confirmed when examined in a Mexican context?”
Rationale to Conduct This Study
As the TMMS-24 was originally adapted from North America to Spain, this tool should be examined psychometrically to account for linguistic and cultural differences among different countries even if the language is technically the same (Hambleton, Merenda, & Spielberg, 2005). Without such evidence, (a) the TMMS-24’s properties cannot be assumed to be stable across the Spanish-speaking countries and (b) its use may result in inaccurate results about individuals or groups given the cultural differences between Mexico and Spain (Carretero-Dios & Perez, 2007).
This work will help to ensure fairness of the outcomes obtained in different cultural contexts even in the presence of the same language (Gregoire & Hambleton, 2009). In fact, examining structural properties of the instrument provides information about construct equivalence across cultures (Hambleton, Yu, & Slater, 1999). Ensuring construct equivalence supports the suitability of the scores in the presence of different cultural and linguistic samples, and when scores are being used for decisions. Furthermore, providing internal structure evidence of the scale follows guidelines for international testing (International Test Commission, 2010).
Method
Participants
The sample for this study has similar characteristics (e.g., educational level, age range) reported when the tool was adapted to Spain (Fernandez-Berrocal et al., 2004). Thus, the generalizability or comparability to those results is not hindered. Participants were high school and college students from Monterrey, Mexico (N = 6,105; 61% female) who ranged in age from 15 to 57 years (M = 17, SD = 2.87). The college sample (23.0%) represented a variety of college majors, including arts, philosophy, psychology, physics and mathematics, civil engineering, nursery, and public health programs. The high school students represented (77.0%) were enrolled in bilingual, vocational, or general high school throughout the city yet in the university system. We note this is different compared with the U.S. education system. Given that the Mexican university is the major higher education institution and students from all income levels and locations across Monterrey attend, it is assumed sociodemographic distribution reflected the population of students in Monterrey and previous studies. This may be limited in scope but reflects the population in which the scale is typically employed. The sample size reflects responses from 4% of the total student body in this local university system (150,000) including high school and college students. The second author is from the university where the data were collected and protection of human subjects was ensured. As the data analyzed were transferred to a university in the United States with no identifiable information (i.e., anonymous data), the Human Subjects Board at that U.S. university ruled the study exempt or not needing human subjects board approval.
Instrument
The TMMS-24 is the Spanish version of the TMMS (Fernandez-Berrocal et al., 2004) that measures three EI components: attention (eight items), clarity (eight items), and repair (eight items). This Likert-type scale asks students to rate the extent they agree to each item on a scale of 1 to 5. Examples of the items include “Presto mucha atención a cómo me siento/I pay a lot of attention to how I feel” (attention); “Casi siempre sé cómo me siento/I almost always know exactly how I am feeling” (clarity); and “Aunque me sienta mal, procuro pensar en cosas agradables/No matter how badly I feel, I try to think about pleasant things” (repair). In Spain, TMMS-24’s Cronbach’s alpha internal consistency reliability estimates were above .85 across domains. These outcomes are consistent with other TMMS’ alternative editions. For instance, the Portuguese and the American editions were within the same range (α = .82-.87; Queirós et al., 2005; Salovey et al., 1995.) Because the TMMS-24 is Spanish version previously administered to similar samples in Monterrey (Paez et al., 2006; Sanchez et al., 2007) with no grammatical problems (Martines et al., 2006), items were not piloted because it was deemed the TMMS-24 presents an standard form of Spanish-language that is spoken in Monterrey . If differences are found, a content and cultural review would have to occur.
Procedure
The TMMS-24 and informed consent were electronically posted on the university board system. Both high school and college students received an invitation through their student account email with a link to respond the scale. The invitation was posted during the academic year. Student’s participation was voluntary without incentive. Time to complete the survey was approximately 30 min. Given that responses from students were saved after the scale was answered completely, samples size represents a 100% response. Surveys having incomplete responses were not saved by the university network automatically, a decision made by the personnel of the Mexican university collecting the data to avoid tracking or identifying participants. That is, the electronic network did not collect incomplete surveys.
The cleaned data set was transferred to United States for statistical analysis, which was processed by a psychometric laboratory located at a university. The data set was separated into two random samples. The first sample was used for the first set of analyses including confirmatory factor analysis (CFA). The second data set was used to cross-validate the results from the first analyses to provide stronger validity evidence. Sample 1 (n = 3,053) and Sample 2 (n = 3,052) were essentially equal in size and participant characteristics.
Analyses
We evaluated the TMMS-24 structurally via CFA in a cross-validation framework and examined the internal consistency of the scale scores. Descriptive statistical analyses were conducted on the data as well as checking of assumptions (e.g., multivariate normality) prior to analysis. Two models were examined in Sample 1. These included a one-factor model (general EI score) and a three-factor model (attention, clarity, and repair), as advocated for by the developer of the TMMS. The best fitting model in Sample 1 was cross-validated using Sample 2.
CFA models were estimated using LISREL v. 8.72 (Jöreskog & Sörbom, 2006) using maximum likelihood estimation and a covariance matrix. Model fit was evaluated using a combination of fit indices. The standardized root mean square residual (SRMR) index was reported with a fit criterion of SRMR < 0.08 (Hu & Bentler, 1999). The comparative fit index (CFI; Bentler, 1990) was reported with values above 0.95 suggesting good fit (Hu & Bentler, 1999). Thus, model fit was determined adequate when the CFI was equal to or greater than 0.95 and the SRMR was below 0.08. This combination of fit indices was empirically derived to minimize Type I and Type II errors (Hu & Bentler, 1999). In addition to reviewing fit indices, parameter estimates were inspected to ensure values were within acceptable limits and statistically significant. The chi-square statistic was used to compute the chi-square difference test for comparing nested models for cross-validation purposes as was a change in the CFI that was greater than 0.01 (e.g., French & Finch, 2006). Once the final model was determined, internal consistency reliability was calculated for the scores related to the factors.
Results
A review of the descriptive statistics revealed that normality assumptions were met. In presence of large samples, absolute values greater than 3.0 and 10.0 indicate problematic skew and kurtosis indices, respectively (Kline, 2005). The set of 24 items presented an absolute value smaller than 2.0. Skewness had a mean of −.019 (range from −0.584 to 0.098), and kurtosis had a mean of 1.34 (range from 0.883 to 1.93).
A one-factor model did not meet fit criteria, χ2(249) = 27,190.37, p < .01, CFI = 0.85, SRMR = 0.12. The three-factor model did meet fit criteria, χ2(252) = 3,543.58, p < .01, CFI = 0.97, SRMR = 0.076, although the chi-square was significant. A significant chi-square can be expected when working with large samples (Hooper, Coughlan, & Muller, 2008). Thus, the three-factor model was considered the best fitting model. No other models were identified empirically or theoretically to examine. Internal consistency reliability estimates for the three factors were as follows: attention = .84, clarity = .83, and repair = .84. Correlations among factors suggest the existence of three related but different factors (attention/clarity = .40, attention/repair = .35, clarity/repair = .63). Examining the three-factor model in Sample 2 revealed that fit criteria were met, χ2(249) = 3,640.8, p < .01, CFI = 0.97, SRMR = 0.077. Moreover, pattern and structure coefficients for Sample 1 and Sample 2 were all statistically significant (p < .01) and strong. Reviewing the structure coefficients (i.e., the correlation of the item with the factor) revealed high correlations with items with their respective factor and lower correlations with the factor not intended to be associated with the factor. Table 1 contains parameter estimates.
Pattern and Structure Coefficients for Sample 1 and Sample 2.
Note. All loadings on their respective factor were significant at p < .05. Bold values indicate pattern coefficient.
Cross-Validation
A cross-validation multisample CFA examined whether the factor structure was invariant across the two samples (e.g., Breidenbach & French, 2012). This analysis fit the general form of the model across groups to evaluate fit and then sequentially constrained model parameters to be invariant across the groups. The first step constrained the factor loadings (lambdas), then the errors terms (theta-deltas), and last the factor variances and covariances. Although there were chi-square significant differences between constraints of model parameters for error variances and covariances, there were no differences in the CFI or SRMR. Thus, differences based on the dual criteria were not met. Moreover, the correlations or relationships between the items and the factors (lambdas) were invariant. See Table 2 for model fit. The most constrained model fit as well as the baseline model present a similar outcome (e.g., Model 4: CFI = 0.97, SRMR = 0.08).
Fit indices for a Three-Factor Model Invariance Examination.
Note. CFI = comparative fit index; NNFI = non-normed fit index; SRMR = standardized root mean square residual; BM = baseline model, λ = factor loadings, δ = error, φ = factor correlation.
Chi-square differences for all models are significant at p < .01.
Discussion
Although evidence exists suggesting the TMMS structure may present additional factors (e.g., a four-factor model; Palmer et al., 2003) across languages and cultures, results for the TMMS-24 version studied here supported a three-factor model that was cross-validated in a hold-out sample for a large sample of high school and college students in Monterrey, Mexico. Examination of pattern and structure coefficients supports this conclusion beyond the fit indices. Moreover, correlations among factors suggest the scale measures three related, yet distinct, domains of EI. The magnitude of the correlations between the three factors is aligned with the theoretical model and empirical results that supported the original design of the TMMS for evaluating the three dimensions of EI (Salovey et al., 1995). Thus, our evidence supports the use of the Spain version of the TMMS-24 to evaluate EI models within the area of Monterey, Mexico (Mayer, Caruso, & Salovey, 2000) and more generally to other areas.
We provide evidence that the three-factor model fits well with students residing in Monterrey, Mexico. When compared with outcomes reported from Spain (Salguero, Fernandez-Berrocal, Balluerta, & Artitzeta, 2010), our results suggest the existence of an equivalent structure for the TMMS-24 when examined with persons speaking a similar language but who are culturally different (e.g., Monterrey). Evidence also suggests further adjustments for the TMMS-24 to this specific context may not be needed. However, given cultural differences can exist in other areas of Mexico, continued validation efforts would be needed to claim broader generalizations of the measure to those contexts.
Although this study provides support for future cross-cultural research using the TMMS-24, as research previously conducted to examine differences on EI skills presented on Hispanic populations within the United States and Mexico (Martines et al., 2006), there remains a need to gather additional validity evidence to more fully support scale equivalence. This study has only examined evidence with regard to the internal structure properties of the scale. To evaluate cultural equivalence, there is a need to examine external validity evidence (e.g., associations with other variables) and invariance models from both populations as well, which requires additional data collection. Such future work will assist to support the scores for inferences about groups and individuals regardless of language spoken or cultural background. The combination of psychometric evidence and theoretical perspectives may provide evidence of reliability and validity for the TMMS-24 scores for use in a variety of settings. Specifically, work to adapt the TMMS-24 to other cultures driven by a priori hypotheses of differences is suggested. While these results support the TMMS-24 in Monterrey, more evidence is needed for use across many Mexican regions.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
