Comment: The Ability Model of Emotional Intelligence: Consistency With Intelligence Theory

Scoring Algorithm

Our primary concern with the MSCEIT/EI literature is that poorly understood issues with the proportion scoring (PS) algorithm may have limited evidentiary support for positioning EI within the Cattell–Horn–Carroll (CHC) three-stratum theory of intelligence (Carroll, 1993). In fairness, Mayer et al. (2016) acknowledge competing representations of the MSCEIT factor structure either as a single g loaded factor or as multiple factors with complex structure. However, they do not acknowledge that these competing results reflect the use of two different scoring algorithms for the MSCEIT rating-based scales (PS vs. shape scoring; Legree et al., 2014). We feel that understanding differences between these two scoring algorithms will help scientific consensus emerge regarding the MSCEIT factor structure and will reinvigorate and extend EI measurement and theory.

The conventional MSCEIT PS algorithm is problematic for its rating-based tests, because resultant scores may be influenced by irrelevant sources of variance (Legree et al., 2014). Specifically, an individual will receive inappropriately low scores for rating-based tests if the respondent’s rating scatter (i.e., variance) or elevation (i.e., mean) differs from that of the scoring key even when the “shape” of the respondent’s pattern of ratings is highly correlated with the values in the scoring key.

We have argued that shape scores should be calculated for the MSCEIT rating-based tasks to control scatter and elevation effects (Legree et al., 2014). Shape scores can be computed as the correlation between a respondent’s ratings for a specific task and the mean ratings for those items. Moreover MSCEIT task descriptions are most consistent with the shape of a respondent’s ratings, not their scatter or elevation (Mayer, Salovey, Caruso, & Sitarenios, 2003).

Regression analyses have demonstrated that MSCEIT PS scores represent linear composites of shape, scatter, and elevation for its rating-based tasks, all Rs > .94 (Legree et al., 2014). Furthermore, each of the four MSCEIT branch scores used two scales that were associated with systematically different levels of scatter, elevation, and shape. For example, perceiving branch scores were computed using scale scores that were primarily influenced by elevation effects, while the facilitating branch scores were heavily influenced by scatter effects. These results thoroughly confound interpretations at the branch level: content versus method.

Consistent with CHC three-stratum expectations, factor analyses of MSCEIT shape scores resulted in a single primary factor with a g loading, r = .79, that approximated g loadings for primary factors computed for conventional cognitive domains (Legree et al., 2014). This result is central to vetting EI as a major domain of cognitive ability within the CHC three-stratum theory. This result also reinforces the conclusion that the EI ability model may not function well as the MSCEIT measurement model, although it may serve as a useful framework for conceptualizing EI abilities (Mayer et al., 2016).

While we emphasize the importance of shape to EI measurement theory, we also suggest that better understanding the MSCEIT scatter and elevation metrics may help extend EI conceptualizations. EI is unusual because it may reflect both knowledge and the strength of one’s convictions. Scatter and elevation metrics may provide insight into the strengths of these convictions and enable research into related domains. Finally, we emphasize that much existing MSCEIT data could be reanalyzed using shape, scatter, and elevation metrics to address the issues discussed before.