The Ego Resiliency Scale-Revised: Confirmatory Factor Analysis and Rasch Models

The Ego Resiliency Scale-Revised (ER89-R)

The ER89-R modification resulted from a series of studies that subjected the ER89 to confirmatory factor analysis and reported a two-factor solution. This provided best fit and was stable across a range of samples (see Alessandri et al., 2008; Menesini & Fonzi, 2005; Vecchione, Alessandri, Barbaranelli, & Gerbino, 2010). Explicitly, Alessandri et al. (2008) found that four items possessed psychometrically inadequate properties. Removal of these resulted in the 10-item ER89-R (Alessandri et al., 2008). Structurally, the ER89-R depicts a higher-order model, where ego resiliency, a second-order factor, affects two first-order components (Openness to Life Experiences and Optimal Regulation), which affect responses to scale items. These factors are conceptually consistent with delineations of ego resiliency (Alessandri et al., 2012). Vecchione et al. (2010) found that this structure was stable and invariant from late adolescence to young adulthood. In a subsequent study, Alessandri et al. (2012) using multigroup confirmatory factor analysis demonstrated invariance (i.e., partial configural, metric and scalar) of the ER89-R across samples from Italy, Spain and the United States.

Despite support for the two-factor ER89-R solution, Farkas and Orosz (2015) propose an alternative three-component model using an 11-item version of the ER89. This comprises a hierarchical model with three factors: Active Engagement with the World (AEW), Repertoire of Problem-Solving Strategies (RPSS) and Integrated Performance under Stress (IPS). These components are distinct and demonstrate distinct relationship patterns with other constructs (e.g., subjective well-being, and state and trait anxiety). Based on this outcome, Farkas and Orosz (2015) advocated that resiliency was a double-faced construct, embracing two functions. The first maintains personality, whereas the other is adaptive and adjusts the personality system to the demands of the dynamically changing environment. From this perspective, stability (permeability) is represented by RPSS and IPS, and flexibility (elasticity or plasticity) by RPSS and AEW.

This approach to scale validation reflects classical test theory (CTT), which emphasises reliability (e.g., internal and test-retest) and validity (e.g., the application of factor analysis to establish internal conceptual coherence) (Mills, Young, Nicholas, Pallant, & Tennant, 2009). Moreover, CTT assumes that in the absence of measurement error, tests produce true scores. Error arises and is inevitable because measurement instruments are imperfect. Hence, observed scores differ as a function of both construct difference and measurement error. CTT advocates that this error is random and that the distribution of error is the same for all individuals (Magno, 2009). In contrast, item response theory (IRT or modern test theory) focuses on item scores. This accent is based on the supposition that items possess different levels of difficulty and that individual variations reflect differences in the latent trait or ability observed (Magno, 2009).

The Present Study

Noting that the development and validation of the ER89-R used CTT, the present study examined the instrument by conducting Rasch scaling in conjunction with CTT (i.e., confirmatory factor analysis). This was necessary since researchers have not previously conducted this form of analysis. In addition, a critical limitation of using CTT in previous research to validate the ER89-R is that observed scores of respondents are dependent on the measure (Magno, 2009). Explicitly, if ER89-R items are too challenging to endorse, then respondents’ mean scores can be reduced, and mean scores can be inflated if items are too easy. Without examining if this bias exists, average scores of the ER89-R can be unreliable when utilised in studies. The ER89 and ER89-R are widely used in research and are examined in relation to significant constructs including mental health and psychological well-being (e.g., Edlina, Arif, Nilesh, & Sonia, 2020; Kubo, Sugawara, & Masuyama, 2021; Milioni et al., 2015; Sadziak, Wiliński, & Wieczorek, 2017; Spurr, Walker, Squires, & Redl, 2021; Tsirigotis & Łuczak, 2018). Therefore, assessment of relationships between the ER89-R, and such constructs can be erroneous, which potentially impacts the conclusions formed concerning the role of ego resiliency. Accordingly, validation of the ER89-R using Rasch techniques that overcome this test-dependent limitation is important. Furthermore, Rasch scaling was particularly appropriate because ER89-R items were constructed over two decades ago, and their source of origin is untraceable (Farkas & Orosz, 2015).

Moreover, Alessandri et al. (2008) claim that the most appropriate conceptualisation of the ER89-R includes the factors of Openness to Life Experiences and Optimal Regulation. In practice, this means that the measure contains two distinct (albeit related) unidimensional scales. Combining Rasch with traditional CFA is a recommended approach for examining the validity of a measure (factorial structure, etc.; Lin & Pakpour, 2017) due to the explicit focus on unidimensionality among other benefits, including the ability to estimate person ability and item difficulty separately. Accordingly, CFA will involve testing one-factor against multidimensional (i.e., the higher-order) models. Failure to confirm a unifactorial solution would support Alessandri et al. (2008) and the presence of two distinct factors/scales. Rasch analysis would provide additional robust tests of unidimensionality.

The Rasch model outlines expected item responses if metric level measurement is achieved (Rasch, 1960). This applies for both dichotomous (Rasch, 1960) and polytomous (Andrich, 1978) responses (Pallant & Tennant, 2007). Rasch models accomplish this by assessing the degree to which observed response patterns match expected values using a probabilistic form of Guttman scaling (Guttman, 1950) and fit statistics (Smith, 2000). Within the model, the probability of a respondent confirming an item is defined as a logistic function. This is the relative distance between the item and respondent location on a linear scale. Thus, the Rasch model views responses as the product of the interaction between test taker scores, which reflect latent trait or ability level, and item difficulty. Thus, the higher the ability relative to item difficulty, the higher the probability of a correct item response. Consistent with this, when latent trait location is equal to item difficulty, there is a 0.5% probability of a correct response (Weller et al., 2013). Within this framework, Rasch analyses characterise a curve, which identifies the ability level at which the item maximally discriminates.

Noting the potential of the Rasch measurement model to convert ordinal observations to interval scaled measurement (Wright & Linacre, 1989), another advantage of the approach is the ability to test for differential item functioning (DIF) (item bias) (Tennant et al., 2004). DIF is important because it indicates that group membership (age, gender, etc.) rather than the underlying latent trait or ability level is influencing responses. Hence, DIF occurs when different groups at the same level demonstrate a different probability of item response (Chen & Revicki, 2014). This is problematic since results are no longer representative of the trait or ability, and therefore cannot be compared objectively (Kopf, Zeileis, & Strobl, 2015). Emerging concern with DIF stems from increased interest in IRT as an alternative to classical test theory. Acknowledging these issues, a Rasch scaling of the ER89-R was undertaken.

Respondents

The sample comprised 2009 respondents, (mean age, M) = 39.81 years, SD = 14.71, range = 18–89. There were 549 males (27%), M = 44.52 years, SD = 16.07, range = 18–88; and 1460 females (73%), M = 38.04 years, SD = 13.75, range = 18–89. Respondent recruitment was via Qualtrics, an online multi-channel management platform for data collection.

The researchers requested a sample of UK-based, non-clinical respondents aged 18 years and over. This was the only exclusion criteria. Recruitment focused on the general public to validate the ER89-R for general use within research and sampled a spread of ages. As the emphasis was on obtaining general public responses, data collection was not restricted to any particular demography. Qualtrics obtains data from recruitment panels, which are derived from a pre-arranged pool of individuals who have consented to respond to surveys in research studies. Data accessed from respondent recruitment panels are generally more diverse and far reaching than traditional student samples. These advantages are not detrimental to quality, and are commensurate with traditional samples in terms of demographics and responses to established surveys (Kees, Berry, Burton, & Sheehan, 2017).

Measures

The Ego Resiliency Scale-Revised (ER89-R; Alessandri, et al., 2008) assesses the capacity to modify responses flexibly in response to changing situational demands, especially under emotionally challenging conditions. Items assess resilience both directly (e.g., ‘I quickly get over and recover from being startled’) and indirectly (e.g., ‘I like to take different paths to familiar places’). The ER89-R presents items as statements, and respondents rate agreement via a 4-point Likert-type scale ranging from 1 (does not apply at all) to 4 (applies very strongly). The ER89-R has demonstrated good reliability and validity (Alessandri, et al., 2008), alongside high correlations with the original measure by Block and Kremen (1996).

Procedure and Ethics

Respondents retrieved study materials using a web-link. Preceding item presentation, respondents received general information about the research project. This explained the investigation and presented details about ethics. To advance, respondents provided informed consent. Respondents then supplied demographic information (i.e., age and preferred gender) before progressing to the items. Procedures asked respondents to thoroughly read and answer all questions, work at their own pace and respond openly and honestly. Furthermore, instructions reduced the potential for evaluation, apprehension and social desirability effects by telling respondents that there were no right or wrong responses. Respondents worked through the items at their own pace until they reached the end of the survey, at which point they received the debrief. A typical survey took approximately 10 min to complete. Ethical approval was granted by the Manchester Metropolitan University Faculty of Health, Psychology and Social Care Ethics Committee.

Analysis

The authors employed confirmatory factor (CFA) and Rasch analyses to assess the validity of the ER89-R. CFA (using AMOS27) tested construct validity via four models: three one-factor models and a higher-order model. The one-factor models comprised a total scale model as a test of the original structure (Block & Kremen, 1996), followed by models with respective latent factors of Openness to Life Experiences and Optimal Regulation. The higher-order model was based on Alessandri et al. (2008) and included the two factors of Openness to Life Experiences and Optimal Regulation, alongside a higher-order construct of Ego Resiliency.

Indices of chi-square, Comparative Fit Index (CFI), Tucker–Lewis Index (TLI), standardised root-mean-square residual (SRMR) and root-mean-square error of approximation (RMSEA) evaluated model fit. Good fit thresholds are CFI ≥ 0.90, TLI ≥ 0.90, SRMR ≤ 0.08 and RMSEA ≤ 0.08 (Browne & Cudeck, 1993). Marginal fit represents CFI ≥ 0.88, TLI ≥ 0.88, SRMR ≤ 0.10 and RMSEA ≤ 0.10 (Bong, Woo, & Shin, 2013). Additionally, for model comparison analysis considered Akaike’s information criterion (AIC), with lower values indicative of superior fit. For interpretation, factor loadings ≥ .30 are satisfactory and representative of the factors (Gliner, Morgan, & Leech, 2016).

Rasch analysis provided follow-up measurement information at the person and item levels. Rasch models can be used ‘as confirmatory tests of the extent to which scales have been successfully developed according to explicit a priori measurement criteria’ (Ludlow, Enterline, & Cochran-Smith, 2008, p. 196). For this study, analysis used the Rasch Rating Scale Model (RRSM; Andrich, 1978). Winsteps software (Linacre, 2018) estimated the parameters for analysis using joint maximum likelihood estimation techniques. Similar to previous Rasch validation studies (e.g., Royal & Elahi, 2011; Wolfe & Smith, 2007), assessment of the ER89-R examined five criteria: rating scale effectiveness, dimensionality, reliability, differential item functioning and item hierarchy.

Confirmatory factor analysis

Assessment of multivariate kurtosis via Mardia’s test indicated a significant departure from normality (i.e., 11.89 > 1.96). However, Mardia’s test is highly sensitive to sample size, and it is recommended to examine kurtosis among individual variables (Stelmack et al., 2009). Values > 3.00 indicate a variable is not normally distributed (Westfall & Henning, 2013). All variables were below 3.00 (i.e., ranged between 0.41 and 1.01).

The one-factor total scale model (Table 1) reported marginal fit on CFI (0.88), unsatisfactory TLI (0.85), good SRMR (0.05) and RMSEA (0.06). Factor loadings were all > 0.3. The other models (i.e., two one-factor models and a higher-order model) demonstrated good fit across CFI (0.96–0.97), TLI (0.93–0.95) and SRMR (0.02–0.03). However, RMSEA reflected a marginally acceptable value of 0.10 for the one-factor Openness to Life Experiences model. Both the higher-order and Optimal Regulation models demonstrated good RMSEA (0.05–0.06).

OPEN IN VIEWER

Table 1.

Fit Indices for ER89-R Factor Models.

Model	χ2	df	CFI	TLI	SRMR	RMSEA (90% CI)	AIC
One-factor total scale	540.97**	35	0.88	0.85	0.05	0.08 (0.08–0.09)	600.97
One-factor Openness to Life Experiences	43.60**	2	0.97	0.93	0.02	0.10 (0.08–0.12)	–
One-factor Optimal Regulation	65.46**	9	0.96	0.94	0.02	0.06 (0.04–0.06)	–
Higher-order model	209.85**	34	0.96	0.95	0.03	0.05 (0.04–0.05)	271.85

Factor loadings were significant and > 0.3, ranging from 0.36 to 0.75. Comparison of AIC between the higher-order and total scale models in addition to the fit indices suggested that the higher-order model was superior (i.e., lower AIC and better overall fit). Figure 1 displays standardised factor loadings for the higher-order model alongside error and R². Omega and alpha reliability for all scales was acceptable (total scale ω = 0.80, total scale α = 0.80; Openness to Life Experiences ω = 0.76, Openness to Life Experiences α = 0.75; and Optimal Regulation ω = 0.70, Optimal Regulation α = 0.70).OPEN IN VIEWER

Rasch analysis

Informed by CFA results and the conclusions of Alessandri et al. (2008), Rasch analysis examined ER89-R as two factors (Openness to Life Experiences and Optimal Regulation). The first step of the Rasch analysis included assessment of rating scale effectiveness. Infit and Outfit mean-square statistics for all response categories were within the acceptable range of 0.6–1.4 (Wright & Linacre, 1994) (Table 2). However, across factors, participants appeared to be slightly more inclined to agree with statements. All items demonstrated Outfit MNSQ and Infit MNSQ between 0.6 and 1.4, indicating a lack of ‘noise’ or randomness within the measure (Table 3). Furthermore, item difficulty ranged from −0.74 to 0.45 logits reflecting a lack of spread, and standard errors were low (from 0.04 to 0.07). These results show that the response scale is functioning appropriately across Openness to Life Experiences and Optimal Regulation, and that the items are productive for measurement.

OPEN IN VIEWER

Table 2.

Rating Scale Effectiveness.

Category	Total Count (%age)	Infit MNSQ	Outfit MNSQ
Openness to life experiences
1. Does not apply at all	941 (12)	1.09	1.10
2. Applies slightly	2269 (28)	0.89	0.87
3. Applies somewhat	2966 (37)	0.89	0.90
4. Applies very strongly	1860 (23)	1.10	1.09
Optimal regulation
1. Does not apply at all	1164 (10)	1.00	1.03
2. Applies slightly	3466 (29)	0.94	0.94
3. Applies somewhat	4837 (40)	0.89	0.92
4. Applies very strongly	2587 (21)	1.09	1.08

OPEN IN VIEWER

Table 3.

Item Fit Statistics.

Item Description	Difficulty	Standard Error	Infit MNSQ	Outfit MNSQ
Openness to Life Experiences
4. I enjoy trying new foods I have never tasted before	−0.20	0.06	1.26	1.24
5. I like to take different paths to familiar places	0.32	0.06	0.88	0.87
6. I am more curious than most people	−0.07	0.07	1.08	1.08
8. I like to do new and different things	−0.05	0.06	0.76	0.77
Optimal Regulation
1. I am generous with my friends	−0.74	0.07	1.17	1.19
2. I quickly get over and recover from being startled	−0.01	0.05	0.85	0.84
3. Most of the people I meet are likeable	0.08	0.05	0.78	0.79
7. I usually think carefully about something before acting	−0.22	0.06	1.13	1.15
9. My daily life is full of things that keep me interested	0.45	0.04	1.00	0.99
10. I get over my anger at someone reasonably quickly	0.44	0.04	1.04	1.04

Note. The MNSQ acceptable limits to productive measurement were 0.6–1.4. Values beyond these limits are considered misfitting.

Figure 2 displays response curves for each category of the survey. This is informative insofar as it indicates level of endorsement. For example, the bell-shaped curve for ‘Applies somewhat’ peaks in the ability range of 0.5 and 1.3 similarly for Openness to Life Experiences and Optimal Regulation, signifying that individuals with Ego Resiliency scores between 0.5 and 1.3 are more likely to endorse this category. It appears from the pattern of responses (see Figure 2) that participants are using all four response categories.OPEN IN VIEWER

The PCA of the residuals examined dimensionality. The observed variance explained by the first extracted dimension for Openness to Life Experiences was 49.4%. The unexplained variance in the first contrast was 19.7% (eigenvalue: 1.6). For Optimal Regulation, the observed variance was 40.3% for the first component. The initial contrast accounted for 13.5% (Eigenvalue: 1.4). An eigenvalue > 2 is suggested to reflect a component (Linacre, 2012). Therefore, the results indicated the presence of one Rasch dimension for each factor, and represented acceptable evidence for unidimensionality.

Reliability and separation estimates reflect the degree of reproducibility in the scores. For Openness to Life Experiences and Optimal Regulation, person reliability was 0.70 and 0.73, signifying acceptable internal consistency. Similarly, item reliability was 0.96 and 0.99, indicating high item reliability. Person separation estimates of 1.54 and 1.66 for Openness to Life Experiences and Optimal Regulation indicated reasonable spread, distinguishing high and low ability in the sample (Souza et al., 2017). Similarly, respective item separation measures of 5.15 and 12.12 existed, inferring good spread of items.

Differential item functioning examines stability in item difficulty in relation to subpopulations (e.g., gender). For Openness to Life Experiences, all items demonstrated acceptable DIF contrasts (i.e., below 0.5 logits; Linacre, 2012). For Optimal Regulation, item 1 displayed a noticeable difference between men and women, and was .67 logits more difficult for men (Mantel–Haenszel p < .001). This study examined how the structure of the ER89-R behaved when administered in this sample. Considering that modifying the scale structure was not the purpose, the authors hope this information will guide use in future research.

Item difficulty is interpretable from the person–item maps (Figure 3). Items positioned in the bottom section of the continuum are the most straightforward, and participants located near these items possess less ability to take part (i.e., less inclination to endorse items). Items positioned near the top of the continuum are the most challenging, and participants close to these possess greater ability. Item 4 was the easiest to agree with for Openness to Life Experiences, whereas item 5 was the most difficult to endorse. Item 1 was the easiest for Optimal Regulation, and items 9 and 10 were the most challenging.OPEN IN VIEWER

Limitations

The current study included a higher proportion of women compared with men (73% vs. 27%). Given that women tend to report higher ER89-R scores (e.g., Alessandri et al., 2008; Caprara et al., 2003), it is possible that an overrepresentation of higher Ego Resiliency scores existed in the sample. Nonetheless, this slight bias is not likely to undermine the calibration of the ER89-R because Rasch analysis permits relatively sample-free measure standardisation in comparison with classical test theory (Tinsley & Dawis, 1975).

In addition, it may have been informative to examine alternative versions of the ER89 (i.e., the original 14-item and the 11-item versions), and compare factor solutions with the ER89-R. As it stands, conclusions can only be made on how the ER89-R performs in isolation, not in comparison with alternative measures that propose different latent structures. Moreover, development of the ER89-R via classical test approaches included cross-cultural comparisons (e.g., Alessandri et al., 2012). Current findings are, however, restricted to the English language version of the ER89-R. It is thus important for future research to replicate these findings in other cultural contexts and with other language versions.