Abstract
We investigated the cross-cultural construct validity of hope, a factor associated with mental health protection and promotion, using the Children’s Hope Scale (CHS). The sample (n = 1,057; 48% girls) included baseline data from three cluster-randomized controlled trials with children affected by armed conflict (n = 329 Burundi; n = 403 Indonesia; n = 325 Nepal). The confirmatory factor analysis in each country indicated good fit for the hypothesized two-factor model. Analysis by gender indicated that configural invariance was supported and that scalar invariance was demonstrated in Indonesia. However, metric and scalar invariance were not supported in Burundi and Nepal. In country comparisons, configural and metric invariance were met, but scalar invariance was not supported. Evidence from this study supports the use of the CHS within various sociocultural settings and across genders, but direct comparisons of CHS scores across groups should be done with caution. Rigorous evaluations of the measurement properties of mental health protective and promotive factors are necessary to inform both research and practice.
Children and adolescents in areas affected by wars and armed conflicts are at an increased risk of developing adverse mental health outcomes (Barenbaum, Ruchkin, & Schwab-Stone, 2004; Mels, Derluyn, Broekaert, & Rosseel, 2010). Given that an estimated one billion children live in countries affected by armed conflict (United Nations Office of the Special Representative of the Secretary-General for Children and Armed Conflict, 2009), this represents a major global public health issue. Research into the individual and contextual factors that can protect children from adverse consequences and promote psychosocial well-being is therefore critically important to better understand the mechanisms between adversity and mental health.
Identifying factors that are associated with better mental health has commonly been done within the paradigm of resilience. Resilience is “a dynamic process encompassing positive adaptation within the context of significant adversity” (Luthar, Cicchetti, & Becker, 2000, p. 543). It is not a static, individual trait. Rather, contemporary research on resilience views it as composed of many factors that interact at multiple levels of the social ecological environment in which children grow up (i.e., intraindividual, families, schools, communities; Betancourt & Khan, 2008; Masten & Narayan, 2012). Both protective factors, or factors known to decrease the probability of suffering from a mental health problem, and promotive factors, or factors associated with actively enhancing positive aspects of mental health, have been researched in association with resilience (Patel & Goodman, 2007). Protective factors are predictive of better outcomes under high-risk conditions. For example, higher levels of social support may be associated with lower levels of depressive symptoms. Promotive factors are predictive of better outcomes in high- and low-risk situations. For example, consistent parenting may be associated with higher levels of self-esteem (Masten & Narayan, 2012). Identification of protective and promotive factors may inform interventions aimed at promoting positive aspects of mental health and preventing mental health problems, thereby bolstering aspects of resilience and aid in making treatments more effective by building on existing strengths.
Hope is a factor that has been identified as important for protection against mental health problems and, more generally, as a positive aspect of well-being. Hope has been conceptualized in various ways in the literature. Snyder (1995) defines hope as “the process of thinking about one’s goals, along with the motivation to move toward and the way to achieve those goals” (p. 355). Schrank, Stanghellini, and Slade (2008) conducted a systematic review aimed at defining hope within the context of mental health research. Their review identified 49 definitions of hope, which were synthesized together to outline the key concepts in hope. The study concluded that hope is a future-oriented expectation of achieving relevant personal goals that are considered subjectively possible, but depend on both personal characteristics (e.g., self-efficacy, courage) and external factors (e.g., resource availability; Schrank et al., 2008).
Having a lack of hope (or hopelessness) is commonly linked to mental disorders. In their seminal work on the hopelessness theory of depression, Abramson, Metalsky, and Alloy (1989) described a subtype of depression etiologically grounded in the beliefs that highly desired outcomes will not occur, highly aversive outcomes will occur, and there is no individual response that can change this trajectory. In other words, hopelessness consists of both negative expectations of the future and expectations of helplessness.
Improving hope through treatment has been shown to have a positive impact on mental health. Promotion of accurate cognitive styles, problem-solving skills, and supportive family relationships aimed at reducing a sense of hopelessness may help reduce morbidity and possibly prevent onset or relapse of mental illness (Klein, Jacobs, & Reinecke, 2007; Stice, Shaw, Bohon, Marti, & Rohde, 2009). In the context of trauma, Hobfoll et al. (2007) identified a sense of hope as an essential principle and empirically supported component of interventions targeting people who have experienced potentially traumatic events.
Psychotherapeutic treatments have been shown to be effective for promoting a sense of hope (Hodgekins & Fowler, 2010; Vilhauer et al., 2013). Specifically among children affected by armed conflict, hope has been shown to be amenable to change in intervention research (Jordans et al., 2010; Khamis, Macy, & Coignez, 2004; Tol et al., 2008; Tol, Komproe, et al., 2010; Tol et al., 2014).
Having a sense of hope is an important protective and promotive factor for mental health (Perry, Taylor, & Shaw, 2007; Ridgway, 2001). Among children affected by armed conflict or violence, research has shown that a sense of hope can contribute to the well-being of children despite adverse armed-conflict experiences (Betancourt, 2005; Cortes & Buchanan, 2007). While research on hope’s role in promoting mental health is more limited, it appears to be a potentially promising promotive factor. Drawing from the field of positive psychology, a field that is focused on understanding and fostering the factors that allow individuals, communities, and societies to thrive, hope has been recognized as part of a core strength of human goodness and character (Kobau et al., 2011). Hope is a part of the core strength of transcendence (i.e., the factors that forge connections and provide meaning; Peterson & Seligman, 2004). Programs designed to build these core strengths may promote mental health and bolster resilience (Kobau et al., 2011). Among Afghan children exposed to violence and armed conflict, a sense of hope was found to be a key component of resilience (Eggerman & Panter-Brick, 2010). The authors found that a sense of hope was directed toward accessing the resources needed to create social and economic opportunities for their families. Hope served as the foundation of resilience by providing the motivation and methods to achieve one’s goals (Panter-Brick & Eggerman, 2012).
Despite the importance of hope in mental health treatment, prevention, and promotion, measuring protective and promotive factors within and across populations has been challenging. Cross-cultural research has generally recognized the importance of testing and evaluating measures of distress in different contexts and for different populations (Kohrt et al., 2011; Yarnell et al., 2013). Yet the same attention has not typically been spent on scales used to measure protective and promotive factors in the context of mental health. If one of the mandates of resilience research is to identify the “critical ingredients” on which to base interventions (Luthar & Brown, 2007), it is necessary to undertake a more thorough investigation into reliable and valid measurement of these “critical ingredients.” Clearly, further research on protective and promotive factors thought to be important in resilience, such as hope, would benefit from knowledge on how to best capture these components empirically in a variety of settings.
The aims of this study were to (a) examine the factor structure of a commonly used measure of hope, the Children’s Hope Scale (CHS; Snyder et al., 1997), using data from three intervention trials aimed at improving mental health of war-affected school children in Burundi, Indonesia, and Nepal and (b) determine the extent to which measurement invariance can be assumed across genders and countries. Based on previous findings (Jordans et al., 2010; Jordans et al., 2013), we hypothesize that the CHS would be invariant across boys and girls. With respect to country, we hypothesized that the CHS would support a two-factor structure in each country. However, we hypothesized that different items from the CHS would contribute differently to the factor structure across the three countries based on sociocultural differences that may affect children’s levels of hope in these settings. For example, the CHS item “When I have a problem, I can come up with lots of ways to solve it” may be influenced by cultural differences in what extent personal agency and problem solving are emphasized in child-rearing practices in each country. Similar differences were found in a previous examination of the construct validity of a psychosocial distress screener in related populations (Jordans, Komproe, Tol, & De Jong, 2009; Jordans, Ventevogel, Komproe, Tol, & de Jong, 2008).
If supported, our hypotheses would indicate that the broad factor structure of a sense of hope composed of two factors related to agency and pathways would exist across settings, but the extent to which scores on the CHS are comparable across populations may vary. Due to previous research examining the CHS among different racial groups (Snyder et al., 1997), an assumption of the universality of hope seemed reasonable. However, research is limited on the degree to which hope is a universal construct across sociocultural settings. The current analysis attempts to address this gap in the literature. While the CHS has been widely used and its construct validation confirmed in studies in the United States (Schmid, Phelps, & Lerner, 2011; Valle, Huebner, & Suldo, 2006), it remains unclear if and how the CHS may be relevant for children in different sociocultural settings, who have been exposed to extreme of adversity, and whether it is appropriate for both boys and girls in these settings. Existing research has examined the invariance of the CHS within individual sociocultural contexts (Edwards, Ong, & Lopez, 2007; Jovanović, 2013). To the best of our knowledge, this is the first study that addresses the construct validity of the CHS across sociocultural contexts and among war-affected youth.
Method
Study Settings and Participants
The sample for the current study included children affected by armed conflict from three different countries. A total of 1,057 children from three different countries—Burundi (N = 329; n = 170 males, n = 159 females), Indonesia (N = 403; n = 209 males, n = 194 females), and Nepal (N = 325; n = 167 males, n = 158 females)—were included in the current analysis. Across all three countries, 52% of participants were male and 48% were female. The average ages were 12.3 years in Burundi and Indonesia and 12.7 years in Nepal. The number of different categories of potentially traumatic events that children were exposed to was assessed in the Burundi and Indonesia samples, with children reporting an average of 4.4 (range: 1-10) and 3.9 (range: 1-9) types of events, respectively. For depression symptoms, the average total score was 10.7 (SD = 5.0) in Burundi, 12.4 (SD = 3.4) in Indonesia, and 21.2 (SD = 2.8) in Nepal. For posttraumatic stress disorder (PTSD) symptoms, children reported average total scores of 15.9 (SD = 8.4) in Burundi, 21.7 (SD = 8.6) in Indonesia, and 32.3 (SD = 7.8) in Nepal (Table 1).
Sample Characteristics.
Trauma exposure was measured using a checklist of 11 items constructed locally through a free listing with 23 staff members from the implementing organization who were ask to list adverse events children may have been exposed to as part of the armed conflict; depression symptoms were measured with the Depression Self-Rating Scale (18 items, 3-point scale, range: 0-36, α = .72, test–retest reliability = .88); PTSD symptoms were measured with the Child Posttraumatic Symptom Scale (17 items, 4-point scale, range: 0-51, α = .84, test–retest reliability = .59). bTraumatic exposure was measured through child-rated checklist of nine dichotomous items created based on a free-listing exercise with staff from the local implementing organization (experienced, yes or no; range: 0-9; test–retest reliability = .61); depression symptoms were measured with the Depression Self-Rating Scale (18 items, 3-point scale, range: 0-36, α = .69, test–retest reliability = .57); PTSD symptoms were measured with the Child Posttraumatic Symptom Scale (17 items, 4-point scale, range: 0-51, α = .85, test–retest reliability = .65). cDepression symptoms were measured using the Depression Self-Rating Scale (18 items, 3-point scale, range: 0-36, α = .60, test–retest reliability = .80); PTSD symptoms measured using the Child Posttraumatic Symptom Scale (17 items, 4-point scale, range: 0-51, α = .81, test–retest reliability = .85).
Data for this analysis were collected as part of three different cluster-randomized controlled trials (RCTs) of a school-based psychosocial intervention aimed at improving mental health of children affected by armed conflict by improving strengths (hope, coping, social support) and lowering psychological symptoms. For this analysis, only baseline (cross-sectional) data were used.
The study in Burundi was conducted in two Northwestern provinces, Bubanza and Cibitoke, between October 2006 and June 2007. These areas experienced cyclical violence between Hutu and Tutsi ethnic groups, with the latest large-scale violence between 1993 and 2006 claiming an estimated 300,000 lives. Participants were selected using multiple steps of random selection. One of the two provinces was randomly selected as the intervention site. The other province served as the control site. Within each province, schools were randomly selected as well (several schools were excluded due to safety issues for the research team). This ultimately resulted in 14 schools (7 for treatment and 7 for control) from which children were recruited. Finally, within each school, children were screened. Children who had been exposed to at least one potentially traumatic event (checklist with yes or no response options) and who scored above the standard cutoff on symptom checklists for PTSD, depression, and/or anxiety were included in the intervention. Common types of exposure to potentially traumatic events included displacement, witnessing killings of family members and violence (bombing and burning of houses), sexual and other forms of gender-based violence, and participation in hostilities. Exclusion criteria were assessed by trained psychosocial counselors and included the inability to function in a group setting (e.g., violent behavior, could not follow instructions, would harm others) and a group of psychiatric problems (mutism, mental retardation, substance abuse, dissociative disorders, epilepsy without medication, panic or phobic disorders, and child psychosis).
Children in Indonesia were selected from the Poso district in Central Sulawesi, Indonesia. The Poso district has a history of communal violence between religious groups, but with a background in economic, demographic, and political changes. At the time of research, major hostilities had ceased but low-intensity conflict ensued through continued, unaccounted murders, occasional bombings, and rumors about renewed communal violence. Previous qualitative work showed the importance of children’s psychological distress (also as indicated by somatic symptoms), morally inappropriate behavior, poverty, and continued interreligious tensions in the area (Tol, Reis, Susanty, & de Jong, 2010). Common types of exposure to potentially traumatic events included displacement, witnessing attacks by village groups on other villages (killings, burning of houses), and witnessing bombings and sniper attacks. Inclusion and exclusion criteria were the same as the study in Burundi. Children were included if they had been exposed to at least one potentially traumatic event (checklist with yes or no response options), and scored above standard cutoff scores on PTSD or anxiety symptom checklists. Exclusion criteria were assessed similar to the study in Burundi.
Participants in Nepal were recruited from four districts in Southwestern Nepal (Banke, Dang, Bardia, Kailali). Nepal is the poorest country in South Asia (World Bank, 2007) and suffered a 10-year armed conflict (1996-2006) between the Communist Party of Nepal (Maoist) and government forces. The conflict was a Maoist insurgency mainly fought in rural areas and was associated with 16,000 deaths and systematic human rights violations such as forced disappearances, torture, conscription of child soldiers, and sexual and other forms of gender-based violence. During the time of data collection, a peace agreement had just been completed. Inclusion criteria was based on a seven-item locally validated version of the Child Psychosocial Distress Screener (Jordans et al., 2008). The Child Psychosocial Distress Screener measures nonspecific psychosocial distress through a combination of indicators (traumatic stress, current distress, school attendance) and strengths (perceived social support and coping). Children were excluded if they showed signs for serious psychiatric problems. More details related to the methods for implementing each RCT are available in detail elsewhere (Jordans et al., 2010; Jordans et al., 2013; Tol et al., 2012; Tol et al., 2014).
Measures
The CHS (Snyder et al., 1997) was used in all three settings. The CHS is based on a conceptualization of hope as consisting of two factors: agency and pathways (Snyder et al., 1997). Agency is conceptualized as the perception by the child that he or she is able to initiate and sustain action toward a goal. The pathways factor involves the child’s capability of producing the means to achieve these goals (Snyder, 1995; Snyder et al., 1997).
The CHS consists of six items that are hypothesized to relate to the two underlying factors of agency and pathways. The three items intended to measure agency are “I think I am doing pretty well,” “I am doing just as well as other kids my age,” and “I think the things I have done in the past will help me in the future.” The three items related to pathways are “I can think of many ways to get the things in life that are most important to me,” “When I have a problem, I can come up with lots of ways to solve it,” and “Even when others want to quit, I know that I can find ways to solve the problem.” Item responses were recorded on a 6-point ordinal scale for frequency, 0 = none of the time, 1 = a little of the time, 2 = some of the time, 3 = a lot of the time, 4 = most of the time, and 5 = all of the time, in Burundi and Indonesia. In Nepal CHS item responses were recorded on a 5-point ordinal scale for frequency: 0 = none of the time, 1 = a little of the time, 2 = some of the time, 3 = most of the time, and 4 = always. For the purposes of this analysis, we collapsed response categories most of the time and always as in Nepal, as always was not included in the options due to harmonization with other scales on the assessment battery.
In all three sites, the CHS (along with all outcome measures for each RCT) was translated to the appropriate local language using a five-step method for preparation of instruments in transcultural settings (Van Ommeren et al., 1999). This process includes bilingual translation, independent bilingual conceptual review, blinded back-translation, focus groups, and piloting with target population. To assist with interpretation of the answering scale, children were provided a visual of glasses with varying levels of water. A trained interviewer read the CHS to each child to account for variation in levels of literacy. For all sites, the CHS was administered after random assignment to treatment and control conditions to all children in both arms of the trial.
In Burundi, the CHS demonstrated internal consistency reliabilities (measured using Cronbach’s alpha [Cronbach, 1951] calculated for the agency and pathways factors separately and using the Spearman–Brown prophesy [Brown, 1910; Spearman, 1910] to adjust reliability back to six items) of .75 for agency and .76 for pathways, and a test–retest reliability of r = .95; in Indonesia, the CHS internal consistency reliability for the agency factor was .61 and for the pathways factor, it was .66, and test–retest reliability was r = .67; and in Nepal, internal consistency reliability was .71 for agency and .78 for pathways, and test–retest reliability was r = .70 (Jordans et al., 2010; Tol et al., 2008; Tol et al., 2014).
Statistical Analysis
We used confirmatory factor analysis (CFA) to examine construct validity of the CHS using only baseline data. Posttreatment data were not analyzed, as the goal of this analysis was to examine only measurement invariance at baseline rather than how hope may change over time. Construct validity represents the degree to which a scale measures what it purports to measure. To explore whether the Hope Scale had the same construct across genders and countries, we used a multiple-group CFA to explore measurement invariance (Park et al., 2012; Steinmetz, Schmidt, Tina-Booh, Wieczorek, & Schwartz, 2009). Invariance testing included configural, metric, and scalar invariance by gender and across countries. The invariance testing by gender (i.e., testing comparing boys and girls) was done within each individual country. In the invariance testing across countries, gender was included as an exogenous variable to account for any gender invariance found during the first step of invariance testing.
Configural invariance tests if the same set of factors is present and indicates if the factor structure of the questionnaire is similar across groups. Metric invariance tests if factor loadings are the same across groups and indicates whether the questions are each correlated with similar magnitudes to the underlying trait across groups. Scalar invariance is more restrictive than metric invariance and tests if item intercepts and factor loadings are the same, which reflects whether there are any systematic differences in the way individuals respond to questions because of group membership. If a scale shows scalar invariance, then summary scores on the scale can be compared across groups.
CFA and multigroup modeling using maximum likelihood estimation were performed using Mplus 7 (Muthén & Muthén, 2012). For each invariance model, the factor means were fixed at 0 and factor variances were fixed at 1. In the configural model, factor loadings, intercepts, and residual variances were free across groups. In the metric model, factor loadings were constrained to be equal across groups, but intercepts and variances were free across groups. In the scalar model, factor loadings and intercepts were constrained to be equal, but residual variances were free, across groups. Model fit was evaluated using standard measures of fit: the root mean square error of approximation, the comparative fit index, the Tucker–Lewis index, and the standardized root mean square residual. Root mean square error of approximation values lower than 0.06, Tucker–Lewis index/comparative fit index values above 0.95, and standardized root mean square residual values lower than 0.08 all are indicative of good model fit (Hu & Bentler, 1998). To compare unconstrained models and constrained models, we performed chi-squared difference tests (Δχ2) to determine whether the difference between model fit was significant. Nonsignificant (p < .05) chi-squared difference tests indicate that the more constrained model does not worsen model fit and may be accepted. Models were also compared using Akaike information criterion (AIC; Akaike, 1974) and Bayesian information criteria (BIC; Schwarz, 1978) values for each model. AIC and BIC provide means for model selection that deal with the tradeoff between model fit and model complexity. AIC takes into account log likelihood and BIC includes further penalization for more parameters. Models with lower AIC or BIC values indicate adequate model fit with the most parsimonious model.
Results
Hope Scale Scores
Average scores on the CHS were 2.6 (SD = 1.0) in Burundi, 2.6 (SD = 1.0) in Indonesia, and 2.2 (SD = 0.6) in Nepal. Response patterns for each of the CHS items, by country and gender, are displayed in Table 2. Overall, response patterns for each item on the CHS were normally distributed across sites. Chi-squared difference tests indicated that there were statistically significantly differences in the proportion of participants who endorsed each category by gender and country.
Percentage Responses to Each Category by Item Across Countries.
CHS01: I think I am doing pretty well; CHS02: I can think of many ways to get the things in life that are most important to me; CHS03: I am doing just as well as other kids my age; CHS04: When I have a problem, I can come up with lots of ways to solve it; CHS05: I think the things I have done in the past will help me in the future; CHS06: Even when others want to quit, I know that I can find ways to solve a problem. bB = Burundi; I = Indonesia; N = Nepal.
Measurement Invariance by Gender
The investigation into measurement invariance by gender is displayed in Table 3. Within each country, configural, metric, and scalar invariance of the CHS across genders showed good model fit. Chi-squared difference tests indicated significant differences between the metric and configural models in Nepal (Δχ2 = 12.13; p = .02) and Burundi (Δχ2 = 12.11; p = .02) but no significant differences between models in Indonesia (Δχ2 = 0.84; p = .92). The AIC and BIC values supported the more parsimonious models (configural invariance) in Nepal and Burundi. Thus, only configural invariance across genders was supported in Nepal and Burundi. Metric and scalar invariance across genders was supported in the Indonesian sample. This indicates that, in both Nepal and Burundi, a model of the general two-factor structure of hope (agency and pathways) had a better fit when contributions of individual items on these two factors were allowed to vary by gender. In Indonesia, a two-factor model in which questionnaire items contributed similarly to factors for both boys and girls, fit the data.
Measurement Invariance by Gender.
Note. p = p-value for chi-squared difference test; df = degrees of freedom; RMSEA = root mean square error of approximation; CFI = comparative fit index; TLI = Tucker–Lewis index; SRMR = standardized root mean square residual; AIC = Akaike Information criterion; BIC = Bayesian information criterion.
Measurement Invariance by Country
Due to the presence of gender invariance in Nepal and Burundi (Table 3), we included gender as an exogenous variable in the models testing invariance by country. Across all country comparisons, model fit statistics indicated support for configural and metric invariance (Figure 1). Chi-squared difference tests and examination of AIC and BIC values indicated no differences between metric and configural models among the three countries when controlling for gender. However, the chi-squared difference tests, AIC values, and BIC values did not support scalar invariance in any of the three comparisons (Table 4). This indicates that, in all three countries, a model with a two-factor structure in which individual items contribute in the same way to these two factors (agency and pathways) appropriately fits the data. However, children needed different levels of hope to endorse the items in the same way across these different countries.
Factor structure and standardized loadings for each country stratified by gender.
Measurement Invariance by Country Controlling for Gender.
Note. p = p-value for chi-squared difference test; df = degrees of freedom; RMSEA = root mean square error of approximation; CFI = comparative fit index; TLI = Tucker–Lewis index; SRMR = standardized root mean square residual; AIC = Akaike information criterion; BIC = Bayesian information criterion.
Given that invariance was identified in the gender analyses in two of the three settings, we present our interpretations of invariance by country for boys and girls separately. For example, in Nepal, boys had an intercept of 2.1 for endorsing the item “I am doing just as well as other kids my age,” while boys in Indonesia had an intercept of 3.0 for this same item. This suggests that boys in Nepal will have lower scores on this item on average than boys in Indonesia who have the same level of hopefulness. If this measurement variance is not accounted for in the scoring, then it will appear that boys of a certain hopefulness in Nepal are less hopeful than similar boys in Indonesia when in fact, they are not. Similar differences were seen for the item “When I have a problem, I can come up with lots of ways to solve it” with boys in Nepal having an intercept of 2.0 for this item compared with intercepts of 2.9 in Indonesia and 2.9 in Burundi.
For girls, scalar invariance was particularly apparent for CHS Items 2, 3, and 4 across settings. Girls in Nepal had intercepts of 2.0 for the items “I can think of many ways to get things in life that are most important to me” and “I am doing just as well as other kids my age,” respectively, while girls in Indonesia had intercepts of 2.7 for these same items. Girls in Nepal, on average, will have lower scores on this item compared with girls in Indonesia despite having the same level of underlying hope. For the item “When I have a problem, I can come up with lots of ways to solve it” girls in Indonesia had an intercept of 2.7 while girls in Nepal and Burundi had intercepts of 1.9 for this item. This indicates that girls in Indonesia have systematically higher scores on this item than girls in the other settings despite being equally hopeful.
Discussion
We aimed to evaluate the construct validity of the CHS among conflict-affected children in Burundi, Indonesia, and Nepal. With regard to gender invariance, our hypothesis that the CHS would not vary across genders was only supported in the Indonesian sample and not supported in the samples from Burundi and Nepal. Full measurement invariance (configural, metric, scalar) was supported across genders in Indonesia. This indicates that in Indonesia, the two-factor structure—with similar factor loadings of items on the factors—fit the data for both boys and girls. In Indonesia, CHS scores can be directly compared for both boys and girls. In Nepal and Burundi, the two-factor model of agency and pathways fit the data for both boys and girls (configural invariance). However, individual items of the questionnaire contributed differently to the two factors (metric variance) and there were systematic differences in how boys and girls scored on each of the items (scalar variance). This suggests that there is measurement bias and direct comparison of CHS scores for boys and girls in these settings should not be done.
The response differences between genders in Nepal and Burundi could indicate that certain items are not as indicative of hope for both boys and girls. For example, in Burundi, the item “I can think of many ways to get the things in life that are most important to me” had a factor loading of .71 among boys and a factor loading of .95 among girls. Perhaps among girls, structural gender inequality means that obtaining the things you want is more difficult; so thinking of ways to do this is more strongly related to having hope. Whereas in boys, obtaining the things you want in life may be relatively easier, meaning that this item is not as strongly related to measuring this aspect of hope in boys compared with girls.
Our hypothesis that the factor structure would vary by country was partially supported. Results indicated that the two-factor solution showed good model fit across different countries. With regard to more restrictive types of invariance across countries, full metric invariance was supported but scalar invariance was not. These results support a common relationship of the items to the factors across country, but systematic differences in level of responses on items between children in different countries. For example, children in Nepal had to have lower levels of hope to endorse the item “I am doing just as well as other kids my age” than children in Indonesia and Burundi. As a result of this scalar invariance, comparing scores on the CHS across these settings may result in biased estimates, as it may appear that children in Nepal have lower scores on the CHS even if they ultimately have the same amount of hope as children in the other settings.
Factors varying by study location appear to confound the relationship between hope and item responses. It is challenging to speculate on the specific processes that may underlie these differences by study location. For example, using the same example of the item “I am doing just as well as other kids my age” makes it appear that children in Nepal need less hope to endorse this item with the same response as children in Indonesia and Burundi. This may be related to factors associated with the armed conflict (e.g., the armed conflict in Nepal may have affected children in classes similarly, rather than affecting specific groups of children particularly—so that less hope is needed to endorse similarity with peers); social factors (e.g., children in the Nepal study might be from economically more homogeneously resourced areas, making it easier for them to endorse this item despite lower levels of hope); or cultural factors (e.g., differences between peers are not emphasized in socialization). Further in-depth qualitative research would be helpful to explore these differences.
These results show that using the underlying two-factor structure of the CHS holds across countries and genders is appropriate, but there may be differences between genders and countries with regard to the scores on individual items. Thus, the use of the CHS to measure the same construct (hope) in children in various sociocultural contexts is possible, but comparing scores on the CHS across countries should be done with caution due to partial measurement invariance.
There are few studies, to our knowledge, that examine the psychometric properties of scales meant to measure resilience processes across contexts. Gana, Daigre, and Ledrich (2013) found that measurement of hope was invariant across genders, which was similar to the results for the Indonesian sample, but not the results from the samples in Burundi and Nepal. However, because of the dearth of research related to measurement of invariance of constructs of the resilience process, we further compared our findings with studies that have looked at psychological symptoms across contexts. Several studies have reported depression scales that demonstrate metric and scalar invariance by nationality (Wu et al., 2012; Zhang et al., 2011). Other studies have examined measurement invariance by gender and found that gender is significantly related to variability in measurement of PTSD and depression symptoms (Armour et al., 2011; Drapeau et al., 2010). However, some studies have found that gender does not account for variability in measurement of symptomology (Contractor et al., 2013; Wang et al., 2013). Overall, it appears that the field has yet to fully understand what explains differences in measurement of symptoms, as well as other key constructs related to mental health.
In the current study, the systematic differences in item responses based on country (scalar invariance) could reflect substantive differences in risk across settings, as well as sociocultural differences. With regard to sociocultural differences, Ungar (2008), as part of his comprehensive International Resilience Project, examined over 1,500 youth in 14 different communities to more fully understand contextual variation in resilience. They propose that resilience processes have both global and culturally specific aspects; that resilience predictors may have different levels of influence on positive mental health and well-being, depending on specific cultures and contexts; that aspects of children’s lives that contribute to resilience are related to each other in ways that reflect the context; and that tensions between individuals and their cultures will influence the way aspects of resilience are grouped together (Ungar, 2008).
Future Directions
Our results broadly support the use of the CHS to measure hope in the three study contexts, given that the same factor loadings were found and items contributed similarly to factors across countries. However, differing levels of measurement variance were found across countries and by gender, indicating that more work would need to be done to derive a measure that can accurately compare levels of hope across subpopulations. Further research could consist of additional qualitative work and psychometric testing to ensure accurate measurement of the construct of hope in multiple subgroup populations. This could be done in a number of ways including formative research aimed at enhancing adaptation and translation of the CHS and exploration of factorial validity through card sort activities (see, e.g., Rasmussen, Katoni, Keller, & Wilkinson, 2011). In addition, future research should look at the invariance of the CHS over time. More broadly, the current analysis demonstrates a useful approach to examining the cross-country invariance of other measures of resilience, with the possibility of also examining how these resilience components fit together across multiple populations and settings.
Limitations
This study was limited by only examining invariance by country and gender. Other variables, such as type of type of traumatic exposure, age, symptom severity, level of education, or other unmeasured cultural variables, may have confounded the degree of measurement variance. Examination of these types of variables was beyond the scope of the current investigation and may be limited by sample sizes. Future research on measurement differences between countries, contexts, or cultures should more fully take into account other factors that may be influencing systematic differences in item responses. We were also unable to investigate whether the CHS was invariant across populations affected and unaffected by war. Given that the CHS was originally created for use among populations who have not experienced the extreme adversity associated with war, the question of whether the instrument performs similarly across war-affected and unaffected populations remains to be answered. More generally, given that children in this study were all affected by armed conflict and recruited as part of intervention trials, the generalizability of the current findings to children without such experiences or mental health problems remains to be determined.
Conclusion
Hope is considered an essential component of resilience processes and an empirically supported element of interventions for people who have experienced significant adversity. This study utilized the CHS and examined its factor structure across three distinct populations, conflict-affected children and adolescents in Burundi, Indonesia, and Nepal. Partial measurement invariance was supported across genders and countries. The evidence from this study supports the use of the CHS to measure a similar construct in various contextual settings and across genders, but it is important to continue to investigate the sociocultural and gender-specific relationship between the CHS items and the underlying factor structure. Moreover, the measurement results from this study indicate that it is not appropriate to compare total scores between sites. If interventions in settings of extreme adversity are going to build on factors thought to be related to positive mental health outcomes, ensuring accurate and reliable measurement should not be limited to only pathological indicators but instead include rigorous evaluations of the components thought to be related to enhancing a person’s ability to overcome significant challenges.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study was funded by PLAN Netherlands.