Abstract
The present study examined the factor structure and construct validity of the Children’s Loneliness Scale (CLS), a popular measure of childhood loneliness, in Belgian children. Analyses were conducted on two samples of fifth and sixth graders in Belgium, for a total of 1,069 children. A single-factor structure proved superior to alternative solutions proposed in the literature, when taking item wording into account. Construct validity was shown by substantial associations with related constructs, based on both self-reported (e.g., depressive symptoms and low social self-esteem), and peer-reported variables (e.g., victimization). Furthermore, a significant association was found between the CLS and a peer-reported measure of loneliness. Collectively, these findings provide a solid foundation for the continuing use of the CLS as a measure of childhood loneliness.
Loneliness is the negative feeling that arises when people perceive their social relations to be deficient, either quantitatively or qualitatively (Perlman & Peplau, 1981). This negative feeling is a problem that affects people of all ages, including children and adolescents (Qualter et al., 2015). Feelings of loneliness have been associated with poorer mental and physical well-being. For example, loneliness in childhood and adolescence has been linked with lower school liking, school dropout, depression, social anxiety, lower self-esteem, peer rejection and victimization, eating disorders, suicide ideation, sleeping problems, and poorer cardiovascular functioning (Heinrich & Gullone, 2006; Mahon, Yarcheski, Yarcheski, Cannella, & Hanks, 2006).
Several instruments have been developed to assess feelings of loneliness. The first scale developed to measure this phenomenon in childhood is the Children’s Loneliness Scale (CLS; Asher, Hymel, & Renshaw, 1984). Originally referred to as the Loneliness and Social Dissatisfaction Questionnaire, the measure was developed in the United States for Grades 3 through 6. Meanwhile, the scale has been used also in middle school (i.e., Grades 7 and 8; Parkhurst & Asher, 1992) and, in a slightly adapted format, from preschool to Grade 2 (Cassidy & Asher, 1992). In a frequently used adaptation of the measure, all items were rephrased to refer to the school context (Asher & Wheeler, 1985). In both its original and adapted versions, the measure has established itself as the “gold standard” in the measurement of childhood loneliness. The Web of Science indicates that the original article (Asher et al., 1984) was referred to 428 times and the article that introduced the school-related version (Asher & Wheeler, 1985) 387 times (information retrieved on August 5, 2015). The CLS has been translated in numerous languages and used in several countries, including Australia, Canada, China, Germany, Greece, India, Israel, Italy, Korea, Turkey, and the United Kingdom, but most psychometric studies on the measure have been conducted in the United States.
Several issues that pertain to key aspects of the validity of the CLS have not been resolved completely. The present article addressed two of these issues for the school-related version of the CLS. Specifically, we aimed to examine whether the instrument exhibits (a) its expected factor structure and (b) substantial correlations with a well-selected set of related constructs.
Factor Structure of the CLS
In the literature, indications for three possible factor structures of the CLS can be found. First, as the scale was intended to tap into a unitary construct of loneliness, the original scale developers aimed to show that all their items loaded on a single factor. Exploratory factor analysis on third- through sixth-grade U.S. children, on a medium-sized sample (N > 500) for the original version and a small sample (N < 300) for the school-related version, indicated that all 16 substantive items effectively loaded on a single factor (Asher et al., 1984; Asher & Wheeler, 1985). Second, two conceptual factors, that is, loneliness and social dissatisfaction, were distinguished using factor analysis. Specifically, exploratory factor analysis on two small samples (N < 300) of ninth-grade U.S. students (Jarvinen & Nicholls, 1996) and fifth- and sixth-grade U.S. children with diverse ethnical backgrounds (Bagner, Storch, & Roberti, 2004) yielded a Loneliness factor, with high loadings for 10 items, and a Social dissatisfaction factor, with high loadings for the remaining 6 items. As all items that loaded on the Loneliness factor were non-reverse–coded (e.g., “I’m lonely at school”) and all items that loaded on the Social dissatisfaction factor were reverse coded (e.g., “I am well liked by the kids in my class”), a third factor structure can be proposed. This structure comprises a single substantive factor that takes into account item wording. A confirmatory factor analysis on a large sample (N > 10,000) of 2nd- through 12th-grade U.S. children modelling one such substantive factor with correlated error terms for the reverse-coded items yielded a superior fit for this factor structure (Ebesutani et al., 2012). However, as this study was conducted in a single state in the Southern part of the United States, replication of the findings in other cultures is required.
Construct Validity
Construct validity may be investigated by examining associations with related constructs. Loneliness, as measured with the CLS, has been associated with various characteristics of the child and its social environment. In childhood, loneliness has been associated with higher levels of depressive symptoms (Toblin, Schwartz, Gorman, & Abou-Ezzeddine, 2005), lower global self-esteem (Aikins, Bierman, & Parker, 2005), and lower social self-esteem (Troop-Gordon & Ladd, 2005). Furthermore, in Kindergarten, loneliness has been positively related with school avoidance and negatively with school liking (Coplan, Closson, & Arbeau, 2007). These relations have not yet been examined in older children. Regarding peer-reported measures, loneliness in childhood has been associated with fewer friendships (i.e., received nominations; Shin, 2010), lower peer acceptance and higher peer rejection (Graham & Juvonen, 1998), and being victimized by peers (Boivin, Hymel, & Bukowski, 1995). In addition to examining related constructs, construct validity may be investigated by examining a loneliness measure completed by a different rater, such as peers.
The present study aims to examine the expected factor structure and construct validity of the school-related version of the CLS in a Belgium. Both the United States and Belgium are Western countries that are rather individualistic. Some cultural differences between these two countries, however, are also noticeable. For example, in Belgium, power inequalities and hierarchy are much more accepted than in the United States (Hofstede, 2001). Moreover, Belgium has one of the highest scores on Hofstede’s (2001) Uncertainty Avoidance dimension, which is the extent to which members of a culture feel threatened by ambiguous or unknown situations, whereas the United States scores below average on this dimension. Because of these different cultural profiles, we cannot assume that results obtained with U.S. samples also hold for samples from other countries such as Belgium. The two psychometric issues, that is, the factor structure and construct validity of the CLS, were addressed in two studies, conducted on a separate sample as described below.
Study 1: Factor Structure
The first study set out to compare the three putative factor structures for the 16 substantive items of the CLS. Model 1 is a single-factor substantive model in which all items load on the same loneliness factor. Model 2 is a two-factor model, with the factors reflecting the non–reverse-coded and reverse-coded items (defined by 10 and 6 items, respectively). Model 3, finally, is a single-factor model with correlated error terms for the reverse-coded items to take into account item wording. Our general expectation, in line with earlier comparisons (Bagner et al., 2004; Ebesutani et al., 2012), was that Model 3 would yield the best fit to the data. If confirmed, this result would indicate that the CLS can be considered a unidimensional measure of loneliness if the effect of item wording is also taken into account.
Method
Participants and Procedure
The sample comprised 422 students (211 girls and 210 boys, 1 did not report gender) from Grade 5 and 334 students (192 girls and 142 boys) from Grade 6, for a total of 756 children. The children were 9 to 15 years old (M = 10.92 years, SD = 0.63). Complete CLS data were available for 88.1% of the participants (9.5% had one missing item and 2.3% had two or three missing items). This large sample was created by merging the three samples described in Goossens and Beyers (2002). Data were collected in 1998, in 13 schools in the Dutch-speaking part of Belgium. Information on the ethnic background of the students was not available, but all schools were known to attract mainly Caucasian students.
School principals gave permission to conduct the study in their schools but children could refuse to participate at any time if they wanted to, in line with ethical standards at the time of data collection. Children completed the CLS during regular classes that were supervised by a trained undergraduate student in psychology.
Measure
Students completed the Dutch translation of the school-related version of the CLS (Asher & Wheeler, 1985). This 24-item scale comprises 16 primary items designed to tap into children’s feelings of loneliness and social dissatisfaction in the school context (sample item: “I feel left out of things in school”) and 8 filler items on children’s hobbies and preferred activities and school subjects (sample item: “I watch TV a lot”). All items were responded to on a 5-point scale ranging from 1 (not at all) to 5 (always). Responses on the primary items were summed. Children’s scores on the CLS, therefore, could range between 16 and 80, with higher scores reflecting higher degrees of loneliness.
Plan of Analyses
Confirmatory factor analyses were conducted in Mplus 6.11 (Muthén & Muthén, 2007) using full information maximum likelihood estimation. The CLS scores were treated as continuous. Fit indices used to evaluate absolute model fit included the root mean square error (RMSEA), the standardized root mean square residual (SRMR), and the comparative fit index (CFI). We followed the guidelines of Hu and Bentler (1999), and considered model fit as good if RMSEA < .06, SRMR < .08, and CFI > .90. In addition, we looked at Akaike’s Information Criterion (AIC) with lower values representing better fit.
Results
Fit indices and standardized factor loadings for the three models examined are presented in Tables 1 and 2, respectively. Inspection of the absolute fit indices revealed that Model 2 and Model 3 showed a good absolute fit to the data. Regarding CFI and AIC, Model 3 showed a somewhat better fit than Model 2. As Model 3 is also more parsimonious, this one-factor model that incorporated wording effects was preferred over Model 2, in line with our general expectation. Moreover, the two factors of Model 2 showed a high correlation (r = .85), which does not support a multidimensional approach.
Fit Indices for Three Factorial Models for the Children’s Loneliness Scale.
Note. RMSEA = root mean square error of approximation; CFI = comparative fit index; SRMR = standardized root mean square residual; AIC = Akaike information criterion; df = degree of freedom.
Standardized Factor Loadings of the Confirmatory Factor Analyses on the Children’s Loneliness Scale (CLS).
Study 2: Construct Validity
The second study investigated construct validity by examining associations with related constructs reported by both the adolescents and their peers. We expected that loneliness was positively related with depressive symptoms, school avoidance, peer rejection, and victimization, and negatively related with social and global self-concept, school liking, friendship quantity, and peer acceptance. Moreover, this study included a peer-reported measure of loneliness, which has not been used in the literature before. A positive association between self- and peer-reported measures of loneliness was expected. When examining the construct validity of the CLS, we controlled for gender. However, we had no specific hypotheses regarding gender effects, as previous findings have been rather inconsistent, with higher scores for girls (Lavallee & Parker, 2009), higher scores for boys (Lackaye & Margalit, 2006), or no differences between boys and girls (Kingery, Erdley, & Marshall, 2011).
Method
Participants and Procedure
The sample comprised 134 students from Grade 5 (66 girls and 68 boys) and 179 students from Grade 6 (99 girls and 80 boys), for a total of 313 children. The children were 10 to 13 years old (M = 11.06 years, SD = 0.73). Data were collected in 2013, in 16 classes in 6 schools in the Dutch-speaking part of Belgium. Information on the ethnic background of the students was not available, but the schools involved were known to attract mainly Caucasian students. Data of two participants were dropped from the current analyses, because they had missing data for one or more subscales. Of the remaining 311 children, 283 (91%) had complete data, whereas the others were missing one (7.4%) or a few items (1.6%). We imputed missing values using the relative mean substitution approach (Raaijmakers, 1999).
Parents were informed about the purpose of the study. In five of the schools, the parents could indicate in writing that they did not want their child to participate in the study (i.e., waiver of parental written consent). In one of the schools, parents had to give their consent in writing before their child could participate in the study (i.e., active parental consent). In all the schools, children could refuse to participate at any time, in line with ethical standards at the time of data collection. The children completed all instrumentation during regular classes that were supervised by an undergraduate student in psychology.
Measures
This study used the CLS and other self-reported questionnaires assessing depressive symptoms, social and global self-esteem, and school liking and avoidance. Furthermore, a set of peer nominations and ratings was used assessing friendships, peer acceptance and rejection, and peer victimization. Finally, a peer-reported measure of loneliness was used.
Self-reported measures
Depressive symptoms were measured using the Dutch Depression Questionnaire–Short Form (De Wit, 1987). This 12-item instrument comprises 9 primary items that tap into depressive symptoms (sample item: “I feel rather down lately”) and 3 filler items about favorite hobbies (sample item: “I like to play on the computer”). All items were responded to in binary fashion (yes, scored as 1, or no, scored as 0). Scores on the primary items are summed to yield an overall score for depressive symptoms. This scale has substantial validity as shown through a significant correlation with another self-reported depression measure for children, that is, the Children’s Depression Scale (Tisher, Lang-Takac, & Lang, 1992).
Both global self-esteem and social self-esteem (eight items each) were measured by means of a Dutch adaptation (Simons & Frisette, 2001) of two subscales of the Self-Description Questionnaire (Marsh, 1988). Sample items are “On the whole, I have a lot to be proud about” and “Most other kids like me” for global and social self-esteem, respectively. All items were responded to on a 5-point scale ranging from 1 (not at all) to 5 (always). Scores for each subscale were averaged to form an overall index of global and social self-esteem, respectively.
School liking and school avoidance were measured by means of the School Liking and School Avoidance Questionnaire (Ladd, 1990; Ladd & Price, 1987). Sample items for the School Liking (nine items) and the School Avoidance (five items) subscales are “Are you happy when you are at school?” and “Would you like to stay home rather than go to school?” All items were responded to on a 5-point scale ranging from 1 (not at all) to 5 (always). Scores for each subscale were averaged to form an overall index of school liking and school avoidance, respectively.
Peer-reported measures
Both peer nominations and peer ratings were used. For the nominations, each participant received a numbered alphabetical list of all class members and unlimited nominations were used. Friendship was measured by the number of nominations received for “being a friend.” Two sociometric items captured social acceptance (i.e., the children in your class you like most) and social rejection (i.e., the children in your class you like least). Children were instructed to read each item, consider the peers in their class who fitted the description best, and then write down the numbers of those peers. The number of received nominations was standardized within each class to account for differences in class size.
To measure peer victimization, peer ratings were used. Each participant had to rate all other class members on three victimization items, using a yes–no format. These items were “gets to hear bad things,” “others act mean to him/her,” and “is beaten or pushed.” For each child, the scores were averaged across classmates and used to create a three-item peer-rated measure of victimization. Peer-reported loneliness was measured in a similar way. For each of their classmates, children rated how lonely they thought each classmate felt. The rating for this item (“Feels lonely”) had to be performed using a 5-point scale ranging from 1 (not at all) to 5 (always). For each participant, the ratings received were averaged across all participating classmates to yield a standardized peer-rated measure of loneliness or loneliness reputation. Children were specifically instructed to refrain from self-ratings and any such ratings were ignored when averaging the received scores.
Results
Internal consistency for and correlations among all study variables are presented in Table 3. As can be seen, all self-report measures and the peer-reported victimization scale showed high levels of internal consistency (ranging between .77 and .92). The CLS exhibited significant correlations with all the other variables. As expected, loneliness was positively associated with depressive symptoms and school avoidance, and negatively with social self-esteem, global self-esteem, and school liking. For the peer-report measures, loneliness showed the expected negative associations with the number of friends and with social acceptance, and the expected positive associations with social rejection and victimization. A significant and positive correlation was found between self-reported and peer-reported loneliness. This peer-related measure was also strongly related with the other peer-reports, especially regarding peer victimization.
Descriptive Statistics, Reliability, and Intercorrelations of Study Variables.
Note. NA = not applicable.
p < .05. **p < .01. ***p < .001.
The results of the hierarchical regression analysis are presented in Table 4. In Step 1, gender was entered as a control variable; in Step 2, all related constructs were entered; and in Step 3, peer-reported loneliness was entered. The variance inflation factors (VIFs) did not indicate problems of multicollinearity (i.e., all VIF values <5; O’Brien, 2007). In all steps, a significant effect for gender was found, indicating that boys reported more loneliness than girls. In Step 2, depressive symptoms, social self-esteem, and peer-reported victimization significantly predicted CLS scores. In Step 3, the effect of peer-rated loneliness was significant, but the effect of peer-reported victimization disappeared. Collectively, the related constructs in Step 2 explained a sizeable portion of the variance in CLS scores (i.e., more than 50%). Adding peer-rated loneliness increased the explained variance by 1.5%.
Hierarchical Regression Analysis Predicting Loneliness.
Note. ΔR2 = .01 (p < .05) for Step 1, ΔR2 = .55 (p < .001) for Step 2, and ΔR2 = .02 (p < .01) for Step 3. VIF = variance inflation factor.
p < .05. **p < .01. ***p < .001.
Discussion
The present study expanded significantly on the extant knowledge base on the psychometric properties of the CLS, the primary measure of childhood loneliness. Specifically, analyses on children in Grades 5 and 6, for a total of 1,069 participants allowed us to fill two gaps in the literature on this scale.
First, we replicated the findings of Ebesutani et al. (2012) in a Belgian sample, providing evidence for the unidimensional nature of the CLS, when item wording is taken into account. This successful replication significantly extends current evidence on the factor structure of the CLS, which had hitherto been restricted, in large part, to studies conducted in the United States. At this moment, combining information across cultures, there is insufficient conceptual and empirical evidence to prefer a multifactor structure over a single-factor structure and researchers should be very cautious about creating subfactors for the CLS.
Second, we found that a substantial portion of the variance in the CLS scores was accounted for by a well-selected set of related constructs. Controlling for the other variables in the model, children experiencing feelings of loneliness, also experienced depressive symptoms and lower social self-esteem more often than their nonlonely peers. Furthermore, a substantial association was found with an alternative measure of the construct that used an innovative peer-report format. As the children were asked to judge their classmates’ internal state of loneliness, they might have based their judgement regarding their classmates’ loneliness on what they can observe, such as bullying and peer rejection. Indeed, the peer-related measure of loneliness was strongly related with the other peer-reports, especially the peer victimization measure, and it added little variance to the hierarchical regression. However, loneliness represents a subjective state that cannot be inferred easily from objective social interactions. Further research is needed to understand to which degree children rely on observed social interactions or expressions of negative affect, for instance, when asked to infer an individual’s internal state such as loneliness. Furthermore, some gender differences in loneliness were found, indicating that boys reported more loneliness than girls. However, this finding needs to be interpreted with caution as the effect was small and previous findings regarding gender differences in loneliness have been inconsistent.
The present study has several strengths, including the fact that two different samples were used and that we did not rely exclusively on self-reports for the measurement of loneliness. There are, however, some limitations that have to be pointed out. First, our research was conducted in a specific country on samples of mainly Caucasian children and our results may not generalize to other countries or to children with a different ethnic background. In our sample, measures related to the peer context were important predictors of loneliness. However, the peer context is not of equal importance in different cultures. Studies in cultures that attach a different value to relationships with peers than is common in the United States and Belgium could yield different results (Liu, Li, Purwono, Chen, & French, 2015). Second, our first study was conducted on relatively old data. However, the fact that our findings replicated more recent findings (Ebesutani et al., 2012) is encouraging. Third, our studies only examined children in Grades 5 and 6 and care should be taken, therefore, not to extend our findings to the entire intended age range of the instrument (i.e., roughly 5-15 years of age or preschool to junior high school).
The peer-reported loneliness measure used in the present study is promising and raises issues regarding the inferential basis of peer-reported loneliness and its link with self-reported loneliness. Additional work could be conducted regarding the associations between standard self-report measures such as the CLS and alternative measures of loneliness, using other types of informants, such as parents (Waaktaar & Torgersen, 2012) or teachers (Heiman, 2002). Pending further comparative research that uses such measures, the present series of studies provides a solid foundation for the continuing use of the CLS in research on children’s loneliness.
Footnotes
Acknowledgements
The authors are greatly indebted to Esther Beckers, An Callewaert, Sofie Derez, Charlotte Maertens, Anneleen Sniekers, Greet Vanderelst, Sara VanVlasselaer, and Tilly Verhoeven for their assistance in data collection.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
