How Universal Is a Construct of Loneliness? Measurement Invariance of the UCLA Loneliness Scale in Indonesia,Germany,and the United States

Abstract

The UCLA Loneliness Scale (ULS-20) and its short version (ULS-8) are widely used to measure loneliness. However, the question remains whether or not previous studies using the scale to measure loneliness are measuring the construct equally across countries. The present study examined the measurement invariance (MI) of both scales in Germany, Indonesia, and the United States (N = 2350). The one-, two-, and three-factor structure of the ULS-20 did not meet the model fit cut-off criteria in the total sample. The ULS-8 met the model fit cut-off criteria and has configural, but not metric invariance because two items unrelated to social isolation were not MI. The final six items (ULS-6) exclusively related to social isolation had complete MI. Participants from the United States scored highest in the ULS-6, followed by participants from Germany and then Indonesia. We conclude that the ULS-6 is an appropriate measure for cross-cultural studies on loneliness.

Keywords

loneliness cross-cultural confirmatory factor analysis measurement invariance UCLA Loneliness Scale social isolation

Loneliness has been increasingly studied in the past few decades. Studies have found loneliness to be an important predictor for physical health (Hawkley & Cacioppo, 2010; Holt-Lunstad et al., 2015), mental health (Gollwitzer et al., 2018; Hawkley & Cacioppo, 2010), and cognitive functioning (Boss et al., 2015). A scale called the University of California, Los Angeles (UCLA) Loneliness Scale (the ULS-20; Russell et al., 1978) and its more recent revisions (Russell, 1996; Russell et al., 1980) has been considered as the cardinal measure of loneliness (Austin, 1983). The ULS-20 consists of 20 statement items with 4-point Likert-type scale that has been used extensively in many countries; for example, United States, Hartshorne (1993), Ireland (Shevlin et al., 2015), Taiwan (Wu & Yao, 2008), Malaysia (Swami, 2009), and Turkey (Yildiz & Duy, 2014). The ULS has been used and validated in adolescents (Shevlin et al., 2015; Wilson et al., 1992), young adults (Hawkley et al., 2005; Russell, 1996), midlife adults (Hawkley et al., 2005), and older adults (Durak & Senol-Durak, 2010). The ULS has not been used in children.

There are several conceptualizations of the ULS-20, including 1-Factor (Russell, 1996), two-factor (Wilson et al., 1992), three-factor (Hawkley et al., 2005), and various short forms of the ULS-20 composed of either three items (ULS-3; Hughes et al., 2004), four items (ULS-4; Russell et al., 1980), or eight items (ULS-8; Hays & DiMatteo, 1987; Roberts et al., 1993). However, no study has compared all of them in one sample. Furthermore, while the ULS-20 has been shown to have good psychometric properties in different countries independently (e.g., United States, Hartshorne, 1993; Ireland, Shevlin et al., 2015; Taiwan, Wu & Yao, 2008; Malaysia, Swami, 2009), we cannot assume cross-cultural validity because so far, no study has investigated measurement invariance (MI; Byrne & Van de Vijver, 2010). Thus, loneliness as measured by the ULS-20 may be understood and expressed differently across countries.

In the literature, the ULS-20 has been used to measure loneliness in different ways. Originally, the ULS-20 was conceptualized to consist of a one-factor structure of loneliness, which has been referred to as Global Loneliness Factor (Russell, 1996). Here, loneliness is defined as a subjective psychological condition that can be felt even in the presence of other people. However, the ULS-20 has also been shown to consist of a two-factor structure (Dodeen, 2015; Knight et al., 1988; Wilson et al., 1992). Wilson et al. (1992) distinguishes between the intensity and frequency of interactions with social units. The first dimension, “Social Other,” represents the individual attachment with social units, while, the second dimension, “Intimate Other,” represents the relational intensity with the social units. Knight et al. (1988) also distinguished between the reverse worded items and positively worded items and labelled those as “Positive” and “Negative.” However, this differentiation may have produced a method effect—in which the structure may be attributable the weakness of reverse worded items rather than to the variance that was intended to be measured, that is, the construct of loneliness (Maul, 2013). A third group of studies found the ULS-20 to consist of a three-factor structure (Durak & Senol-Durak, 2010; Dussault et al., 2009; Hawkley et al., 2005; Kwiatkowska et al., 2017; Shevlin et al., 2015; Zarei et al., 2016), which differentiates between interpersonal and collective attachment with social units as well as their isolatedness with such units. The first dimension, “Isolation,” relates to the disengagement of individuals with the social units, the second dimension, “Relational Connectedness,” represents interpersonal attachment of a person with other individuals and the third dimension, “Collective Connectedness,” relates to a person’s attachment with collective groups.

Aside from the various dimensional structures, shorter versions of the ULS-20 have also been proposed. One of them consists of eight items (ULS-8) and is similar to the “Isolation” dimension proposed by Hawkley et al. (2005). With the exception of two items (“I am an outgoing person” and “People are around me but not with me”)—which have been noted to have a different meaning (Wu & Yao, 2008)—the ULS-8 focuses mostly on the actual condition of isolation rather than on subjective feelings of social disconnectedness (see Table 1). Thus, previous authors have suggested that it reflects perceived social isolation rather than loneliness (Hays & DiMatteo, 1987; Yildiz & Duy, 2014), most items indeed deal primarily with the actual condition of disengagement from social networks The ULS-8 has been shown to have good reliability and validity in Asian contexts (Swami, 2009; Wu & Yao, 2008). Although there are further short versions, such as the ULS-4, Russell et al. (1980) or the ULS-3; Hughes et al. (2004), the ULS-8 has the most convincing psychometric properties and is ideal for a comparison of perceived social isolation with the broader construct of loneliness as presented in the ULS-20, which would allow to examine whether people from various cultural contexts construe both loneliness and the perception of social isolation in a similar manner.

Table 1.

Various Factorial Structures of UCLA Loneliness Scale.

#	Items	One-factor (Russell, 1996)	Two-factor (Wilson et al., 1992)		Three-factor (Hawkley et al., 2005)			ULS-8 (Hays & DiMatteo, 1987)	ULS-6
#	Items	Global loneliness dimension (1)	Social other (1)	Intimate other (2)	Isolation (1)	Relational connectedness (2)	Collective connectedness (3)	Social isolation (1)	Cross-cultural social isolation(1)
	α	.91	.88	.89	.88	0.87	.72	.80	.82
1	I felt in tune with the people around me	V	V				V
2	I lacked companionship	V		V	V			V	V
3	There was no one I can turn to	V		V	V			V	V
4	I did not feel alone	V		V	V
5	I felt part of a group of friends	V	V				V
6	I had a lot in common with the people around me	V	V				V
7	I was no longer close to anyone	V		V	V
8	My interests and ideas were not shared by those around me	V		V			V
9	I was an outgoing person	V	V				V	V
10	There were people I feel close to	V	V			V
11	I felt left out	V		V	V			V	V
12	My social relationships were superficial	V		V	V
13	No one really knew me well	V		V	V
14	I felt isolated from others	V		V	V			V	V
15	I could find companionship when I wanted it	V	V			V		V	V
16	There were people who really understood me	V		V		V
17	I was unhappy being so withdrawn	V		V	V			V
18	People were around me but not with me	V		V	V			V	V
19	There were people I could talk to	V	V			V
20	There were people I could turn to	V	V			V

Note. The ULS-6 is a scale made by eliminating two items from the ULS-8 in this study.

As it has become clear from the different factor solutions that loneliness is a complex phenomenon, the meaning of the word loneliness may be conceptually be different in various cultural contexts and different languages. Although all these different versions of the ULS have been used in various cultural contexts, so far we cannot draw any conclusion with regard to its cross-cultural validity. In the English language, the word loneliness is used to describe a negative psychological condition. This is not necessarily the same in other languages. Although mostly used in a negative context, the German word for loneliness (“Einsamkeit,” n.d.) is sometimes used in a positive manner to reflect a state of seeking peace and quietness; to be oriented toward higher goals (de Jong Gierveld, 1998). For example, the German romantic philosophers Goethe used Einsamkeit to describe the desirable condition of withdrawal from superficiality of the society and to hear the voice of the inward attachment (Von Goethe, 1995). The Indonesian word for loneliness is Ke-sepi-an, which is from Sepi meaning quietness and lack of crowd. Therefore, the emphasis is on the objective condition of lack of sounds and/or people rather than on the psychological condition. Because of how the word for loneliness is used differently in the three languages, it is possible that loneliness is not represented as a similar construct across languages.

Loneliness was demonstrated to predict physical and mental health outcomes similarly across Indonesia, Germany, and United States (Beutel et al., 2017; Gerst-Emerson & Jayawardhana, 2015; Peltzer & Pengpid, 2019; Stickley et al., 2014). However, loneliness may manifest differently depending on the individual’s culture. Individuals living in countries who have a higher need for existential security (e.g., countries in poverty or postconflict conditions) such as Indonesia (a low-and-middle-income country) have been shown to prioritize societal obligations, in-group harmony, and intergroup experiences, rather than interpersonal closeness that is central to the concept of loneliness in the literature (Rokach, 1998, Seepersad et al., 2008). On the other hand, individuals living in countries with better existential security such as Germany and the United States have been shown to prioritize interpersonal closeness (Rokach, 1998). Furthermore, the absence of interpersonal relationships has been demonstrated to be closely linked to the concept of loneliness in individualistic societies, while the absence of ties with collective groups (e.g., family) rather than absence of interpersonal relationships are more likely to be associated with loneliness in collectivistic societies (Lykes & Kemmelmeier, 2014).

Loneliness experience may be more prevalent in the societies where modernization occurs. It was argued by Sundström et al. (2009) that in the highly developed countries such as the United States and Germany, individuals are more likely to be isolated from larger communities. They assumed that in such societies, household atomization occurs where there are increasing number of single households, “never married” people, and increase in individualistic lifestyle. In other words, within such societies, the experience of social isolation is intensified as opposed to the societies where modernization is not quite prevalent. There, people are less likely to be isolated from larger communities (e.g., extended family, close-knit communities). Recent research that collected a global data demonstrated that people in individualistic well-developed cultures were more prone to the subjective feelings of isolatedness compared with those in the collectivistic cultures (Barreto et al., 2021).

In order to examine whether a construct is similar across cultural contexts and different languages, we need to compute MI. MI assesses the equivalence of measurement attributes across different conditions, such as different cultures (Steenkamp & Baumgartner, 1998). Specifically, MI refers to a statistical analysis that is employed to examine whether measurement operations yield measures of the same attributes throughout different observed conditions or groups (Horn & McArdle, 1992). There are four levels of MI (Horn & McArdle, 1992): (1) configural invariance, which refers to a condition in which two or more groups construe the ULS-20 items with the respective factor structure similarly; (2) metric invariance, which refers to the question of whether the factor loadings of the ULS-20 are of similar strength across countries; (3) scalar invariance, which refers to a condition, in which individual mean scores for the latent structure of the construct are consistent with the observed variables across all groups. For the ULS-20, this would imply that individuals who have the same mean scores on the latent construct of loneliness would obtain the same score for the observed variable items regardless of country; (4) error variance invariance, which refers to a condition, in which the same level of measurement error is found across all groups. The process of obtaining MI can be considered to be like staircase in which that one step must be passed before moving on to the next step which reflects the more restrictive the criteria for equivalence (Byrne & Van de Vijver, 2010).

In the present study, we tested four versions of the ULS-20 (one-factor ULS-20, two-factor ULS-20, three-factor ULS-20, and the ULS-8) in samples from three countries (Germany, Indonesia, and the United States). After testing for valid factorial structures in the three different countries, we compared whether the valid factorial structures vary across countries using MI analysis. Achieving MI of the scale is important for an unbiased mean comparison between groups, without a scale that is MI mean differences between groups may reflect measurement error differences rather than true group differences. In other words, only when the ULS items have been regarded as invariant we can know whether the difference in loneliness score is due to countries rather than measurement error. Therefore, we conducted a group comparison using mean scores from the original one-factor ULS-20 and the version of ULS that achieve satisfactory MI using analysis of variance (ANOVA). This may reveal whether previously reported differences of mean scores between cultures may be reflective of measurement error differences or loneliness differences.

Method

Participants and Procedure

All participants from Germany, Indonesia, and the United States were recruited using CrowdFlower, internet forums, and social networking websites. Participants recruited from CrowdFlower were rewarded US$ 0.50 for their participation, while participants recruited from internet forums and social networking websites were not given compensation. The participants were asked for their consent before participating in the study and only individuals older than 18 years were allowed to participate. The Ethics Committee at the Universität Hamburg approved the survey. Initially, a total of 2,501 participants completed the questionnaires. However, we excluded 151 participants due to duplicate entries (n = 98), inconsistent answers (n = 7), and responding with same answers 50 times (n = 46). Ultimately, we analyzed 2,350 participants (M_age = 32.53, n_female = 37.8%). There was a total of 786 participants from Germany, M_age (SD) = 26.59 (8.90), n_female = 41.8%, 844 participants from Indonesia, M_age (SD) = 29.23 (8.41), n_female = 78.7%), and 720 participants from the United States, M_age (SD) = 34.35 (12.30), n_female = 39.8%. On average, the participants were 32.53 years old and 37.8% were women. A total of 6.8% participants were unemployed. Details of the participants’ education and income in each country are provided in Supplementary Material B (available online). The original study included further outcome measures that are reported in previous publications based on this data set (Jaya, 2017; Jaya et al., 2018; Jaya et al., 2020; Jaya & Lincoln, 2016). We report how we determined our sample size, all data exclusions, all manipulations, and all measures in the study.

UCLA Loneliness Scale–20 (ULS-20)

The ULS-20 consists of 20 items, in which each item is rated on a 4-point Likert-type scale (1 = “never,” 2 = “rarely,” 3 = “sometimes,” 4 = “often”). Table 1 shows where the 20 items are assigned into the various versions of the ULS-20. The English version of the ULS-20 was back translated into German and Indonesian for this study. The internal consistencies of the various versions of the ULS-20 in the total sample were good (see Table 1). The mean and standard deviations of the four versions of ULS-20 are shown in Table 2.

Table 2.

Descriptive Statistics of Various Version and Dimensions of the UCLA Loneliness Scale Across Three Countries.

Measure	All samples (N = 2,350)		Germany (n = 786)		Indonesia (n = 844)		United States (n = 720)
Measure	M	SD	M	SD	M	SD	M	SD
ULS-20, One-factor
Global loneliness	44.43	11.76	43.48	11.27	42.66	10.38	46.84	13.37
ULS-20, Two-factor
Social other	17.51	5.61	17.73	5.53	16.61	5.34	18.33	5.85
Intimate other	26.91	7.95	26.43	7.74	26.20	7.03	28.29	8.97
ULS-20, Three-factor
Isolation	22.28	7.01	21.94	6.82	21.53	6.29	23.55	7.82
Relational connectedness	10.52	3.81	10.24	3.85	10.34	3.57	11.03	4.01
Collective connectedness	11.63	3.25	11.98	3.12	10.94	3.00	12.04	3.52
ULS-8
Social isolation	17.95	5.17	17.89	4.89	17.02	4.54	19.11	5.88
ULS-6
Cross-cultural social isolation	13.14	4.29	12.78	4.20	12.36	3.83	14.16	4.56

Note. ULS = UCLA Loneliness Scale.

Data Analysis

In order to determine what dimensional structure of loneliness fits the data from all three countries, we conducted a series of confirmatory factor analyses (CFAs). Because the one-factor ULS-20 was the first structure created by its designer (Russell, 1996; Russell et al., 1980), we first conducted a CFA for the one-factor ULS-20. Then, we compared it with other dimensional structures in the literature, that is, the two-factor ULS-20 (Wilson et al., 1992), the three-factor ULS-20 (Hawkley et al., 2005), and the ULS-8 (Hays & DiMatteo, 1987). After discovering the best dimensional structure(s) that fits the data, we conducted a MI analysis to examine whether the dimensional structure was interpreted in a similar manner by participants from the three countries and thus whether the mean scores across countries for this structure can be compared with each other.

First, we conducted the CFA using mean adjusted weighted least square (WLSM) to examine the factorial structures for all four versions of the ULS-20 (one-factor ULS-20, two-factor ULS-20, three-factor ULS-20, and ULS-8) in both the total sample and within each country sample. WLSM performs better when the responses are ordinal (equal or lesser than a 7-point Likert-type scale; Schmitt, 2011) and it is also the better estimator for a sample size of more than 500 (Yu & Muthén, 2002). We refrain from using other estimators (e.g., maximum likelihood, mean-adjusted maximum likelihood, mean and variance-adjusted maximum likelihood) because they may underestimate factor loadings and may not be optimal in estimating the χ² test of fit when the responses are ordinal (Tarka, 2017). The following fit indices, along with the proposed cut-off criteria, were used to assess model fit: confirmatory fit index (CFI) > 0.95, root mean square error of approximation (RMSEA) < 0.08, and standardized root mean square residual (SRMR) < 0.08 (Marsh et al., 2004; van Overveld et al., 2011). In addition, we used the Tucker–Lewis index (TLI) as another proposed index of model fit. We used the recommended cut-off criteria of TLI > 0.95, as suggested by Hu and Bentler (1999). To establish configural MI, we followed the recommended stringent criteria from a simulation study because of our large sample size (n > 300), ΔCFI ≤ 0.010, supplemented by ΔRMSEA ≤ 0.015 or ΔSRMR ≤ 0.030 (Chen, 2007). As for metric and error variance MI, we followed the recommended ΔCFI ≤ 0.010, supplemented by ΔRMSEA ≤ 0.015 or ΔSRMR ≤ 0.010 (Chen, 2007). We then conducted the MI analysis step-by-step, again using WLSM. We examined the modification indices when the cut-off criteria were not achieved at each step and deleted the recommended items as suggested by the indices (Putnick & Bornstein, 2016). When the model was reported to be adequate with modifications, we regard it as having partial MI. When all of the MI standards were achieved (with or without modifications), we tested the mean score difference for three separate sample groups using ANOVA statistics. We utilized lavaan package of R version 3.4.3 to compute all statistics.

Results

Participants’ Characteristics

A detailed description of the demographic characteristics of the participants have been reported elsewhere (e.g., Jaya et al., 2017; Jaya & Lincoln, 2016). Here, we describe them briefly. On average, the participants were 32.53 years old and 37.8% were women. A total of 6.8% participants were unemployed. The participants tend to have higher education. Details of the participants’ education and income in each country are provided in Supplementary Material B (available online). There was a total of 786 participants from Germany, M_age (SD) = 34.19 (12.51), n_female = 241 or 30.7%, 844 participants from Indonesia, M_age (SD) = 29.55 (8.43), n_female = 213 or 25.2%, and 720 participants from the United States, M_age (SD) = 34.19 (12.35), n_female = 434 or 60.3%.

Factorial Structure of the Four Versions of ULS-20 in the Total Sample

Table 3 shows the goodness-of-fit criteria of the four versions of ULS-20 in the total sample. We found that the one-factor ULS-20 did not achieve goodness-of-fit for all indicators. The two-factor ULS-20 had a better fit for all indicators when compared with the one-factor ULS-20. Even so, the two-factor ULS-20 did not achieve a good fit. Moreover, we found that the thee-factor ULS-20 had better fit in all indicators in comparison with both the one-factor ULS-20 and the two-factor ULS-20. Still we found that the three-factor ULS-20 did not meet our proposed fit criteria. The ULS-8 met all of our proposed fit criteria. All factor loadings for four versions of the ULS-20 are illustrated in Table 4 and Supplementary Figures 1, 2, 3, and 4 (available online).

Table 3.

Factorial Structure in the Total Sample (N = 2,350).

#	Measure	χ² (df)	SRMR	CFI	TLI	RMSEA [95% CI]
1.	ULS-20, One-factor	15726.223 (170)	0.131	0.851	0.833	0.142 [0.140, 0.144]
2.	ULS-20, Two-factor	9372.550 (169)	0.097	0.919	0.909	0.105 [0.103, 0.107]
3.	ULS-20, Three-factor	8571.659 (167)	0.091	0.929	0.920	0.099 [0.097, 0.100]
4.	ULS-8	501.729 (20)	0.047	0.979	0.971	0.064 [0.059, 0.069]
5.	ULS-6	35.509 (9)	0.025	0.996	0.994	0.035 [0.024, 0.048]

Note. df = degree of freedom; SRMR = standardized root mean square residual; CFI = confirmatory fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; CI = confidence interval.

Table 4.

Factor Loadings for All Dimensional Structures.

#	Items	One-factor (Russell, 1996)	Two-factor (Wilson et al., 1992)		Three-factor (Hawkley et al., 2005)			ULS-8 (Hays & DiMatteo, 1987)	ULS-6
#	Items	Global loneliness dimension (1)	Social other (1)	Intimate other (2)	Isolation (1)	Relational connectedness (2)	Collective connectedness (3)	Social isolation (1)	Cross-cultural social isolation (1)
	α	.91	.88	.89	.88	.87	.72	.80	.82
1	I felt in tune with the people around me	0.60	0.72				0.71
2	I lacked companionship	0.50		0.57	0.58			0.61	0.58
3	There was no one I can turn to	0.64		0.70	0.71			0.68	0.53
4	I did not feel alone	0.44		0.40	0.40
5	I felt part of a group of friends	0.66	0.79				0.77
6	I had a lot in common with the people around me	0.61	0.73				0.72
7	I was no longer close to anyone	0.64		0.71	0.72
8	My interests and ideas were not shared by those around me	0.44		0.51			0.41
9	I was an outgoing person	0.35	0.44				0.42	0.16
10	There were people I feel close to	0.57	0.70			0.73
11	I felt left out	0.67		0.75	0.77			0.80	0.35
12	My social relationships were superficial	0.57		0.64	0.65
13	No one really knew me well	0.65		0.73	0.74
14	I felt isolated from others	0.73		0.81	0.83			0.83	0.33
15	I could find companionship when I wanted it	0.52	0.63			0.65		0.29	0.86
16	There were people who really understood me	0.64		0.60		0.80
17	I was unhappy being so withdrawn	0.43		0.50	0.51			0.58
18	People were around me but not with me	0.62		0.70	0.71			0.73	0.48
19	There were people I could talk to	0.61	0.76			0.80
20	There were people I could turn to	0.64	0.77			0.82

MI Between Sample Groups

On the grounds that only the ULS-8 met all our proposed fit criteria in the total sample, we conducted the MI analysis solely on the ULS-8. As presented in Table 5, we found configural MI for the ULS-8 as it met all of our fit cut-off criteria, χ²(60) = 638.05, SRMR = 0.05, CFI = 0.98, TLI = 0.97, RMSEA [confidence interval: CI] = 0.07 [0.07, 0.08]). We then tested for metric MI for which we did not find evidence, χ²(70) = 1159.26, SRMR = 0.09, CFI = 0.89, TLI = 0.87, RMSEA = 0.14 [0.13, 0.15]. Thus, we consulted the modification indices and found that Item 9 and Item 17 of ULS-20 (Item 3 and Item 7 of the ULS-8) were the reason for non-MI. Items 9 and 17 refer to the individual’s perception of his or her interactional disposition (e.g., being outgoing, being withdrawn), whereas the other six items refer to be the perception of being isolated versus nonisolated. This difference in meaning has also been noted by other researchers (Wu & Yao, 2008). Also, despite their similarity, Item 17 loaded on the isolation factor in the three-factor loneliness (which was similar to what ULS-8 measure, i.e., Social Isolation), whereas Item 9 loaded on Collective Connectedness factor, which also raises questions with regard to the construct they assess. Thus, we decided to delete both items, which resulted in the ULS-6 meeting all of our proposed fit criteria for both configural, MI, χ²(27) = 142.23, SRMR = 0.03, CFI = 0.98, RMSEA [CI] = 0.07 [.04, .05] and metric MI, χ²(70) = 324.59, ΔSRMR = 0.03, ΔCFI ≤ 0.01, ΔRMSEA ≤ 0.01. Having metric MI, we then tested the ULS-6 for scalar MI for which we found evidence, χ² (82) = 438.19, SRMR = 0.07, CFI = 0.96, TLI = 0.96, RMSEA [CI] = 0.07 [0.07, 0.08]. Finally, we tested for error variance MI, for which we also found evidence, χ²(98) = 561.65, SRMR = 0.08, CFI = 0.96, TLI = 0.96, RMSEA = 0.08 [0.07, 0.08]). Thus, the ULS-6 met all criteria for MI across Germany, Indonesia, and the U.S. samples.

Table 5.

Measurement Invariance of the ULS-6 Between Participants From Indonesia, German, and United States.

Model	Robust χ²			CFI	ΔCFI	RMSEA	ΔRMSEA	SRMR	ΔSRMR	Decision
Model	χ²	df	p	CFI	ΔCFI	RMSEA	ΔRMSEA	SRMR	ΔSRMR	Decision
Configural invariance model (ULS-6)	142.23	27	<.01	0.98		0.07		0.03		Accept
Metric invariance model (ULS-6)	324.59	70	<.01	0.98	<.01	0.07	<.01	0.06	0.03	Accept
Scalar invariance model (ULS-6)	438.19	82	<.01	0.96	0.02	0.07	<0.01	0.07	0.01	Accept
Error variance invariance model (ULS-6)	561.65	98	<.01	0.96	<0.01	0.08	0.01	0.08	0.01	Accept

Note. ULS-6 = cross-cultural social isolation measure; df = degrees of freedom; SRMR = standardized root mean square residual; CFI = confirmatory fit index; RMSEA = root mean square error of approximation.

Mean Comparison Across Countries

Having found all models of MI to be adequate, we then tested for mean differences of the ULS-6 between the samples from Germany, Indonesia, and the United States. We computed ANOVA and found that there was indeed a difference of mean scores between samples of three countries, F(2, 2347) = 37.57, p < .001. Tukey’s HSD post hoc analysis revealed that the mean scores of the participants from the United States were significantly higher than the mean scores of participants in Germany (p < .001, Cohen’s d = 0.30) and in Indonesia (p < .001, Cohen’s d = 0.42). We also found that the mean scores of participants from Germany were significantly higher compared with participants from Indonesia although this difference was not large (p = .04, Cohen’s d = 0.12). The difference pattern was not similar for when we computed the ANOVA for one-factor ULS-20. In spite of the statistical significance, F(2, 2347) = 30.12, p < .001, Tukey’s HSD analysis revealed that there was no difference between the mean scores of participants from Germany and Indonesia (p = .36, Cohen’s d = 0.08). There were mean scores differences between participants from the United States and Germany (p < .001, Cohen’s d = 0.27), as well as the United States and Indonesia (p < .001, Cohen’s d = 0.35). These results demonstrated that comparing participants across three countries using the scale with unsatisfactory MI (ULS-20) could overestimate the cultural differences compared with the scale which achieved MI (ULS-6). Thus, it is not recommended to conduct mean comparison using the scale without testing for its MI.

Discussion

This study compared the different factor structures of the widely used measure of loneliness, ULS-20, and its MI across three countries from three continents. We found that the ULS-8 had a good fit to the data in comparison with the other versions of ULS-20. We also found ULS-8 had full configural invariance across Germany, Indonesia, and the United States. This means that the construct of loneliness as measured by ULS-8 is understood similarly across the three countries. However, we failed to find evidence of full metric invariance for ULS-8. We found full configural invariance, metric invariance, scalar invariance, and error variance invariance for a modified version of ULS-8 that consists of six items called ULS-6. Thus, here we report a cross-culturally equivalent measure of loneliness based on the six items of ULS (ULS-6) that can be used for cross-culture comparison of loneliness.

Past work suggests that the historical context of a country may affect how people construe loneliness in their minds (Rokach et al., 2001). We argue that psychological essentialization or the belief that individuals possess the essence-like unchangeable properties such as fixed personality is less prevalent in Asian countries such as Indonesia. Thus, the belief that one’s outgoing personality is consistent may not be shared in such context (Tsukamoto et al., 2015) which contributes to dissimilar views about the essentialization of self when compared with the United States or Germany. Thus, the deleted two items in the ULS-8 may lack metric invariance simply because it represents psychological essentialization (e.g., I am outgoing) which may not be common in non-Western contexts.

Also, it is possible that the reason why Item 17 was perceived as not similar across different countries is due to how German culture may sometimes regard withdrawing from the society as positive psychological experiences. Einsamkeit, which is the direct translation of loneliness in German, may be understood as a condition that can lead to the fulfillment of higher goals or withdrawal from the superficiality of the masses. The language being used in the cultural context reflects the cognitive processing for the people within the context (Strickland, 2017). Words and grammatical structures being used in specific cultures have gone through a long evolutionary process and this shaped how people think in their respective society.

Because the ULS-6 was found to be MI across the Indonesian, German, and United States sample, we are able to compare the level of loneliness between these countries based on the ULS-6. We found that the participants from the United States reported the highest level of loneliness and the participants from Indonesia reported the lowest. This is consistent with the notion that loneliness is intensified in the context where modernization occurs (Sundström et al., 2009). Within highly developed, more individualistic countries, such as the United States and Germany, societal modernization brought household atomization, in which individuals are more strongly isolated from larger communities. On the other hand, it may not be as straightforward as inferring that loneliness is generally higher in individualistic cultures because at least one study found that people in a collectivistic culture scored higher in loneliness than those in an individualistic culture (Lykes & Kemmelmeier, 2014). Thus, it might be interesting for future studies to explore whether collectivistic cultures with recently occurring strong modernization (e.g., Japanese, Korean, etc.) are particularly prone to loneliness. This type of research would enable us to answer whether the observed differences in our study are due to individualistic versus collectivistic cultures or well-developed versus underdeveloped nations.

The six-item version of scale was found to be cross-culturally adequate. However, this is not the briefest version of the scale. The briefest version of the scale is ULS-3 consisting of three items that have been shown to measure loneliness adequately for practical telephone and email use where number of items is an issue (Hughes et al., 2004). Although ULS-6 includes all items from ULS-3 (see Hughes et al., 2004, for the list of the items), ULS-3 should be the reference for short-version measure of loneliness and ULS-6 as the cross-culturally adequate measure of loneliness.

In terms of limitations, we must note the possibility that our method of data collection may have overestimated the similarities across countries because we obtained the data from a similar population across the three countries, that is, internet users. In addition, it is important to investigate other cultures in other continents, such as Africa, Australia, and South America. Such investigations will establish a stronger claim of universality or cultural relativism. Moreover, our method of data collection may not enable a firm cultural distinction between the countries because we did not attempt to obtain a representative data across all subcultures within the countries.

Conclusion

To summarize, we found that compared with the various factorial structures of the ULS-20, the ULS-8 stands as a stronger measure across three countries (Indonesia, Germany, and the United States). We found the ULS-8 to have configural invariance, which means that participants across countries experience a similar kind of loneliness. However, the ULS-8 does not appear to have metric invariance, which means that participants across the three countries emphasize different ways in expressing loneliness through the items of the ULS-8. This means that any group differences between countries using the ULS-20 and the ULS-8 may reflect differing psychometric properties, rather than true differences. Since two items of the ULS-8 were assumed to be associated with psychological essentialization as opposed to an objective condition, we then deleted those two items and found the final six items to have MI in all levels. When we compare the mean score of ULS-6 across three countries, we found statistically significant differences between countries. Participants from Indonesia were the least lonely, followed by Germany. Participants from the United States mean score was highest compared with the other two countries. The non-MI ULS-20 erroneously showed no significant differences in loneliness between participants from Germany and Indonesia. Thus, the new ULS-6 is a valid measure for cross-cultural studies on loneliness and can be used in cross-cultural comparison.

Supplemental Material

sj-pdf-1-asm-10.1177_10731911211034564 – Supplemental material for How Universal Is a Construct of Loneliness? Measurement Invariance of the UCLA Loneliness Scale in Indonesia, Germany, and the United States

Supplemental material, sj-pdf-1-asm-10.1177_10731911211034564 for How Universal Is a Construct of Loneliness? Measurement Invariance of the UCLA Loneliness Scale in Indonesia, Germany, and the United States by Joevarian Hudiyana, Tania M. Lincoln, Steffi Hartanto, Muhammad A. Shadiqi, Mirra N. Milla, Hamdi Muluk and Edo S. Jaya in Assessment

Supplemental Material

sj-pdf-2-asm-10.1177_10731911211034564 – Supplemental material for How Universal Is a Construct of Loneliness? Measurement Invariance of the UCLA Loneliness Scale in Indonesia, Germany, and the United States

Supplemental material, sj-pdf-2-asm-10.1177_10731911211034564 for How Universal Is a Construct of Loneliness? Measurement Invariance of the UCLA Loneliness Scale in Indonesia, Germany, and the United States by Joevarian Hudiyana, Tania M. Lincoln, Steffi Hartanto, Muhammad A. Shadiqi, Mirra N. Milla, Hamdi Muluk and Edo S. Jaya in Assessment

Supplemental Material

sj-pdf-3-asm-10.1177_10731911211034564 – Supplemental material for How Universal Is a Construct of Loneliness? Measurement Invariance of the UCLA Loneliness Scale in Indonesia, Germany, and the United States

Supplemental material, sj-pdf-3-asm-10.1177_10731911211034564 for How Universal Is a Construct of Loneliness? Measurement Invariance of the UCLA Loneliness Scale in Indonesia, Germany, and the United States by Joevarian Hudiyana, Tania M. Lincoln, Steffi Hartanto, Muhammad A. Shadiqi, Mirra N. Milla, Hamdi Muluk and Edo S. Jaya in Assessment

Supplemental Material

sj-pdf-4-asm-10.1177_10731911211034564 – Supplemental material for How Universal Is a Construct of Loneliness? Measurement Invariance of the UCLA Loneliness Scale in Indonesia, Germany, and the United States

Supplemental material, sj-pdf-4-asm-10.1177_10731911211034564 for How Universal Is a Construct of Loneliness? Measurement Invariance of the UCLA Loneliness Scale in Indonesia, Germany, and the United States by Joevarian Hudiyana, Tania M. Lincoln, Steffi Hartanto, Muhammad A. Shadiqi, Mirra N. Milla, Hamdi Muluk and Edo S. Jaya in Assessment

Supplemental Material

sj-pdf-5-asm-10.1177_10731911211034564 – Supplemental material for How Universal Is a Construct of Loneliness? Measurement Invariance of the UCLA Loneliness Scale in Indonesia, Germany, and the United States

Supplemental material, sj-pdf-5-asm-10.1177_10731911211034564 for How Universal Is a Construct of Loneliness? Measurement Invariance of the UCLA Loneliness Scale in Indonesia, Germany, and the United States by Joevarian Hudiyana, Tania M. Lincoln, Steffi Hartanto, Muhammad A. Shadiqi, Mirra N. Milla, Hamdi Muluk and Edo S. Jaya in Assessment

Supplemental Material

sj-pdf-6-asm-10.1177_10731911211034564 – Supplemental material for How Universal Is a Construct of Loneliness? Measurement Invariance of the UCLA Loneliness Scale in Indonesia, Germany, and the United States

Supplemental material, sj-pdf-6-asm-10.1177_10731911211034564 for How Universal Is a Construct of Loneliness? Measurement Invariance of the UCLA Loneliness Scale in Indonesia, Germany, and the United States by Joevarian Hudiyana, Tania M. Lincoln, Steffi Hartanto, Muhammad A. Shadiqi, Mirra N. Milla, Hamdi Muluk and Edo S. Jaya in Assessment

Footnotes

Acknowledgements

We thank Salima for her support in proofreading the article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Joevarian Hudiyana

Steffi Hartanto

Supplemental material

Supplemental material for this article is available online.

References

Austin

B. A.

(1983). Factorial structure of the UCLA Loneliness Scale. Psychological Reports, 53(3), 883-889. https://doi.org/10.2466/pr0.1983.53.3.883

Barreto

Victor

Hammond

Eccles

Richins

M. T.

Qualter

(2021). Loneliness around the world: Age, gender, and cultural differences in loneliness. Personality and Individual Differences, 169(February), Article 110066. https://doi.org/10.1016/j.paid.2020.110066

Beutel

M. E.

Klein

E. M.

Brähler

Reiner

Jünger

Michal

Wiltink

Wild

P. S.

Münzel

Lackner

K. J.

Tibubos

A. N.

(2017). Loneliness in the general population: Prevalence, determinants and relations to mental health. BMC Psychiatry, 17(1), 1-7. https://doi.org/10.1186/s12888-017-1262-x

Boss

Kang

D. H.

Branson

(2015). Loneliness and cognitive function in the older adult: A systematic review. International Psychogeriatrics, 27(4), 541-553. https://doi.org/10.1017/S1041610214002749

Byrne

B. M.

Van de Vijver

F. J.

(2010). Testing for measurement and structural equivalence in large-scale cross-cultural studies: Addressing the issue of nonequivalence. International Journal of Testing, 10(2), 107-132. https://doi.org/10.1080/15305051003637306

Chen

F. F.

(2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464-504. https://doi.org/10.1080/10705510701301834

de Jong Gierveld

. (1998). A review of loneliness: concept and definitions, determinants and consequences. Reviews in Clinical Gerontology, 8(1), 73-80. https://doi.org/10.1017/S0959259898008090

Dodeen

(2015). The effects of positively and negatively worded items on the factor structure of the UCLA loneliness scale. Journal of Psychoeducational Assessment, 33(3), 259-267. https://doi.org/10.1177/0734282914548325

Durak

Senol-Durak

(2010). Psychometric qualities of the UCLA loneliness scale-version 3 as applied in a Turkish culture. Educational Gerontology, 36(10-11), 988-1007. https://doi.org/10.1080/03601271003756628

10.

Dussault

Fernet

Austin

Leroux

(2009). Revisiting the factorial validity of the Revised UCLA Loneliness Scale: A test of competing models in a sample of teachers. Psychological Reports, 105(3), 849-856. https://doi.org/10.2466/PR0.105.3.849-856

11.

Einsamkeit. (n.d.). In Duden. https://www.duden.de/rechtschreibung/Einsamkeit

12.

Gerst-Emerson

Jayawardhana

(2015). Loneliness as a public health issue: The impact of loneliness on health care utilization among older adults. American Journal of Public Health, 105(5), 1013-1019. https://doi.org/10.2105/AJPH.2014.302427

13.

Gollwitzer

Wilczynska

Jaya

E. S.

(2018). Targeting the link between loneliness and paranoia via an interventionist-causal model framework. Psychiatry Research, 263(May), 101-107. https://doi.org/10.1016/j.psychres.2018.02.050

14.

Hartshorne

T. S.

(1993). Psychometric properties and confirmatory factor analysis of the UCLA Loneliness Scale. Journal of Personality Assessment, 61(1), 182-195. https://doi.org/10.1207/s15327752jpa6101_14

15.

Hawkley

L. C.

Browne

M. W.

Cacioppo

J. T.

(2005). How can I connect with thee? Let me count the ways. Psychological Science, 16(10), 798-804. https://doi.org/10.1111/j.1467-9280.2005.01617.x

16.

Hawkley

L. C.

Cacioppo

J. T.

(2010). Loneliness matters: A theoretical and empirical review of consequences and mechanisms. Annals of behavioral Medicine, 40(2), 218-227. https://doi.org/10.1007/s12160-010-9210-8

17.

Hays

R. D.

DiMatteo

M. R.

(1987). A short-form measure of loneliness. Journal of Personality Assessment, 51(1), 69-81. https://doi.org/10.1207/s15327752jpa5101_6

18.

Holt-Lunstad

Smith

T. B.

Baker

Harris

Stephenson

(2015). Loneliness and social isolation as risk factors for mortality: a meta-analytic review. Perspectives on Psychological Science, 10(2), 227-237. https://doi.org/10.1177/1745691614568352

19.

Horn

J. L.

McArdle

J. J.

(1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18(3), 117-144. https://doi.org/10.1080/03610739208253916

20.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118

21.

Hughes

M. E.

Waite

L. J.

Hawkley

L. C.

Cacioppo

J. T.

(2004). A short scale for measuring loneliness in large surveys: Results from two population-based studies. Research on Aging, 26(6), 655-672. https://doi.org/10.1177/0164027504268574

22.

Jaya

E. S.

(2017). Confirmatory factor analysis of the Indonesian version of community assessment of psychic experiences. Makara Human Behavior Studies in Asia, 21(1), 1-12. https://doi.org/10.7454/mssh.v21i1.3495

23.

Jaya

E. S.

Ascone

Lincoln

T. M.

(2017). Social adversity and psychosis: The mediating role of cognitive vulnerability. Schizophrenia Bulletin, 43(3), 557-565. https://doi.org/10.1093/schbul/sbw104

24.

Jaya

E. S.

Ascone

Lincoln

T. M.

(2018). A longitudinal mediation analysis of the effect of negative-self-schemas on positive symptoms via negative affect. Psychological Medicine, 48(8), 1299-1307. https://doi.org/10.1017/S003329171700277X

25.

Jaya

E. S.

Lincoln

T. M.

(2016). Social adversities and psychotic symptoms: A test of predictions derived from the social defeat hypothesis. Psychiatry Research, 245(November), 466-472. https://doi.org/10.1016/j.psychres.2016.09.002

26.

Jaya

E. S.

Wüsten

Alizadeh

B. Z.

van Amelsvoort

Bartels-Velthuis

A. A.

van Beveren

N. J.

Bruggeman

Cahn

de Haan

Delespaul

Luykx

J. J.

Myin-Germeys

Kahn

R. S.

Schirmbeck

Simons

C. J. P.

van Haren

N. E.

van Os

van Winkel

Fonseca-Pedrero

. . . Luykx

J. J.

(2020). Comparing psychotic experiences in low-and-middle-income-countries and high-income-countries with a focus on measurement invariance. Psychological Medicine. Advance online publication. https://doi.org/10.1017/S0033291720003323

27.

Knight

R. G.

Chisholm

B. J.

Marsh

N. V.

Godfrey

H. P.

(1988). Some normative, reliability, and factor analytic data for the revised UCLA Loneliness Scale. Journal of Clinical Psychology, 44(2), 203-206. https://doi.org/10.1002/1097-4679(198803)44:2<203::AID-JCLP2270440218>3.0.CO;2-5

28.

Kwiatkowska

M. M.

Rogoza

Kwiatkowska

(2017). Analysis of the psychometric properties of the Revised UCLA Loneliness Scale in a Polish adolescent sample. Current Issues in Personality Psychology, 3(1), 1-21. https://doi.org/10.31234/osf.io/m6udc

29.

Lykes

V. A.

Kemmelmeier

(2014). What predicts loneliness? Cultural difference between individualistic and collectivistic societies in Europe. Journal of Cross-Cultural Psychology, 45(3), 468-490. https://doi.org/10.1177/0022022113509881

30.

Marsh

H. W.

Hau

K. T.

Wen

(2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11(3), 320-341.

31.

Maul

(2013). Method effects and the meaning of measurement. Frontiers in Psychology, 4, 169. https://doi.org/10.3389/fpsyg.2013.00169

32.

Peltzer

Pengpid

(2019). Loneliness correlates and associations with health variables in the general population in Indonesia. International Journal of Mental Health Systems, 13(1), 1-11. https://doi.org/10.1186/s13033-019-0281-z

33.

Putnick

D. L.

Bornstein

M. H.

(2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41(September), 71-90. https://doi.org/10.1016/j.dr.2016.06.004

34.

Roberts

R. E.

Lewinsohn

P. M.

Seeley

J. R.

(1993). A brief measure of loneliness suitable for use with adolescents. Psychological Reports, 72(3 Suppl.), 1379-1391. https://doi.org/10.2466/pr0.1993.72.3c.1379

35.

Rokach

(1998). The relation of cultural background to the causes of loneliness. Journal of Social and Clinical Psychology, 17(1), 75-88. https://doi.org/10.1521/jscp.1998.17.1.75

36.

Rokach

Orzeck

Cripps

Lackovic-Grgin

Penezic

(2001). The effects of culture on the meaning of loneliness. Social Indicators Research, 53(1), 17-31. https://doi.org/10.1023/A:1007183101458

37.

Russell

D. W.

(1996). UCLA Loneliness Scale (Version 3): Reliability, validity, and factor structure. Journal of Personality Assessment, 66(1), 20-40. https://doi.org/10.1207/s15327752jpa6601_2

38.

Russell

D. W.

Peplau

L. A.

Cutrona

C. E.

(1980). The revised UCLA Loneliness Scale: Concurrent and discriminant validity evidence. Journal of Personality and Social Psychology, 39(3), 472-480. https://doi.org/10.1037/0022-3514.39.3.472

39.

Russell

D. W.

Peplau

L. A.

Ferguson

M. L.

(1978). Developing a measure of loneliness. Journal of Personality Assessment, 42(3), 290-294. https://doi.org/10.1207/s15327752jpa4203_11

40.

Schmitt

T. A.

(2011). Current methodological considerations in exploratory and confirmatory factor analysis. Journal of Psychoeducational Assessment, 29(4), 304-321. https://doi.org/10.1177/0734282911406653

41.

Seepersad

Choi

M. K.

Shin

(2008). How does culture influence the degree of romantic loneliness and closeness? Journal of Psychology, 142(2), 209-220. https://doi.org/10.3200/JRLP.142.2.209-220

42.

Shevlin

Murphy

(2015). The latent structure of loneliness: Testing competing factor models of the UCLA Loneliness Scale in a large adolescent sample. Assessment, 22(2), 208-215. https://doi.org/10.1177/1073191114542596

43.

Steenkamp

J. B. E.

Baumgartner

(1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 25(1), 78-90. https://doi.org/10.1086/209528

44.

Stickley

Koyanagi

Koposov

Schwab-Stone

Ruchkin

(2014). Loneliness and health risk behaviours among Russian and U.S. adolescents: A cross-sectional study. BMC Public Health, 14(1), Article 366. https://doi.org/10.1186/1471-2458-14-366

45.

Strickland

(2017). Language reflects “core” cognition: A new theory about the origin of cross-linguistic regularities. Cognitive Science, 41(1), 70-101. https://doi.org/10.1111/cogs.12332

46.

Sundström

Fransson

Malmberg

Davey

(2009). Loneliness among older Europeans. European Journal of Ageing, 6(4), Article 267. https://doi.org/10.1007/s10433-009-0134-8

47.

Swami

(2009). Psychometric analysis of the Malay version of the UCLA Loneliness Scale (ULS-8) and a comparison of loneliness among sojourning and non-sojourning Malaysian students. International Journal of Culture and Mental Health, 2(1), 38-50. https://doi.org/10.1080/17542860802560397

48.

Tarka

(2017). The comparison of estimation methods on the parameter estimates and fit indices in SEM model under 7-point Likert scale. Archives of Data Science: Series A 2 (1), 1-16.

49.

Tsukamoto

Holland

Haslam

Karasawa

Kashima

(2015). Cultural differences in perceived coherence of the self and ingroup: A Japan–Australia comparison. Asian Journal of Social Psychology, 18(1), 83-89. https://doi.org/10.1111/ajsp.12090

50.

van Overveld

de Jong

P. J.

Peters

M. L.

Schouten

. (2011). The Disgust Scale-R: A valid and reliable index to investigate separate disgust domains? Personality and Individual Differences, 51(3), 325-330. https://doi.org/10.1016/j.paid.2011.03.023

51.

Von Goethe

J. W

. (1995). Goethe 9: Wilhelm Meister's apprenticeship (Vol. 9). University Press. https://doi.org/10.1515/9780691213378

52.

Wilson

Cutts

Lees

Mapungwana

Maunganidze

(1992). Psychometric properties of the revised UCLA Loneliness Scale and two short-form measures of loneliness in Zimbabwe. Journal of Personality Assessment, 59(1), 72-81. https://doi.org/10.1207/s15327752jpa5901_7

53.

C. H.

Yao

(2008). Psychometric analysis of the short-form UCLA Loneliness Scale (ULS-8) in Taiwanese undergraduate students. Personality and Individual Differences, 44(8), 1762-1771. https://doi.org/10.1016/j.paid.2008.02.003

54.

Yildiz

M. A.

Duy

(2014). Adaptation of the short-form of the UCLA Loneliness Scale (ULS-8) to Turkish for the Adolescents. Journal of Psychiatry and Neurological Sciences, 27, 194-203. https://doi.org/10.5350/DAJPN2014270302

55.

C. Y.

Muthen

(2002). Evaluation of model fit indices for latent variable models with categorical and continuous outcomes. In annual meeting of the American Educational Research Association, New Orleans, LA.

56.

Zarei

Memari

A. H.

Moshayedi

Shayestehfar

(2016). Validity and reliability of the UCLA loneliness scale version 3 in Farsi. Educational Gerontology, 42(1), 49-57. https://doi.org/10.1080/03601277.2015.1065688

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.09 MB

0.15 MB

0.14 MB

0.04 MB

0.05 MB

0.14 MB