Measuring Loneliness in Autistic Adults: A Collaborative Approach to Refining the UCLA Loneliness Scale

Abstract

Background:

Loneliness is a significant yet under-investigated health risk for Autistic individuals. It is important that loneliness measures are valid for use among both Autistic and non-autistic adults. This study aimed to (1) adapt the UCLA Loneliness Scale for Autistic adults, (2) provide evidence based on test content for the adapted scale, (3) provide psychometric evidence using an international sample, (4) investigate differential item functioning (DIF) between Autistic and non-autistic adults, and (5) assess convergent and discriminant evidence with other loneliness and well-being variables.

Methods:

To build evidence for test content, we thoroughly examined the UCLA Loneliness Scale items in partnership with Autistic individuals. As the scale was already designed for non-autistic people, we made adaptations to ensure there was evidence for use with Autistic people. This process included cognitive interviews (n = 10) and expert reviews (n = 13). Following the adaptation phase, we gathered data (N = 628) to demonstrate a degree of psychometric evidence using item response theory to assess psychometric properties, including autistic-based DIF. We used latent variable modeling to assess convergent and discriminant evidence.

Results:

The UCLA Loneliness Scale—Modified (UCLA-m) resulted from the adaptation phase and was used to survey 628 adults. We found UCLA-m has sufficient evidence for being essentially unidimensional, with total scores demonstrating preliminary evidence of high reliability, internal validity, and convergent evidence. Autistic adults in our study reported higher levels of loneliness, social anxiety, and depression compared with non-autistic adults.

Conclusion:

The evidence from this study serves as a precursor for studying loneliness in Autistic adults and provides evidence about the validity of a measurement tool that will enable further research into better supporting Autistic adults who are experiencing loneliness. Additionally, many of the adaptations made during this process will increase the overall accessibility of the UCLA-m for all individuals, not just Autistic adults.

Community Brief

Why is this an important issue?

Loneliness can have serious health consequences, such as depression, obesity, and heart disease. Studies show that Autistic individuals often feel lonelier than those who are not Autistic. To address this, we needed a better tool to help health professionals and researchers assess loneliness in Autistic adults. Currently, loneliness questionnaires are not adapted to Autistic people, so the main focus of the study was to make sure the scale could be used with either someone who was Autistic or someone who was non-autistic.

What was the purpose of this study?

This study aimed to adapt a widely used loneliness questionnaire, originally designed for the general population, to be more suitable for Autistic adults in addition to non-autistic adults. This was done by talking with Autistic people, their families, and researchers to ensure the tool was user-friendly and relevant.

What did the researchers do?

To make the questionnaire more autism friendly, we engaged directly with Autistic people to gather their insights on loneliness and how it should be described. This feedback led to the modification of the UCLA Loneliness Scale—Modified (UCLA-m). The language was simplified and made more relatable to Autistic experiences. A total of 23 contributors, including Autistic individuals, researchers, and family members, provided input on these adaptations. Then, we tried the new scale with 628 people including 416 Autistic people and 212 non-autistic people.

What were the results of the study?

We learned that Autistic people in our study reported higher levels of loneliness than people who were not Autistic. We learned that the UCLA-m is a reliable and valid method for measuring loneliness in Autistic adults. Reliable means that the survey gives consistent scores every time it is used. A valid tool measures what it is supposed to measure. This means that the UCLA-m can be confidently used by both Autistic and non-autistic individuals.

What do these findings add to what was already known?

The findings align with previous research indicating that Autistic people often experience more loneliness, which is linked to anxiety and depression. The major contribution of this study is the creation of the UCLA-m, which was developed with direct input from the Autistic community. This tool allows for a better understanding of loneliness in Autistic adults and can help guide efforts to reduce loneliness in both Autistic and non-autistic people. Importantly, the adaptations made the survey more accessible for everyone, not just Autistic individuals.

What are the potential weaknesses in the study?

While the study included a diverse group of participants from various countries and age-groups, the sample was not fully representative. A significant portion of participants were women (69.6%) with high levels of education, and only a small number (1.6%) reported having an intellectual disability. Thus, the findings may not fully represent the experiences of all Autistic individuals.

How will these findings help autistic adults now or in the future?

This study is significant because it provides a validated tool to better understand and measure loneliness in Autistic adults, a group often experiencing high levels of loneliness. The UCLA-m can be used by communities and researchers to explore ways to reduce loneliness, ultimately improving the well-being of Autistic individuals.

Introduction

Loneliness has surged to the forefront of global health concerns, recognized as an epidemic with profound health and economic implications.¹ According to a longitudinal survey conducted between 2014 and 2018 in Australia, 34% of adults have reported experiencing either episodic (over transient periods) or chronic (over a period of time) loneliness, highlighting the pervasive nature of this issue.² Certain population groups may be even more likely to experience loneliness. For instance, emerging evidence has indicated that Autistic individuals encounter heightened and more frequent experiences of loneliness compared with their non-autistic counterparts.^3–6 Many Autistic people desire social connection but can experience challenges in relationships,⁷ contributing to increased loneliness and social isolation.^3,6,8–10 This increased risk is concerning as loneliness is linked to significant mental health concerns and poor well-being.⁸,⁹ For Autistic adults, research has shown that higher levels of self-reported loneliness are related to elevated levels of depression, anxiety,^6,7,11,12 and suicidal ideation,¹³ as well as substance use and a range of physical health conditions^14,15 such as obesity, increased incidence of cancer, cardiovascular diseases,¹⁴,¹⁶ and higher mortality.¹¹ Finally, loneliness is uniquely associated with social anxiety; in a 2016 longitudinal study, Lim and colleagues¹⁷ found social anxiety to be the only predictor of future loneliness, a finding that may have implications for Autistic people who often experience social anxiety.¹⁸

Although researchers are starting to understand variations in the extent and nature of loneliness experiences among Autistic individuals as a group, there is still much to learn about the disparities within these experiences in Autistic adults. To successfully achieve this goal, reliable and valid data must be intentionally collected from Autistic adults using a range of quantitative and qualitative methodologies (see review¹). Unfortunately, the quantitative research on the loneliness experiences of Autistic adults to date has relied on loneliness measures that were developed for the general population and have not yet been validated for use with Autistic adults.³ This substantially limits this critical line of research and may put into question the findings from studies that previously relied on such tools.^19–20 In fact, Grace and colleagues²¹ investigated the viewpoints of 203 Autistic individuals from the United Kingdom regarding two commonly used loneliness questionnaires and found that these measures contained difficult item wording (e.g., confusing terms and phrases), and the items did not fully capture the concept of loneliness as experienced by Autistic individuals.²¹ This finding is consistent with feedback from Autistic adults in other domains (e.g., quality of life²²). It cements the importance of devoting research that focuses on gathering psychometric evidence to use patient-reported outcome measures (PROMs) within critical mental health domains, such as loneliness. Furthermore, advice from Nicolaidis and colleagues¹⁹ suggests that instruments for the general population are regularly “inaccessible to Autistic adults or people with intellectual disability” (p. 67). Across studies, community partners in their study commonly expressed concerns about (1) language complexity and pragmatics, (2) Likert scales, (3) anxiety about providing accurate answers, (4) the potential for varied responses depending on different situations, and (5) the inability to fully capture constructs.¹⁹ Additionally, Jones²⁰ identified the need to ensure these instruments are not only reliable and valid by psychometric standards but also relevant to Autistic people.

Purpose of the study

The purpose of our study was to create a modified version of the UCLA Loneliness Scale that would be suitable for both Autistic and non-autistic individuals with intentional codesign elements. The overarching aim was to adapt a commonly used PROM (i.e., UCLA Loneliness Scale) for use with Autistic adults (the scale adapted for this study is called the UCLA-modified Loneliness Scale; UCLA-m Loneliness Scale). To do this, we sought to provide evidence of test content for the adapted scale through cognitive interviews and expert review processes and then provide psychometric evidence based on an international sample of Autistic and non-autistic adults. Psychometric evidence for the adapted scale included item response theory (IRT) analyses to examine the internal structure and an investigation of differential item functioning (DIF) between Autistic and non-autistic adults. These IRT analyses fulfill the same purposes as factor structure and measurement invariance testing using confirmatory factor analysis. Finally, we provided correlational evidence (convergent validity) by examining the relationship between loneliness and well-being variables, including social anxiety¹⁷ and depression,^7,9,11,23,24 which are posited to be positively correlated, as suggested by evidence from previous research. We also predict scores from the UCLA-m Loneliness Scale will be higher for Autistic people in our study compared with non-autistic people.^7,25 Our study, therefore, was guided by two research questions: (1) Do items on the UCLA-m Loneliness Scale reflect a unidimensional structure? and (2) Do items on the UCLA-m Loneliness Scale manifest DIF between Autistic and non-autistic participants? Additionally, we tested the hypothesis that scores on the UCLA-m Loneliness Scale would have a strong positive relationship with scores from a social anxiety scale and a scale measuring depression. We also expected Autistic participants to self-report higher levels of loneliness compared with non-autistic participants.

Methods

Community involvement statement

We intentionally used community engagement throughout the study’s life cycle, as our team was a mixed-neurotype team where Autistic and non-autistic researchers worked together to answer a question of interest that was common to all team members.^26–27 The research reflects participatory community engagement at the “engaging” level because our research team initiated the research project and invited the community to work with the team at different stages.²⁶ The conceptualization for the project came from members of the research team, including a parent of Autistic individuals (V.G.), an autism researcher and sibling of an Autistic young adult (A.M.A.L.), an autism researcher (R.Y.C.), and two Autistic researchers (C.E. and K.G.). M.H.L. served as a content expert for the area of loneliness and also has experience as a therapist working with Autistic individuals. D.M.D. does not bring any direct connection to autism but collaborated with M.D.T. (a parent of an Autistic individual) to serve as the team psychometricians. Throughout the study, the first and second authors worked as an Autistic (C.E.) and non-autistic (A.M.A.L.) pair to break the cycle of scale development through an exclusively neurotypical lens. Community members were included in the cognitive interviewing and expert review phases. Both Autistic researchers and community members influenced the language used in the study’s design, the research questions, all phases of scale adaptation, and the interpretation of the findings.

Procedure

The scale adapted for our study is the UCLA Loneliness Scale Version 3, a 20-item scale designed to measure subjective feelings of loneliness and social isolation (see Table 1).²⁸ To build evidence for test content, our research team conducted a thorough examination of the items of the UCLA Loneliness Scale Version 3.²⁸ This process included a cognitive interview and an expert review process where Autistic adults, parents/caregivers of Autistic people, psychometricians, and experts in loneliness contributed to the adaptation. Based on information gathered during this examination, we adapted the items to ensure they were easier to understand and reflect the loneliness construct.

Table 1.

Adapting the UCLA Loneliness Scale Version 3 to the UCLA Loneliness Scale—Modified

UCLA Loneliness Scale Version 3		UCLA Loneliness Scale—Modified (UCLA-m)
The following statements describe how people sometimes feel. For each statement, please indicate how often you feel the way described by writing a number in the space provided. Here is an example: “How often do you feel happy? If you never felt happy, you would respond “never”; if you always feel happy, you would respond “always.”		When thinking about the last 12 months, indicate how often each of the statements below is descriptive of you. We have provided descriptions or “hotlinks” for some of the terms used. You can view these descriptions by clicking on or hovering over the blue underlined words or phrases. If you find it difficult to complete, please try to choose the answers that are most close to your subject experiences
20 items		17 items		15 items
UCLA1*	How often do you feel that you are “in tune” with the people around you?	UCLA-m1	How often do you and the people in your life understand each other?	UCLA-m1	How often do you and the people in your life understand each other?
UCLA2	How often do you feel that you lack companionship?	UCLA-m2	How often do you feel that you lack companionship when you want it?	UCLA-m2	How often do you feel that you lack companionship when you want it?
UCLA3	How often do you feel that there is no one you can turn to?	UCLA-m3	How often do you feel like you have nobody you can ask for help when you need it?	Item removed
UCLA4	How often do you feel alone?	UCLA-m4	How often do you feel alone when you do not want to be?	UCLA-m3	How often do you feel alone when you do not want to be?
UCLA5*	How often do you feel part of a group of friends?	UCLA-m5	How often do you feel like you belong to a group of people?	UCLA-m4	How often do you feel like you belong to a group of people?
UCLA6*	How often do you feel that you have a lot in common with the people around you?	UCLA-m6	How often do you feel that you have things in common with the people in your life?	UCLA-m5	How often do you feel that you have things in common with the people in your life?
UCLA7	How often do you feel that you are no longer close to anyone?	UCLA-m7	How often do you feel that you lack a connection with the people in your life?	UCLA-m6	How often do you feel that you lack a connection with the people in your life?
UCLA8	How often do you feel that your interests and ideas are not shared by those around you?	UCLA-m8	How often do you feel that your interests are not shared by people in your life?	UCLA-m7	How often do you feel that your interests are not shared by people in your life?
UCLA9*	How often do you feel outgoing and friendly?	Item removed
UCLA10*	How often do you feel close to people?	UCLA-m9	How often do you feel a sense of connection with people in your life?	UCLA-m8	How often do you feel a sense of connection with people in your life?
UCLA11	How often do you feel left out?	UCLA-m10	How often do you feel left out or excluded in social situations?	UCLA-m9	How often do you feel left out or excluded in social situations?
UCLA12	How often do you feel that your relationships with others are not meaningful?	UCLA-m11	How often do you feel that your relationships with others are meaningful?	UCLA-m10	How often do you feel that your relationships with others are meaningful?
UCLA13	How often do you feel that no one really knows you well?	UCLA-m12	How often do you feel that people in your life know you well?	UCLA-m11	How often do you feel that people in your life know you well?
UCLA14	How often do you feel isolated from others?	UCLA-m13	How often do you feel isolated from others when you don’t want to be?	UCLA-m12	How often do you feel isolated from others when you don’t want to be?
UCLA15*	How often do you fee1 you can find companionship when you want it?	UCLA-m14	How often do you feel that you can find companionship when you want it?	UCLA-m13	How often do you feel that you can find companionship when you want it?
UCLA16*	How often do you feel that there are people who really understand you?	UCLA-m15	How often do you feel that there are people who understand you?	UCLA-m14	How often do you feel that there are people who understand you?
UCLA17	How often do you feel shy?	Item removed
UCLA18	How often do you feel that people are around you but not with you?	UCLA-m16	How often do you feel alone even when you are in a group of people?	Item removed
UCLA19*	How often do you feel that there are people you can talk to?	UCLA-m17	How often are there people you can ask for help when you need it?	UCLA-m15	How often are there people you can ask for help when you need it?
UCLA20	How often do you feel that there are people you can turn to?	Item removed

Note. The UCLA items with an asterisk are reverse scored; For both the original and the adapted measure, high scores indicate higher levels of self-reported loneliness; In the study, the phrases that are underlined in this table were highlighted in blue in the participants version to indicate a hotlink; Participants responded with a 4-point Likert-type response scale ranging from “never” to “often.”

Following ethics approval with the Griffith University Human Research Ethics Committee (project number 2023/363), we recruited participants for each stage of feedback (see Fig. 1) through social media and mailing lists, autism support networks, and word of mouth. We targeted an international audience to recruit a diverse sample of participants, enhancing our psychometric study’s cultural relevance and generalizability. Interested participants could access information about the study on a website, contact the research team for more information, or use a QR code and link to access the participant information sheet and survey. Participants were eligible for all stages of the project if they (1) spoke English, (2) had a professional diagnosis of Autism or self-identified as Autistic, and (3) were 18 years of age or older. For all surveys in this study, we collected and managed the data using Research Electronic Data Capture (REDCap) hosted at Griffith University. REDCap is a secure, web-based software platform designed to support data capture for research studies, providing (1) an intuitive interface for validated data capture, (2) audit trails for tracking data manipulation and export procedures, (3) automated export procedures for seamless data downloads to common statistical packages, and (4) procedures for data integration and interoperability with external sources.^29,30 Participants were given the participant information statement at the beginning of the survey and asked to provide informed consent before proceeding to the survey. Participants who completed the survey were given the option to select one of five international charities to receive a $2 donation (AUD) on behalf of the research participant.

FIG. 1.

Process for gathering psychometric evidence to adapt a measure of loneliness that is accessible for Autistic adults.

Scale adaptation procedures

Cognitive interviews

Following a detailed review of the findings of Grace and colleagues²¹ and critical discussions about the adaptations that may be needed by members of the research team, a stage of cognitive interviewing took place to review the 20-item UCLA Loneliness Scale. Cognitive interviews are a process where the target participants, in this case Autistic adults, participate in “think-aloud” interviews with a researcher to provide insight on their thinking. At the same time, they read and respond to a questionnaire or survey. The interviewer uses a structured protocol (see Supplementary Data) and probes the participant for more information when items or item phrasing are identified as confusing or challenging.^31–32 As participants “think aloud,” we noted consistency with responses and problematic wording or phrases of the loneliness items. In addition to item-level feedback, participants were asked to define loneliness broadly and reflect on the scale as a whole, determining if any element of loneliness was left out from their perspective. Each interview was carried out remotely via video conferencing and led by the first author, who possesses expertise in this interviewing method but who does not identify as Autistic. Therefore, the second author, an Autistic researcher, was responsible for note-taking and posed clarifying questions to ensure appropriate interpretation of the responses through an Autistic lens.

In total, 10 Autistic adults from three countries participated in the cognitive interviewing phase, including 6 participants who identified as female and 4 who identified as males. Two participants had an intellectual disability. Initially, we conducted six interviews and analyzed the data, discussing each stage with the larger research team. Then, we conducted four more interviews until no additional modifications were suggested. On the basis of these interviews, of the 20 items, 10 items were recommended for adaptation (UCLA Items 1, 2, 5, 6, 8, 12–16, and 19) and three items were removed (UCLA Items 9, 17, and 20), resulting in a remaining 17-item UCLA-m Loneliness Scale (see Table 1). Despite these revisions to item language, many phrases frequently used in the UCLA items (i.e., “people in your life”) continued to be problematic for cognitive interview participants. Participants noted that providing a single answer for an item containing those phrases was challenging, as it required choosing one answer to reflect diverse relationships such as friends, family, and work colleagues. To help address this problem, we added “hotlinks” (the ability for participants to click on a phrase or word to obtain more information) to frequently used terms to reduce confusion around these phrases. To increase accessibility, the survey also included an audio option, and hotlinks were included as recommended by Nicolaidis and colleagues¹⁹ as a resolution to challenges with language complexity and anxiety associated with how to respond. The participants did not identify any specific need for additional items; instead, they discussed related constructs to loneliness that were important to them, including Autistic burnout, companionship with animals, and camouflaging.

Consultation with experts

Next, 13 participants responded to a REDCap survey hosted at Griffith University and served as expert reviewers for the now 17-item UCLA-m Loneliness Scale. Categories of experts were chosen to identify participants who shared similar demographics as future participants and as individuals who were specialists in the field and included Autistic adults (n = 5), researchers with expertise in loneliness (n = 2), psychometrics (n = 2) or autism (n = 1), and parents/carers of Autistic adults (n = 3). After reading a participant information sheet and completing a consent form, we asked experts to review the items on the scale by rating them using a 4-point Likert-type response format for both (1) clarity and (2) importance to the construct of loneliness. Participants could also provide open-response feedback about the items. After receiving data from all participants, we examined the scores and open-ended responses for both items addressing clarity and importance. Specifically, we flagged items with scores <2 or items that were identified as problematic by 50% or more of the expert reviewers. Items that were flagged were then discussed with the larger research team. Following the interpretation of the feedback from the experts as well as consultation with the larger research team, two items were adapted (UCLA Items 5 and 17), additional hotlinks were added to increase clarity, and a 17-item UCLA-m loneliness scale went on to our data collection phase (see Table 1).

Participants

Participants of our study were 628 adults (65.59% self-identified as female, M_age = 41.23) including 312 (49.68%) adults who self-reported a professional diagnosis of autism, 104 (16.56%) who self-identified as Autistic, and 212 (33.76%) who were not Autistic. All participants were required to be English-speaking. Our sample consisted of participants from 25 countries including Australia (n = 402), Canada (n = 13), Ireland (n = 7), New Zealand (n = 5), United Kingdom (n = 64), and the United States (n = 131; see Table 2 for full demographic information). There were 160 participants with missing data, and the rates of missing data by scale varied from 1.1% to 14.6%.

Table 2.

Participant Demographics

	Professional diagnosis of autism (n = 312)	Self-identify as Autistic (n = 104)	Not Autistic (n = 212)
Gender
Female	194 (63.2%)	71 (68.9%)	172 (81.1%)
Male	61 (19.9%)	18 (17.5%)	37 (17.5%)
Nonbinary	41 (13.3%)	11 (10.7%)	1 (0.0%)
Other	11 (3.6%)	3 (2.9%)	2 (0.1%)
Relationship status
In a relationship	59 (19.0%)	25 (25.2%)	42 (20.0%)
Married	89 (28.7%)	23 (22.3%)	118 (56.2%)
Single	162 (52.3%)	54 (52.4%)	50 (23.8%)
Living arrangement
Alone	80 (25.8%)	23 (22.1%)	32 (15.1%%
Family	109 (35.2%)	36 (34.6%)	106 (50.0%)
Roommate	104 (33.5%)	40 (38.5%)	71 (33.5%)
Support	17 (5.5%)	5 (4.8%)	3 (1.4%)
Education level
Graduate degree	108 (34.6%)	38 (36.5%)	94 (44.3%)
Bachelor’s degree	103 (33.0%)	32 (30.8%)	74 (34.9%)
Some postsecondary	56 (17.9%)	21 (20.2%)	30 (14.2%)
Secondary	38 (12.2%)	11 (10.6%)	13 (6.1%)
Less than secondary	4 (1.3%)	1 (1.0%)	1 (0.5%)
Employment status
Full-time	103 (33.0%)	43 (41.3%)	122 (57.5%)
Part-time	91 (29.2%)	25 (24.0%)	64 (30.2%)
Unemployed	65 (20.8%)	25 (24.0%)	6 (2.8%)
Retired	16 (5.1%)	5 (4.8%)	10 (4.7%)
Other	37 (11.9%)	6 (5.8%)	10 (4.7%)
Cooccurring conditions^a
Anxiety	248 (79.5%)	78 (75.0%)	94 (44.3%)
Depression	224 (71.8%)	71 (68.3%)	87 (41.0%)
OCD	43 (13.8%)	15 (14.4%)	13 (6.1%)
Speech/lang impairment	25 (8.0%)	6 (5.8%)	3 (1.4%)
Sensory disorder	62 (19.9%)	12 (11.5%)	1 (0.5%)
ADHD/ADD	157 (50.3%)	32 (30.8%)	28 (13.2%)
Seizure/epilepsy	9 (2.9%)	2 (1.9%)	2 (0.9%)
Intellectual disability	4 (1.3%)	4 (3.8%)	2 (0.9%)

Participants could select more than one cooccurring condition. They could also select “other” and fill in an open response with their specific cooccurring condition(s).

Instruments

UCLA Loneliness Scale— Modified

The UCLA-m was adapted from the UCLA Loneliness Scale Version 3.²⁸ For this instrument and our modified instrument, participants respond using a 4-point Likert-type response format (never = 0, rarely = 1, sometimes = 2, and always = 3). Negatively oriented items are reverse-coded before summing item scores to create a total scale score. The UCLA Loneliness Scale Version 3 was revised from the UCLA Loneliness Scale³³ and Revised UCLA Loneliness Scale³⁴ and was chosen because it has been most frequently used with Autistic people.²¹ Russell²⁸ reported high internal consistency for the Revised UCLA Loneliness Scale Version 3 across multiple studies (Cronbach’s α = 0.89–0.94) and reported an acceptable test–retest correlation (r = 0.73). Our sample’s overall coefficient alpha for UCLA-m was 0.94 (Autistic α = 0.89; non-autistic α = 0.89).

Social and Emotional Loneliness Scale for Adults

The Social and Emotional Loneliness Scale for Adults (SELSA-S)^35,36 is a 15-item self-report instrument designed to measure loneliness. Participants respond in a 7-point Likert-type format ranging from “strongly disagree” to “strongly agree.” The instrument assesses three factors of loneliness, including social, family, and romantic. A higher total score on the SELSA-S represents higher levels of loneliness. An example item reads, “I feel part of a group of friends” (social subscale). DiTommaso and colleagues³⁶ reported internal consistency ranging from α = 0.87 to 0.90, evidence for validity based on relationship to a version of the UCLA Loneliness Scale,³⁴ and evidence for a three-factor solution across items representing the three subscales. The SELSA-S has been used in Autistic samples,³⁷ although the instrument currently does not have validity evidence for use with Autistic samples. Sample coefficient alpha in the present study was 0.890 for the social factor (α = 0.852 Autistic; α = 0.883 non-autistic), 0.903 for the family factor (α = 0.884 Autistic; α = 0.891 non-autistic), and 0.919 for the romantic factor (α = 0.906 Autistic; α = 0.934 non-autistic).

Social Interaction Anxiety Scale—Straightforward items

The Social Interaction Anxiety Scale – Straightforward (S-SIAS)^38–39 describes anxiety-related reactions to a variety of social interaction situations to measure social anxiety. Respondents use a 4-point Likert-type response format that ranges from “not at all” to “extremely.” It has been used with Autistic adults.^40–41 An example item reads, “I become tense if I have to talk about myself or my feelings.” The authors reported internal consistency (α > 0.89) for the straightforwardly worded items.³⁹ Overall sample reliability was 0.951 (Autistic α = 0.916; non-autistic α = 0.953). Importantly, we used the full scale (15 items), but following feedback from several participants who accurately felt the item was not inclusive, we removed Item 14 (“I have difficulty talking to attractive persons of the opposite sex”). This decision is consistent with authors Linder and colleagues⁴⁰ who suggest a nonheteronormative phrasing of Item 14 (“I have difficulty talking to attractive persons of the sex/sexes that I am interested in”). At the time of our survey, we used the original item wording but dropped the item postdata collection. We believe strongly that future iterations should always use the nonheteronormative phrasing.

Patient Health Questionnaire

The Patient Health Questionnaire (PHQ-9)⁴² was a 9-item norm-referenced instrument used to measure depression symptoms. The PHQ-9 uses a 4-point Likert-type response format ranging from “not at all,” which has a score of “0,” to “nearly every day,” which receives a score of “3.” Items are summed to obtain total scores ranging between 0 and 30. Cut-off scores of 20, 15, 10, and 5 represent severe, moderately severe, moderate, and mild depression, respectively. A score of 10 or above had both a sensitivity and a specificity of 0.88 for major depression. The PHQ-9 has been used extensively with Autistic samples, and evidence for the psychometric properties of the PHQ-9 in Autistic adults has been demonstrated.^43–44 Sample reliability was 0.901 (Autistic α = 0.880; non-autistic α = 0.890).

Autistic traits

The Autistic traits (AQ-28)⁴⁵ is an abbreviated version of the full 50-item screening questionnaire used to measure Autistic traits.⁴⁶ Each item is rated on a 4-point Likert-type scale ranging from definitely agree (score of 1) to definitely disagree (score of 4). After reverse-coding items, item scores are summed with scores ranging from 28 to 112. The cut-off score for autism is 65, according to Hoekstra and colleagues.⁴⁵ Sample correlation with the 50-item AQ is very high, with r ranging from 0.93 to 0.95 and comparable estimates of sensitivity (0.97) and specificity (0.82) for autism using an AQ-Short cut-off score of 65. Overall coefficient alpha in our sample was 0.920 (Autistic α = 0.833; non-autistic α = 0.862).

Data analyses

We managed data and analyses primarily in R. Cleaned data and primary analysis scripts are available at (note: This is a blinded link that will be replaced before publication) https://osf.io/edyth/?view_only=6bd3061272254c0b93540c8bbe10e9d0.

Validating Autism identification groups

To provide evidence for self-reported autism identification groups, we used analysis of variance (ANOVA) to compare AQ scores across the three autism diagnosis classification groups: those who reported a professional diagnosis of autism, those who self-identified as Autistic, and those who do not identify as Autistic. If the ANOVA showed an overall statistically significant difference, then we conducted Tukey’s pairwise group comparisons.

IRT analyses

Prior to conducting IRT analyses of UCLA-m data, we assessed the assumption of unidimensionality by computing the eigenvalues of the data’s polychoric correlations matrix. Next, a unidimensional graded response IRT model was fit to the UCLA-m data using the MIRT package for R.⁴⁷ The assumption of item fit was tested⁴⁸ using S-χ²; statistically significant (p < 0.01) violations of item fit were inspected with empirical fit plots. Additionally, the assumption of local independence was tested using Q3⁴⁹; item pairs with Q3 > |.20| or more than two standard deviations (SDs) away from the mean Q3 were inspected.⁵⁰ Items were removed as appropriate to ensure a well-fitting model.

Given that the original UCLA Loneliness Scale is often modeled using a bifactor model, where positive and negatively oriented items are assigned to respective specific factors,⁵¹ a bifactor model was also considered if the unidimensional model did not exhibit adequate fit. To assess the extent to which data were acceptably unidimensional, explained common variance (ECV) and hierarchical omega (OmegaH) for the general factor were computed to determine whether the interpretation of an overall score was appropriate (ECV > 0.70 and OmegaH > 0.80).⁵² As Omega indices are not generally computed from IRT models, a bifactor item factor analysis model was fit using Mplus,⁵³ and bifactor indices were computed from this model using the Bifactor Indices Calculator package for R.⁵⁴

Differential item functioning

We used the LORDIF package⁵⁵ for R to evaluate DIF based on autism diagnosis classification within an ordinal logistic regression framework. LORDIF provides p values for uniform DIF, nonuniform DIF, and overall DIF. Due to the large number of statistical tests (three for each item), p values were compared with a conservative 0.01 for statistical significance testing. Also, the unsigned item difference in sample and expected score standardized difference effect sizes for DIF were computed for each item.⁵⁶ A multigroup model was fit in which item parameters were held constant across groups except for the items flagged for DIF. Graphs of expected item score versus trait score were inspected to assess the extent of DIF at different levels of loneliness.

Revisiting IRT analyses

If tests of DIF resulted in the removal of additional items, a new IRT model was fit to the data and evaluated. Following the establishment of a final model, item parameters and option response functions were inspected to ensure all items were productive (i.e., good model-data fit and acceptable parameter estimates). The empirical reliability of trait scores⁵⁷ was computed, and conditional standard errors of measure were plotted for all complete cases.

Relationship with external variables

Loneliness scores across groups were compared based on the UCLA-m observed total score and scores from the SELSA-S family, romantic, and social domains. Since UCLA-m was designed to assess social components of loneliness, we expect the standardized difference (i.e., Cohen’s d) between Autistic and non-autistic adult groups to be similar for UCLA-m and SELSA-S’s social subdomain. The statistical significance of these differences was tested using t tests. Pearson correlations between scores on measures of loneliness (UCLA-m and SELSA-S) and scores on measures of related constructs including social interaction anxiety (S-SIAS) and depression (PHQ-9) were computed separately within each autism diagnosis group. We expected correlations involving UCLA-m to be similar in magnitude and sign to relationships involving the SELSA-S social subdomain. Within each autism classification group, correlations of UCLA-m with S-SIAS and PHQ-9 were statistically compared with correlations of SELSA-S subdomains with S-SIAS and PHQ-9. Due to the large number (12) of significance tests being performed, the Benjamini–Hochberg procedure was applied to control the false discovery rate.

Missing data

Item-level missing data on the UCLA-m were minimal, with <1% of cases missing for each item. Scale-level missingness for other measures ranged from 8% to 13%. We conducted measurement analyses on the UCLA-m in an IRT framework and therefore utilized full information maximum likelihood handling of missing data, which assumes missing data were missing at random. We performed correlational tests using a pairwise deletion approach to missing data, which assumes missing data were missing completely at random (MCAR). Little’s MCAR test was nonsignificant, p = 0.210, suggesting the MCAR assumption was tenable.

Results

Validating autism identification groups classification

The initial three groups were non-autistic, formally identified (e.g., professional diagnosis) Autistic individuals, and self-identified Autistic. ANOVA indicated a statistically significant overall difference in AQ scores across autism diagnosis groups, F(2, 533) = 266.20, p < 0.001, η_p² = 0.50. Follow-up tests revealed statistically significant differences between the Autistic (M = 84.9, SD = 10.9) and non-autistic (M = 60.3, SD = 12.0) groups (d = 2.17, p < 0.001) and between the self-identified Autistic (M = 82.2, SD = 11.5) and non-autistic (M = 60.3, SD = 12.0) groups (d = 1.86, p < 0.001) but not between the Autistic and self-identified Autistic groups (d = 0.24, p = 0.13). Due to the minimal difference between the Autistic and self-identified Autistic groups, we performed all further analyses involving the autism classification group by combining those two groups into a single Autistic group. As a sensitivity analysis, all analyses were additionally performed using the original three autism groups. However, the results did not show any substantive difference from those obtained in the two-group analysis.

IRT analyses

The first three eigenvalues of the 17-item UCLA-m data were 10.44, 1.39, and 0.79, suggesting an essentially unidimensional dataset with a possible minor secondary dimension. As such, a unidimensional graded response model was fit to the data. No items were flagged for concerning item fit according to S-χ². However, 29 item pairs were found to have Q3 > |.20| or Q3 more than two SDs away from mean Q3. Inspection of these item pairs revealed that high Q3 was mostly associated with item pairs of the same orientation (i.e., both positive or both negative). In contrast, low Q3 was always associated with item pairs of mixed orientation. In fact, pairs of positively oriented items had a mean Q3 of 0.09, pairs of negatively oriented items had a mean Q3 of 0.10, and item pairs with mixed orientation had a mean Q3 of −0.15. These findings are consistent with a bifactor structure. Hence, a bifactor-graded response IRT model was fit to the UCLA-m item data.

No items within the bifactor model were flagged for concerning item fit according to S-χ². However, items UCLA-m3 (How often do you feel like you have nobody you can ask for help when you need it?) and UCLA-m17 (How often are there people you can ask for help when you need it?) exhibited high Q3 = 0.21. Because these items are essentially oppositely oriented versions of each other, we decided they were redundant and removed UCLA-m3 from all further analyses. UCLA-m3 was chosen for removal as it is more complicated in its phrasing than UCLA-m17. Evidence provided during the expert review phase also indicated that this item’s wording was more complex, and the idea of “feeling like you have nobody” could be challenging for some individuals. Following the removal of UCLA-m3, a new bifactor IRT model was fit to the 16-item UCLA-m data.

Like the original bifactor model fit analysis, no items were flagged for concerning item fit according to S-χ². Likewise, no concerning item pairs were flagged by the Q3 fit statistic. Bifactor indices of ECV = 0.76 and OmegaH = 0.83 suggest the 16-item UCLA-m is sufficiently unidimensional to interpret a total score.

Differential item functioning

Next, we addressed the second research question regarding DIF testing for the Autistic and non-autistic samples. Results of DIF testing can be found in Table 3. Note that DIF was evaluated using a unidimensional model because the bifactor IRT model results indicated that UCLA-m was adequately unidimensional to justify interpreting a total score. Items UCLA-m5, UCLA-m8, UCLA-m10, and UCLA-m16 were flagged for uniform DIF, whereas UCLA-m4 was flagged for nonuniform DIF. Graphs of expected item score versus trait score in each group can be found in Figure 2. The nonoverlapping lines indicate where DIF is observed across groups for certain levels of the trait score (loneliness).

FIG. 2.

DIF—Expected item score versus trait score in each group. DIF, differential item functioning. Note: The non-overlapping lines indicate where DIF observed across groups for certain levels of the trait score (loneliness).

Table 3.

Differential Item Functioning of UCLA-m Items

Item	p (uniform DIF)	p (nonuniform DIF)	p (overall DIF)	UIDS	ESSD
UCLA-m1	0.038	0.157	0.043	0.06	0.18
UCLA-m2	0.953	0.613	0.878	0.02	0.00
UCLA-m4	0.970	<0.001^*	0.001^*	0.19	0.16
UCLA-m5	<0.001^*	0.558	<0.001^*	0.27	0.46
UCLA-m6	0.955	0.189	0.422	0.03	0.02
UCLA-m7	0.933	0.592	0.863	0.03	0.02
UCLA-m8	0.001^*	0.095	0.001^*	0.27	0.58
UCLA-m9	0.804	0.294	0.559	0.06	0.05
UCLA-m10	<0.001^*	0.673	0.002^*	0.18	0.34
UCLA-m11	0.421	0.863	0.713	0.05	0.11
UCLA-m12	0.847	0.697	0.910	0.04	0.05
UCLA-m13	0.108	0.185	0.115	0.16	0.26
UCLA-m14	0.174	0.202	0.176	0.05	0.05
UCLA-m15	0.122	0.998	0.302	0.09	0.14
UCLA-m16	<0.001^*	0.021	<0.001^*	0.37	0.73
UCLA-m17	0.394	0.057	0.113	0.09	0.11

Note. p < 0.01; DIF, differential item functioning; ESSD, expected standardized score difference; UIDS, unsigned item difference in sample.

Effect sizes of DIF for UCLA-m4 are small, as expected with nonuniform DIF. Additionally, the impact of DIF on expected item score was very small for levels of loneliness for which both groups have substantial density. Accordingly, UCLA-m4 was retained despite the presence of DIF.

A close inspection of the item wording for items UCLA-m5 and UCLA-m16 both deals with “group of people,” and UCLA-m5 additionally addresses “belonging.” Effect sizes of DIF are moderate for UCLA-m5 and moderate to large for UCLA-m16. We chose to retain UCLA-m5 due to its importance in construct representation (i.e., it is the only item addressing belonging). However, UCLA-m16 was removed from further analyses due to its more substantial DIF and construct overlap with UCLA-m5.

DIF in UCLA-m8 was anticipated due to the documented tendency of Autistic individuals to develop intense, highly specific, and often uncommon interests.^58–59 Consequently, Autistic individuals often come to expect that others may not fully share or understand these interests.⁵⁷ Accordingly, after controlling for other aspects of loneliness, it makes sense that Autistic people are less likely to feel their interests are not shared by other people. However, this lack of shared interest still contributes to feelings of loneliness, and thus DIF in UCLA-m8 was deemed to be construct relevant and therefore we retained UCLA-m8.

Finally, UCLA-m10 exhibited a small-to-moderate amount of DIF and was retained. The likely reason behind this item exhibiting DIF is its potential double-barreled nature (“feel left out” versus “excluded”). However, cognitive interviews revealed that participants found the additional word “excluded” to help explain what it meant to be “left out.” Moreover, this is the only item focusing on exclusion, a significant concern among Autistic adults according to the interview phases, and feelings of being left out or excluded are key components of loneliness. Therefore, it was retained to ensure comprehensive content coverage.

Revisiting IRT analyses

Following the removal of UCLA-m16, a new bifactor IRT model was fit to the 15-item UCLA-m data. Item parameter estimates from this model can be found in Table 4. No items within this model were flagged for concerning item fit according to S-χ². Likewise, no concerning item pairs were flagged by Q3 as being concerning for local dependence.

Table 4.

Item Parameter Estimates from Final Bifactor IRT Model of the 15-Item UCLA-m

Item	a (general)	a (positive)	a (negative)	b1	b2	b3
UCLA-m1	1.543		1.252	−2.817	2.908	6.327
UCLA-m2	2.783	1.445		−5.598	−2.262	2.830
UCLA-m4	3.401	2.324		−5.850	−1.771	4.489
UCLA-m5	2.137		1.001	−3.090	−0.141	2.711
UCLA-m6	2.013		1.296	−2.714	1.399	5.480
UCLA-m7	2.693	−0.016		−4.333	−1.890	2.525
UCLA-m8	2.091	−0.467		−4.351	1.798	2.044
UCLA-m9	2.470		1.606	−2.998	1.883	5.671
UCLA-m10	2.226	.004		−4.382	−2.015	1.829
UCLA-m11	2.385		1.748	−2.714	2.117	5.551
UCLA-m12	2.163		1.579	−3.284	0.440	4.145
UCLA-m13	4.028	2.235		−7.028	−2.658	4.409
UCLA-m14	2.032		0.883	−2.721	0.147	3.143
UCLA-m15	3.091		2.246	4.270	0.756	5.982
UCLA-m17	2.327		1.431	−2.186	1.376	4.593

Note. a, discrimination; b, threshold; IRT, item response theory.

The empirical reliability of the general factor was 0.895 for the entire sample; empirical reliabilities were 0.857 for the autistic subsample and 0.872 for the non-autistic subsample. A graph of standard errors of measure versus trait score can be found in Figure 3. Except for extreme scores, all persons are measured with an effective reliability of at least 0.80. Bifactor indices of ECV = 0.76 and OmegaH = 0.83 suggest UCLA-m is sufficiently unidimensional to interpret a total score.

FIG. 3.

Conditional standard errors of measurement for final 15-item UCLA-m.

Relationship with external variables

Pearson correlations between UCLA-m observed total scores and related variables can be found in Table 5. The osf repository contains an expanded version of Table 5 with confidence intervals for each correlation. Loneliness scores based on UCLA-m were higher for Autistic participants compared with non-autistic participants (d = 1.307, p < 0.001). A statistically significant difference was also found for each SELSA-S subdomain: d = 1.072, p < 0.001 for the social domain, d = 0.525, p < 0.001 for the romantic subdomain, and d = 0.938, p < 0.001 for the family subdomain. Autistic participants also had higher levels of self-reported depression as measured by the PHQ-9 (d = 0.908, p < 0.001) and social interaction anxiety as measured by the S-SIAS (d = 1.476, p < 0.001). Regarding associations with social interaction anxiety as measured by the S-SIAS and depression as measured by the PHQ-9, scores on the UCLA-m were positively correlated with scores on the S-SIAS and scores on the PHQ-9, as hypothesized. Pearson correlations involving UCLA-m were all statistically significantly higher than correlations involving any of the SELSA-S subdomains.

Table 5.

Correlations and Summary Statistics of Measures by Autism Classification

	Autistic						Not Autistic
	UCLA-m	SELSA-S social	SELSA-S romantic	SELSA-S family	S-SIAS	PHQ-9	UCLA-m	SELSA-S social	SELSA-S romantic	SELSA-S family	S-SIAS	PHQ-9
UCLA-m	(0.89)						(0.94)
SELSA-S social	0.70	(0.85)					0.73	(0.88)
SELSA-S Romantic	0.41	0.25	(0.88)				0.53	0.35	(0.89)
SELSA-S family	0.48	0.32	0.20	(0.91)			0.62	0.58	0.46	(0.93)
S-SIAS	0.35	0.26	0.02	0.17	(0.92)		0.66	0.51	0.23	0.33	(0.95)
PHQ-9	0.44	0.28	0.21	0.34	0.32	(0.88)	0.62	0.45	0.27	0.45	0.54	(0.89)
Mean	40.83	22.39	22.17	20.51	38.16	12.89	30.57	14.37	16.71	13.01	19.39	7.12
SD	7.54	7.53	10.39	8.13	12.40	6.64	8.44	7.39	10.38	7.73	13.29	5.75

Note. Values in parenthesis on diagonals are Cronbach’s alpha estimates of internal consistency; All correlations are Pearson correlations; PHQ-9, Patient Health Questionnaire; SD, standard deviation; SELSA-S, Social and Emotional Loneliness Scale for Adults; S-SIAS, Social Interaction Anxiety Scale—Straightforward; UCLA-m, UCLA Loneliness Scale—Modified.

Discussion

Our study aimed to modify a widely used instrument for measuring loneliness in the general population so that it could be effectively used with an Autistic sample and a non-autistic sample. Crucially, our study incorporated participatory approaches, acknowledging the importance of involving Autistic individuals in the scale development and modification processes. This involvement ensures that the item wording is relevant and clear for the population of interest. Without community involvement, as was employed in our study, confidence in findings is limited, as existing scales are often developed and normed based on psychometrics derived from non-autistic samples.^19–20

To develop and provide initial psychometric evidence for the validity of the UCLA-m, we first modified items based on data from cognitive interviews with Autistic adults and feedback from an expert review process. During this phase, two items were removed, and modifications were made to several of the original UCLA items to ensure clarity and appropriateness for the intended population. Then, psychometric evidence of the items was analyzed using IRT and DIF. During this phase, two items were removed; specifically, we removed items UCLA-m3 and UCLA-m16, which relate to Items 3 and 18 on the original UCLA, respectively.³⁰ The decision to remove these items was supported by coordinating psychometric analysis with content validity evidence, ensuring that the final instrument had a degree of evidence for reliability and validity for use with both Autistic and non-autistic individuals. Interestingly, no items were added during our process. Participants were asked to identify gaps in the scale during the cognitive interview and expert review phase; however, no consistency and need were reflected in the participants’ responses. Sum scores of the resulting 15-item instrument have initial evidence for construct validity and high reliability. Psychometric evidence should always be considered ongoing; specifically, several items that displayed mildly concerning psychometric properties were retained in our study (i.e., UCLA-m4, UCLA-m5, UCLA-8, and UCLA-m10). These items should be carefully considered in future research involving the UCLA-m. Specifically, UCLA-m4 showed DIF due to nonuniform effects with minimal impact on item scores, UCLA-m5 due to its focus on “belonging” and “group of people,” UCLA-m8 due to Autistic individuals’ unique experiences with shared interests, and UCLA-m10 likely from its double-barreled phrasing addressing “left out” and “excluded.” To correct these concerns, future research should conduct cognitive interviews with Autistic adults to understand the reasons better and ultimately refine item phrasing and comprehension. Doing so could enhance the psychometric properties of these moderately concerning items, thereby improving the overall psychometric strength of the UCLA-m. Furthermore, research should investigate the sources of DIF related to group membership (Autistic vs. non-autistic adults) on the UCLA-m. This could begin by analyzing patterns in the autistic classification DIF-flagged items identified in this study and those observed in other research on loneliness or related measures. Subsequently, confirmatory DIF analyses could be conducted to test these patterns. Such efforts would contribute to identifying additional sources of group DIF associated with autistic classification and inform the development of guidelines and principles for reducing group DIF in loneliness measures or psychological assessments more broadly.

Overall, the IRT results showed excellent fit and measurement precision for the items on the UCLA-m. To provide evidence for convergent validity, we examined a correlational analysis of the relationship between the adapted loneliness scale and scores on related. As expected, results indicated that individuals who reported higher levels of loneliness also reported higher levels of social anxiety and depression. Scores on the UCLA-m had stronger associations with scores on the SELSA-S and the PHQ-9 social anxiety and depression than the SELSA-S, which is another commonly used instrument used to measure self-reported levels of loneliness, providing further evidence for the modified instrument. Also, as expected, the UCLA-m had lower correlations with SELSA-S-family and SELSA-S-romantic than with SELSA-S-social. This could indicate that changes made to the UCLA-m helped to capture the construct of loneliness better.

Additionally, all correlations among scales for the non-autistic group are descriptively higher than correlations for the Autistic group. This raises the question as to whether these mental health adjacent constructs are less strongly connected for Autistic individuals. Another possibility is that lower measurement precision in the Autistic group resulted in attenuated correlations. However, the differences in reliability estimates between the Autistic and non-autistic groups were too small to explain the differences in correlations. Therefore, we suspect that these constructs may be less strongly connected for Autistic than non-autistic people or that other constructs are more strongly related. Another possible explanation is construct representation for Autistic people, or the idea that Autistic people’s experiences of loneliness may not be represented as well by the UCLA-m, when compared with non-autistic people, as suggested by Grace.²¹ However, evidence from our processes and consultation with Autistic participants did not reveal the necessity for additional items. Overall, results of the process revealed that based on the degree of evidence accumulated, the instrument can be used to assess self-reported loneliness in both Autistic and non-autistic samples. We recommend that future studies gather additional psychometric evidence with diverse samples and aim to replicate the findings reported herein.

Autistic individuals and loneliness

Consistent with the original UCLA Loneliness Scale, the UCLA-m captures the unidimensional construct of loneliness. The data from our study showed a clear distinction between individuals who are Autistic and those who are non-autistic in their self-reported rates of loneliness. Specifically, Autistic participants reported higher levels of loneliness, social anxiety, and depression. These results are similar to previous findings^6,7,24 and continue to provide justification for an urgency into research that addresses loneliness and well-being for Autistic adults.

Practical application

By modifying the UCLA Loneliness Scale to better suit Autistic people, practitioners now have a more accurate and reliable PROM for assessing loneliness in their Autistic clients. This tailored instrument allows for better identification of self-reported loneliness levels, which can inform targeted interventions and support strategies. Additionally, the validation process ensures that the modified scale is both effective and relevant for both Autistic and non-autistic adults, fostering more precise data collection and analysis, including comparative analysis, in future research. Ultimately, this study enhances the ability of allied health workers including health care providers, therapists, and support services to understand and address the unique experiences of loneliness in Autistic individuals, leading to more personalized and effective care.

Other researchers may wish to use a similar process outlined in this article when considering scale modification or development. Each scale used in a study must provide appropriate validity evidence with an Autistic sample, and feedback on wording and content can only strengthen confidence in the results of a study. The key steps of this process include seeking feedback on the test content and clarity of items through interviews, obtaining expert reviews, and combining this qualitative advice with evidence from advanced psychometric analyses. By following a research procedure like the one outlined in this study, measures can become more accessible and relevant for all individuals.

The evidence from this study serves as a precursor for studying loneliness in Autistic adults and provides evidence about the validity of a measurement tool that will support further research into how communities (e.g., support services, parents/caregivers, and therapists) can better support Autistic adults who are at risk of experiencing loneliness. Before this study was conducted, psychometric evidence was only emerging regarding the use of any validated PROM to measure loneliness with Autistic adults. Based on the evidence in this study, the UCLA-m can be used alongside lived experiences⁶⁰ to explore important questions within this field that go beyond examining the prevalence of loneliness to exploring ways in which to alleviate it, given the associated health and well-being outcomes. For instance, attention could be devoted to understanding the balance between finding social connection and ensuring restorative alone time.⁶¹ Collaborations between researchers, practitioners, and the Autistic community are needed to develop and implement these opportunities and interventions. Additionally, funding and resources must be allocated to support these initiatives, ensuring they are accessible and effective. By combining rigorous research with immediate, practical applications, we can better address the loneliness experienced by Autistic adults and improve their overall quality of life.

Limitations

As acknowledged in the community participation section, this article included the involvement of Autistic community members. However, the project did not meet the standards for community-controlled research, due to the input and shared power by non-autistic researchers.⁶² However, a strength of this research is the shared partnership of non-autistic and Autistic researchers and community members who came together from different cultures and countries to complete this research. For all stages of research, self-report data were used. While PROMs are valuable and critical within the field of loneliness and other well-being constructs, self-report measures come with inherent limitations including the potential for social desirability and recall bias, where participants may not accurately remember past experiences or events, leading to inaccuracies in their responses. Additionally, the participants in our study were female majority (69.6%), and our sample included a higher number of individuals with higher education than in the general population, limiting the generalizability of our findings. Only a small percentage of the full sample reported an intellectual disability (1.6%), and only 3 of the 23 individuals who gave us feedback reported an intellectual disability. This means that the results of our study may not apply to all Autistic people. Finally, we collected an international sample; however, we limited participants to English-speaking Western countries. Therefore, our findings are not generalizable to other non-English-speaking countries where the construct of loneliness may be experienced differently than by people of other countries or by individuals who could not independently access the survey.

Continual evolution of measurement tools

The authors of this research intentionally carried out this project with community involvement from Autistic people to ensure that change in the language, development of new items, and discussions around a separate Autistic construct of loneliness were made with consideration to lived experiences and community perspectives. This is a strength of the study, and the methods used can be replicated for other scale development studies. As recommended by Jones²⁰ and Nicolaidis and colleagues,¹⁹ tools designed to measure aspects of Autistic lives must be made in collaboration with Autistic people. Scales like the UCLA-Loneliness Instrument were made before coproduction and Autistic voices were valued; therefore, it is time that they are revised.

Many of the adaptations made during this process will increase the overall accessibility of the scale for all individuals, not just Autistic adults. These modifications ensure that the language and structure of the scale are more comprehensible and user-friendly, which is likely to enhance the accuracy and reliability of self-reported data across diverse populations. This broader applicability means that the scale can be a valuable tool in various contexts, contributing to more inclusive research practices and potentially informing the development of interventions aimed at reducing loneliness across different groups. The practice of modifying and updating instruments for diverse samples is critical. Researchers should move beyond the routine use of instruments solely based on existing psychometric evidence and instead ask: Can this scale be improved before using it to measure my construct of interest? If so, will that increase my confidence in the findings? Engaging in this step first, in collaboration with individuals who have lived experiences, will enrich research findings and ensure that outdated scales are not used indiscriminately. This approach promotes the continual evolution of measurement tools to better capture the nuances of diverse populations and enhances the validity and relevance of research outcomes.

Footnotes

Acknowledgments

Our research team would like to thank the input by Autistic researchers and community members who served as expert reviewers or cognitive interview participants. This study was only made possible because of your generous sharing of your lived experiences and constructive feedback on the questionnaire.

Authorship Confirmation Statement

A.M.A.L.: Project administration, conceptualization and design, data collection, interpretation of results, and original writing. C.E.: Conceptualization and design, data collection, interpretation of results, and review and editing. D.M.D.: Methodology, analysis, interpretation of results, and review and editing. R.Y.C.: Conceptualization and design, interpretation of results, and review and editing. K.G.: Conceptualization, content expert, interpretation of results, and review and editing. M.D.T.: Methodology, analysis, interpretation of results, and review and editing. M.H.L.: Content expert, interpretation of results, and review and editing. V.G.: Conceptualization and design, supervision, and review and editing. The article has been submitted solely to Autism in Adulthood.

Author Disclosure Statement

The authors do not have any conflict of interest to disclose.

Funding Information

The research was supported by Autism Spectrum Australia.

Supplementary Material

References

Surkalim

, Luo

, Eres

, et al. The prevalence of loneliness across 113 countries: Systematic review and meta-analysis. BMJ. 2022; 376:e067068; doi: 10.1136/bmj-2021-067068

Lim

, Qualter

, Thurston

, et al. A global longitudinal study examining social restrictions severity on loneliness, social anxiety, and depression. Front Psychiatry. 2022; 13:818030; doi: 10.3389/fpsyt.2022.818030

Grace

, Remington

, Lloyd-Evans

, Davies

, Crane

. Loneliness in autistic adults: A systematic review. Autism. 2022; 26(8):2117–2135; doi: 10.1177/13623613221077721

Hymas

, Badcock

, Milne

. Loneliness in autism and its association with anxiety and depression: A systematic review with meta-analyses. Rev J Autism Dev Disord. 2024; 11(1):121–156; doi: 10.1007/s40489-022-00330-w

Scheeren

, Howlin

, Pellicano

, Magiati

, Begeer

. Continuity and change in loneliness and stress during the COVID‐19 pandemic: A longitudinal study of autistic and non‐autistic adults. Autism Res. 2022; 15(9):1621–1635; doi: 10.1002/aur.2787

Quadt

, Williams

, Mulcahy

, et al. “I’m trying to reach out, I’m trying to find my people”: Loneliness and loneliness distress in autistic adults. Autism Adulthood. 2023; doi: 10.1089/aut.2022.0062

, Hwang

, Reppermund

, et al. Loneliness in adults on the autism spectrum. Autism Adulthood. 2019; 1(3):182–193; doi: 10.1089/aut.2018.0038

Hedley

, Uljarević

, Wilmot

, Richdale

, Dissanayake

. Understanding depression and thoughts of self-harm in autism: A potential mechanism involving loneliness. Res Autism Spectrum Dis. 2018; 46:1–7; doi: 10.1016/j.rasd.2017.11.003

Schiltz

, Gohari

, Park

, Lord

. A longitudinal study of loneliness in autism and other neurodevelopmental disabilities: Coping with loneliness from childhood through adulthood. Autism. 2024; 28(6):1471–1486; doi: 10.1177/13623613231217337

10.

Leigh-Hunt

, Bagguley

, Bash

, et al. An overview of systematic reviews on the public health consequences of social isolation and loneliness. Public Health. 2017; 152:157–171; doi: 10.1016/j.puhe.2017.07.035

11.

Mazurek

. Loneliness, friendship, and well-being in adults with autism spectrum disorders. Autism. 2014; 18(3):223–232; doi: 10.1177/1362361312474121

12.

Schiltz

, McVey

, Wozniak

, et al. The role of loneliness as a mediator between autism features and mental health among autistic young adults. Autism. 2021; 25(2):545–555; doi: 10.1177/1362361320967789

13.

van Bentum

, Sijbrandij

, Huibers

, Begeer

. Occurrence and predictors of lifetime suicidality and suicidal ideation in autistic adults. Autism. 2024; 28(9):2282–2294; doi: 10.1177/13623613231225901

14.

Holt-Lunstad

, Smith

, Layton

. Social relationships and mortality risk: A meta-analytic review. PLoS Med. 2010; 7(7):e1000316; doi: 10.1371/journal.pmed.1000316

15.

Holt-Lunstad

, Smith

, Baker

, Harris

, Stephenson

. Loneliness and social isolation as risk factors for mortality. Perspect Psychol Sci. 2015; 10(2):227–237; doi: 10.1177/1745691614568352

16.

Valtorta

, Kanaan

, Gilbody

, Ronzi

, Hanratty

. Loneliness and social isolation as risk factors for coronary heart disease and stroke: Systematic review and meta-analysis of longitudinal observational studies. Heart. 2016; 102(13):1009–1016; doi: 10.1136/heartjnl-2015-308790

17.

Lim

, Rodebaugh

, Zyphur

, Gleeson

JFM

. Loneliness over time: The crucial role of social anxiety. J Abnorm Psychol. 2016; 125(5):620–630; doi: 10.1037/abn0000162

18.

Spain

, Sin

, Linder

, McMahon

, Happé

. Social anxiety in autism spectrum disorder: A systematic review. Research in Autism Spectrum Disorders. 2018; 52:51–68; doi: 10.1016/j.rasd.2018.04.007

19.

Nicolaidis

, Raymaker

, McDonald

, et al. Creating accessible survey instruments for use with autistic adults and people with intellectual disability: Lessons learned and recommendations. Autism Adulthood. 2020; 2(1):61–76; doi: 10.1089/aut.2019.0074

20.

Jones

. Measuring the wrong thing the right way? Time to rethink autism research tools. Autism Adulthood. 2022; 4(2):104–109; doi: 10.1089/aut.2021.0050

21.

Grace

, Remington

, Davies

, Crane

. Evaluating measures to assess loneliness in autistic adults. Autism. 2024; 28(8):1959–1971; doi: 10.1177/13623613231217056

22.

Beck

, Terhorst

, Greco

, Kulzer

, Skidmore

, McCue

. Item understanding of common quality of life measures for use with autistic adults. J Autism Dev Disord. 2024; 54(7):2625–2635; doi: 10.1007/s10803-023-05945-4

23.

Jackson

, Hart

, Brown

, Volkmar

. Brief report: Self-reported academic, social, and mental health experiences of post-secondary students with autism spectrum disorder. J Autism Dev Disord. 2018; 48(3):643–650; doi: 10.1007/s10803-017-3315-x

24.

Williams

, Everaert

, Gotham

. Measuring depression in autistic adults: Psychometric validation of the beck depression Inventory–II. Assessment. 2021; 28(3):858–876; doi: 10.1177/1073191120952889

25.

Elmose

. Understanding loneliness and social relationships in autism: The reflections of autistic adults. Nordic Psychology. 2020; 72(1):3–22; doi: 10.1080/19012276.2019.1625068

26.

den Houting

, Higgins

, Isaacs

, Mahony

, Pellicano

. ‘I’m not just a guinea pig’: Academic and community perceptions of participatory autism research. Autism. 2021; 25(1):148–163; doi: 10.1177/1362361320951696

27.

Nicolaidis

, Raymaker

, Kapp

, et al. The AASPIRE practice-based guidelines for the inclusion of autistic adults in research as co-researchers and study participants. Autism. 2019; 23(8):2007–2019; doi: 10.1177/1362361319830523

28.

Russell

. UCLA loneliness scale (Version 3): Reliability, validity, and factor structure. J Pers Assess. 1996; 66(1):20–40; doi: 10.1207/s15327752jpa6601_2

29.

Harris

, Taylor

, Thielke

, Payne

, Gonzalez

, Conde

. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009; 42(2):377–381; doi: 10.1016/j.jbi.2008.08.010

30.

Harris

, Taylor

, Minor

, et al; REDCap Consortium. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform. 2019; 95:103208; doi: 10.1016/j.jbi.2019.103208

31.

Murphy

, Hollinghurst

, Salisbury

. Qualitative assessment of the primary care outcomes questionnaire: A cognitive interview study. BMC Health Serv Res. 2018; 18(1):79; doi: 10.1186/s12913-018-2867-6

32.

Willis

, Boeije

. Reflections on the cognitive interviewing reporting framework: Efficacy, expectations, and promise for the future. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences. 2013; 9(3):123–128; doi: 10.1027/1614-2241/a000074

33.

Russell

, Peplau

, Ferguson

. Developing a measure of loneliness. J Pers Assess. 1978; 42(3):290–294; doi: 10.1207/s15327752jpa4203_11

34.

Russell

, Peplau

, Cutrona

. The revised UCLA Loneliness Scale: Concurrent and discriminant validity evidence. J Pers Soc Psychol. 1980; 39(3):472–480; doi: 10.1037/0022-3514.39.3.472

35.

Cramer

, Barry

. Conceptualizations and measures of loneliness: A comparison of subscales. Personality Individual Differ. 1999; 27(3):491–502; doi: 10.1016/s0191-8869(98)00257-8

36.

DiTommaso

, Brannen

, Best

. Measurement and validity characteristics of the short version of the Social and Emotional Loneliness Scale for adults. Educational Psychol Measurement. 2004; 64(1):99–119; doi: 10.1177/0013164403258450

37.

Hijab

MHF

, Al-Thani

, Banire

. A Multimodal Messaging App (MAAN) for adults with autism spectrum disorder: Mixed methods evaluation study. JMIR Form Res. 2021; 5(12):e33123; doi: 10.2196/33123

38.

Mattick

, Clarke

. Development and validation of measures of social phobia scrutiny fear and social interaction anxiety. Behav Res Ther. 1998; 36(4):455–470; doi: 10.1016/S0005-7967(97)10031-6

39.

Rodebaugh

, Heimberg

, Brown

, et al. More reasons to be straightforward: Findings and norms for two scales relevant to social anxiety. J Anxiety Disord. 2011; 25(5):623–630; doi: 10.1016/j.janxdis.2011.02.002

40.

Boulton

, Guastella

. Social anxiety symptoms in autism spectrum disorder and social anxiety disorder: Considering the reliability of self‐report instruments in adult cohorts. Autism Res. 2021; 14(11):2383–2392; doi: 10.1002/aur.2572

41.

Linder

, Martell

, Bergstrom

, Andersson

, Carlbring

. Clinical validation of a non-heteronormative version of the Social Interaction Anxiety Scale (SIAS). Health and Quality of Life Outcomes. 2013; 11:1–6.

42.

Kroenke

, Spitzer

, Williams

JBW

. The PHQ-9. J Gen Intern Med. 2001; 16(9):606–613; doi: 10.1046/j.1525-1497.2001.016009606.x

43.

Arnold

SRC

, Uljarević

, Hwang

, Richdale

, Trollor

, Lawson

. Brief report: Psychometric Properties of the Patient Health Questionaire-9 (PHQ-9) in autistic adults. J Autism Dev Disord. 2020; 50(6):2217–2225; doi: 10.1007/s10803-019-03947-9

44.

Robeson

, Brasil

, Adams

, Zlomke

. Measuring depression and anxiety in autistic college students: A psychometric evaluation of the PHQ-9 and GAD-7. Autism. 2024; 28(11):2793–2805; doi: 10.1177/13623613241240183

45.

Hoekstra

, Vinkhuyzen

AAE

, Wheelwright

, et al. The construction and validation of an abridged version of the Autism-Spectrum Quotient (AQ-Short). J Autism Dev Disord. 2011; 41(5):589–596; doi: 10.1007/s10803-010-1073-0

46.

Baron-Cohen

, Wheelwright

, Skinner

, Martin

, Clubley

. The autism-spectrum quotient (AQ): evidence from Asperger syndrome/high-functioning autism, males and females, scientists and mathematicians. J Autism Dev Disord. 2001; 31(1):5–17; doi: 10.1023/a:1005653411471

47.

Chalmers

. mirt: A Multidimensional Item Response Theory package for the R Environment. J Stat Soft. 2012; 48(6); doi: 10.18637/jss.v048.i06

48.

Orlando

, Thissen

. Further investigation of the performance of S - X2: An item fit index for use with dichotomous item response theory models. Applied Psychol Measurement. 2003; 27(4):289–298; doi: 10.1177/0146621603027004004

49.

Yen

. Effects of local item dependence on the fit and equating performance of the Three-Parameter logistic model. Applied Psychol Measurement. 1984; 8(2):125–145; doi: 10.1177/014662168400800201

50.

Kim

, De Ayala

, Ferdous

, Nering

. The comparative performance of conditional independence indices. Applied Psychol Measurement. 2011; 35(6):447–471; doi: 10.1177/0146621611407909

51.

Ozdemir

, Tan

. Meta-analytic factor analysis of the UCLA Loneliness Scale. Curr Psychol. 2024; 43(20):18307–18318; doi: 10.1007/s12144-024-05681-7

52.

Rodriguez

, Reise

, Haviland

. Evaluating bifactor models: Calculating and interpreting statistical indices. Psychol Methods. 2016; 21(2):137–150; doi: 10.1037/met0000045

53.

Muthen

, Muthen

. MPlus Users Guide. 8th edition. Muthén & Muthén; 2024.

54.

Dueber

. Bifactor Indices Calculator. CRAN: Package. May 12, 2021. Available from: https://CRAN.R-project.org/package=BifactorIndicesCalculator [Last accessed: May 8, 2024].

55.

Choi

, Gibbons

, Crane

. lordif: An R Package for detecting differential item functioning using iterative hybrid ordinal logistic regression/item response theory and Monte Carlo simulations. J Stat Softw. 2011; 39(8):1–30; doi: 10.18637/jss.v039.i08

56.

Meade

. A taxonomy of effect size measures for the differential functioning of items and scales. J Appl Psychol. 2010; 95(4):728–743; doi: 10.1037/a0018966

57.

Brown

, Croudace

. Scoring and estimating score precision using multidimensional IRT Models. In: Handbook of Item Response Theory Modeling. Routledge/Taylor & Francis Group; 2014;pp. 307–333; doi:10.4324/9781315736013.ch15

58.

Grove

, Hoekstra

, Wierda

, Begeer

. Special interests and subjective wellbeing in autistic adults. Autism Res. 2018; 11(5):766–775; doi: 10.1002/aur.1931

59.

Nowell

, Bernardin

, Brown

, Kanne

. Characterization of special interests in Autism Spectrum Disorder: A brief review and pilot study using the special interests survey. J Autism Dev Disord. 2021; 51(8):2711–2724; doi: 10.1007/s10803-020-04743-6

60.

Grace

. Towards a Better Understanding of Loneliness in Autistic Adults: Examining Measurement Tools and Lived Experiences. 2023. Available from: https://discovery.ucl.ac.uk/id/eprint/10164705/2/07.02.23_Final%20thesis.pdf

61.

Neville

, Sedgewick

, McClean

, White

, Bray

. Reacting, retreating, regulating, and reconnecting: How Autistic adults in the United Kingdom use time alone for well-being. Autism in Adulthood. 2024; doi: 10.1089/aut.2024.0148

62.

den Houting

. Participatory and Inclusive Autism Research Practice Guides. Autism CRC: Brisbane; 2021. Available from: https://www.autismcrc.com.au/best-practice/sites/default/files/resources/Participatory_and_Inclusive_Autism_Research_Practice_Guides.pdf

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.01 MB

0.00 MB