#Bias: The Opportunities and Challenges of Surveys That Recruit and Collect Data of Autistic Adults Online

Abstract

Internet-based online surveys are a crucial tool for researchers to learn about the understudied and often overlooked population of autistic adults. The recruitment and administration of online surveys can be cheaper, quicker, and have a wider reach compared with more traditional in-person methods. As online surveys become more prevalent, it is important to place strengths in the context of limitations and biases that can arise when recruiting and administering surveys online. In this perspective, we discuss two common issues that often appear in studies that use online tools to recruit and administer surveys to autistic adults and nonautistic volunteers: selection bias and sample identifiability. Selection bias is the distortion in effect estimates (e.g., relative risk, risk ratio, incidence rate) resulting from the factors that influence why a person chose to participate or how the researcher recruits and selects participants in a study. Sampling identifiability is the ability (or inability) to quantify and define the population of interest. We use a case example of an online survey study of suicidal ideation in autistic adults and describe how issues in selection bias and sample identifiability arise and may lead to challenges unique to studying autistic adults. We conclude with recommendations to improve the quality and utility of online survey research in autistic adults. Using online resources to recruit and collect data on autistic adults is an incredible tool with great potential; yet, authors need to consider the limitations, potential biases, and tools to overcome systematic error at each stage of the study.

Lay summary

What is the purpose of this article?

Our purpose was to describe challenges in conducting and analyzing data from surveys of autistic adults, recruited and completed online.

What is already known on the topic?

Health outcomes for autistic adults are understudied by crucial areas of autism research. While researchers are interested in the outcomes of autistic adults, this type of research is difficult because many autistic adults are not formally diagnosed, so not available to recruit for studies through clinic registries. Furthermore, study participation can be a long, inconvenient, and stressful process. It is not surprising then that we are seeing internet surveys of autistic adults become a popular tool to reach this population. We wanted to offer an overview and recommendations of these issues to researchers and people who read research about topics pertaining to autistic adults.

What are the perspectives of the authors?

We are epidemiologists at Boston University and the University of Wisconsin-Madison. We both conduct research centered in improving health and well-being for autistic people across the life span. As people who study research methods, we have seen a lot of new research using survey methods. This research is intriguing, but all too often the articles need more information so we can be sure that the research is high quality. We want to share ways to improve this type of research and to help people in understanding the strengths and limitations of online survey research.

What do the authors recommend?

We offer a few considerations for researchers working in this area. (1) Make the steps you took to do the research as clear as possible. (2) Be specific about who you intend to study and who you ended up studying. (3) Present the demographics and characteristics of the participants. (4) If possible, consider using data analysis methods to account for selection bias and sample identifiability issues. (5) Do not make statements that are not supported for your study results. (6) Acknowledge that we are at the beginning of studying autistic adults. (7) Advocate for more funding for research in autistic adults.

How will these recommendations help autistic adults now or in the future?

Online surveys are an important tool for researchers to generate hypotheses and connect with a wider range of participants. However, online surveys have unique methodological challenges. We hope that this perspective raises the topic of bias and misinterpretation in online surveys and researchers continue to produce valid and meaningful work that is crucial to improving lives of autistic adults.

In the past two decades, autism, as a diagnosable condition, culture, identity, and area of research, has been greatly impacted by the near-universal access, increased speed, and growing capability of the internet. While the internet has greatly impacted all areas of life and society, there are many concrete examples of how internet technologies and online communities have altered the autism world. Whether it be the proliferation of the autism-vaccine hoax,¹ the use of social media and internet forums to foster community among autistic adults,^2,3 a resource for autistic adults, caregivers, and parents seeking advice and support,⁴ or a medium to provide intervention to underserved areas,⁵ the internet has played a considerable role in the development and growth of the autism community. From the research perspective, the increasing need to focus on adult outcomes^6,7 and the burgeoning online autistic adult community⁸ has led to a boom in internet-based surveys that can reach a population of autistic adults, often overlooked in research studies.⁹ To illustrate, the Autism Science Foundation, a nonprofit research funder in the United States, alone has posted upward of 20 active online research surveys one can elect to participate in at the time of our writing.

Online surveys are an enticing tool to better understand sensitive and pressing issues in the larger community of autistic adults, such as gender expression¹⁰ or mental health.¹¹ In general, compared with data collected in a clinic or through intensive interviews, an online survey is cheaper, quicker, and less labor-intensive.¹² Surveys are not reliant on the participant having attended a clinic or being available to participate at a certain time, which widens the potential pool of participants. The speed and expediency of online surveys are crucial for the pressing public health crises (e.g., suicidality), and online surveys can be useful tools for generating hypotheses to push toward prevention, intervention, and treatment for conditions that affect autistic adults. With social media as a research and recruiting tool,¹³ plus a movement toward more remote meetings and communication after COVID-19,¹⁴ online surveys will continue to be a major tool for much human participant research going forward. Specifically in regard to autism, the vibrant and diverse population of autistic adults who use internet platforms to communicate and find community, with a fairly broad range of verbal language abilities¹⁵ can be leveraged to create more inclusive and representative study samples. Online surveys can be taken from a location of one's choosing, which may reduce or eliminate some logistical difficulties or social anxiety that especially affects autistic people and may prevent them from participating in a study.¹⁶ Online surveys can also reach participants without a formal diagnosis of autism, which can be difficult for many undiagnosed adults to attain.¹⁷

While online surveys have strengths, the ease of implementation and nearly limitless opportunity for participation and recruitment can create bias and lead to misinterpretation in research studies.¹⁸ By enrolling large samples that can be drawn, in theory, from the full population of diagnosed or self-identified autistic adults with internet access, new issues arise. Many of the challenges are similar to issues in survey research in general, including relying on participant self-report and poor description of study methods¹⁹; however, the paucity of research in autistic adults leaves us without the baseline knowledge needed to evaluate internal and external validity of online surveys. Challenges in online survey research can also be similar to those in nonpopulation-based clinical research, including generalizability issues²⁰; however, the major difference when comparing challenges in other clinical samples is the huge sample size and reach possible when leveraging the internet. An online survey can have hundreds of thousands of respondents. For example, Warrier et al. analyzed the association between autistic traits and gender expression in 641,860 respondents.²¹ With that level of response, there is the desire to make population-level inferences. However, if the study design has systematic errors, the inference can be incorrect at a population level. Often, challenges arise that are specific to research in autistic adults and require special consideration, including accessibility and inclusion, lack of baseline population data, and autism-specific selection bias.

In this perspective, we present a case study of an online survey assessing the risk of suicidal ideation in autistic adults. We explore two sources of error in online survey studies that can be a result of methodologic limitations: selection bias and sample identifiability. When these sources of error are not considered and addressed, results from online surveys of autistic adults may be biased or misleading. We then propose steps researchers should take to minimize bias and improve the generalizability of results, to improve the quality of autism research. Our goal is to discuss online survey research with open online recruitment, that is, any adult who sees the advertisement can participate. Studies that recruit online and collect data through means other than online survey, or studies that recruit through nononline means and collect data online, are outside the scope of this article. Furthermore, the focus of this perspective is on public health research with inferences for the population rather than the individual. While many complexities of this line of research are unique for the autistic adult population, the biases and challenges are often applicable to other research topics. The issues we discuss may not be applicable to other nonsurvey online research²² or more qualitative studies.²³

Case Study

We will use the association between autism and suicidal ideation (thinking about, considering, or planning suicide)²⁴ as the basis for understanding potential issues in online survey methods. The true underlying prevalence of suicide ideation in autistic adults varies widely across studies. A systematic review by Hedley and Uljarević found that the prevalence of suicidal ideation ranged from 11% to 66%²⁵ and Cassidy et al.¹¹ reported that 72% of autistic adults scored above the cutoff for suicide risk in an online survey. The ranging prevalence illustrates a challenge in this area, as outcomes of studies in autistic adults are often impacted by population characteristics, such as gender, age, educational attainment, and other medical conditions.²⁵ Demographic differences in samples may lead to differential associations in subgroups of autistic adults.²⁶ Further complicating research is that identifying suicidal ideation may be more challenging in autistic adults compared with peers since few autism-specific suicidality measures exist.²⁷

As a public health issue, it is important to accurately describe the extent of suicidal ideation in autistic people so that we have accurate estimates of occurrence of ideation and prevalence of factors associated with increased risk. With that information, researchers can develop programs for prevention and intervention that can act on community and system levels.²⁸ However, recruiting, enrolling, and collecting data on a sample of autistic adults in a cohort study, let alone one focused on a sensitive issue such as suicide, are difficult. Administrative data may miss autistic adults who lack access to care because of known disparities, autistic adults without an autism diagnosis, or autistic adults who chose not to disclose their autism,^29–31 all of whom may be at the highest risk for suicide ideation.²⁶ With the importance of acting quickly to change policy and practice,³² using efficient online surveys in an unbiased and methodologically sound manner becomes more vital. We frame suicidal ideation as one issue where online survey methods can help advance our understanding of a crucial topic, but thoughtful consideration of bias and sampling in online surveys for all topics is critical to improve the quality and utility of research.

Selection Bias

Selection bias is the distortion in effect estimates (e.g., relative risk, risk ratio, incidence rate ratio) resulting from the factors that influence why a person chooses to participate in a study, or how the researcher recruits and selects participants into a study.³³ There is some ambiguity in the autism literature on selection bias, with the phrase sometimes being used to describe issues in generalizability.³⁴ We opt for the definition of selection bias used in the epidemiology literature. Selection bias differs from generalizability, or the transportability of inferences from one sample to another, in that under this definition, selection bias may compromise internal validity.³³ As an example, a well done study on risk for disordered eating that restricts to autistic adults without intellectual disability is internally valid to the population of autistic adults without intellectual disability. There may be an error in generalizing the findings to autistic adults with intellectual disability, but based on the sampling criteria, the results would be valid for the sample under study. However, if people with intellectual disability can be included in the autistic group but do not participate in the comparison group, selection bias can confound results. In the discussion of generalizability and selection bias, the lack of representation of people with intellectual disability in online surveys is a major issue. Autistic adults with intellectual disability need to be included in the research of pressing topics that affect autistic adults. However, if a study excludes people with intellectual disability, this does not necessarily mean that results are not internally valid. We point toward Russell et al.'s³⁴ review on the inclusion of autistic adults with intellectual disability and the implications for generalizability to the autistic population.

In online surveys of autistic adults, selection bias is a major issue and is often brought on by self-selection, study advertisement, and recruitment. We stress that selection bias is an issue in all online survey research, not just in autistic adults. Participants are more likely to participate in surveys on topics that interest them,³⁵ so people with no history of suicide ideation or mental health conditions may be less likely to participate in a survey that does not mirror their lived experience. Conversely, there may be “healthy volunteer” effects, where the nonautistic comparison group, whose motivation in responding is out of altruism and desire to participate in research, may be healthier than those more motivated by the topic.³⁶ If there are monetary incentives, that may differentially affect the response between autistic adults and other participants. Online sampling may also lead to a noncomparable comparison group; if a study is assessing suicidality in autistic adults and includes people with co-occurring anxiety in the autistic group, there need to be people with anxiety without autism in the comparison group or we may see an effect that is attributable to anxiety rather than autism. The oversampling of high-risk autistics and low-risk nonautistics may lead to a misrepresentation of suicidal ideation in some demographic subsets, distorting associations and leading to erroneous results.

We used a directed acyclic graph³⁷ (Fig. 1) to illustrate the causal question: is being autistic a cause of suicidal ideation? For our purpose, the figure has been simplified and does not include many of the covariates that may confound or mediate the relationship. The causal pathway from the “exposure” (autism) to the “outcome” (suicidal ideation) is represented using directed arrows. The dotted arrow represents the association in question—is there a direct causal pathway from autism to suicide ideation? The solid lined arrows illustrate causal pathways that include intermediary variables, for example, through disparity and inequality autistic people develop depression,³⁸ which can lead to suicidal ideation.³⁹ If two arrows collide at a variable (see selection in Fig. 1, which represents forces by which participants enter the study) and that variable is controlled for (selection is the controlling of who enters the study), the path now appears to be causal (referred to as “backdoor pathway,” gray arrow).³⁷ Due to self-selection and the recruitment strategy, there are more people with depression in the study than in the target population because of their interest in the topic, the study sample no longer mirrors the underlying distribution of depression in the population of autistic adults. Because of this bias, there would appear to be a causal direct effect from autism to suicidal ideation, independent of depression. Without proper adjustment for selection bias, we would come to the wrong conclusion, which may lead to misguided and public health action.

FIG. 1.

Representation of selection bias in the association between autism and suicidal ideation. The DAG shows causal pathways, confounding factors, and selection bias in the etiology from an “exposure” (here autism) to an “outcome” (suicidal ideation). This simplified example DAG highlights how selection into the study based on suicidal ideation creates a backdoor pathway, or a confounding pathway, from autism to suicidal ideation not through depression. DAG, directed acyclic graph.

The quantifiable effect of selection bias is presented in Figure 2A–D. These fabricated data represent a study in which participants were recruited through an existing research registry and online advertisements. Here, the selection bias is a differential association between recruitment source and suicidal ideation. The recruitment source as a cause of selection bias is still hypothetical in autistic adults, as research has not been done comparing respondents from different recruiting techniques. However, there is a literature in the general population illustrating how small changes in recruitment material and source can impart selection effects.^40,41

FIG. 2.

Example illustrating how different response rates across recruiting sources can effect estimates. By varying the percentage enrolled from each recruitment source (registry or advertisements) and the prevalence within each recruitment source, we can see how this variation can change odds ratios. In each example, there is the same total prevalence of suicidal ideation in each group (50% autism and 20% comparison group) and the impact of selection bias changes our results when prevalence differs by source. (A) Half of the participants in the autism and comparison groups are from each recruitment source (registry vs. online advertisements). Within groups, risk is the same regardless of recruitment source: autism prevalence is 50% and comparison prevalence is 20%. (B) Half of the participants in the autism and comparison groups are from each recruitment source. In the autism group, there is a higher prevalence of suicidal ideation from the online source (90%) compared with the registry source (10%). The comparison group has a prevalence of 20% in each recruitment source. (C) In both the autism and comparison groups, 80% of participants are from online advertisement recruitment and 20% are from the registry. In the autism group, there is a higher prevalence of suicidal ideation from the online source (90%) compared with the registry source (10%). The comparison group has a prevalence of 20% in each recruitment source. (D) In both the autism and comparison groups, 80% of participants are from online advertisement recruitment and 20% are from the registry. Within the autism and comparison groups, risk is the same regardless of recruitment source: autism prevalence is 50% and comparison prevalence is 20%.

In Figure 2, we have created four different scenarios. All four scenarios have 100 autistic adult respondents and 100 nonautistic adult respondents and the weighted prevalence (i.e., when accounting for source of enrollment) of suicidal ideation is 50% in autistic adults and 20% in the comparison group. The four scenarios are as follows: (A)

Half of the participants in the autism and comparison groups are from each recruitment source (registry vs. online advertisements). Within the autism and comparison groups risk is the same regardless of recruitment source: autism prevalence is 50% and comparison prevalence is 20%.

(B)

Half of the participants in the autism and comparison groups are from each recruitment source. In the autism group there is a higher prevalence of suicidal ideation from the online source (90%) compared with the registry source (10%). The comparison group has a prevalence of 20% in each recruitment source.

(C)

In both the autism and comparison groups 80% of participants are from online advertisement recruitment and 20% are from the registry. In the autism group there is a higher prevalence of suicidal ideation from the online source (90%) compared with the registry source (10%). The comparison group has a prevalence of 20% in each recruitment source.

(D)

In both the autism and comparison groups 80% of participants are from online advertisement recruitment and 20% are from the registry. Within the autism and comparison groups risk is the same regardless of recruitment source: autism prevalence is 50% and comparison prevalence is 20%.

If we consider A to be our referent unbiased scenario, we see in B and C that with the same number of autistic individuals sampled, there are different odds ratios if prevalence differ between sampling sources. In B, we see starkly different odds ratios within strata, and when pooled, the overall estimate is close to the estimate in A. However, what happens when the proportions from each enrollment source change, as in scenario C? The difference in odds ratio is exaggerated when one source is sampled more than the other. Unless the proportion of online recruits compared with registry members mirrors the demographics in the underlying population, there is high probability for bias. D illustrates that if there is no effect of recruitment source, there will not be bias in the odds ratio due to sampling. The example here is greatly simplified since we did not address confounding by other factors or the possibility that prevalence of suicide ideation differs by source in the comparison group as well. This is also a constructed example in which we set the parameters. Yet, we can see how the interplay between different response rates and prevalence by sampling sources can lead to stark differences in effect estimates.

Sample Identification

In many online surveys of autistic adults, the goal is to understand a health condition or personal experience to better serve and improve the quality of life for autistic adults. Yet, that end goal is not achievable unless we have (or can create) a representative sample to make inferences on. The heterogeneity of autism, the differences in services and opportunities at the local, state, and national level, and the nonrandom sampling of an online survey can interact to make the results of a survey of autistic adults difficult to place in the proper context. In other explorations of survey methodology, this issue can be referred to as sampling bias.⁴² Here, we refer to sampling identifiability as the ability or inability to quantify and define the population of interest. We make the distinction because “bias” implies erroneous estimation and a lack of internal validity.³⁶ Sample identification is an issue even when results are internally valid since the valid finding is unique to a sample that does not represent an existing population. An online survey can have participants from six continents, but how does one interpret findings in the context of federal policies and country-specific disparities? In clinic samples, the sampling area is often limited to a nearby geographic area, and in epidemiologic studies, while effort is made to recruit a population-representative sample, often samples are still drawn from a specified geographic area. With the entire internet to sample from, online surveys on health and well-being can result in findings that are not interpretable or actionable for a population or subpopulation.

The issue of sample identifiability is one of clearly defining the target, source, and sample population. The target population is the underlying universe from which samples are drawn, and the overarching goal of public health research is to make inferences for the target population.⁴³ As presented in Figure 3, the target population for many surveys of autistic adults is all autistic adults and a demographically similar group of nonautistic adults. The target population is theoretical: it is not enumerable because it is dynamic, and if it were fixed, would be enormously costly to research.⁴⁴ The source population is the group in which study participants are drawn.⁴⁵ While ideally the source population is demographically similar to the target population, it is often the case that the source is a subset of the target, often because not all people in the target population can or desire to participate in the research.

FIG. 3.

Target, source, and sample populations in surveys.

A good example of differences between target and source populations would be that if the target is all autistic adults and the method of data collection is an online survey, the source will be conditioned on having internet access. The sample population is the subset of the source population selected and who agreed to participate in the study.⁴³ Ideally, the sample population is a random subset of the source population, which is representative of the target population. There are methods, such as inverse probability weighting and oversampling, to adjust for difference between the sample and source populations. Briefly, weighting and oversampling methods give more value to less-represented subgroups, and so, the sample (when weighted) has a more similar distribution to the source population.⁴⁶ However, weighting methods require knowledge about the makeup of the source population, which can lead to sample identification issues in online surveys. We know who participated in the survey, but we cannot quantify or describe the pool of people the participants came from. Without knowing information about who is missing, certain confounding factors cannot be adjusted for and our inference is not generalizable to the target population. If 80% of respondents to our survey on suicidal ideation in diagnosed autistics report being of cis-gender women, we can be certain we are missing a large number of males (based on the higher prevalence of autism in males compared with females⁴⁷) and we have no way of knowing whether the small proportion of males we sampled is representative of males not sampled.

It should be noted that both selection bias and sample identification are more easily addressed for registry-based or panel-surveys such as the Interactive Autism Network (IAN)⁴⁸ or SPARK,⁴⁹ although these designs also have limitations in generalizability and nonresponse. These projects recruit a large cohort to enroll in a database and then they are contacted about participating in online surveys. These cohorts are not necessarily population-representative and have funding and goals that may not align with the aims of some in the autistic community, which may cause some autistic adults to elect not to participate. However, the source population is enumerated (all enrolled in IAN or SPARK). Researchers know who elected to participate in a given online survey from all the members in the registry. With that information, participant responses can be upweighted and downweighted to reflect the larger registry. Without knowing who could have potentially participated in online surveys, we cannot account for nonrandom entry from the source into the sample. There still may be issues in generalizability as the registry may not reflect the true underlying autistic population, but statistical adjustment can be made to control for bias that impacts internal validity.

In regard to the case study of suicidal ideation, we illustrated how two issues in online survey methodology can bias and hamper studies. With the extreme importance and urgency of preventing suicide in the autistic population, the autism community does not have the time or resources for spurious and biased findings leading the scientific community down the wrong path. While all studies of autistic adults have limitations, the vast samples and reach of online surveys may give the impression of a population-level sample, and that may lead to population-level policies and interventions. Our best science is needed to prevent suicidal ideation and suicide attempts and that requires the careful consideration of selection bias and sample identifiability.

Recommendations

We provide a few recommendations to guide the description, interpretation, and analysis of online surveys going forward. Many of these recommendations are applicable to all online survey research, with some being more specific to the study of autistic adults. The recommendations are also useful for nonscientists as important aspects to evaluate when determining the quality of a study.

1. Provide sufficient methodological detail on how the survey was administered to allow for reproduction

It is common for journals to state that an article's methods section should be of a detail in which a reader could replicate the study if desired. That should be the case for the description of online surveys. We do not necessarily mean that the study should or needs to be replicated, but that a comprehensive level of methodological detail should be reported. Step-by-step information on how the study was done will allow the reader to identify the strengths and weaknesses of the methodology. Information that should be provided includes the recruitment strategy, dates the survey was open, where and how the study was advertised, any exclusion criteria, percent that started but did not complete the survey, and any information about a registry or existing participant resource used for recruiting. The data should be assessed for any inconsistencies that might suggest manipulation (e.g., bots, trolls), especially if there is a financial incentive to participate. Specific to studying autistic adults, researchers should highlight ways in which the recruitment or survey methodology was designed to be more inclusive of the full autism spectrum (e.g., tailoring text to be accessible to those with intellectual disability), or why they selected to use certain exclusion criteria. In addition, sufficient detail should be provided on the recruitment of nonautistic participants as that can be a large source of selection bias.

2. Clearly outline the target, source, and sample population from which the survey recruited

To aid the reader in understanding the context of the study design and the results, authors should clearly define the populations they wished to research (target), the participants whom they had access to (source), and those who participated (sample). For example, “Our objective was to examine the association between autism and internet gaming disorder in US adults (target). We recruited a sample from the ‘Autism and Gaming’ Facebook group (source) and administered an online survey to 40% of the group that consented to participate (sample).” While this advice is applicable to all population-based research, it is especially key for studying autistic adults since the heterogeneity of community and disparities in internet access can lead to different results if different subgroups are assessed.

3. Describe the demographics of those who participated in the survey and consider whether that may bias results compared with those who did not participate

Demographic information should be presented to place the sample in the context of what is known about the target sample. Age, gender, race/ethnicity, country, educational attainment, socioeconomic status, and intellectual disability status should be reported as these variables are likely to be associated with selection into the study and the outcome of interest. If the survey is housed within a research registry, how do the demographics of the study sample compare with the demographics of the registry? If not in a registry, how does the demographic distribution compare to what we know about the autistic population? For example, many online surveys of autistic adults have more women respondents than would be expected based on the male to female autism sex ratio. Could that impart bias?

4. Evaluate whether statistical methods can be used to account for selection bias and improve sample identifiability when relying on potentially biased survey samples.

In some cases, selection bias and problems with sample identifiability are inevitable. There are statistical methods that can be used to account for bias and issues in generalizability. Researchers should consider the causal pathways of selection bias and statistically control for variables to block backdoor and confounding causal paths. Using our case study of suicidal ideation, for example, analyses should adjust for recruitment source, as the act of selection creates the biased path from autism to suicidal ideation. If the source population is enumerated, inverse probability weighting or standardization can align the sample population to the demographics of the source.

5. Avoid overgeneralizing results from the survey sample to the target population

As we have described, the sample size of an online survey may give the researchers the impression that they have a representative sample of autistic adults. Since research in autistic adults is of crucial public health importance but poorly funded, it is tempting to make broad statements. Claims such as “We found that autistic adults are more likely…” or titles such as “Risk factors for eating disorders autistic adults” would be more accurate and reduce overinterpretation by using more measured language such as “We found that in a sample of autistic adults recruited online…” or “Risk factors for eating disorders in a sample of autistic adults.”

6. Acknowledge the newness of survey research of autistic adults

The concerted study of outcomes for autistic adults is relatively new compared with the study of autistic children or research in other disability groups.⁵⁰ There are very few gold standard measures to characterize outcomes in autistic adults, as there are issues in adapting tools to autistic populations, identifying autistic adults, and capturing autism's heterogeneity (e.g., participation of autistic adults with intellectual disability).^51,52 Methods for studying autistic adults have limitations and strengths, unique to the newness of the topic. As the field advances, methods will improve, both for online surveys and other approaches to studying autistic adults. We recommend authors acknowledge that the field is still growing and innovative work in this area needs to be continued.

7. Continue to advocate for funding for research in issues that impact autistic adults

The paucity of funding for important research that affects autistic adults is an obstacle for all methodologically sound research in this area, not just online surveys. With more funding, studies can be done to validate findings from online surveys, improve precision of recruiting, and develop analytic methods to account for systemic error.

Conclusions

As a research field, we should strive to do work that is of high importance to the autistic community that results in near-term policy and practice change; this goal is not in conflict with doing sound and unbiased science. Online surveys are an important tool for researchers to generate hypotheses and connect with a wider range of participants. We hope that this perspective raises the topic of bias and misinterpretation in online surveys and researchers continue to produce valid and meaningful work that is crucial to improving lives of autistic adults.

Footnotes

Author Disclosure Statement

No competing financial interests exist.

Funding Information

This study was supported in part by a core grant to the Waisman Center from the National Institute of Child Health and Human Development (U54 HD090256).

References

Tomeny

, Vargo

, El-Toukhy

. Geographic and demographic correlates of autism-related anti-vaccine beliefs on Twitter, 2009-15. Soc Sci Med. 2017; 191:168–175.

Mazurek

. Social media use among adults with autism spectrum disorders. Comput Human Behav. 2013; 29(4):1709–1714.

Bellon-Harn

, Ni

, Manchaiah

. Twitter usage about autism spectrum disorder. Autism. 2020; 24(7):1805–1816.

Mohd Roffeei

, Abdullah

, Basar

. Seeking social support on Facebook for children with autism spectrum disorders (ASDs). Int J Med Inform. 2015; 84(5):375–385.

Parsons

, Cordier

, Vaz

, Lee

. Parent-mediated intervention training delivered remotely for children with autism spectrum disorder living outside of urban areas: Systematic review. J Med internet Res. 2017; 19(8):e198.

Robison

. Autism prevalence and outcomes in older adults. Autism Res. 2019; 12(3):370–374.

Frazier

, Dawson

, Murray

, Shih

, Sachs

, Geiger

. Brief report: A survey of autism research priorities across a diverse community of stakeholders. J Autism Dev Disord. 2018; 48(11):3965–3971.

Abel

, Machin

, Brownlow

. Support, socialise and advocate: An exploration of the stated purposes of Facebook autism groups. Res Autism Spectr Disord. 2019; 61:10–21.

Amaral

, Anderson

, Bailey

, et al. Gaps in current autism research: The thoughts of the autism research editorial board and associate editors. Autism Res. 2019; 12(5):700–714.

10.

Pecora

, Hancock

, Hooley

, et al. Gender identity, sexual orientation and adverse sexual experiences in autistic females. Mol Autism. 2020; 11(1):57.

11.

Cassidy

, Bradley

, Shaw

, Baron-Cohen

. Risk markers for suicidality in autistic adults. Mol Autism. 2018; 9:42.

12.

Stern

, Bilgen

, Dillman

. The state of survey methodology. Field Methods. 2014; 26(3):284–301.

13.

Kapoor

, Tamilmani

, Rana

, Patil

, Dwivedi

, Nerur

. Advances in social media research: past, present and future. Inf Syst Front. 2017; 20(3):531–558.

14.

Townsend

, Nielsen

, Allister

, Cassidy

. Key ethical questions for research during the COVID-19 pandemic. Lancet Psychiatry. 2020; 7(5):381–383.

15.

Gillespie-Lynch

, Kapp

, Shane-Simpson

, Smith

, Hutman

. Intersections between the autism spectrum and the internet: Perceived benefits and preferred functions of computer-mediated communication. Intellect Dev Disabil. 2014; 52(6):456–469.

16.

Bejerot

, Eriksson

, Mortberg

. Social anxiety in adult autism spectrum disorder. Psychiatry Res. 2014; 220(1–2):705–707.

17.

Huang

, Arnold

SRC

, Foley

K-R

, Trollor

. Diagnosis of autism in adulthood: A scoping review. Autism. 2020; 24(6):1311–1327.

18.

Ball

. Conducting online surveys. J Hum Lact. 2019; 35(3):413–417.

19.

Bennett

, Khangura

, Brehaut

, et al. Reporting guidelines for survey research: An analysis of published guidance and reporting practices. PLoS Med. 2010; 8(8):e1001069.

20.

Huebschmann

, Leavitt

, Glasgow

. Making health research matter: A call to increase attention to external validity. Annu Rev Public Health. 2019; 40:45–63.

21.

Warrier

, Greenberg

, Weir

, et al. Elevated rates of autism, other neurodevelopmental and psychiatric diagnoses, and autistic traits in transgender and gender-diverse individuals. Nat Commun. 2020; 11(1):3959.

22.

Stevenson

, Harp

, Gernsbacher

. Infantilizing autism. Disabil Stud Q. 2011; 31(3):dsq-sds.org/article/view/1675/1596.

23.

Livingston

, Shah

, Happe

. Compensatory strategies below the behavioural surface in autism: A qualitative study. Lancet Psychiatry. 2019; 6(9):766–777.

24.

Klonsky

, May

, Saffer

. Suicide, suicide attempts, and suicidal ideation. Annu Rev Clin Psychol. 2016; 12:307–330.

25.

Hedley

, Uljarević

. Systematic review of suicide in autism spectrum disorder: Current trends and implications. Curr Dev Disord Rep. 2018; 5(1):65–76.

26.

Cassidy

, Bradley

, Robinson

, Allison

, McHugh

, Baron-Cohen

. Suicidal ideation and suicide plans or attempts in adults with Asperger's syndrome attending a specialist diagnostic clinic: A clinical cohort study. Lancet Psychiatry. 2014; 1(2):142–147.

27.

Cassidy

, Bradley

, Bowen

, Wigham

, Rodgers

. Measurement properties of tools used to assess suicidality in autistic and general population adults: A systematic review. Clin Psychol Rev. 2018; 62:56–70.

28.

David-Ferdon

, Crosby

, Caine

, Hindman

, Reed

, Iskander

. CDC grand rounds: Preventing suicide through a comprehensive public health approach. MMWR Morb Mortal Wkly Rep. 2016; 65(34):894–897.

29.

Lai

, Baron-Cohen

. Identifying the lost generation of adults with autism spectrum conditions. Lancet Psychiatry. 2015; 2(11):1013–1027.

30.

Kirby

, Bakian

, Zhang

, Bilder

, Keeshin

, Coon

. A 20-year study of suicide death in a statewide autism population. Autism Res. 2019; 12(4):658–666.

31.

Burke

, Heller

. Disparities in unmet service needs among adults with intellectual and other developmental disabilities. J Appl Res Intellect Disabil. 2017; 30(5):898–910.

32.

Cassidy

, Rodgers

. Understanding and prevention of suicide in autism. Lancet Psychiatry. 2017; 4(6):e11.

33.

Rothman

, Greenland

, Lash

. Modern Epidemiology, 3rd Edition. Philadelphia, PA: Lippincott, Williams & Wilkins; 2008.

34.

Russell

, Mandy

, Elliott

, White

, Pittwood

, Ford

. Selection bias on intellectual ability in autism research: A cross-sectional review and meta-analysis. Mol Autism. 2019; 10(1):9.

35.

Groves

, Presser

, Dipko

. The role of topic interest in survey participation decisions. Public Opin Q. 2004; 68(1):2–31.

36.

Delgado-Rodriguez

, Llorca

. Bias. J Epidemiol Community Health. 2004; 58(8):635–641.

37.

Hernan

, Hernandez-Diaz

, Robins

. A structural approach to selection bias. Epidemiology. 2004; 15(5):615–625.

38.

Rai

, Heuvelman

, Dalman

, et al. Association between autism spectrum disorders with or without intellectual disability and depression in young adulthood. JAMA Netw Open. 2018; 1(4):e181465.

39.

Hedley

, Uljarevic

, Wilmot

, Richdale

, Dissanayake

. Brief report: Social support, depression and suicidal ideation in adults with autism spectrum disorder. J Autism Dev Disord. 2017; 47(11):3669–3677.

40.

Choi

, Milne

, Glozier

, Peters

, Harvey

, Calvo

. Using different Facebook advertisements to recruit men for an online mental health study: Engagement and selection bias. Internet Interv. 2017; 8:27–34.

41.

Thornton

, Batterham

, Fassnacht

, Kay-Lambkin

, Calear

, Hunt

. Recruiting for health, medical or psychosocial research using Facebook: Systematic review. Internet Interv. 2016; 4:72–81.

42.

Tyrer

, Heyman

. Sampling in epidemiological research: Issues, hazards and pitfalls. BJPsych Bull. 2016; 40(2):57–60.

43.

A Dictionary of Epidemiology, Fifth Edition. Porta M, ed. New York: Oxford University Press; 2008.

44.

Morabia

Invited commentary: Do-it-yourself modern epidemiology—At last!. Am J Epidemiol. 2014; 180(7):669–672.

45.

Checkoway

, Pearce

, Kriebel

. Selecting appropriate study designs to address specific research questions in occupational epidemiology. Occup Environ Med. 2007; 64(9):633–638.

46.

Seaman

, White

. Review of inverse probability weighting for dealing with missing data. Stat Methods Med Res. 2013; 22(3):278–295.

47.

Lai

, Lombardo

, Auyeung

, Chakrabarti

, Baron-Cohen

. Sex/gender differences and autism: Setting the scene for future research. J Am Acad Child Adolesc Psychiatry. 2015; 54(1):11–24.

48.

Daniels

, Rosenberg

, Anderson

, Law

, Marvin

, Law

. Verification of parent-report of child autism spectrum disorder diagnosis to a web-based autism registry. J Autism Dev Disord. 2012; 42(2):257–265.

49.

SPARK Consortium. Electronic address: pfeliciano@simonsfoundation.org. A US cohort of 50,000 families to accelerate autism research. Neuron. 2018; 97(3):488–493.

50.

Wise

. Aging in autism spectrum disorder. Am J Geriatr Psychiatry. 2020; 28(3):339–349.

51.

Underwood

JFG

, Kendall

, Berrett

, et al. Autism spectrum disorder diagnosis in adults: Phenotype and genotype findings from a clinically derived cohort. Br J Psychiatry. 2019; 215(5):1–7.

52.

Roestorf

, Bowler

, Deserno

, et al. “Older adults with ASD: The consequences of aging.” Insights from a series of special interest group meetings held at the International Society for Autism Research 2016–2017. Res Autism Spectr Disord. 2019;63:3–12.