Letter to the Editor: A possible threat to data integrity for online qualitative autism research

Abstract

Researchers are increasingly relying on online methods for data collection, including for qualitative research involving interviews and focus groups. In this letter, we alert autism researchers to a possible threat to data integrity in such studies: “scammer” participants, who may be posing as autistic people and/or parents of autistic children in research studies, presumably for financial gain. Here, we caution qualitative autism researchers to be vigilant of potential scammer participants in their online studies and invite a broader discussion about the implications of such fraudulent acts.

Lay abstract

Doing research online, via Zoom, Teams, or live chat, is becoming more and more common. It can help researchers to reach more people, including from different parts of the world. It can also make the research more accessible for participants, especially those with different communication preferences. However, online research can have its downsides too. We have recently been involved in three studies in which we had in-depth discussions with autistic people and/or parents of autistic children about various topics. It turns out, though, that some of these participants were not genuine. Instead, we believe they were “scammer participants”: people posing as autistic people or parents of autistic children, possibly to gain money from doing the research. This is a real problem because we need research data that we can trust. In this letter, we encourage autism researchers to be wary of scammer participants in their own research.

Keywords

fraud imposter research participation

Online data collection methods have been game-changing—especially during the COVID-19 pandemic (Lobe et al., 2020)—allowing large samples to be recruited across geographic limits with relative ease and proliferating more inclusive practices. There is, however, a potentially stark disadvantage to these methods, especially those that offer participant incentives (Jones et al., 2021; Teitcher et al., 2015).

We have heard of—and been victims of—fraudulent participation in studies using online, survey-based methods (see Lawlor et al., 2021, for a guiding framework). Online methods are usually asynchronous in nature, providing “scammer” participants¹ with an easy and anonymous way to supplement income at a time of cost-of-living crises. In our recent experience, these fraudulent acts have extended to online synchronous methods, including in-depth interviews and focus groups, concentrating on intensely personal issues in populations the general public do not understand.

We have now been involved in three online qualitative autism research studies (in Australia, United States, and the United Kingdom), in which we conducted a significant number of interviews with participants we believe were posing as autistic people and/or parents of autistic children (via Zoom, Teams, or live chat). It is unclear whether each interview was with a unique participant or whether there were a small number of people repeatedly creating alternate identities to take part in our research.

Although such fraudulent acts are a growing problem in online qualitative health-related research (Ridge et al., 2023; Roehl & Harland, 2022), there are good reasons to believe that certain important characteristics of online qualitative autism research, specially designed to make it easier for people to participate, may increase the risk of attracting scammer participants. In this letter, we caution qualitative autism researchers to be vigilant and have an open discussion on the implications.

About our studies

Several of our studies’ features align with those of other autism research by using social media as one key approach to recruitment, including an online prescreening questionnaire to determine eligibility, professionally diagnosed (without formal verification) and self-identified autistic participants to allow for the prohibitive nature of diagnostic processes (Sarrett & Kapp, 2018), using multiple researchers to conduct one-on-one interviews, and offering remuneration through gift vouchers to acknowledge participants’ time, effort, and expertise (ranging from 10 to 25 USD). Other features, however, were specifically prompted by our participatory approach (Fletcher-Watson et al., 2019; Nicolaidis et al., 2019), ensuring our methods were responsive to participants’ needs for a range of interview formats, including text-based interviews (see also Ashworth et al., 2021; Cascio et al., 2020), and enhancing research quality (Pellicano et al., 2022). For example, we recruited via our community partners’ extensive online connections, provided a range of interview formats, and distributed questions in advance to accommodate processing differences (see Gillespie-Lynch et al., 2014).

For each project, several issues aroused the suspicion of our autistic and nonautistic research teams. Researchers noted the following for suspect participants:

Email addresses were often identically configured from the same email platform.

Emails were short, curt, and similar in format and style across purportedly different participants, sometimes without subject lines.

Booking data suggested that participants were based in countries different from those they said they were based in.

There was an abundance of participants claiming to be from highly underrepresented groups (whom we were often purposively sampling).

Interview scheduling was significantly more straightforward than usual (unlimited participant availability, keenness for the process to happen quickly).

Cameras were kept off during Zoom/Team interviews, and participants were difficult to understand, sometimes due to patchy internet connections.

Interviews were short in duration (<30 min) compared with others and elicited vague and confused responses, including an apparent lack of familiarity with autistic experiences.

There were a series of sharp inconsistencies in participants’ stories or accounts of their own perspectives, including differences between prescreening questionnaire responses and personas presented in the interview/focus group (switching from parent of autistic child to autistic person, stating discrepant ages, email addresses not matching stated names).

Similarities in participant voices, mannerisms, and responses across different focus groups/interviews.

Participants making more frequent-than-usual enquiries about payments, including the timing and type of voucher offered.

The efforts made by these participants—from completing a preinterview questionnaire and scheduling an interview to taking part in the interview or focus group (with other people) and follow-up emails with researchers—presumably for monetary gain, was striking. This may result from contemporary cost-of-living pressures in the countries where studies were conducted or from the fact that even the smaller amount of compensation offered for such participation can be of significant value to those living in lower-income countries.² Whatever the causes, however, team members were conscious of the need to screen and remove data due to a lack of trustworthiness.

This experience has prompted much reflection about how best to ensure the integrity of our qualitative data, while maintaining trust with genuine participants, increasing accessibility of research, and avoiding stereotyping any particular group of participants as more or less likely to be a scammer participant. Table 1 stems from our discussions on how we might navigate this problem. These potential strategies seek to strike the right balance between the need to make research open and inclusive while maintaining the integrity of the research and dissuading scammer participants from entering the research process (see Teitcher et al., 2015, for discussion of ethical concerns).

Table 1.

The pros and cons of potential strategies to mitigate threats to data integrity in online qualitative autism research.

Research phase	Strategy	Possible advantages to implementation	Possible disadvantages to implementation
Recruitment	Do not offer incentives for study participation	• May eliminate the financially motivated “study scammer” issue. • Enables feasibility of study for researchers who lack funding (e.g., students and early career researchers). • May promote more effective long-term community commitment to research participation through altruism (Titmuss, 2018).	• May suggest we do not value participants’ time and contribution. • Could be extractive and exploitative for an autism community more likely to live in poverty and be oversaturated with research. • Potentially damages trust between researchers and community. • May exclude lower-income participants and bias sample toward people with more comfortable financial situations and intrinsic interest.
	Include in the information letter and consent form that steps will be taken to identify scammer participants (see Teitcher et al., 2015).	• Disclosing this to participants respects their rights.• May dissuade potential scammer participants that read the information letter.	• Could deter legitimate participants from taking part.
	Restrict/close recruitment from particular sources (e.g., Twitter, Facebook) (Ridge et al., 2023).	• Suspect participants or bots might find it more difficult to identify research studies.	• Potentially excludes a significant portion of the autistic and autism communities from participating.
	Replace terms such as “gift card” or “payment” in recruitment materials with more generic terms that describe compensation for time.	• It might make the study adverts less “findable” by scammer participants who may use specific search terms.	• Introduces a lack of clarity during the consent process and may go against good ethical practices.
Before data collection	For preinterview screening questionnaires, implement features identified as important for preventing fraudulent responses within quantitative research, including using CAPTCHA, restricting responses from the same IP address, requiring responses for all questions, and setting minimum limits for free-text boxes (Lawlor et al., 2021; Teitcher et al., 2015).	• Many fraudulent participants often use bots to undertake the initial questionnaires or expressions of interest.• Having a “red flag” system for data received through the initial online survey allows a relatively objective way to conduct initial screening.	• Participants should have the choice whether to respond to particular questions.• Bots are now more proficient at CAPTCHAs than particular groups of disabled people, and other similar methods can be just as inaccessible to those who are blind or dyslexic.• Same IP address bans can result in exclusion of participants from the same family and students (who make up the bulk of psychology studies).
	If IP addresses are not restricted, check carefully for potentially fraudulent responses (e.g., multiple responses from the same IP address). Similarly, if using an online booking system (e.g., Calendly), check which time zone the booking was made in if only focusing on particular geographic areas.	• It could rule out multiple responses from the same IP address.• Could provide one way to rule out those who did not use VPNs.	• Not a fool-proof solution since people can mask their true IP address using VPNs and can have IP addresses or set time zones outside of their geographical location.• Same IP address bans can result in exclusion of family members or students from the same university.
	Not following up with people with email addresses that followed a highly similar pattern, or writing to say recruitment has closed.	• Could prevent suspect participants from engaging further in the research process.	• Not following up with people might mean that legitimate participants are inadvertently excluded, especially those with greater executive function difficulties.
	During an expression-of-interest phase, ask participants to include a brief description of themselves (with a minimum number of characters; Jones et al., 2021).	• Including at least one open question for which a “bot” might not have a preprogrammed response, but that can be answered by a genuine participant, may assist with screening.	• It would increase participant burden.• Scammer participants could overcome this feature through the use of artificial intelligence (e.g., ChatGPT).• Could deter genuine participation because it may seem invasive and unnecessary or make responding difficult for people with certain disabilities to participate.
	Require potential participants to provide documentation (e.g., diagnostic information) that they meet the study’s inclusion criteria.	• It would allow researchers to confirm study eligibility and also potentially identify potential suspect participants.	• Diagnostic details are sensitive information, which could deter legitimate participants from participating.• Potential issues with data privacy.• Excludes people who self-identity as autistic from participating, which may inadvertently and disproportionately affect working-class, socioeconomically disadvantaged people, women, gender minorities, and racial minorities.
	Include a preinterview briefing over videoconferencing or telephone to go through eligibility criteria and the consent process. Researchers could forewarn potential participants about this aspect in the consent form: “If you are keen to take part, we will arrange a preinterview chat, at a time that suits you. This will either be over the phone or online using Zoom, during which I will ask you some questions to make sure that you fit the study criteria. Your responses will be used to confirm your identity and to discourage scammer participants” (adapted from Roehl & Harland, 2022).	• This process might identify potential scammer participants while, at the same time, helping to establish rapport with legitimate participants. We know that autistic participants feel “catered for” and “safe” when relational aspects of the research process have been implemented (Pellicano et al., 2022).	• Autistic people and parents of autistic children often have busy lives, and including a preinterview might deter participants from taking part.• Including a preinterview requires additional researcher and participant time.• Misinterpretation by researchers could exclude genuine participants who do not communicate as expected from the research.
Data collection	Requiring participants’ cameras to be on momentarily, during the preamble of the interview, for example, but giving the option to keep it off for the rest of the interview—to deter people from doing the interview multiple times, not to establish whether someone is autistic or a parent of an autistic person. Researchers could include information on this aspect of the research in the consent form: “Interviews will be done via Zoom. I will have my camera on, and I will want to see your face for the first few moments of the interview, until we begin recording. You can choose whether we have our cameras on or off for the rest of the interview” (adapted from Roehl & Harland, 2022).	• It could deter suspect participants from continuing with the research or prevent suspect participants from participating repeatedly.	• It could deter legitimate participants from taking part. Being on camera, even for a short time, might make autistic people anxious.
	If the interviewer has doubts about the authenticity of the participant, introduce an interview question that was not in the original interview schedule but that autistic participants would likely not find difficult to answer (e.g., Can you describe what being autistic means for you, or some of your traits/characteristics that you feel are part of being autistic? Can you describe what makes you autistic?) (see also Roehl & Harland, 2022).	• Scammer participants might find it challenging to answer unexpected questions. Researchers could use confusing or vague responses to probe with follow-up questions or finish the interview early should they have strong suspicions about the participant’s authenticity.	• Scammer participants can “Google” autistic traits before interviews to prepare.• Clear or inconsistent responses does not always indicate fraudulent behavior.• Variation does not necessitate fraud. Autistic people are heterogeneous and experience being autistic in a wide variety of ways. This works against the purpose of sending out questions in advance to accommodate processing differences.
	Taking in-depth reflective notes following the interview, reflecting particularly on the trustworthiness of the person’s responses.	• Keeping reflexive journals and memos are important for ensuring trustworthiness in qualitative research.	• Requires additional researcher time (although this should be par for the course for qualitative researchers).• This is not a consistent or objective way to detect fraud as interpretation and opinions may vary across researchers.
	Meeting regularly with other interviewers and with the broader team to debrief and discuss concerns around data rigor specifically.	• It would allow the team to share experiences, review the data, and identify any suspect behavior as quickly as possible.	• Requires additional researcher time.• Is difficult when working in larger cross-continental teams.
After data collection	Only offer nontransferable vouchers that must be texted to a mobile phone or posted to a physical address within the geographic limits of the study. Alternatively, consider payments directly into bank accounts, as opposed to vouchers (Ridge et al., 2023).	• These mitigating features may make it less likely for vouchers to be sold on or are of limited use to participants from outside the country where recruitment is taking place.	• Increases administrative burden for both researchers and participants.• Introduces data privacy issues and increased liability for researchers and their institutions.• Discriminates against those without a permanent mailing address or mobile phone as housing instability and financial insecurity are common in disability populations.• Can deter autistic people from taking part, as form filling and administrative tasks can be challenging; and this is one of the easier barriers for scammer participants to work around if using bots.
Reporting	Researchers may consider notifying research ethics committees and/or funders about any fraudulent activities, which may have financial implications and threats to data integrity.	• Research ethics committees can track the severity of the problem.	• If research ethics committees react poorly, researchers may be hesitant to report fraudulent activity in future.
	Reporting to participants to inform them of potential scammer participants.	• Ensures transparency.	• Needs careful consideration given the unlikelihood of incontrovertible evidence that those we suspect of being scammer participants were so and the likely absence of concrete harm to honest participants because of their interactions with potentially dishonest ones.• May raise anxiety and/or reduce trust in autism science.
	Be transparent in dissemination materials (reports, papers) about the extent and nature of fraud in the study, how it was dealt with by researchers (including the standard used to justify removal), and how it might have influenced the results.	• Such reporting is in the spirit of open, transparent (autism) science (Fletcher-Watson et al., 2021).	• Open, transparent research is not always interpreted as such by the lay public, and some participants may be deterred from participation if they think the study is compromised or researchers are not being diligent enough about protecting study integrity.

This is complex, as some suggestions for mitigating against fraud in online qualitative studies could exclude legitimate autistic participants from taking part. Moreover, these measures could disproportionately disadvantage autistic people, whose testimony has often been questioned (see Jaswal & Akhtar, 2018), and whose experiences of autism research have not always been positive (Botha, 2021; Pellicano et al., 2014; Pukki et al., 2022). Steps taken to secure data quality and integrity must therefore ensure they are dismantling, rather than enacting, epistemic injustice to create the conditions for safe and trusting interactions (Cascio et al., 2020).

Conclusion

As qualitative researchers, we are acutely aware of the need to be reflective and reflexive in research (Braun & Clarke, 2019), and these recent experiences have convinced us of the need to open a broader conversation with our peers. We urge discussion, reflection, and collaborative action to ensure high-quality data, without limiting genuine participation (cf. Ridge et al., 2023).

Footnotes

Acknowledgements

We are very grateful to our broader team members who have contributed to ongoing discussions about data integrity, including Rachel Bowen, Tendai Dawkins, Jac den Houting, Martin Downes, Andrew Frakes, Honey Heussler, Melanie Heyworth, Maggie Johnson, Kelsie Daley, Jessica Paynter, Bec Poulsen, Hannah Rapaport, Kate Simpson, Diana Tan, David Trembath, and Marleen Westerveld.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The studies on which this letter is based were supported by an Australian Research Council Future Fellowship (FT190100077) awarded to EP, a Medical Research Future Fund (MRFF) from the Australian Government awarded to DA and EP (MRF201614), an Autism Intervention Network on Physical Health (AIR-P) Scholars & Pilot and Feasibility grant awarded to CH, and an Autism Science Foundation Suzanne Wright Memorial Research Accelerator Grant to CH. This latter project is supported by the Health Resources and Services Administration (HRSA) of the U.S. Department of Health and Human Services (HHS) under the UT2MC39440 grant. The information, content and/or conclusions are those of the authors and should not be construed as the official position or policy of, nor should any endorsements be inferred by HRSA, HHS or the U.S. Government.

Community involvement statement

This letter to the editor is a collaboration between nonautistic and autistic researchers who have been directly subject to scammer participants, involved in discussions about detecting and preventing such fraud, including the pros and cons and ethical implications, and/or contributed to the writing of the manuscript.

ORCID iDs

Elizabeth Pellicano

Dawn Adams

Laura Crane

Connie Allen

Notes

References

Ashworth

Crane

Steward

Bovis

Pellicano

(2021). Towards empathetic autism research: Developing an autism-specific Research Passport. Autism in Adulthood, 3, 280–288.

Botha

(2021). Academic, activist or advocate? Angry, entangled and emerging: A critical reflection on autism knowledge production. Frontiers in Psychology, 12, 727542.

Braun

Clarke

(2019). Reflecting on reflexive thematic analysis. Qualitative Research in Sport and Exercise Health, 11, 589–597.

Cascio

M. A.

Weiss

J. A.

Racine

., & the Autism Research Ethics Taskforce. (2020). Person-oriented ethics for autism research: Creating best practices through engagement with autism and autistic communities. Autism, 24, 1676–1690.

Fletcher-Watson

Adams

Brook

Charman

Cusack

Crane

Leekam

Milton

Parr

J. R.

Pellicano

(2019). Making the future together: Shaping autism research through meaningful participation. Autism, 23, 943–953.

Fletcher-Watson

Bölte

Crompton

C. J.

Jones

Lai

M.-C.

Mandy

Pellicano

Stahmer

Taylor

Mandell

(2021). Publishing standards for promoting excellence in autism research. Autism, 25, 1501–1504.

Fronek

Briggs

(2018). Faking participant identity: Vested interests and purposeful interference. Research Ethics, 14, 1–5.

Gillespie-Lynch

Kapp

S. K.

Shane-Simpson

Smith

D. S.

Hutman

(2014). Intersections between the autism spectrum and the internet: Perceived benefits and preferred functions of computer-mediated communication. Intellectual and Developmental Disabilities, 52(6), 456–469.

Jaswal

V. K.

Akhtar

(2018). Being vs. appearing socially uninterested: Challenging assumptions about social motivation in autism. Behavioral and Brain Sciences, 42, 1–73.

10.

Jones

Caes

Rugg

Noel

Bateman

Jordan

(2021). Challenging issues of integrity and identity of participants in non-synchronous online qualitative methods. Methods in Psychology, 5, 100072.

11.

Lawlor

Thomas

Guhin

A. T.

Kenyon

Lerner UCAS Consortium, & Drahota

(2021). Suspicious and fraudulent online survey participation: Introducing the REAL framework. Methodological Innovations, 14, 20597991211050467. https://doi.org/10.1177/20597991211050467

12.

Lobe

Morgan

Hoffman

K. A.

(2020). Qualitative data collection in an era of social distancing. International Journal of Qualitative Methods, 19, 1609406920937875. https://doi.org/10.1177/1609406920937875

13.

Nicolaidis

Raymaker

Kapp

S. K.

Baggs

Ashkenazy

McDonald

Weiner

Maslak

Hunter

Joyce

(2019). The AASPIRE practice-based guidelines for the inclusion of autistic adults in research as co-researchers and study participants. Autism, 23, 2007–2019.

14.

Pellicano

Dinsmore

Charman

(2014). Views on researcher-community engagement in autism research in the United Kingdom: A mixed-methods study. PLOS ONE, 9, e109946.

15.

Pellicano

Lawson

Hall

Mahony

Lilley

Heyworth

Clapham

Yudell

(2022). “I knew she’d get it, and get me”: Participants’ perspectives of a participatory autism research project. Autism in Adulthood, 4, 120–129.

16.

Pukki

Bettin

Outlaw

A. G.

Hennessy

Brook

Dekker

Doherty

Shaw

S. C. K.

Bervoets

Rudolph

Corneloup

Derwent

Lee

Rojas

Y. G.

Lawson

Gutierrez

M. V.

Petek

Tsiakkirou

Suoninen

Minchin

. . . Yoon

W. H.

(2022). Autistic perspectives on the future of clinical autism research. Autism in Adulthood, 4, 93–101.

17.

Ridge

Bullock

Causer

Fisher

Hider

Kingstone

Gray

Riley

Smyth

Silverwood

Spiers

Southam

(2023). “Imposter participants” in online qualitative research, a new and increasing threat to data integrity? Health Expectations, 26, 941–944. https://doi.org/10.1111/hex.13724

18.

Roehl

J. M.

Harland

D. J.

(2022). Imposter participants: Overcoming methodological challenges related to balancing participant privacy with data quality when using online recruitment and data collection. Qualitative Reports, 27, 2469–2485.

19.

Sarrett

J. C.

Kapp

S. K.

(2018). Self-identification and self-diagnosis in the autistic community. In Heller

Parker Harris

Gill

Gould

(Eds.), Disability in American life (pp. 588-591). ABC-CLIO.

20.

Teitcher

J. E. F.

Bockting

W. O.

Bauermeister

J. A.

Hoefer

C. J.

Miner

M. H.

Klitzman

R. L.

(2015). Detecting, preventing and responding to “fraudsters” in internet research: Ethics and tradeoffs. Journal of Law, Medicine and Ethics, 43, 116–133.

21.

Titmuss

(2018). The gift relationship: From human blood to social policy. Policy Press.