Abstract
The author explores interactions with one research subject who feigns credentials and invents stories in order to participate in social science research interviews online. The possibility of intentional deception among interviewees in virtually mediated fieldwork is a critical consideration in the context of the recent extensive pivot to online-based fieldwork during the need for social distancing associated with the coronavirus disease 2019 pandemic. Following this rapid shift in what is generally accepted as the “gold standard” for social science research interviews, widespread use of online-based interviewing methods will likely endure as equivalent to in-person methods. A methodological case study with implications for virtually mediated fieldwork, this article highlights some of the advantages and disadvantages of virtually mediated interviews and provides practical suggestions.
When the coronavirus disease 2019 (COVID-19) pandemic shut down in-person research studies, enterprising and creative researchers who previously conducted in-person research pivoted en masse to a virtual or hybrid model, including virtually mediated interviews. This is great news for social science, and no doubt this pivot and newfound acceptance for additional sites for social research is permanent. Nevertheless, as interview methods evolve, new vulnerabilities arise. In this article I consider the addition of virtually mediated interaction to the field’s “gold standard” for research interviews. In particular, I focus on conversations with an interview respondent who has participated in hundreds of research interview studies virtually; how she is able to convince researchers that she is the “real deal”; and practical suggestions for recruiting, interviewing, verifying, and paying interviewees in virtually mediated studies.
The COVID-19 pandemic provided a decisive nudge for qualitative social science researchers to be more open to virtually mediated methods (Hidalgo and Khan 2020; Murphy, Jerolmack, and Smith 2021). However, even before the pandemic, calls to more fully examine virtual realms and virtual methods abounded (e.g., Lane 2018). Debates on methodological questions remain in flux, but the consensus is that virtually mediated communication and social life, including virtually mediated methods of data collection, should be included in qualitative analysis (Murphy et al. 2021). Practical and theoretical questions concerning how to attain “good” data, how to undertake “good” analysis, and the interaction between these continue to evolve (Silverman 2017).
The Virtually Mediated Interview
Virtually mediated video interviews have been used for more than a decade (Adams-Hutcheson and Longhurst 2017; Edwards and Holland 2020; Hanna 2012; Janghorban, Roudsari, and Taghipour 2014; Jenner and Myers 2018; Krysan and Couper 2003; Mirick and Wladkowski 2019; Namey et al. 2019; Sedgwick and Spiers 2009; Sullivan 2012; Thunberg and Arnell forthcoming; Vogl 2013; Weinmann et al. 2012). Until recently, however, the relatively untested ground of virtually mediated interviews meant that in-person interviews remained the research “gold standard” (Deakin and Wakefield 2014; Gerson and Damaske 2020; Howlett 2021; Murphy et al. 2021). 1 With improvements in technology, and because of practical matters related to the need for social distancing in the wake of the COVID-19 pandemic, virtually mediated interviews have rapidly gained an equivalence to in-person interviews, which is likely to endure (Howlett 2021).
Prior to the pandemic, methodological literature focused on what is lost and what is gained through interviews that do not take place in person. Synchronous interviews with video chat, text chats, and audio only, as well as asynchronous interviews through email or comment streams, all have particular vulnerabilities. A number of technological issues are challenging for both subjects and interviewers, including lagging data, dropped calls, and incomprehensibility. Other issues include the possibility of a larger number of no-shows or last-minute cancellations, or participants who refuse to turn on their cameras (Deakin and Wakefield 2014). In virtual interviews, the balance of power between the interviewer and interviewee shifts, which may lead to differences in data gathered compared with in-person methods (Adams-Hutcheson and Longhurst 2017; Krysan and Couper 2003; Vogl 2013). Nevertheless, most studies of virtual interviews indeed showed that they were just as methodologically useful as in-person interviews, especially video interviews, as many issues and critiques related to virtually mediated interviews are resolved by the visibility of interviewees gained with use of a camera (e.g., Edwards and Holland 2020; Hanna 2012; Howlett 2021; Janghorban et al. 2014; Jenner and Myers 2018; Mirick and Wladkowski, 2019; Namey et al. 2019; Sedgwick and Spiers 2009).
Online interviews present unique challenges but also opportunities. In a virtual conference held following almost a year of pandemic lockdowns, a panel of graduate student qualitative researchers discussed some of the clear advantages, as well as initial challenges, of pivoting to virtually mediated research (Slaymaker et al. 2021). One researcher discussed how it was difficult to have verbal conversations with some subjects because they were stuck at home with people who were dangerous to them; the researcher reported switching to text conversations at times and found it helpful (Slaymaker et al. 2021). Another researcher recruited interview subjects through online ads and was thrilled with the quantity and quality of interviews she was able to complete in a relatively short amount of time (Slaymaker et al. 2021). Such a thing would not have been possible if all interview subjects had to make time in their schedules to travel to be in the same room as the researcher for the interview (Slaymaker et al. 2021). In my own experience, I have found that the ability to keep up relationships through multiple interviews with the same subject, to chat with subjects on their lunch breaks, or to video-conference with them in their apartments has exponentially increased. As a bonus, getting to know spaces, habits, partners, and mothers with dementia adds depth rather than taking away from it. These are all things that would have been mostly impossible in in-person interviews. 2
At times, virtually mediated interviewing necessitates methodological considerations that are not analogous to in-person interviews. For example, the existing literature discusses various types of data and metadata that may be lost or are unavailable during virtually mediated interviews. Acknowledging that interviewees are not “dopes,” 3 researchers must continue to pursue a critical understanding of subjects and data in virtual spaces (Atkinson and Silverman 1997; Cicourel 1982; Silverman 2017). 4 For example, intangibles of in-person interviews allow interviewers to understand when interviewees are uncomfortable, withholding, or acceding to interview questions. Pauses, diversions, pacing, and avoidance in interview exchanges can be meaningful for analysis (see, e.g., Duneier 2011). In this domain, analysis of in-person versus virtual behavior in interviews may lead to different paths. Additionally, recruiting online and conducting interviews online opens up the possibility that a person residing in one geographic location may claim to be somewhere else and, as in the case of Colleen presented here, may participate in many studies in a short period of time without traveling. 5 The removal of constraints naturally presented by physical travel shifts the meaning of incentive structures and opens up new possibilities.
The Case of Colleen
In this section, I present a case study based on conversations with Colleen (a pseudonym), an interviewee who feigned credentials and invented stories during an initial online interview, pretending to be a recently married public housing resident living in New York. I also interviewed the “real” Colleen, who spoke candidly about her participation in “hundreds” of primarily socially oriented research studies. I first met Colleen while recruiting New York City public housing residents to discuss marriage in a virtual interview through an online advertisement. In previous studies involving public housing, I had recruited interviewees through my contacts at organizations that worked within public housing, as well as using an inconvenience sample 6 of residents randomly recruited from online ads and then interviewed in person. This method of recruitment was thus not alien to me, and it felt convenient following a year of online-based lockdown life. Nevertheless, in conducting previous interviews with public housing residents, even when recruiting took place online, the interviews themselves were conducted in person.
A huge number of people live in public housing in New York City—about the same number of people as live in Detroit—so I felt that my chances of finding people through online advertisements who were both qualified and willing to be interviewed were fairly good. I received a number of responses, and most of the interviews resulting from this recruitment attempt were plausible. Colleen, who at the time claimed to be a recently engaged resident of public housing in New York City, was one of the first people who responded to my online advertisement, writing, Your ad is cool. Yes, I’m living in [public] housing with my husband- we were just married a few weeks ago. Just wondered if you happened to pay via Paypal instead of a gift card or if the gift card could be a visa gift card? As you can imagine, we could use the money.
Her response had a few red flags, which I pinpoint in retrospect. In previous research involving interviews with public housing residents (which was not focused on love or marriage), I heard many love stories, but I had not met many people who had actually tied the knot and, at the time, no newlyweds. Speaking with this applicant would add further depth to my research. I was excited to have found Colleen and looked forward to speaking with her. On close reading, though, even in her initial contact, Colleen expected a certain sensibility between the researcher and the research subject: “we could use the money.” This statement was emotionally fraught for an initial response to an anonymous Internet advertisement. However, as the one offering payment (even if euphemistically referred to as an “incentive” or a “gift card”), the researcher does hold influence over respondents who would like to have that payment. Still, mentioning money at all is a bit odd, most people just do not do so. Add Colleen’s request for details of how the money could be delivered, and her initial contact was not just concerned with money but almost entirely focused on it. In my excitement in recruiting someone like Colleen to my study, and an attempt to be mindful of my own concerns around frank conversation about money, I overlooked these initial red flags.
I responded to Colleen first by email, explaining what my research was about and how the interview would go. I had designed a few questions to narrow down serious respondents—simple things that any genuine respondent would be able to answer without much work, such as which housing complex they lived in, simple details about their marriage and engagement, and whether they were able to speak via Zoom. These questions narrowed down the field quite a bit. Only about a quarter of initial respondents wrote back a second time. Colleen was more verbose than other respondents, who were often monosyllabic in regard to these questions. 7 I then scheduled a time to meet, as I attempted to do with anyone who responded to the questions and seemed to meet the study’s criteria.
When Colleen arrived, on time, for our virtual video call, she told me the camera on her computer was broken. 8 Our connection was not good and she requested that we speak via phone instead. I agreed and proceeded with the interview. Starting with a second set of questions designed to covertly screen candidates, I was 99 percent sure that she was not being honest with me a few minutes in, but I continued the interview for several minutes before deciding to politely end it. At that point, I thanked Colleen for her time and paid her (through PayPal). She wrote me afterward in an email, “It was nice to speak with you today. Thank you so much for your prompt payment. I appreciate it very much!”
I was amused when during our conversation she described the process of getting her midtown Manhattan public housing unit as happening within a few weeks (it generally takes years) and the neighborhood as very scary at night (this is subjective, so it is possible, but she is the only person I have heard describe midtown that way). When I asked her to describe the things her neighbors did for a living, she replied, “Well, you know what they say . . . snitches get stitches.” Although I was highly suspicious, I could not objectively prove anything until months later, when I put up another advertisement for an unrelated preliminary study with completely different parameters, and the very first response I got was from Colleen. I was thrilled to be able to confirm my suspicions and wrote back to Colleen, explaining that we had already spoken previously about a different study. I proposed that we schedule a time to talk, but this time about the “real” Colleen. Colleen was fortunately willing to speak with me. 9
Colleen told me that she had participated in hundreds of online interviews and focus groups with researchers following the beginning of the COVID-19 pandemic. She found research opportunities by combing through advertisements on various social and job Web sites.
10
When looking for a job, Colleen tried to be the most polite, most on-time interview subject, an extremely willing chameleon. Colleen said, I’ve always been a good person for improv, acting, that type of thing, I feel that I’m able to keep very quickly on my feet. I’m able to answer their questions with them thinking that what I’m telling them is the truth.
Colleen was talkative and detail focused, and I understood how she could be experienced by researchers as the perfect subject. Notably, Colleen has experience as an academic, having earned a PhD and worked as an untenured university lecturer. She may thus feel comfortable speaking to researchers or be familiar with what is expected in research interviews.
11
“I’m a people pleaser,” she explained, “I intuitively know how to say what people want to hear. I’ve always been that way.” She also does research on her own to pass under the radar, and she relies on her real-life experiences and wits for the rest. She explained that before contacting me, she had done research to answer my questions: I had to research. I researched about living in housing. I had to research the name of the [public housing development] place. I had to make up [my husband]—because he doesn’t exist. . . . I made him up to be the sexiest Puerto Rican guy.
But beyond a few specific researched tidbits, Colleen lies in interviews primarily by telling the truth. She said that she might “switch around demographics” or use her experiences with jobs, friends, growing up, pets, relationships, or travel put into different contexts. The husband she told me about was modeled after a celebrity she likes.
During a conversation with Colleen, I pulled up a number of online advertisements for interviewees. When I asked if she would apply to take part in a study recruiting participants with obsessive-compulsive disorder, Colleen responded, “Oh, no, never any medical stuff.” Colleen said that she does not participate in medical studies because in her experience they require identification. I looked for social scientific studies, pulling up one discussing sexual assault. I asked her about that one. Probably not, “not the heavy stuff,” she sighed. Instead, she looks for studies that focus on, for example, interpersonal relationships, emotions, work, effects of the pandemic, or consumer habits.
When speaking with Colleen in our first interview, I had an advantage. I know what New York looks like, what it feels like in midtown at night (or Manhattan Valley or Brownsville or Far Rockaway or Mott Haven), and this information came from in-person, physical experience and observation. I know many stories about getting public housing, keeping public housing, and the day-to-day experiences of people living in public housing. To put it succinctly, although my knowledge is imperfect, I am familiar with the field. Knowing the field allowed me to quickly pick up that Colleen’s interview was implausible. 12 Getting to know the field is necessarily different virtually, but it must be just as carefully planned for through research design as in-person research. 13
According to Colleen, I was the only researcher to figure out that she was feigning qualifications and inventing stories. In our original interview, when I realized that Colleen was most likely not who she said she was, I ended the interview early and paid her without informing her that I suspected her deception. I believe others may have done the same. Regardless, research subjects who feign qualifications and invent stories present implications for virtually mediated fieldwork.
The Vulnerable Research Subject
The initial Colleen seemed preoccupied with payment, and the “real” Colleen wanted to clarify that she does these interviews out of true necessity. 14 Colleen told me that she was laid off from her job as a nontenured full-time lecturer at a public university because of the shutdowns during the COVID-19 pandemic, losing her status as a visa holder that allowed her to reside legally in the United States. With a decline in her health that left her in too much discomfort to work outside her home (and amid broader concerns about catching the coronavirus), she began participating in online social research to earn income. She specified that she did not use the money she earned from these studies to buy extravagant things but to pay the rent and buy food. This work is temporary, she said, until she can get back on her feet and hopefully move back to the country where she has citizenship.
A substantial literature explores the economic effects of the COVID-19 pandemic on vulnerable populations. COVID-19 not only spread more quickly among vulnerable populations, but it had graver health effects and greater economic consequences (Maestripieri 2021). Although highly educated, Colleen is an immigrant who lacked legal status in the United States, she is also disabled and a woman, all of which heightened her vulnerability to effects of the pandemic (Maestripieri 2021; Trout and Kleinman 2020). Workplace closures and stay-at-home orders, such as those that affected Colleen’s lecturer position, had some of the largest economic effects, which would not be mitigated until later in the pandemic (Deb et al. 2020). Colleen was also an untenured lecturer, which is a fragile position generally and was particularly so during the initial stages of the pandemic (Herder 2022). More fragile workers were more likely to halt working and to suffer more immediate income loss at the start of the pandemic, exacerbating preexisting inequalities (Blundell et al. 2020). Furthermore, it is possible that individuals without documentation, such as Colleen, were excluded from key income support provisions associated with the pandemic, leaving them particularly vulnerable to poverty (Folley 2020; National Low Income Housing Coalition 2021). 15
Nevertheless, Colleen was able to leverage her skills and advantages to become part of the teleworking elite, despite lacking a work visa, experiencing profound precarity, and being in an unpredictable employment market. Social science research often seeks to include economically vulnerable participants, and it often offers remuneration for participation in studies. It may be one way for individuals who are generally unqualified for high-skilled white-collar positions, or unable to access telework-based careers, to make an income online.
In terms of push and pull factors, the push of income insecurity paralleled the pull of newly available and relatively untested opportunities at the beginning of the pandemic. With Colleen’s familiarity with higher education and research, she was well suited to developing a new telework niche participating in virtually mediated social science research, even as many researchers were just gaining their bearings in novel fields of inquiry. These circumstances do not excuse Colleen’s dishonesty—it is beyond the scope of this article to speculate on moral implications—but they do provide a framework for understanding Colleen’s motivations and apparent success.
Practical Suggestions
This article brings up the question of how to identify red flags while recruiting and interviewing individuals virtually. What can be done to prevent the participation of people who feign credentials? When and why does it matter? How can researchers identify such individuals? How should credentials be confirmed? How should we contextualize Colleen’s avoidance of more emotionally taxing or knowledge-specific research? The following sections address these questions with practical considerations around evaluating data and its sources, recruiting interviewees, paying interviewees, confirming credentials, and addressing an implausible interview if encountered.
Recruiting Interviewees
Colleen used a number of platforms to find opportunities to participate in research, including sites designed to facilitate contact between participants and researchers. For any online advertising platform recruiting research participants, there will be downsides and specific considerations concerning access, orientation, and usership. In some circumstances, researchers may benefit from using targeted advertisements on social media platforms, where researchers can set parameters on the basis of respondents’ gender, job title, behaviors, interests, age, and geographic location as determined by Internet protocol address. Using targeted advertisements may build in a layer of protection against those inclined to feign qualifications during the recruitment process. 16 However, targeted advertising may not be advisable or feasible for certain projects. 17
Additionally, virtual recruitment methods analogous to in-person methods may provide layers of protection against research subjects who might feign qualifications. Snowball sampling, for example, is sometimes criticized as generating data that lack diversity (Kirchherr and Charles 2018). However, snowball sampling is an excellent recruitment method once rapport is established with virtually recruited interviewees, insofar as those recruited by such a method will most likely be, at the very least, part of the researcher’s field. Recruiting interviewees through organizations that work directly with targeted populations may also be extrapolatable to virtually mediated contexts. It seems less likely Colleen would have been recruited as a snowball contact or through organizations that work directly with public housing residents in New York.
Still, Gerson and Damaske (2020) suggested that the best recruitment method is random. 18 I recruited Colleen through a nontargeted online advertisement on a well-used community-based platform that specializes in classified advertisements (Craigslist New York). Colleen responded to the advertisement by email, and I responded by email in turn, asking Colleen an initial set of questions that were specific to her qualifications and required specific types of answers confirming she met the criteria for the study. She was able to answer these convincingly with (she told me later) a bit of research that, given the lag, she could complete in her own time without the pressure of synchronous, verbal conversation. This initial email with relatively straightforward questions about interviewee suitability did screen out a large number of applicants (who either did not respond or responded in ways that disqualified them), but it did not screen out Colleen.
The recruitment process itself may not screen out those who willfully intend to deceive. The initial screening criteria I used were relatively lax, yet stricter screening criteria might have suppressed the number of respondents. It was not in the initial email contact and screening that I understood Colleen to be presenting implausibly but while speaking with her during a second, synchronous set of screening questions. During the synchronous portion of the interview, her answers were not as convincing. Even for a practiced research participant, questions that require detailed knowledge of the field (e.g., “Can you tell me the process you went through to get your current apartment?”) require an erudition that is difficult to invent synchronously.
Confirming Interviewee Identity
The availability of software that facilitates real-time interviews has made it easier to engage interview participants face to face synchronously (Howlett 2021). Nevertheless, the necessity of ascertaining a subject’s identity must be thought through (Lo Iacono, Symonds, and Brown 2016). Colleen reported that she did not participate in virtually mediated medical studies because they required her to provide identification documents; such requirements may be a natural deterrent for those who are feigning their identity or suitability for a study. However, Lo Iacono et al. (2016) pointed out that asking for an identification document (perhaps held up to the camera) is detrimental to establishing rapport. Moreover, such documents may easily be faked, especially in the context of virtual relationships. At any rate, knowing a person’s proper name does not necessarily guarantee their suitability for a study. Lo Iacono et al. sought identification in the context of making sure the interviewee was a particular person (i.e., a politician or an attorney). In seeking to interview members of a group bounded by geography and institutional affiliation (e.g., public housing residents in New York), and not a specific person, identification is a different matter.
Case in point: some interview subjects recruited online who met qualifications initially contacted me using false names, as they were cautious about revealing their true names. This is not an unreasonable precaution when contacting unknown individuals virtually, particularly if a study involves sensitive subject matter. Furthermore, Colleen gave me her “real name” when she applied to both studies, and again when she agreed to speak about the “real” Colleen. Beyond the problem that some of the most vulnerable research participants may not have government-issued identification, Colleen’s real name had no bearing on other qualifications, such as residency in public housing in a particular city, which are difficult to confirm in a virtual setting. For example, even lacking a scanned lease document may not have meant that Colleen did not live there; several of my interviewees lived off-lease in public housing apartments (it is estimated that hundreds of thousands do) and would not have been on or even had access to a lease document. In seeking to confirm that respondents meet study parameters, it is not necessarily advisable or necessary to ask for government identification or other documents that “prove” identification, especially when such documents may be inaccessible to more vulnerable groups (Gaskins and Iyer 2012) or increase feelings of vulnerability among respondents in virtual contexts.
Confirming a respondent’s qualifications (rather than identification) for the research study may be a more fruitful and rapport-building approach. This may be done in any number of ways. My initial email contact screening with Colleen asked her to confirm certain parameters of qualification so that there was no misunderstanding of the requirements at the outset. She confirmed she was a resident in a specific type of public housing and the particular development, that she was married, and that she was able to speak via videoconferencing (although she did not turn on her camera during the interview; perhaps I should have been more specific!). This gave me a number of qualification-related items to probe in our interview and to look into more deeply. For example, it would have been difficult for Colleen to convincingly and synchronously (without research) answer questions such as “What are some popular restaurants or stores near your home?” 19 Such questions would also provide an opportunity to verify or triangulate information easily and somewhat objectively.
Paying Interviewees
Colleen was preoccupied with payment. It stands to reason that if money is removed from the equation, it would remove the possibility of a participant engaging in research because of financial motivation. Some researchers believe that payment in field research introduces methodological vulnerabilities (Surmiak 2020). Moreover, research has shown that payment of research subjects is associated with deception (Fernandez Lynch et al. 2019). Ideally, qualitative researchers would cultivate relationships on the basis of trust, understanding, and time invested. However, offering a small incentive for something like an interview is a successful and frequently used recruitment tool. 20 In many if not most cases, offering an incentive or payment is also ethically justified (Gelinas, White, and Bierer 2020; Head 2009; Polacsek, Boardman, and McCann 2016).
Nevertheless, offering remuneration, including details of payment method, is a methodologically meaningful act and must be carefully thought through. In a conversation a few years ago among graduate students about to embark on their first foray into the field (prepandemic), I noticed a shift in thoughts about paying interviewees, from favoring gift cards that could more easily be thought of as a token of appreciation or honorarium to direct payments of cash. The reasoning behind this was that gift cards may not be as useful, and interviewees deserved to be remunerated for their time as a matter of dignity. Virtually mediated interviews may not be able to accommodate the cash approach, but flexibility in working with respondents to find a payment method that works may address the intention of providing a useful and highly fungible payment. For example, I offer a menu of payment methods to interviewees, including direct cash transfers through apps that do not require bank accounts and on which nicknames can be used, as well as gift cards or cashier’s checks sent virtually or to a physical address.
Furthermore, some methods of virtual payment, such as PayPal, 21 may confirm respondents’ given names or other details (which could compromise anonymity if sought or confirm a given name, although again this is not definitive). 22 Payment method, names, and nicknames introduce additional data points that may either confirm or negate other data, leading to a fuller picture of the interviewee.
Planning for Implausibility
How can researchers plan for implausible interviews methodologically? Small (2009) outlined a process by which interviewers may respond to their growing knowledge of a field by, for example, changing or deepening interview questions as they proceed through a study. Described as achieving “saturation” of data in the field, the process should include retaining “the power to determine how to refine and re-evaluate” through multiple interviews so as to move a study to show results “beyond a reasonable empirical doubt” (pp. 26–27).
Gerson and Damaske (2020) pointed out that whether it is the first interview or the hundredth, the next interview may lead to something new. However, I was able to recognize Colleen’s answers as implausible, and not merely “new,” primarily because I had interviewed a great many public housing residents before her. If I had met Colleen early on in my research, I may not have had the knowledge to judge whether the data she gave were “new” or implausible. Because of the type of question (e.g., how Colleen acquired her apartment) and my previous experience, I had attained a “saturation” around particular issues and understood enough about the possible range of experiences in attaining a public housing apartment in New York City. My previous experience made the comparison possible; however, this leaves questions about how researchers who are new to a field should approach data. Because the process of working toward saturation can provide new insights, early interviews may require special scrutiny for implausible data.
Notably, Colleen avoided research that involved heavily detailed or emotional knowledge about which the researcher had expertise, because these kinds of responses are more difficult to invent. It was in the questions requiring specific knowledge and detail that I was able to understand she was fabricating her experiences. It may have been more difficult to recognize that Colleen was not a public housing resident if I had asked only survey-type questions (requiring short answers or single-word confirmations, e.g., “You live in the Marcy Houses in Brooklyn?”) or broad, open-ended questions that did not require specific knowledge of the field. For example, when I asked Colleen, “How did you meet your partner?” She was able to give a convincing, detailed recitation of her relationship that, on its own, was completely plausible. 23 Rather, it was in evaluating questions that required detailed, comparable knowledge 24 that I was able to make the decision to exclude Colleen’s data. Asking how, for example, Colleen came to get her public housing apartment was a question that (1) could not be answered coherently and synchronously without preexisting detailed, field-specific knowledge and (2) could be compared and standardized against answers from other interviewees.
When Colleen told me, for example, that it had taken only a few weeks to obtain her apartment, her story became implausible because I was able to compare it with other accounts of how apartments were attained. I could compare Colleen’s answer with a broad range of nonuniform responses (from “My mother got the apartment in the ’80’s” to, most often, detailed responses involving paperwork and long, long waiting lists), and Colleen’s answer (that she filled out some paperwork online and got her apartment after a couple of weeks) was very much an outlier. 25 Each field and inquiry will require development of its own set of detailed, field-specific, comparable questions, but this may be one tool in planning for implausibility in the field.
Verifying Data
Qualitative researchers may be in a uniquely advantageous position to identify deceptive data because of their relatively intimate familiarity with their research subjects and data, compared with quantitative researchers or those with larger sample sizes. Researchers with larger data sets may not have the same familiarity with their data because of the nature of their methods, analysis, and inquiries (e.g., Fernandez Lynch et al. 2019). As such, qualitative methodologists are and should be concerned with the necessity of confirming or verifying data (e.g., Dean and Whyte 1958; Duneier 1999; Freese and Peterson 2017; Lubet 2018; Murphy et al. 2021). 26
Colleen had a fair amount of success participating in virtually mediated research: she participated in hundreds of projects and did not believe that researchers suspected her of feigning credentials or inventing stories (i.e., she was not confronted or questioned about her honesty). 27 Such success points out that data should not be taken for granted. How can researchers evaluate data gathered online, and to what extent should data be verified? What decisions should be made concerning the reliability of data? Duneier (2011) encouraged researchers to approach such decisions as if they may be asked to defend them in a methodological “court.” In defending methodological choices, researchers have a duty to ensure high standards for data and to plan for the possibility that outright fabrications are possible.
If an interviewee can successfully be outright deceptive about the most basic and salient aspects that frame the way data are gathered, this is a problem for researchers and for research. This is not to say that one should seek to understand data as either “right” or “wrong”; however, a rigorous approach to methods does typically mean making plans to ensure data quality, as well as imposing and regulating limitations or parameters in studies. 28 Moreover, Gerson and Damaske (2020:144) pointed out that even though interviewers seek “honest and full disclosure,” narrators are often unreliable. However, while descriptions and interpretations from interviewees necessarily vary, this does not mean participants “deliberately deceived” (p. 144). In addition to such considerations about the nature of interview data, researchers must grapple with interviewees such as Colleen, who invent their qualifications or willfully deceive in their interviews (Fujii 2010; Rosie 1993). Virtually mediated interviewing may introduce a higher risk for encountering research subjects who are intentionally dishonest, and it is important to minimize errors in data sets. To this end, distinguishing wholesale false or misleading data is requisite to analysis.
Colleen deliberately deceived, feigning her qualifications for the study and inventing stories. In cases in which interviewees are suspected of feigning qualifications or inventing stories, it may make sense to simply exclude data. 29 That Colleen was not a participant in the field of inquiry, or at least seemed to have no direct or verifiable knowledge of the field, made excluding the data she gave straightforward. However, in some circumstances, an implausible interview may have methodological or theoretical value, providing additional context (see Luker 2008). In in-person field research, respondents who feign qualifications and invent stories will still most likely be present in the field of inquiry. For example, in interviewing survivors of the Rwandan genocide, Fujii (2010) discussed one interview subject who feigned qualifications and invented stories. In that case, the interview subject was also a field participant, and Fujii found that although the interviewee’s story and qualifications were invented, there was a “story behind the story” (p. 235) that helped better shape understanding of the field. 30
Verification of data may take different forms. While interviewing Colleen, I considered her data alongside the entire body of data gathered from the field. This allowed me to understand that the data she gave were implausible, likely fabricated, and additional “confirmation” from Colleen was not necessary before making the decision to exclude her data. I marked my notes to exclude Colleen’s data. 31 I strongly suspected, but did not take additional steps to verify, that Colleen was feigning her qualifications. 32 Of course, this was resolved when she applied for another, unrelated research study in the weeks after speaking with her initially.
The approach to data verification in qualitative research is not uniform, and researchers have varying considerations. Small (2009) noted that one of the best ways to “confirm” or verify data in interviews is by assessing them comparatively with data previously gathered through other interviews. 33 Becker and Geer (1957) wrote that distortions are part of the data-gathering process and that being attuned to this is advantageous. Murphy et al. (2021) pointed out that the ethnographer Matthew Desmond hired a professional fact checker to look into his data gathered through ethnographic observation. 34 And Lane (2018) and Stuart (2020) used social media feeds to further contextualize and triangulate data gathered in person (Murphy et al. 2021).
Even if Colleen had been able to maintain plausibility in an initial interview, she may not have been able to do so in a second or third contact. Making multiple contacts during the recruitment and interview process may have the dual benefit of building rapport between the researcher and research participant and providing the researcher with additional data to compare and verify against other data (Lo Iacono et al. 2016; Weller 2017). Similarly, some researchers ask “repeated question sequences within a single interview” to help them verify data (Lamont and White 2008:71). Qualitative researchers also reapproach interviewees to confirm data (Lamont and White 2008; Murphy et al. 2021). If I was merely unsure about Colleen’s data, rather than fairly certain that her data were fabricated, reapproaching and seeking further data that could not be verified through other sources may have provided me with further insight.
Conclusion
The case of Colleen is important for a number of reasons, although perhaps especially as a provocation for how researchers orient themselves in the process of virtual methodological design. Colleen was able to capitalize on the availability of online research opportunities to pose as a qualified research subject when she did not actually meet the qualifications. Although Colleen may have been particularly attuned to telling researchers “what they wanted to hear” by virtue of her individual skill or academic background, her success highlights that researchers must consider not only methodological realities but also the nature of interview data as constructed and in context (Atkinson and Silverman 1997; Cicourel 1982; Silverman 2017). The case of Colleen also highlights the necessity of focusing on practical considerations of conducting research online. In this article, I outlined some methodological lessons learned in virtually mediated interviewing in the social sciences, which may hold implications for virtually mediated fieldwork generally. A case study, this article does not survey the interview methods of researchers in virtual fields or their experiences with confronting implausibility. Such a study could prove fascinating. Additionally, further study may expound on payment amounts, their role in recruitment of research participants, and ideologies in interview-based studies, especially in virtual contexts.
The virtually mediated interview is a new spin on a time-tested methodology, opening some new flanks but also highlighting some old ones. The capabilities associated with virtually mediated methodologies are extraordinary. For one, recruitment and interviewing online allow more efficient uses of time and resources, opening up opportunities where there previously were higher barriers to entry and allowing research to take place even in the midst of global crises requiring social distancing measures. New groups and connections may be accessed, and new forms of inequality may be assessed. It also opens up a variety of new considerations, including those related to data use and protection, access, and data quality. None of these considerations are new to interviewing or social science in general; however, although many questions in virtual contexts may be addressed analogously through in-person contexts and best practices, others require more careful scrutiny. Accounting for participant qualifications and sincerity is one such consideration that takes on new forms in virtually mediated contexts.
