Face-to-Face Compared With Online Collected Accounts of Health and Illness Experiences: A Scoping Review

Abstract

Advocates of online alternatives to face-to-face interviewing suggest online approaches save money and time, whereas others have raised concerns about the quality and content of the resulting data. These issues affect researchers designing and costing their studies and application reviewers and research funders. We conducted a scoping review of English language articles describing the range of online alternative approaches. Furthermore, we systematically identified studies directly comparing online alternatives with face-to-face approaches. Synthesis of these 11 articles (565 participants) suggests that online alternatives should not be viewed as a straightforward replacement for face-to-face, a particularly important finding given the rapid communication changes occurring in the COVID-19 pandemic. When applied with consideration of the evolving evidence on their strengths and weaknesses, online methods may increase the likelihood of obtaining the desired sample, but responses are shorter, less contextual information is obtained, and relational satisfaction and consensus development are lower.

Keywords

communication field methods focus groups health internet interview research design online technology use in research qualitative United States of America North America North Americans systematic reviews research strategies qualitative methods research design

Narrative interviews, semistructured interviews, and focus groups are widely used in qualitative research on experiences of health and illness. Over the past 20 years, the use of the internet as a supplement to, or even replacement for, in-person interaction has grown rapidly, including within the realm of research interviews, a term we use here to encompass both one-on-one and focus group methods. Many of these new approaches appear to hold promise as efficient and cost-effective alternatives, but evidence about their accuracy and effectiveness as a replacement for in-person interviews is still emerging. Understanding the applicability of these different methods will be beneficial. Researchers, designing studies based on theoretical paradigms or perspectives that shape their strategy of inquiry, need to select the most appropriate methods for their studies. Grant reviewers and potential funders, in turn, need to be able to assess the pros and cons of these approaches when making funding decisions. For example, online alternatives to the in-person interview have been noted to expand opportunities for inclusion (Ayling & Mewse, 2009), and analysis of existing blogs or discussion forums could provide useful insights into a phenomenon without the effort and expense of arranging face-to-face meetings (Lee, 2017; Owen et al., 2017). But concerns have also arisen, including the notion that web-based data reflect a specific online persona, which may differ from the more intimate and sometimes contradictory self that may emerge during an interview conducted face-to-face (Hallett & Barber, 2014). All these questions have taken on new importance as the COVID-19 pandemic has changed our ability to interact in person.

The earliest articles and books on this topic appeared in the second half of the 1990s, shortly after Netscape came online in 1995 as the first easy-to-use web browser incorporating images and text (Berners-Lee, 1999), and Hotmail emerged in 1996 as the first free public email service. Marketing researchers appear to have been the first to use the internet for research purposes, writing about the pros and cons of the internet as a medium through which to obtain information about people’s opinions and experiences (Curasi, 2001; Jones, 1999; Solomon, 1996). By the mid-2000s, social media emerged as part of the landscape of the web, with Facebook, YouTube, and Twitter becoming available in 2004, 2005, and 2006, respectively (Snelson, 2016). These interactive media, and the many others now available, have supported the rise of the internet as a key place for interaction and exchange on multiple aspects of daily life, including experiences of health and illness. A 2016 review describing trends in social media research identified 229 articles, concluding that the most common online research approaches were the use of interviews, focus groups, or surveys on social media platforms, with the second most common being content analysis of existing online data sources. These publications have mostly appeared recently (Snelson, 2016), and used a mixed methods approach, as per Creswell and Plano Clark’s (2011) mixed methods typology. Specifically, the studies used either a convergent design, in which quantitative and qualitative data are directly combined; an explanatory sequential design, in which qualitative data are collected to explain quantitative findings, or vice versa; or an exploratory sequential design, in which qualitative data are collected to explore quantitative findings, or vice versa.

The objective for this project was to evaluate the current range of online approaches used as alternatives to face-to-face interviews for qualitative health research, and to synthesize the evidence regarding the quality and content of these online alternatives compared with face-to-face formats. Because this is still a young field, we used a scoping review approach, as described by Arksey and O’Malley (2005) and refined by Levac et al. (2010). Scoping reviews are particularly helpful in emerging fields and provide a framework for describing the range and nature of the evidence, as well as summarizing and disseminating research findings (Levac et al., 2010). Here, we first identify the range of online approaches that have been used as alternatives to a face-to-face format for qualitative health research. Following this, we systematically analyze and synthesize the findings from articles that directly compared data generated when using a face-to-face approach compared to an online approach for research on experiences of health and illness.

Method

The Levac et al. (2010) framework guided the design and conduct of this scoping review. In Step 1, the development of the research question began with a specialist librarian-assisted broad literature search and work among coauthors to identify the range of online approaches currently being used, which might represent alternatives to face-to-face interviews and focus groups. Recent example articles using these approaches were then analyzed to identify the unique aspects of each approach, and to create a general taxonomy.

As part of this work, we used an iterative process to create boundaries around what we considered to qualify as potential alternatives to face-to-face interviews or focus groups. The included approaches are described in detail in the “Taxonomy” section of the results. The excluded approaches were photovoice, a specific approach in which images are the focus of the activity to gather qualitative data, as well as the broad category of online ethnography—the ethnographic study of online communities and groups that often crosses over into analyses of, and comparisons with, “offline” experience. This is a large and diverse area of research with different aims from those typical in face-to-face interviews and focus group studies. This field has been recently reviewed by Costello et al. (2017). We also excluded online approaches to ecologic momentary assessment, a method developed in the 1980s to capture periodic self-reported data (often quantitative) of people’s experiences. Before the widespread availability of the internet and software applications, studies using this approach were labor intensive. But research questions that require this technique can now be done through specifically designed applications or by using “big data” analytic techniques on existing data (Dodds et al., 2011; Dogan et al., 2017; Keedle et al., 2018; Larson & Csikszentmihalyi, 1983; Moskowitz & Young, 2006; Stone & Shiffman, 1994).

In Step 2, we narrowed and refined the research question:

Research Question 1: What is the direct evidence regarding the quality and content of face-to-face interviews and focus groups when compared with online alternatives for qualitative research questions on the experience of health and illness?

We identified relevant studies by working with research librarians to construct and perform a systematic search, using the eligibility criteria described below. In Steps 3 to 5, also described in detail below, studies were selected, charted, and critically reviewed to create a synthesis of the findings. We performed the optional Step 6 by presenting interim results of Steps 1 and 2 for feedback during the 2018 Database of Individual Patient Experiences (DIPEx) meeting in Groningen, the Netherlands. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Extension for Scoping Reviews, PRISMA-ScR (Tricco et al., 2018), was used for reporting the work.

Eligibility Criteria

To be included in the synthesis, studies had to directly compare face-to-face interviews or focus groups with an online alternative. The articles had to use the same research question for the two modes in comparison, it could not be a reflection on an approach without an explicit comparator. In addition, the topic had to be the experience of health and illness. Participants of all ages and subgroups were included; articles were limited to English language only.

Information Sources, Search Criteria, and Search Process

To identify candidate documents for the review, the following databases were searched up through January 7, 2019: PubMed, PsycInfo (EBSCO), CINAHL (EBSCO), Web of Science (Clarivate Analytics), and Embase (Ovid 1974 to current). The search strategies were designed by two experienced research librarians and using the results of Step 1 of the scoping review, in which we had identified the range of online alternatives to face-to-face interviews and focus groups. (The supplemental material contains detailed information regarding the search strategy). Few directly applicable Medical English Subject Heading (MESH) terms were available to guide the search, necessitating a broad approach. We used the following terms: Focus group OR Interview OR Qualitative, restricted to: Digital OR On-line OR internet OR Online OR Virtual OR Chat based OR Electronic OR Web-based OR Web chat OR Web focus group OR Web forum OR Web group OR Web portal OR Face to face OR In person OR On-site OR Personal OR Traditional OR Conventional, and further restricted to article titles that included the terms Compar* OR Versus OR vs. OR Contrast* OR Difference*. The final search results included 947 articles and these were imported into EndNote with duplicates removed by library staff. If an article was found to meet criteria, we used the “Similar Articles linking” tool on PubMed and Google Scholar as well as the “Cited by” function to expand our search from relevant articles. Additional studies and reports were identified by searching bibliographies of identified articles and through hand searches of the literature, including the Social Science Citation Index (Clarivate Analytics).

Selection of Sources of Evidence

All articles captured by the search strategy were independently evaluated for potential inclusion by two people. Each screened titles, abstracts, and when appropriate, full text of publications. Rayyan software for systematic reviews (Qatar Computing Research Institute, Doha, Qatar) was used to screen each article for inclusion or exclusion and to identify conflicts. In the instance of disagreement, cases were resolved by consensus and discussion. See Supplemental Figure S1 for details.

Data Collection Process

Through initial review of the identified articles, a data collection form was iteratively designed to capture the relevant variables for extraction. This standardized form included fields for all relevant study information (described below under “Data Items”), as well as necessary data points to compare the two modes of collecting qualitative data (details below). This form was then used by both to independently extract the information, with a discussion to resolve any initial differences of interpretation.

Data items

We extracted article information (year of publication, country of origin), study design (modes of data collection compared, study population size), study question, population studied, and principal findings.

Synthesis of Results

Studies were first organized by whether the comparison of an online alternative was with a face-to-face interview or focus group approach, and by year of publication, and then a coding structure was developed. The included articles reported a variety of comparisons between online and face-to-face approaches in their findings, and so as each article was reviewed, categories of study findings were iteratively identified through inductive coding. This coding structure was then applied to each reviewed article so that, where possible, reporting of similar (or contradictory) findings could be made across articles. Each major type of finding was tabulated and ordered in the tables from most common (i.e., described by five separate studies) to least common. When a study described one mode of data generation as having a particular characteristic, it was noted whether this was supported by data from the study or was an unsupported statement. Once coding was complete, the categories of findings were further analyzed and we grouped them into three domains: data and sampling, group dynamics, and logistics.

Results

Focusing on the most recent and salient publications, we identified the range of types of online data that we hypothesized might be alternatives to face-to-face interaction for qualitative research on experiences of health and illness. We created from this a general taxonomy of online alternatives to face-to-face formats for qualitative research questions on experiences of health and illness. After developing the taxonomy, we next identified and summarized the direct comparative evidence of online alternatives with face-to-face formats.

Taxonomy

Using an iterative approach, we divided the data sources into two broad categories—“extant online data” and “researcher obtained online data.” Extant data are the content of the internet that exist without prompting from a researcher presence. Researcher-directed online data refer to qualitative online data created as a result of researcher action or mediation, and these data are designed to answer a specific research question posed by the researcher—the researcher is present as a participant in the process.

Extant online data

Analysis of extant data parallels investigations of public or private records such as personal diaries to understand a culture or phenomena. Extant data are amenable to (at least) thematic analysis, content analysis (Snelson, 2016), and conversational analysis (Giles et al., 2015), and have also been used as a source of triangulation of data obtained through face-to-face approaches.

First of the extant data types is the “single voice” source: blogs, vlogs, and personal web sites. Because they represent a single voice, they can provide unique insight into coping mechanisms and psychological support. It has been noted that this data source has the potential drawback of self-censoring, or may employ a storytelling approach, which may be seen as a drawback (Kurtz et al., 2017; Owen et al., 2017). However, stories are the data form for many kinds of qualitative data analysis, and so the larger drawback may be better described as the lack of opportunity to clarify or explore ideas. We did not find any studies in our subsequent systematic search in which extant data from a single voice were directly compared with face-to-face approaches using the same research question.

A second type of extant data, which we call “co-created, topic focused,” have been tested and used as an alternative to face-to-face interaction. Support group forums and discussion boards with specific threads of discussion fall into this category. The data are the interactions among participants, with topics focused around a specific experience or idea. Strengths of these data sources include that the range of participants may be broad—caregivers and others with a stake in the experience may also participate. In addition, members may post over long periods of time, allowing for longitudinal data analysis (Jamison et al., 2018). However, recent articles also note that researchers should be aware that forums often have third-party moderators, who may influence the content of cocreated, topic-focused data (McKechnie et al., 2014). An additional concern notes the danger of only certain narrative threads being supported, those which are considered acceptable to the assembled group (Lee, 2017). However, these drawbacks can also be present in face-to-face formats.

We term the last type of extant data “co-created, non-topic focused.” Included in this category are data derived from Facebook (Reich, 2015) and other social media platforms; mobile messaging systems (Tagg et al., 2017); Wikipedia entries and their revision histories (Pentzold, 2017); Yik Yak—a closed messaging systems for students, no longer in existence (Byrne, 2017); Second Life—a website that allows people to create and have online interactions using created personas; and web sites from “the dark web” (Ferguson et al., 2017), that is, sites on the internet accessible only to persons with specialized software, allowing users and site operators to remain anonymous and untraceable. These studies focused primarily on how people related to one another in online settings. We did not find any studies in our subsequent systematic search in which data from these sources were directly compared with face-to-face approaches using the same research question.

Researcher-directed online data

This is created when web-based approaches are used as the platform to generate data that in a previous era would have been conducted using face-to-face approaches. Many creative designs have been devised, as described below, and these were the most common data sources we identified in our subsequent systematic search for studies directly comparing face-to-face with online approaches addressing the same research question.

We have divided researcher-directed online data into talk and text based. Talk-based approaches include video interviews using technologies such as Skype, as well as focus groups using software to accommodate multiple participants who can all see each other on camera. Video approaches allow people across a broad geography to participate but require robust internet connections. Text-based approaches include email interviews as well as synchronous and asynchronous online focus groups (Gaiser, 2017). In email interviews, researchers pose questions by email, responding with follow-up questions and probes based on participant responses. In synchronous text-based online focus groups, participants all log into a specific online platform at the same time. They participate by text-based “chat” for a specific period, typically an hour or so, answering questions posed by the moderator and responding to one another. In asynchronous text-based online focus groups, participants log into a specific online platform at a time of their choosing to answer the questions posed by the moderator, while also responding to others who have already commented. Asynchronous text-based online focus groups may occur over a week or longer, with questions posed every few hours, days, or even weeks. A variation on this approach is the creation of a researcher-guided forum, which is a combination of an open forum for invited participants to discuss questions of interest to them (e.g., the participants in a larger research project), combined with moderated questions by the researcher. Both observational data of spontaneous concerns and researcher-driven responses are captured using this method (Im & Chee, 2012)

Direct Comparative Evidence

A search was conducted to identify studies that evaluated the quality and content of face-to-face interviews and focus groups as compared with the online alternatives for qualitative research questions on experiences of health and illness. The search resulted in 947 references. After duplicates were removed, the titles and abstracts of the remaining 936 were reviewed. Of these, 916 were excluded, 16 were included, and two conflicted, leading to individual review. Full text review and hand searching ultimately resulted in inclusion of 11 articles (Supplemental Figure S1). We excluded book reviews, abstracts that did not link to completed works, “how-to” articles about conducting interviews and focus groups online without a critical evaluation component, thought pieces on a particular approach (such as the use of public forums as a source of data) without a methods section, articles on the process of doing ethnography in an “online space,” articles about what people do online that were not specifically about health-related issues, machine learning of online data, articles not about health/illness, articles that advanced theories of qualitative research (e.g., using grounded theory, discourse analysis, or conversation analysis to analyze online data), articles that were solely reflections on the ethics of online qualitative research methods, and articles that compared closed-item survey results when administered face-to-face versus using a “computer-assisted self-interview” approach (closed-item survey completion using a computer).

Results of sources of evidence

The 11 studies meeting inclusion criteria were published between 2001 and 2018 (Supplemental Table 1). The articles covered a broad range of topics, from colon cancer to childhood disability and chronic illness, to intimate partner violence. One study was conducted on children, whereas the remainder included only adults. All the studies used the data from both approaches to answer the research question of interest. The total sample size of the 11 studies altogether was “at least” 565 participants, as one study provided no sample size information, and another used public forum data as a comparator (reliable participant counts could not be captured). Thus, conservatively, 243 (43%) of at least 565 participants’ data were from an online source, with the remainder from face-to-face approaches.

Seven of the 11 studies we included compared face-to-face focus groups with online focus groups, in the form of synchronous online text-based focus groups, asynchronous online text-based focus groups, or online video focus groups. These studies compared data from the online alternative with as few as one face-to-face focus group, up to as many as four.

The other four studies compared face-to-face interviews with email interviews, analysis of extant cocreated, topic-focused data from public forums, or online video interviews. These studies compared interviews (sample size range seven to 140 participants) with one or two different online alternatives (sample size range 23–49+ participants; reliable participant counts could not be captured from the public forums).

Synthesis of Results

The 11 included articles tested theories about online and face-to-face approaches that we categorized into three domains: sampling, data quantity/quality, and topical coverage (Jenner, 2019; McCoyd, 2006; Seale et al., 2010; Synnot et al., 2014; Woodyatt et al., 2016); interpersonal dynamics of online compared with face-to-face approaches (Graffigna, 2006; Schneider et al., 2002); and feasibility and logistics (Kramish Campbell et al., 2001; Nicholas et al., 2010; Rupert et al., 2017). Although each article focused on one research question in their primary domain of interest, most also reported findings on the other domains, allowing synthesis of data across the articles.

Demographics

Data such as age, sex, educational levels, and racial/ethnic makeup of the participants were inconsistently reported in the included articles, limiting interpretation of these characteristics. Some of the earlier articles documented that online participants were younger, or more affluent. For example, Kramish Campbell et al. (2001) reported that the online population was wealthier, and more educated. By 2017, however, Rupert et al. (2017) reported in their study that the online sample was more diverse in terms of race/ethnicity, education, and household income than the face-to-face sample. Jamison et al. (2018) reported that their online cohort was significantly younger than their face-to-face cohort. The adoption of online technology across time and within patient groups is a likely influence on the participant demographics. These findings suggest that the use of online methods can have a varied effect on participant demographics and inclusion depending on setting and topic.

Broad findings reported across the included articles are organized by domain: sampling and data, logistics, and interpersonal dynamics (Supplemental Table 2).

Sampling and data

Seven of the 11 articles reported that sensitive topics seemed to be more easily discussed in the online alternatives. Overall, responses in the online alternatives were generally shorter and more on topic, with one article describing responses as more “concrete” (Synnot et al., 2014). Some articles noted that there was less contextual information provided in the answers, with three articles (on focus group alternatives) noting that there was less chance for probe/follow-up questions than in person. The email interview (McCoyd, 2006) was a notable exception—providing better opportunities for probe/follow-up. However, the study noted their self-selected population for the online group, which was accustomed to text-based support group interaction. They further noted that the email interview allowed for less capture of emotional reactions than a face-to-face approach. Last, one article noted that online contributions were more often written in the present tense, compared with face-to-face approaches that were in the past tense. This suggests that data from an online forum was more likely to describe a recent event, perhaps with less interpretation and self-reflection than face-to-face approaches (Seale et al., 2010).

Interpersonal dynamics

Face-to-face formats were reported to result in greater relational satisfaction for participants (Nicholas et al., 2010). This finding was also reported qualitatively by the participants and captured in closed-item survey results (Synnot et al., 2014) and discussed without the inclusion of supporting data in other articles (Jamison et al., 2018; Kramish Campbell et al., 2001; Schneider et al., 2002). Schneider noted that fewer alliances in opinion were formed in online than face-to-face focus group approaches, suggesting differences in relational interaction between the approaches. However, Graffigna in 2006, reported that an asynchronous text-based focus group followed by a synchronous text-based focus group provided an improved sense of group belonging, an articulate discourse, and fewer conflicts/miscommunications. It is unclear whether simply having two interactions or having the two different types of interaction was the reason for improvement.

Logistics

Online alternatives were reported to be more convenient, to allow greater geographical diversity of participants, and to support participation of people who could not easily travel, such as those with illness or physical disability. Depending on the platform and approach used, the cost and time required to recruit and complete online interviews or focus groups were not necessarily less than using a face-to-face approach. One article reported that online study participants had higher cancelation and no-show rates than face-to-face formats (Rupert et al., 2017), whereas another article reported the opposite (Jenner, 2019). One article reported analysis as more challenging for online data, which precluded “placing a face to a name” (Synnot et al., 2014).

Discussion

This scoping review creates a taxonomy of online alternatives to face-to-face formats for qualitative research questions on experiences of health and illness, and is the first to systematically analyze studies that have made a direct comparison (i.e., using the same research question) of online alternatives with face-to-face approaches for these types of research questions. This review demonstrates the areas of methodological consideration that might influence decisions about how and when to use each approach singly or in combination, and provides particular value given evolving changes in communication being brought about as a result of the COVID-19 pandemic. We identified three domains of particular concern from qualitative researchers working in health and illness: sampling, interpersonal dynamics, and feasibility and logistics.

Eleven studies with a total sample size of 565+ participants (43% online) conducted direct comparisons of the data obtained using online alternatives with face-to-face interviews or focus groups for qualitative research on experiences of health and illness. The approaches included both talk-based platforms such as Skype, Web-ex, or Adobe Connect, and text-based media, such as online chats or forums. In the case of text-based formats, both synchronous and asynchronous designs have been used, with some data collection periods extending over days to weeks. Online participant responses were typically shorter and more to the point, without contextual detail. Furthermore, online approaches were reported to result in lower levels of relationship building and relational satisfaction. Although a broader range of participants was reached using online approaches, the cost and efficiency were not always considered better. Coverage of sensitive topics has in the past been noted to be more extensive in online than in face-to-face approaches (Seale et al., 2006). Notably, this may be evolving: In the most recently published article included in this review, participant’s perceived level of privacy was hypothesized to be more important than whether the approach was face-to-face or online (Jenner, 2019).

Focusing on the last 5 years, which more closely reflects current internet function and usage, several articles have been written describing the unique considerations that must be incorporated into decisions about using online approaches compared with face-to-face. For example, researchers now have to choose the best technology from an array of choices for a particular research question and population, different skills may be needed for online interactions, as well as sufficient knowledge of the population under study to anticipate their online usage patterns and norms (Lo Iacono et al., 2016; Tuttas, 2015; Vicsek, 2016; Williams et al., 2012). Together with the findings of our systematic review, this suggests to us that online approaches should not be viewed as a direct replacement for face-to-face approaches. Different researcher skill sets are needed and may affect results. Furthermore, the contradictory findings from our systematic analysis regarding response length and quality, range of participant ages, education, and socioeconomic status suggests that variance might also be due to factors such as technology development over time, the research question, the population under study, and secular events. We note especially the COVID-19 pandemic as a secular event that has caused a great increase in the use of online approaches to communication, and based on these observations is likely to affect study results in the future.

There are additional issues for researchers to consider when weighing the merit of online approaches. Wilkerson et al. (2014) note the significance of administrative considerations, including skills of the overall research team and budget. Online approaches require reading and technological literacy of participants, as well as access to technology; however, they may be beneficial in reducing threats to sampling such as barriers of geography, time, and research costs that would otherwise exclude some participants. In the case of asynchronous online text-based groups, for example, participants who work irregular hours or who tire easily and need to participate in short bursts can be easily accommodated. A final important consideration is the need for personal connection. Although online formats can increase the perception of privacy, they limit relationship building between participants and between participant and researcher, which may be an undesirable side effect that ultimately makes an online format unappealing (Reisner et al., 2018).

Our search found only one study (Seale et al., 2010) that directly compared extant, cocreated, topic-focused data (a support group forum) with face-to-face interaction. The use of extant material means that the researcher is, by definition, not present to guide or influence the topics discussed. This can be both a strength and a weakness. While showing how topics are shaped and meanings are created in online discussions, it may not be able to answer directly the research question of interest. However, identifying the range and content of topics discussed in online settings may help researchers to frame subsequent research. A caution in using extant data from the internet was described in recent work by Germain et al. (2017). They found that it was possible to reidentify people by performing Google searches of specific quotes. The risk of reidentification in their study required special consideration because even though the speech they were using for their research was public—posted to an open forum—to use it for research might be seen as eavesdropping on a private conversation in a public place, which is ethically problematic. In addition, online settings are increasingly recognized as less anonymous or private, as internet behaviors are tracked (Manjoo, 2019) and personal data are used and sold by hosting companies for other purposes, as covered in extensive news reporting.

Less discussed in social science literature but commonly described by qualitative researchers is the risk of losing data “richness” when people are not physically copresent. Data “richness” is hard to define but Charmaz holds that rich data “reveal participants’ thoughts, feelings, intentions, and actions as well as context and structure . . . [and] afford views of human experience that etiquette, social conventions, and inaccessibility hide or minimize in ordinary discourse” (Charmaz, 2003, p. 89). Researchers also point to the personal connections that can be made when interviewing in person, which may result in better/richer data through more candid or elaborate answers. They also describe the importance of being able to provide comfort when covering a sensitive or difficult topic, as well as the more nuanced understanding and subsequent questioning that can come from the ability to fully observe nonverbal cues, further enriching data quality (Ogden & Cornwell, 2010). These concerns are valid. Depending on how the video is done and how the feed is smoothed (often to optimize bandwidth usage), the loss of eye contact and altered angle at which visual cues are received can adversely affect interpretations of interactions and relationship building (Murphy, 2020; Nguyen & John, 2007). Observation of the interview setting is also described by researchers as valuable information if it was chosen by the participant, which may be relevant to the research question or the qualities of the participant. However, all of this presupposes that relationship building, nuanced communication, and direct observation are necessary precursors to rich data. Although for particular settings and topics, research relationships may be compromised by online data collection, some people with mental illness or learning and communication disabilities may prefer to, or may only be able to, interact comfortably when they are not face-to-face (Ryan, 2013). In these cases, online approaches may allow one to collect data that previously were not accessible.

Looking to other fields, there is some evidence that a lack of copresence does not always have to compromise interaction quality. For example, the field of psychiatry has incorporated telemedicine, performing trials to compare the effect of in-person and remote visits on relationship building and creating a therapeutic alliance. Findings show that relationship development for therapeutic work is possible but success is neither certain nor easily predicted (Backhaus et al., 2012; Godleski et al., 2012; Greene et al., 2010). It is not clear how far these observations from clinical practice may relate to research endeavors, but it seems likely that a population habituated to online interactions may be comfortable being interviewed in this manner. Lo Iacono, who gathered data on the participant experience of using Skype for research questions related to health and illness, noted that participants were aware of the differences but wanted to be part of the project and so were willing to accept the limitation, saying, for example, that “you are not getting my energy, you see, there are details you are not catching . . . but . . . you wouldn’t be able to interview me right now if it wasn’t for Skype . . . ” (Lo Iacono et al., 2016, p. 106). A comparison from the marketing field, which has also tested the use of alternatives to face-to-face interaction (for nonsensitive topics unrelated to health and illness) found that self-disclosure appeared similar, and noted that a larger number of ideas were generated in the online format (Reid & Reid, 2005).

In-person interaction is not the only quality that affects data richness. The particular qualities inherent to the topic, interviewer, and interviewee are also relevant (Ogden & Cornwell, 2010). In an empiric study to define data richness, Ogden and Cornwell (2010) found that questions regarding a topic that was personal, less specific, and positive tended to yield richer data than those that were not. Professionals who were interviewed about their own field tended to give “richer” answers than lay people giving answers about personal topics. Questions that were open ended, contained more than one question, and that were asked later in the interview were all associated with richer data (Ogden & Cornwell, 2010). We note, however, that an in-person interaction is no guarantee. If the relationship is not effectively established during the interview, or if important nonverbal communications are missed, the data may be less rich than if a different approach had been used. Also, for people who prefer to communicate without face-to-face interaction, assumptions about the need for in-person communication will not hold.

Limitations

Our analysis only includes the direct, comparative findings reported by the authors. There may be other factors that the authors chose not to report, and response bias should be considered. All the data are observational, rather than randomized. This is inevitable because neither sequential comparison (with each participant taking part in two different approaches) nor a randomized design (in which each participant would be allocated to a different approach) could yield robust data for the desired comparison. Participants in the 11 studies were either invited to participate in the offered approach, or in some studies, were invited to select their method of choice. This may introduce an aspect of selection bias, as those opting for online approaches may already have greater comfort with this mode of communication. None of the studies reported characteristics of those who chose the different approaches.

It is worthwhile to recall that at one time, telephone interviews were “new” and, therefore, subject to similar concerns and comparative research to test their utility (Carr & Worth, 2001; Sturges & Hanrahan, 2004). It was only in the 1990s that they became widely accepted, and although still not a preferred method for many qualitative studies, they have been recognized for their efficiency, utility, and affordances of privacy and security for both parties. For telephone interactions, we have now accumulated a generation of experience and the recognition that the approach has a place in some qualitative research. Loss of voice fidelity and delays in response time are typically low for telephone, an advantage over video. However, it is possible that as video use increases with the COVID-19 pandemic, these limitations may lessen as the issues are recognized and demands for improvements of video fidelity to in-person interaction increase.

Conclusion

We identified three domains of methodological concern about online compared with face-to-face approaches on questions of health and illness: sampling, interpersonal dynamics, and feasibility and logistics. Key findings are that in online methods, participant responses were typically shorter and often lacked contextual information. In addition, online approaches were reported to result in lower levels of relational satisfaction and consensus development. A broader range of participants was reported with online approaches, but the cost and efficiency were not necessarily better. Thus, online approaches to the collection of qualitative data on health and illness can expand the ability to reach certain populations, but with less relationship and (in focus groups) consensus-building potential. Online approaches are not necessarily less expensive, faster, or less prone to cancelation than face-to-face approaches. They require knowledge of the study population’s use of the medium, as well as technical skills by both parties. The growing comparative literature on online approaches, as summarized here, as well as the increasing access to technology, provide opportunity to make purposeful and evidence-based decisions about when online and face-to-face approaches will each be preferred for answering specific questions or overcoming particular methodological challenges. Continued studies comparing online and face-to-face formats should test the consequences for inclusion, recruitment, logistics, topics/content and indicators of data richness, trust, and engagement, particularly in light of recent secular trends in online usage for interactions of all kinds during the COVID-19 pandemic. Meanwhile, although online formats have opened up new spaces and opportunities that did not exist before, these spaces will themselves be woven with different levels of accessibility, engagement, and opportunity that should contribute to the tools and techniques available to qualitative researchers. This discussion around online and face-to-face methods will likely continue within the broader framework of critical discussion regarding qualitative research methods and the role of the interview in data generation.

Supplemental Material

sj-docx-1-qhr-10.1177_1049732320935835 – Supplemental material for Face-to-Face Compared With Online Collected Accounts of Health and Illness Experiences: A Scoping Review

Supplemental material, sj-docx-1-qhr-10.1177_1049732320935835 for Face-to-Face Compared With Online Collected Accounts of Health and Illness Experiences: A Scoping Review by Louise Davies, Karissa L. LeClair, Pamela Bagley, Heather Blunt, Lisa Hinton, Sara Ryan and Sue Ziebland in Qualitative Health Research

Supplemental Material

sj-pdf-2-qhr-10.1177_1049732320935835 – Supplemental material for Face-to-Face Compared With Online Collected Accounts of Health and Illness Experiences: A Scoping Review

Supplemental material, sj-pdf-2-qhr-10.1177_1049732320935835 for Face-to-Face Compared With Online Collected Accounts of Health and Illness Experiences: A Scoping Review by Louise Davies, Karissa L. LeClair, Pamela Bagley, Heather Blunt, Lisa Hinton, Sara Ryan and Sue Ziebland in Qualitative Health Research

Supplemental Material

sj-pdf-3-qhr-10.1177_1049732320935835 – Supplemental material for Face-to-Face Compared With Online Collected Accounts of Health and Illness Experiences: A Scoping Review

Supplemental material, sj-pdf-3-qhr-10.1177_1049732320935835 for Face-to-Face Compared With Online Collected Accounts of Health and Illness Experiences: A Scoping Review by Louise Davies, Karissa L. LeClair, Pamela Bagley, Heather Blunt, Lisa Hinton, Sara Ryan and Sue Ziebland in Qualitative Health Research

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Louise Davies

Supplemental Material

Supplemental Material for this article is available online at . Please enter the article’s DOI, located at the top right hand corner of this article in the search bar, and click on the file folder icon to view.

Author Biographies

Louise Davies is an associate professor of surgery at the Dartmouth Institute for Health Policy and Clinical Practice, Hanover, New Hampshire, USA.

Karissa L. LeClair, BS, is a medical student at the Geisel School of Medicine at Dartmouth in Hanover, New Hampshire, USA.

Pamela Bagley is the coordinator of biomedical research support at the Biomedical Libraries at Dartmouth in Hanover, New Hampshire, USA.

Heather Blunt is a librarian at the Biomedical Libraries serving Geisel School of Medicine at Dartmouth in Hanover, New Hampshire, USA.

Lisa Hinton is a senior associate at THIS Institute, University of Cambridge. Until December 2019, she led applied research for the Health Experiences Research Group, NDPCHS, University of Oxford.

Sara Ryan is a senior research lead in Nuffield Department of Primary Care Health Sciences (NDPCHS), University of Oxford.

Sue Ziebland is a professor of medical sociology in NDPCHS at the University of Oxford and an National Institute for Health Research (NIHR) senior investigator.

References

Arksey

O’Malley

(2005). Scoping Studies: Towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19–32. https://doi.org/10.1080/1364557032000119616

Ayling

Mewse

A. J.

(2009). Evaluating internet interviews with gay men. Qualitative Health Research, 19(4), 566–576. https://doi.org/10.1177/1049732309332121

Backhaus

Agha

Maglione

M. L.

Repp

Ross

Zuest

Rice-Thorp

N. M.

Lohr

Thorp

S. R.

(2012). Videoconferencing psychotherapy: A systematic review. Psychology Services, 9(2), 111–131. https://doi.org/10.1037/a0027924

Berners-Lee

(1999). Weaving the web: The original design and ultimate destiny of the world wide web, by its inventor. Harper.

Byrne

(2017). Anonymous social media and qualitative inquiry: Methodological considerations and implications for using Yik Yak as a qualitative data source. Qualitative Inquiry, 23(10), 799–807. https://doi.org/10.1177/1077800417731081

Carr

E. C. J.

Worth

(2001). The use of the telephone interview for research. NTResearch, 6(1), 511–524.

Charmaz

(2003). Grounded theory. In Smith

(Ed.), Qualitative psychology: A practical guide to research method, (ch.4, pp. 89–110). Sage.

Costello

McDermott

M. L.

Wallace

(2017). Netnography: Range of practices, misperceptions, and missed opportunities. International Journal of Qualitative Methods, 16, 1–12. https://doi.org/10.1177/1609406917700647

Creswell

V. L.

Plano Clark

V. L.

(2011). Designing and conducting mixed methods research (2nd ed.). Sage.

10.

Curasi

C. F.

(2001). A critical exploration of face-to-face interviewing vs. computer-mediated interviewing. International Journal of Market Research, 43(4), 361–375.

11.

Dodds

P. S.

Harris

K. D.

Kloumann

I. M.

Bliss

C. A.

Danforth

C. M.

(2011). Temporal patterns of happiness and information in a global social network: Hedonometrics and Twitter. PLoS ONE, 6(12), Article e26752. https://doi.org/10.1371/journal.pone.0026752

12.

Dogan

Sander

Wagner

Hegerl

Kohls

(2017). Smartphone-based monitoring of objective and subjective data in affective disorders: Where are we and where are we going? Systematic review. Journal of Medical Internet Research, 19(7), Article e262. https://doi.org/10.2196/jmir.7006

13.

Ferguson

M. A.

Kitterick

P. T.

Chong

L. Y.

Edmondson-Jones

Barker

Hoare

D. J.

(2017). Hearing aids for mild to moderate hearing loss in adults. Cochrane Database of Systematic Reviews, 9, CD012023. https://doi.org/10.1002/14651858.CD012023.pub2

14.

Gaiser

(2017). Online focus groups. In Fielding

Lee

R. M.

Blank

(Eds.), The SAGE handbook of online research methods (2nd ed., pp. 290–306). Sage.

15.

Germain

Harris

Mackay

Maxwell

(2017). Why should we use online research methods? Four doctoral health student perspectives. Qualitative Health Research, 28(10), 1650–1657.

16.

Giles

Stommel

Paulus

Lester

Reed

(2015). Microanalysis of online data: The methodological development of “digital CA.” Discourse Context & Media, 7, 45–51. https://doi.org/10.1016/j.dcm.2014.12.002

17.

Godleski

Darkins

Peters

(2012). Outcomes of 98,609 U.S. Department of Veterans Affairs patients enrolled in telemental health services, 2006-2010. Psychiatry Services, 63(4), 383–385. https://doi.org/10.1176/appi.ps.201100206

18.

Graffigna

G. B. A.

(2006). The influence of setting on findings produced in qualitative health research: A comparison between face-to-face and online discussion groups about HIV/AIDS. International Journal of Qualitative Methods, 5(3), 55–76. https://doi.org/10.1177/160940690600500307

19.

Greene

C. J.

Morland

L. A.

Macdonald

Frueh

B. C.

Grubbs

K. M.

Rosen

C. S.

(2010). How does tele-mental health affect group therapy process? Secondary analysis of a noninferiority trial. Journal of Consultative Clinical Psychology, 78(5), 746–750. https://doi.org/10.1037/a0020158

20.

Hallett

R. E.

Barber

(2014). Ethnographic research in a cyber era. Journal of Contemporary Ethnography, 43(3), 306–330. https://doi.org/10.1177/0891241613497749.

21.

E. O.

Chee

(2012). Practical guidelines for qualitative research using online forums. Computers, Informatics, Nursing, 30(11), 604–611. https://doi.org/10.1097/NXN.0b013e318266cade

22.

Jamison

Sutton

Mant

De Simoni

(2018). Online stroke forum as source of data for qualitative research: Insights from a comparison with patients’ interviews. BMJ Open, 8(3), Article e020133. https://doi.org/10.1136/bmjopen-2017-020133

23.

Jenner

B. M. M. K.

(2019). Intimacy, rapport, and exceptional disclosure: A comparison of in-person and mediated interview contexts. International Journal of Social Research Methodology, 22(2), 165–177. https://doi.org/10.1080/13645579.2018.1512694

24.

Jones

(Ed.). (1999). Doing internet research—Critical issues and methods for examining the net. Sage.

25.

Keedle

Schmied

Burns

Dahlen

(2018). The design, development, and evaluation of a qualitative data collection application for pregnant women. Journal of Nursing Scholarship, 501(1), 47–55. https://doi.org/10.1111/jnu.12344

26.

Kramish Campbell

Meier

Carr

Enga

James

A. S.

Reedy

Zheng

(2001). Health behavior changes after colon cancer: A comparison of findings from face-to-face and on-line focus groups. Family and Community Health, 24(3), 88–103. https://doi.org/10.1097/00003727-200110000-00010

27.

Kurtz

L. C.

Trainer

Beresford

Wutich

Brewis

(2017). Blogs as elusive ethnographic texts: Methodological and ethical challenges in qualitative online research. International Journal of Qualitative Methods, 16(1). https://doi.org/10.1177/1609406917705796

28.

Larson

Csikszentmihalyi

(1983). The experience sampling method. New Directions for Methodology of Social and Behavioral Science, 15, 41–56.

29.

Lee

(2017). Don’t give up! A cyber-ethnography and discourse analysis of an online infertility patient forum. Culture, Medicine, Psychiatry, 41(3), 341–367. https://doi.org/10.1007/s11013-016-9515-6

30.

Levac

Colquhoun

O’Brien

K. K.

(2010). Scoping studies: Advancing the methodology. Implementation Science, 5, Article 69. https://doi.org/10.1186/1748-5908-5-69

31.

Lo Iacono

Symonds

Brown

D. H. K.

(2016). Skype as a tool for qualitative research interviews. Sociological Research Online, 21(2), 103–117. https://doi.org/10.5153/sro.3952

32.

Manjoo

(2019, August 23). I visited 47 sites. Hundreds of trackers followed me. The New York Times. https://www.nytimes.com/interactive/2019/08/23/opinion/data-internet-privacy-tracking.html

33.

McCoyd

J. L. M.

(2006). Conducting intensive interviews using email: A serendipitous comparative opportunity. Qualitative Social Work, 5(3), 389–406. https://doi.org/10.1177/1473325006067367

34.

McKechnie

Barker

Stott

(2014). The effectiveness of an Internet support forum for carers of people with dementia: A pre-post cohort study. Journal of Medical Internet Research, 16(2), Article e68. https://doi.org/10.2196/jmir.3166

35.

Moskowitz

D. S.

Young

S. N.

(2006). Ecological momentary assessment: What it is and why it is a method of the future in clinical psychopharmacology. Journal of Psychiatry & Neuroscience, 31(1), 13–20.

36.

Murphy

(2020, May 4). Why zoom is terrible. The New York Times, 23. https://www.nytimes.com/2020/04/29/sunday-review/zoom-video-conference.html

37.

Nguyen

John

(2007). Multiview: Improving trust in group video conferencing through spatial faithfulness. CHI ‘07 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, San Jose, CA, United States. https://dl.acm.org/doi/proceedings/10.1145/1240624.

38.

Nicholas

D. B.

Lach

King

Scott

Boydell

Sawatzky

B. J.

Reisman

Schippel

Young

N. L.

(2010). Contrasting Internet and face-to-face focus groups for children with chronic health conditions: Outcomes and participant experiences. International Journal of Qualitative Methods, 9(1), 105–121. https://doi.org/10.1177/160940691000900102

39.

Ogden

Cornwell

(2010). The role of topic, interviewee and question in predicting rich interview data in the field of health research. Sociology of Health & Illness, 32(7), 1059–1071. https://doi.org/10.1111/j.1467-9566.2010.01272.x

40.

Owen

M. I.

Braun

L. T.

Hamilton

R. J.

Grady

K. L.

Gary

R. A.

Quest

T. E.

(2017). Weblogs: A complex data source for qualitative research. Journal of Cardiac Failure, 23(11), 826–827. https://doi.org/10.1016/j.cardfail.2017.08.453

41.

Pentzold

(2017). “What are these researchers doing in my Wikipedia?” Ethical premises and practical judgment in internet-based ethnography. Ethics and Information Technology, 19(2), 143–155. https://doi.org/10.1007/s10676-017-9423-7

42.

Reich

J. A.

(2015). Old methods and new technologies: Social media and shifts in power in qualitative research. Ethnography, 16(4), 394–415. https://doi.org/10.1177/1466138114552949

43.

Reid

F. J. M.

(2005). Online focus groups an in-depth comparison of computer mediated and conventional focus group discussions. International Journal of Market Research, 47(2), 131–162. https://doi.org/10.1177/147078530504700204

44.

Reisner

S. L.

Randazzo

R. K.

White Hughto

J. M.

Peitzmeier

DuBois

L. Z.

Pardee

D. J.

Marrow

McLean

Potter

(2018). Sensitive health topics with underserved patient populations: Methodological considerations for online focus group discussions. Qualitative Health Research, 28(10), 1658–1673. https://doi.org/10.1177/1049732317705355

45.

Rupert

D. J.

Poehlman

J. A.

Hayes

J. J.

Ray

S. E.

Moultrie

R. R.

(2017). Virtual versus in-person focus groups: Comparison of costs, recruitment, and participant logistics. Journal of Medical Internet Research, 19(3), Article e80. https://doi.org/10.2196/jmir.6980

46.

Ryan

(2013). Engagement and inclusivity in researching patients’ experiences. In Ziebland

(Ed.), Understanding and using health experiences: Improving patient care (1st ed., pp. 135–147). Oxford University Press.

47.

Schneider

S. J.

Kerwin

Frechtling

Vivari

B. A.

(2002). Characteristics of the discussion in online and face-to-face focus groups. Social Science Computer Review, 20(1), 31–42. https://doi.org/10.1177/089443930202000104

48.

Seale

Charteris-Black

MacFarlane

McPherson

(2010). Interviews and internet forums: A comparison of two sources of qualitative data. Qualitative Health Research, 20(5), 595–606. https://doi.org/10.1177/1049732309354094

49.

Seale

Ziebland

Charteris-Black

(2006). Gender, cancer experience and internet use: A comparative keyword analysis of interviews and online cancer support groups. Social Science in Medicine, 62(10), 2577–2590. https://doi.org/10.1016/j.socscimed.2005.11.016

50.

Snelson

C. L.

(2016). Qualitative and mixed methods social media research: A review of the literature. International Journal of Qualitative Methods, 15(1). https://doi.org/10.1177/1609406915624574

51.

Solomon

(1996). Targeting trendsetters: Cyberstudies capture the perceptions and insights of early adopters. Marketing Research, 8(2), 9.

52.

Stone

Shiffman

(1994). Ecological momentary assessment in behavioral medicine. Annals of Behavioral Medicine, 16(3), 199–202.

53.

Sturges

Hanrahan

K. J.

(2004). Comparing telephone and face-to-face qualitative interviewing: A research note. Qualitative Research, 4(1), 107–118. https://doi.org/10.1177/1468794104041110

54.

Synnot

Hill

Summers

Taylor

(2014). Comparing face-to-face and online qualitative research with people with multiple sclerosis. Qualitative Health Research, 24(3), 431–438. https://doi.org/10.1177/1049732314523840

55.

Tagg

Lyons

Rock

(2017). The ethics of digital ethnography in a team project. Applied Linguistics Review, 8(2–3), 271–292. https://doi.org/10.1515/applirev-2016-1040

56.

Tricco

A. C.

Lillie

Zarin

O’Brien

K. K.

Colquhoun

Levac

Moher

Peters

M. D. J.

Horsley

Weeks

Hempel

Akl

E. A.

Chang

McGowan

Stewart

Hartling

Aldcroft

Wilson

M. G.

Garritty

Straus

S. E.

(2018). PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and explanation. Annals of Internal Medicine, 169(7), 467–473. https://doi.org/10.7326/M18-0850

57.

Tuttas

C. A.

(2015). Lessons learned using Web conference technology for online focus group interviews. Qualitative Health Research, 25(1), 122–133. https://doi.org/10.1177/1049732314549602

58.

Vicsek

(2016). Improving data quality and avoiding pitfalls of online text-based focus groups: A practical guide. Qualitative Report, 21(7), 1232–1242. https://nsuworks.nova.edu/tqr/vol21/iss7/4

59.

Wilkerson

J. M.

Iantaffi

Grey

J. A.

Bockting

W. O.

Rosser

B. R.

(2014). Recommendations for internet-based qualitative health research with hard-to-reach populations. Qualitative Health Research, 24(4), 561–574. https://doi.org/10.1177/1049732314524635

60.

Williams

Clausen

M. G.

Robertson

Peacock

McPherson

(2012). Methodological reflections on the use of asynchronous online focus groups in health research. International Journal of Qualitative Methods, 11(4), 368–383. https://doi.org/10.1177/160940691201100405

61.

Woodyatt

C. R.

Finneran

C. A.

Stephenson

(2016). In-person versus online focus group discussions: A comparative analysis of data quality. Qualitative Health Research, 26(6), 741–749. https://doi.org/10.1177/1049732316631510

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.08 MB

0.25 MB