Emergent Design and Unanticipated Ideas in Asynchronous Online Focus Groups: Finding an Unexpected Silver Lining in Apparent Methodological Compromise

Abstract

This article explores methodological advantages and limitations of using asynchronous online focus groups (AOFGs) as an approach to investigating community attitudes and values relating to biotechnology in food production. We describe our experiences using AOFGs during the COVID-19 pandemic as a form of emergent design. We highlight the methodological advantages and limitations of such research approaches, particularly to explore unanticipated ideas relating to community attitudes and values associated with biotechnology in food production. Although AOFGs are not a direct substitute for traditional methodologies, there are clear advantages that can outweigh typical concerns, depending on the research questions and goals, particularly when research aims to involve participants from diverse demographic groups and engage in more emergent methods allowing unanticipated ideas to be explored. As a result of having to pivot our methods to produce the required analyses during the COVID-19 pandemic, this paper provides valuable insights into the use of AOFGs to capture qualitative data when use of traditional methods is restricted or may be less desirable.

Keywords

focus groups qualitative research asynchronous discussion groups online qualitative research empirical methods biotechnology and food diverse demographic recruitment

Introduction

Focus group methodology is commonly used by social science researchers to collect data through group interactions on a predetermined topic. Focus groups provide added value compared to one-on-one interviews and survey methodologies as they permit elicitation of ideas through facilitated participant-to-participant interactions. These ideas may not have been anticipated in the original script or schedule and can provide nuance about understandings of focal topics.

Our research group uses qualitative methods such as focus groups and interviews to explore attitudes toward issues about which most community members have limited knowledge and which require approaches that allow for broad exploration of responses through elicitation of unanticipated ideas. Like most, our research was heavily impacted by the COVID-19 pandemic: with contracts already in place, we pivoted our methods before we commenced the empirical research to produce the required analyses despite public health mandates such as social distancing and lockdowns. Since the onset of the COVID-19 pandemic, many qualitative researchers have shared their experiences of using online focus group methods (e.g., Adom et al., 2020; Dodds & Hess, 2020; Frey & Bloch, 2023; Keen et al., 2022; Lathen & Laestadius, 2021; Newman et al., 2021; Roberts et al., 2021; Varma et al., 2021). We initially chose asynchronous online focus groups (AOFGs) to meet our research objectives and due to the greatly different time zones of participants (across Australia and New Zealand) but learned that they permitted more productive research than we initially anticipated, as explored in this paper.

Online technologies allow researchers to collect large quantities of data with ease and relatively low cost, including from participants who are otherwise difficult to access. As methods such as ethnographic or observational techniques, interviewing, and focus groups are increasingly successfully adapted for use in online settings, more qualitative researchers need to share their experiences with novel research methods “to make transparent the experience and demonstrate the viability of the method” (Fox et al., 2007, p. 539).

The purpose of this paper is to discuss the advantages and limitations of using AOFGs to investigate community attitudes and values. It explores claims that AOFGs are less capable of generating participant-to-participant interactions than their face-to-face counterparts (Fox et al., 2010) or that discussions in them are lacking in conceptual detail (Nicholas et al., 2010), and consequently that unanticipated issues are less likely to be raised. We challenge these claims by highlighting the benefits of AOFGs and illustrating our design and process decisions that encouraged participant-to-participant interactions and engagement, and thus enabled unanticipated issues to be discussed, drawing on our AOFG experiences in two projects exploring Australian and New Zealand community attitudes to novel biotechnologies in food production.

Background

Online Focus Groups – Synchronous Versus Asynchronous

Both online and traditional face-to-face focus groups gather data through discussion of a specific set of issues: they are thus “focused” and involve a collective activity (Kitzinger, 1994, p. 104). Online focus groups are computer-mediated communication events (Terrance et al., 1993, p. 53) that use text, voice, and/or video to mimic face-to-face interactions (Lobe, 2017; Williams et al., 2012). Online focus groups are not straightforward replacements for traditional face-to-face techniques; they provide a different set of tools with their own unique advantages and limitations (Graffigna & Bosio, 2006; Williams et al., 2012). As noted in a recent literature review of research comparing online and face-to-face approaches, there is no clear consensus to date as to whether face-to-face or online focus groups hold specific advantages in terms of the data produced and the resources required (Jones et al., 2022).

Online focus groups can occur synchronously or asynchronously and can take advantage of multiple modes of communication. Synchronous focus groups occur in real time, requiring participants to contribute over a defined time period (Fox et al., 2007), using audio, video, and/or text-based methodologies such as chat rooms or messenger systems (Williams et al., 2012). Asynchronous focus groups are conducted over a longer period of time, often several days (Fox, 2017), allowing participants to engage with discussions at their convenience (Gaiser, 1997; Williams et al., 2012; Zwaanswijk & van Dulmen, 2014). Asynchronous focus groups often usually rely on online discussion boards or fora but can also involve multimedia responses.

Emergent Design and Unanticipated Ideas

The research process for qualitative researchers is emergent, no matter what methods are utilised. Emergent¹ design refers to the ability to adapt and incorporate new ideas, concepts, or findings (what we refer to as “unanticipated ideas”) that arise while conducting qualitative research (Palithorpe, 2017). Unlike more structured approaches such as close-ended surveys, emergent design encourages unanticipated ideas, which not only add colour and richness to the data but also allow exploration of the meanings and values that people bring to the phenomena in question (Denzin & Lincoln, 2011). Consequently, the initial plan for research cannot be too tightly prescribed, recognising that the process or focus may shift once data collection begins (Creswell & Poth, 2018). Such more inductive approaches are particularly useful for understudied topics, such as those which were our focus, and is likely to be the case with rapidly emerging concepts or issues (such as the COVID-19 pandemic or new technologies) or where initial literature reviews reveal limited previous research.

Semi-scripted approaches are a feature of emergent design, allowing for flexibility within the research. This flexibility defocuses the researchers, allowing participants to respond to the topic or question in their own diverse ways. “Defocusing” means providing space for unanticipated ideas to emerge and permitting exploration of topics that might initially seem less relevant to the research (Timmermans & Tavory, 2022). Unanticipated ideas can highlight nuances in an argument or present new ways to frame existing arguments, and hence build on key themes of the research in ways that the researchers do not anticipate; they also can enable deeper understanding of participants’ attitudes and values.

In face-to-face interactions, qualitative researchers are typically constrained by the time allotted for focus groups and need to follow the script to ensure all topics are (in some sense) “covered.” Time constraints often prevent apparently tangential ideas to be explored in detail, some of which might prove to be useful if pursued. In AOFGs, unanticipated ideas can be explored as part of what we term “back-of-the-room conversations” that do not detract from other conversations. While unanticipated ideas may be so abstract or tangential that they are not directly useful for purposes of analysis, encouraging such ideas to be discussed adds nuance and breadth to the conversations and data, which is particularly important when doing research in domains that may be unfamiliar to or uncomfortable for participants. Hence permitting these conversations produces the type of crosstalk and interactions that are often considered critical in face-to-face focus groups particularly because they can produce divergent or conflicting viewpoints, and thus richer data (Liamputtong, 2011).²

Potential Limitations of AOFGs

There are some commonly recognized issues with AOFGs: most notably, technological barriers are likely to be experienced by some participants, including access and comfort with online technologies (O’Connor et al., 2008); hence study planning must carefully include consideration of these issues when recruiting as well as assessing potential resulting sample bias. Problems associated with digital literacy and the so-called “digital divide” including unequal access to virtual devices, tools, and Internet connectivity can affect and limit active participation in any online focus groups. These differences result in the need to consider researchers’ positionality when conducting focus groups and analysing the results (Roberts et al., 2021). However, it is important to note that many of these types of criticisms in the scholarly literature are now dated and may well not be reflective of current levels of public use of and comfort levels with communication in online spaces.

Focus group participants engage in sustained dialogue around a set of themes of interest in the research. Past scholars have noted that levels of interaction are lower in asynchronous focus groups than in synchronous options (Fox et al., 2010) due to the disparity in the times at which participants are online, with greatly differing levels of participation in AOFGs ranging from highly interactive people to “lurkers” who simply read posts but do not respond. In types of research where participant interactions are particularly useful, such as in situations where little is known about the topic (Kitzinger, 1994), many scholars consider AOFGs to be unsuitable because they are claimed to result in more structured and simpler responses, and less reflection (e.g., Murgado-Armenteros et al., 2012). Some scholars have noted that the group interactions that often occur in synchronous focus groups may be harder to replicate in asynchronous environments such as AOFGs (Zwaanswijk & van Dulmen, 2014). Others claim that there are fewer opportunities to ask meaningful follow-up questions or use probes than might occur in a synchronous focus group (e.g., Gordon et al., 2021). Although moderators can post follow-up questions and encourage discussion amongst participants, there is a need to consider best practices about these types of prompts, in order to maximize the quality of the data produced and not expect too much of participants.

Many scholars emphasise the benefit of synchronous video exchanges over text-based mediums when attempting to establish rapport (Archibald et al., 2019; Calefato & Lanubile, 2010; Deakin & Wakefield, 2013; Lo Iacono et al., 2016; Sy et al., 2020) as non-visual approaches are considered antisocial and impersonal. Removing the visual and verbal cues associated with face-to-face focus groups reduces the ability to read body language, interactions between participants, and non-verbal cues (Lobe, 2017). Facial expressions or other movements that express emotion may affect individuals’ participation in focus groups (Gaiser, 2008), and lack of visual cues can limit researchers’ abilities to overcome potential barriers to participation introduced through non-verbal cues.

There is no doubt that group dynamics are very different in AOFGs than in face-to-face focus groups, and between-participant interactions can be more limited; however, facilitation techniques can be used to foster more interactions between participants and more engagement with the discussions, as we discuss in more detail below. Nonetheless, as with some face-to-face focus groups, some groups will be more interactive than others. It also should be noted that some distinct limitations are likely to occur when projects pivot mid-course from other formats to AOFGs (see e.g. Saberi, 2020), but we do not explore this issue in any detail, as our research was designed to be performed in this format from the start.

Potential Benefits of AOFGs

Much existing literature on virtual qualitative research has focused on the technology and tools utilised (e.g., Adom et al., 2020; Archibald et al., 2019; Moylan et al., 2015), especially for synchronous focus groups (e.g., Dodds & Hess, 2020; O'Connor et al., 2008; Roberts et al., 2021). Some of the advantages of synchronous online focus groups are shared with AOFGs while others are unique.

Hosting focus groups online makes it easier to recruit diverse groups of participants (Gaiser, 2008), target different population segments (Lobe, 2017), and access hard-to-reach groups (Fox, 2017; Tates et al., 2009), including from geographically remote areas. Recruitment is always difficult for face-to-face focus groups, particularly when researchers wish to engage participants from a variety of population subgroups (Lobe, 2017). While no-shows and drop-offs often plague face-to-face focus group research, participants are much easier to replace for AOFGs given their flexibility in terms of participation timelines. All types of online focus groups can be particularly useful methods in locales where it is difficult and expensive to conduct face-to-face focus groups due to highly geographically dispersed populations requiring researchers and participants to travel considerable distances (see also Keen et al., 2022).

While online focus groups generally provide a more convenient mode of participation for members of hard-to-reach populations and for whom face-to-face meetings may be difficult (Fox, 2017; Williams et al., 2012), AOFGs are even more convenient for participants, as they allow them to join conversations at their convenience and thus are likely to be more inclusive for those with non-standard schedules (LaForge et al., 2022). Researchers can also engage other hard-to-reach groups, such as those with language or communication barriers or disabilities (Fischbein et al., 2024) or stigmatized people (Marley et al., 2023), using a wide range of online data collection methods (Creswell & Poth, 2018). Text-based exchanges such as those used in AOFGs require minimal bandwidth, which is useful for those in areas with reduced Internet capacities and minimise the impacts of potential differences of abilities related to written expression (Oringdorrf, 2004; Tates et al., 2009; Williams et al., 2012; Turner et al., 2021).

Several typical facilitation challenges are made easier in AOFGs, such as compensating for dominant personalities among participants and making space for other voices, among other issues associated with power relations (Kornbluh, 2023). Participants may feel that they have greater freedom to speak online due to a sense of anonymity, thus reducing the level of vulnerability, and encouraging more forthright participation (Lobe, 2017; Mann & Stewart, 2000; Stewart & Williams, 2005; Zwaanswijk & van Dulmen, 2014). The inherent physical and psychological distance between participants in AOFGs allows individuals who might otherwise hesitate to participate to do so from the safety, distance, and comfort of their own computer (Reid & Reid, 2005; Reips, 2000; Gaiser, 2008; Lijadi & van Schalkwyk, 2015; Dodds & Hess, 2020). Facilitators in online focus groups have more subtle means of intervening in participants’ domination of conversation, for instance through private messaging particularly to encourage more reflective, constructive, and equitable exchanges (Kornbluh, 2023). Private messaging can be used by researchers to communicate with each other during AOFGs, which can improve data gathering processes (Roberts et al., 2021) and allow more coordinated facilitation.

The opportunity for respondents to reflect on their responses is an advantage of AOFGs. Writing about asynchronous interviews, James and Busher (2006) observe that asynchrony gave participants the opportunity of “drafting and redrafting what they wanted to write” (p. 406). Web-based platforms enable participants to provide deeper reflection on the discussion topics by affording them the time and opportunity to research a topic before they respond (Creswell & Poth, 2018). Joinson (2005) argues that considered responses of this type are more likely to be “socially desirable” than the more honest utterances that occur with spontaneity. However, the perceived anonymity that comes with being in an asynchronous online focus group has been claimed to largely mitigate these sorts of negative effects (Eynon et al., 2008).

Our AOFG Design

Many of our own preconceptions about AOFGs were challenged by our recent experiences conducting them. Since these preconceptions are likely shared with many qualitative researchers, we believe that the following reflections will be valuable across multiple fields.

We conducted 6 AOFGs between August 2020 and June 2021 as part of two research projects exploring community attitudes toward the use of novel biotechnologies in food production. For the first project, our 4 AOFGs were conducted to understand Australian attitudes toward the potential use of gene editing in sheep and beef cattle production (MEAT), with a total of 119 participants.³ The second project (FOOD) explored Australian and New Zealand consumers’ attitudes toward the use of new breeding techniques in food production and involved 79 participants.⁴ Two AOFGs were conducted simultaneously for this research, which built on the methods used in the first project.⁵

Unsurprisingly, we found that careful consideration of focus group design was essential for group functionality. We also noted that certain processes allowed us to elicit unanticipated ideas and foster more participant-to-participant interactions during the focus groups. We outline these considerations and processes below, in support of our broader argument that AOFGs have considerable potential not just as a fallback when face-to-face research is difficult or impossible but are excellent ways to engage diverse participants and promote discussion of unanticipated ideas, thus allowing more emergent design processes.

Sampling and Recruitment

In designing the AOFGs, we took advantage of the opportunity to include participants who might not normally be able or willing to participate in in-person focus groups. In the MEAT project, we were able to include participants from across all Australian states and from urban, regional, rural, and remote locations, which was particularly important because this type of diversity has previously been hypothesised as related to contrasting views on relevant topics such as meat consumption and animal agriculture (Ankeny, 2008; Buddle et al., 2021, 2022). In the FOOD project, we were able to include both Australia- and New Zealand-based participants as we could engage with participants from different time zones where times differed by as much as 4 hours. To engage participants from varying demographic backgrounds, we used a specialist recruitment company,⁶ thus eliminating the normal complexity of recruiting participants and ensuring recruitment from demographics that were broadly representative of the relevant national population (e.g., including First Nations peoples and recent migrants) as the company had existing panels from which they could draw.

Each focus group had up to 5 facilitators engaged throughout the process, with a primary and secondary facilitator active in the fora at any given time. Utilising a team of facilitators allowed us to present ourselves as having different types of expertise and distinct personalities with which we engaged participants. The online format also allowed us to engage interstate-based researchers as facilitators for the discussions, thus reducing the usual time, costs, and impacts associated with travel. We were conscious of our positionality as researchers not just in terms of geographic location but other factors such as socioeconomic and educational backgrounds and were able to use these differences to foster conversation. We did not find considerations relating to positionality to be more problematic by being in an online group as compared to a standard, face-to-face focus group (cf. Roberts et al., 2021). Having researchers located in different time zones was extremely useful as allowed individual researchers to be engaged online at more convenient times or for shorter periods of time.

We ran each focus group over a three-day period. To help foster participation, we clearly defined what qualified as participation and our expectations from the beginning. Participants were instructed that they needed to complete all tasks on each of the three days, participate online (by reading or providing comments) for at least 15 minutes per day, and complete all pre-set questions by the end of the final day to receive their honoraria.

We involved a relatively large number of participants in the AOFGs, many more than we would have engaged in face-to-face focus groups, which permitted us to include participants from a greater diversity of demographics and allow adequate buffer for any dropouts. Having larger numbers of participants in real-time focus groups can pose challenges, such as difficulties in engaging with complex issues due to the speed at which the conversation is moving (Mann & Stewart, 2000). Use of multiple facilitators and the ease of navigation of the platform allowed us to keep conversations going across multiple threads.

The fact that the research took place over several days meant that a relatively high drop-out rate might have been expected (Lobe, 2017). However, only one person (1%) dropped out of the FOOD AOFG and 19 (16%) from the MEAT AOFG. We suspect these relatively low numbers might in part be due to COVID-19 limitations during the research, which meant that many participants were at home and had more time than usual to participate. However, the convenience of participation at a time selected by each participant no doubt also contributed. An added benefit of the larger size of the AOFG was that it reduced the overall number of focus groups required and enabled researchers to engage a greater number of participants in a shorter time period.

Platform Design

The platform structure used was similar to an online forum or discussion board. Participants could post a thread (i.e., a new comment on the board) which allowed other participants or facilitators to follow up by commenting on existing threads. Threads were shown on the board in the order in which they were posted, with newer threads at the top of the boards. “Pinned” threads (selected by the facilitators) were always shown at the top of the board. Comments in threads were listed sequentially in the order in which they were posted, with the newest comments shown at the bottom of the thread. Several participants posted images as comments to supplement their written responses, although images were not specifically requested.

The platform encouraged interaction and engagement by notifying participants when there were unread comments in the threads which they had initiated or to which they had previously replied. These prompts encouraged them to return to the board to read responses and answer follow-up questions.

Privacy and confidentiality issues should be addressed at the outset of AOFGs, including any prohibition on saving content through screenshots or similar (Lobe et al., 2020). Other concerns, such as whether other participants will compromise the discussions’ confidentiality, are similar in all types of focus groups: if of particular concern, researchers should ask that the privacy of other participants be respected by not discussing who was present or what was said with non-participants, and remind participants that confidentiality cannot be guaranteed in this type of setting. For our projects, we specifically asked participants to talk about the AOFG discussions with family or close friends and used these conversations as part of our research and dialogues during the focus groups to generate even richer data.

Pre-set Activities

Each focus group lasted three days, with activities introduced each day exploring a new theme which permitted focused discussions and required engagement by each participant. These activities involved responding to a variety of pre-set questions including multiple choice or Likert-style questions (not primarily as a form of quantitative research but to provide material for discussion) and providing short written responses in dialogue with one another and facilitators. We also used scenarios and questions to spark discussion.

Participation was self-paced, meaning that participants could log on and off the platform as often as they wished and at any time during the three days. This flexibility made it possible for people to participate who might otherwise not have participated if the focus group was conducted at a specific time of day and/or in person. The boards for earlier days were not locked, allowing participants and facilitators to revisit and respond to previous questions and comments. However, the activities on subsequent days were locked until their scheduled release, thereby preventing participants from moving beyond the current board and completing all the tasks in one sitting or without engaging with others. This structure meant that participants moved through the focus group boards and their associated topics at a similar rate, which created a good environment for discussion amongst the participants.

In both projects, the biotechnology and its potential uses were introduced using animated videos. We chose this approach based on literature indicating that visual aids can promote conversations about difficult, abstract, or unfamiliar topics (Hurworth et al., 2005). The video created for the MEAT project was also used to prompt discussions about bias and credibility in communications materials. In the FOOD project, participants were asked to give detailed feedback on draft communication materials, which was a specific request from our research sponsor.

The principles related to ending face-to-face groups (e.g., Kruger & Casey, 2000) can be applied to online focus groups. We structured our online forum so that participants were able to move through the content at their own pace, but we provided a definitive end to the forum by asking participants to share any last thoughts about the topic and thanking them for their time once they reached the end of the content.

Key Lessons Learned

The Importance of Onboarding: Creating Norms for a Safe Space

To encourage participants to feel relaxed about and engaged in the AOFGs, and to begin to mimic some of the relational dynamics that are desirable in face-to-face focus groups, it was important to create a safe and welcoming online space where they could express their views. Several methods were used: when first logging into the fora, participants were introduced in plain language to the project, its aims, objectives, and background, including information about the funding sources and the projects’ ethics approval details. A short video designed to establish ground rules for participation was included at the start of the focus groups to attach a “friendly face” to the research. We stressed that there were no expectations about certain responses being correct or socially desirable but that our interests were in their views, ideas, concerns, and questions.

Participant-to-participant interactions tend to be more robust when researchers create a safe environment in which participants feel comfortable expressing their views and talk about their experiences. One way to do this and build trust is to ensure all participants are aware of who is online (Roberts et al., 2021). O’Connor and Madge (2003) recommend asking each participant to post a photograph and brief biography at the beginning to provide contextual information to other group members to which they can relate. As a “gentle beginning” (Elwyn et al., 2016), we asked our participants and facilitators to introduce themselves and answer a simple ice-breaker question related to the focus group topic (e.g., “what comes to mind when you see the term ‘gene’?”). These initial tasks familiarised participants with the functionality of the online platform, including how to post comments and replies, and how to navigate between the boards associated with various questions or activities. They also permitted the establishment of a constructive group dynamic and rapport, similar to the ice breakers typically used in face-to-face focus groups.

Using AOFGs to Generate Interaction

Analysis of participant-to-participant interactions in each of the focus groups allowed us to identify different types of interactions as well as commonalities in the design features and processes that enabled them.

The most frequent type of direct interaction between participants consisted of one participant commenting “I agree” or similar in a thread started by another participant. Often this did not spark further discussion between the participants, even if facilitators stepped in to ask follow-up questions. Other types of interactions that exhibited a similar level of one-way peer-to-peer interactions occurred when participants referred to arguments or information from others in their own threads. For example, after watching the MEAT introductory video, participants were asked what came to mind: a participant from the third AOFG started an unprompted thread stating that “Absolutely I agree with J⁷ (…) that our genetic makeup should be left the way we were made (…)”. These interactions illustrate that participants did engage with each other’s comment threads and were not disinterested in the views of others. Many participants who initially seemed to be “lurking” (see Fox et al., 2010) would later interact in these ways, or by replying with a simple “…sigh…,” or emojis added to the text (Cabraal, 2022; Dunlap et al., 2016). Such comments can help to supplement the findings and analysis, and should be taken as an important part of the data (Gaiser, 2008).

Another common type of interaction occurred when a participant commented on a point made by another participant, either with expansion on the original point or, less frequently, to ask follow-up questions. These types of comments were more likely to generate responses from other participants than were “agree”-type interactions. These types of interactions often contributed to the emergence of unanticipated ideas: for instance as shown in Figure 1, the original poster (participant no. 33), the other participants (nos. 15, 18, and 29), and the facilitator, Emily, are discussing the potential use of gene editing to make cattle more resistant to heat through introduction of the naturally occurring SLICK gene. In this thread, participant no. 33 asks how the effectiveness of this alteration would be assessed. Both participants nos. 15 and 29 offer alternative possibilities, one highly technical and the other experiential based on the quality of the meat.

Figure 1.

An example of a common type of interaction that occurred when a participant commented on a point made by another participant, either with expansion on the original point or, less frequently, to ask follow-up questions.

The AOFG format also allowed on-and-off topic conversations to take place between smaller groups or pairs of participants, which did not serve to answer the posed question or the current line of discussion. In these interactions, new information was presented and discussed by participants (e.g., from a web search or from participants’ own experiences). We have termed these types of threads “back-of-the-room conversations,” as they correspond with the type of side “chatter” which might be discouraged in face-to-face focus groups or would be impossible within a synchronous focus group. However in the AOFG, these conversations did not distract from the main purpose of the discussion, and in fact were recorded and analysed as they were often a site where unanticipated ideas emerged. Additionally, these types of conversations helped to build rapport between participants.

Participant engagement in our AOFGs on average was very high, where engagement is defined by the percentage of questions answered, replies to facilitators, and the length and richness of responses (LaForge et al., 2022). Many participants came back online multiple times each day and stayed well beyond the 15 minutes minimum per day commitment, providing responses and discussing issues. In contrast, interaction, defined as responses made to or referencing a previous response by another participant (LaForge et al., 2022), was on average not as high. We contend that the data produced was nonetheless rich based on Charmaz’s definition (2003) of data richness as revealing participants’ thoughts, understandings, feelings, intentions, and the context and structures in which they are embedded. Our data was especially rich because it often took the discussions in directions that had not been anticipated by us when planning the research, including in existing literature on related topics.

The Critical Role of the Facilitators

Our experiences with AOFGs highlighted the importance of the roles that facilitators can play even in participant-to-participant interactions. The platform gave focus group facilitators an overview of the discussions that were occurring across multiple participants and boards, and they could create impromptu “pinned” discussion threads that brought together issues that were shared by multiple participants.

A roster system was set up for the facilitators which allowed continuous monitoring of the focus group boards between 8.00 am and 11pm AEST, to ensure timely responses and maximise the amount of engagement from participants while they were active on the forum. Previous studies highlight that when facilitators are active in the forum, they can encourage sustained engagement and conversations between participants (de Jong et al., 2012; Fox, 2017). Conversely, we used a different technique to manage “overly enthusiastic” participants who felt the need to comment on every post; these participants typically were not rude (though perhaps lacked familiarity with social media norms) but could be disruptive. We generally allowed the participant to engage and only stepped in when absolutely necessary, as we noted that other participants often opted simply to ignore their comments.

We also set up a facilitators’ chat using a “back channel” to provide updates to other facilitators when they were not active in the forum, to discuss any problems arising including technical ones, and to do hand-off when the facilitation team changed over. This “back channel” was also used to summarise the discussions facilitators had with or observed amongst participants, discuss possible emerging topics and issues, and ask for assistance if necessary. The asynchronous format provided greater opportunities to manage inappropriate or tangential comments, particularly as facilitators could send private messages as required to participants.

The facilitators also occasionally pointed out an argument or observation made by another participant and directed participants to each other’s threads, often resulting in participant-to-participant interactions. The asynchronous format of the focus groups allowed both participants and researcher/facilitators greater amounts of time to think and process what was being asked of them before responding, which we contend produced more measured opinions and refined contributions to the conversation. It also gave us opportunities to follow individual trains of thought through in detail with participants and raise related issues with the broader group where themes seem to be emerging. These opportunities often are lost in an active, real-time focus group.

Learnings on Limitations and Difficulties Associated with AOFGs

The online format did provide some limitations in terms of engagement on the board. To successfully conduct AOFGs, both the facilitators and the participants need to have Internet access, but Internet connectivity in Australia and New Zealand can be highly variable. During periods of high activity (e.g., when several participants were using the board simultaneously and responding to comments across several days), the refresh speed was sometimes very slow, resulting in facilitators having difficulties keeping up with the influx of comments from participants who were online at that time. The platform had various technical limitations which might have limited further engagement by participants: for instance, it did not allow them to pin or bookmark a discussion thread on which they had not commented, which might have discouraged them from engaging with new discussions or following existing conversations.

New participants occasionally joined the focus groups on days 2 and 3 to replace participants who dropped out and to meet demographic sampling requirements. These additions might have stifled peer-to-peer interactions because some participants finished the focus group tasks for a particular day before others had begun that day’s activities. In addition, although we do not have direct evidence, we speculate that participants were less likely to return to the board once their tasks were completed and they had finished the wrap-up section.

Upon reflection, we are now aware that in our eagerness to keep participants engaged, we might have been too active on the board as facilitators, thereby pre-empting follow-up questions that might otherwise have been asked by other participants. In the first four focus groups in particular, facilitation was heavy-handed in the hopes that this approach would encourage participants to stay active on the board. There clearly was a delicate balance between facilitators continuously engaging with individual participants and stifling participant-to-participant interactions.

It is difficult to speculate on how changing the communication modality from speech as in face-to-face focus groups to a written form may have altered the group dynamics: we did find that a rapport appeared to be established between most participants and the facilitators, and that on later days of the AOFGs, the conversations flowed more naturally. We suspect that increased familiarity with these sorts of online forms of communication is growing amongst many groups due to its use in many social media platforms, and hence this type of communication is less foreign to many participants. The relatively large size of the AOFGs initially seemed difficult to manage but various subgroups tended to form depending on time of day and specific interests in the topics under discussion, again in turn permitting conversations that arguably more precisely mimic everyday face-to-face conversations as compared to typical facilitated focus groups, including “back-of-the-room” conversations.

Conclusions and Recommendations

The obvious major benefit of using AOFGs was the ability to collect rigorous empirical qualitative data from diverse participants representing national demographic patterns in Australia and New Zealand at the height of the COVID-19 pandemic. Social isolation measures during the pandemic meant that many people remained at home and experienced a shift in the demands on their time amid life stressors and responsibilities (Ravitch, 2020; Newman et al., 2021). The shift to online focus groups were one of the only means available for collecting data during this period, but we were surprised by the benefits of AOFGs compared to face-to-face focus groups. AOFGs offered considerable advantages, particularly in permitting participation from diverse groups of people with differing schedules and commitments, allowing the goals of the research to be more robustly addressed. Such diversity is particularly important in highly geographically dispersed locales such as Australia and New Zealand, permitting more inclusion of remote and regional communities than is typically possible.

One substantial benefit of AOFGs is the reduced cost (Fox, 2017; Gaiser, 2008; Williams et al., 2012; but cf. Rupert et al., 2017; Davies et al., 2020). Although there were costs associated with using the commercial platform, developing the content, setting up the online forum, training facilitators, and significant facilitation time, there was no need to locate and pay for a physical meeting space or for travel. We were also able to offer lower incentives for participation as the online focus groups were less of an imposition than asking someone to attend in person. Text-based asynchronous methodology allowed data to be directly extracted and eliminated costly and time-consuming transcription (Tates et al., 2009).

Ironically, one downside of this approach is the sheer amount of data generated, as many participants spent far more than the required 15 minutes per day in the fora. At times, the amount of participation made it difficult for facilitators to immediately respond to each participant, particularly during peak times, and generated a large amount of material for participants to read. However, we found that the AOFGs provided somewhat more substantial answers and richer data than we typically receive via face-to-face focus groups on similar topics, particularly in terms of the relevance to research aims, provision of the reasoning behind participants’ responses, and the considered nature of replies, paralleling previous direct comparisons of AOFGs and face-to-face groups (e.g., Ingram & Steger, 2015; Murgado-Armenteros et al., 2012).

AOFGs are not a direct substitute for traditional methodologies, for instance due to the loss of visual cues and capacity for sustained participant dialogue. However, there are clear advantages that may outweigh these considerations, depending on the questions and goals, particularly if the research requires participants from a wide mix of demographic subgroups. The use of an online platform allowed participants to react to diverse types of content (including video and written prompts) and permitted multiple facilitators to present distinct perspectives. We found that more participants engaged, with much less dominance of a small number or particular voices than occurs in even well-facilitated face-to-face focus groups.

It is clear that participant interactions varied considerably, including across the subgroups that formed due to timing (i.e., when certain participants came onto the board) or interests. As has been noted in previous research (Gordon et al., 2021), interactions in AOFGs can vary widely within the same study, even when using similar engagement and facilitation processes. An advantage of AOFGs is that participants can read others’ responses and interact with them (or ignore them), whereas traditional focus groups are much more limited in this regard. Participants may be influenced by the responses that have already been provided, but we used facilitation to attempt to break this tendency down and to actively question any signs of “group think”; note that this problem is not significantly different from what can occur in face-to-face focus groups.

Most importantly, our AOFGs permitted us to explore unanticipated ideas introduced by participants relating to community attitudes and values associated with biotechnology in food production resulting from our emergent design approach together with our decisions about how to structure and facilitate the AOFGs. Obviously one goal associated with many forms of qualitative, open-ended research is for new ideas to emerge in an inductive manner: however based on our past experiences, we contend that our AOFGs exceeded what usually occurs in this regard both in quantity and in diversity, in part because of our design approach as well as the structuring of activities and facilitation techniques.

We summarise our key decisions and lessons learned in the following recommendations to other qualitative researchers:

Don’t	Do
View AOFGs as a substitute or second-best options	Think through the potential advantages of AOFGs particularly for engaging a more diverse group of participants and exploring understudied questions
Underestimate the importance of onboarding participants	Actively create a safe environment in which participants can express their views and talk about their experiences
Assume designs from other contexts can be straightforwardly transferred	Consider diverse design options, including advantages available in AOFGs such as use of multimedia prompts, and pilot in advance
Discount data other than text or not directly responsive to research questions	Consider how to analyse emojis, reactions, and other non-textual materials, and include consideration of “back-of-the-room” and tangential conversations which can reveal unanticipated ideas
Assume that facilitation will be the same as for a face-to-face focus group	Train facilitators in advance and provide mechanisms for communication and constant reflection between facilitators about approaches and methods
Overfacilitate!	Allow participants to engage with the facilitators and other participants, and give them time to reflect and ask questions

Our use of AOFGs was arguably a process of “learning through missteps” (Fujii, 2018, p. 5), involving active reflexivity (Soedirgo & Glas, 2020), as the COVID-19 pandemic and its uncertainties required us to be particularly agile. Using examples of our own research investigating community attitudes and values, we have highlighted how AOFGs can provide not only a viable alternative to face-to-face focus groups, but one that may be more appropriate than more traditional means of doing research, depending on its goals, aims, and context.

Footnotes

Acknowledgements

We wish to thank Kelly McKinley for her assistance with the online focus groups.

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: None of the authors have any conflicts of interest to declare. The research was conducted without limits by the funders beyond overall research questions, and all findings are the authorial team’s responsibility.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by Meat and Livestock Australia (MLA) via their Genetics Research Development and Adoption Project of the National Livestock Genetics Consortium (project L.GEN.2003), and by Food Standards Australia New Zealand (FSANZ).

Ethical Statement

ORCID iDs

Emily A. Buddle

Rebekah J. Harms

Heather J. Bray

Data Availability Statement

The datasets generated during and/or analysed during the current study are not publicly available due an internal embargo but are available from the corresponding author on reasonable request.*

Notes

References

Adom

Osei

Adu-Agyem

(2020). COVID-19 lockdown: A review of an alternative to the traditional approach to research. Research Journal in Advanced Social Sciences, 1, 1–9. https://doi.org/10.58256/rjass.v1i.107

Ankeny

R. A.

(2008). The moral economy of red meat in Australia. In Friedland

(Ed.), Food and morality: Oxford symposium on food and cookery 2007 (pp. 20–28). Propsect Books.

Archibald

M. M.

Ambagtsherr

R. C.

Casey

M. G.

Lawless

(2019). Using Zoom videoconferencing for qualitative data collection: Perceptions and experiences of researchers and participants. International Journal of Qualitative Methods, 18, 1609406919874596. https://doi.org/10.1177/1609406919874596

Braun

Clarke

(2013). Successful Qualitative Research: A practical guide for beginners. London: Sage.

Buddle

E. A.

Bray

H. J.

Ankeny

R. A.

(2021). “Of course we care!”: A qualitative exploration of Australian livestock producers’ understandings of farm animal welfare issues. Journal of Rural Studies, 83, 50–59. https://doi.org/10.1016/j.jrurstud.2021.02.024

Buddle

E. A.

Bray

H. J.

Ankeny

R. A.

(2022). Values of Australian meat consumers related to sheep and beef cattle welfare: What makes a good life and a good death? Food Ethics, 8(1), 5. https://doi.org/10.1007/s41055-022-00114-2

Cabraal

(2022). Using emoji for qualitative data analysis. Quirkos. Retrieved 30 September 2023 from. https://www.quirkos.com/blog/post/using-emoji-for-qualitative-data-analysis/

Calefato

Lanubile

(2010). Communication media selection for remote interaction of ad hoc groups. Advances in Computers, 78, 271–313. https://doi.org/10.1016/S0065-2458(10)78006-2

Charmaz

(2003). Grounded theory. In Smith

J. A.

(Ed.), Qualitative psychology: A practical guide to research methods (pp. 81–110). Sage.

10.

Creswell

J. W.

Poth

C. N.

(2018). Qualitative inquiry and research design: Choosing among five approaches (4th ed.). Sage.

11.

Davies

LeClair

K. L.

Bagley

Blunt

Hinton

Ryan

Ziebland

(2020). Face-to-face compared with online collected accounts of health and illness experiences: A scoping review. Qualitative Health Research, 30(13), 2092–2102. https://doi.org/10.1177/1049732320935835

12.

Deakin

Wakefield

(2013). Skype interviewing: Reflections of two PhD researchers. Qualitative Research, 14(5), 603–616. https://doi.org/10.1177/1468794113488126

13.

de Jong

Reinders-Messelink

H. A.

Jassen

W. G. M.

Poelma

M. J.

van Wijk

van der Sluis

C. K.

(2012). Activity and participation of children and adolescents with unilateral congenital below elbow deficiency: An online focus group study. Journal of Rehabilitation Medicine, 44(10), 885–892. https://doi.org/10.2340/16501977-1027

14.

Denzin

N. K.

Lincoln

Y. S.

(2011). Introduction: The discipline and practice of qualitative research. In The Sage handbook of qualitative research (4th ed., pp. 1–19). Sage.

15.

Dodds

Hess

A. C.

(2020). Adapting research methodology during COVID-19: Lessons for transformative service research. Journal of Service Management, 32(2), 203–217. https://doi.org/10.1108/JOSM-05-2020-0153

16.

Dunlap

J. C.

Bose

Lowenthal

P. R.

York

C. S.

Atkinson

Murtagh

(2016). What sunshine is to flowers: A literature review on the use of emoticons to support online learning. In Tettegah

S. Y.

Gartmeier

(Eds.), Emotions, technology, design and learning (pp. 163–182). Elsevier. https://doi.org/10.1016/B978-0-12-801856-9.09993-X

17.

Elwyn

Greenhalgh

Mcacfarlane

(2016). Groups: A guide to small group work in healthcare, management, education and research. CRC Press.

18.

Eynon

Fry

Schroeder

(2008). The ethics of internet research. In Fielding

Lee

R. M.

Blank

(Eds.), The Sage handbook of online research methods (pp. 23–41). Sage.

19.

Fischbein

Bracken

Bhambra

Congeni

Hameed

(2024). Beyond universal research design: Lessons for developing accessible virtual focus group research. International Journal of Qualitative Methods, 23, Article 16094069241257942. https://doi.org/10.1177/16094069241257942

20.

Fox

F. E.

(2017). Meeting in virtual spaces: Conducting online focus groups. In Braun

Clarke

Gray

(Eds.), Collecting qualitative data: A practical guide to textual, media and virtual techniques (pp. 275–299). Cambridge University Press. https://doi.org/10.1017/9781107295094

21.

Fox

F. E.

Morris

Rumsey

(2007). Doing synchronous online focus groups with young people: Methodological reflections. Qualitative Health Research, 17(4), 539–547. https://doi.org/10.1177-1049732306298754

22.

Fox

F. E.

Morris

Rumsey

(2010). How do young people use disclosure in real-time and non-real-time focus groups? International Journal of Web Based Communities, 6(4), 337–348. https://doi.org/10.1504/IJWBC.2010.035837

23.

Frey

T. K.

Bloch

B. S.

(2023). Using Microsoft Teams to facilitate asynchronous online focus groups. International Journal of Qualitative Methods, 22, Article 16094069231211251. https://doi.org/10.1177/16094069231211251

24.

Fujii

L. A.

(2018). Interviewing in social science research: A relational approach. Routledge.

25.

Gaiser

(1997). Conducting online focus groups: A methodological discussion. Social Science Computer Review, 15(2), 135–144. https://doi.org/10.1177/089443939701500202

26.

Gaiser

(2008). Online focus groups. In Fielding

Lee

R. M.

Blank

(Eds.), The Sage handbook of online research methods (pp. 290–306). Sage.

27.

Gordon

A. R.

Calzo

J. P.

Eiduson

Sharp

Silverstein

Lopez

Thomson

Reisner

S. L.

(2021). Asynchronous online focus groups for health research: Case study and lessons learned. International Journal of Qualitative Methods, 20, 1609406921990489. https://doi.org/10.1177/1609406921990489

28.

Graffigna

Bosio

A. C.

(2006). The influence of setting on findings produced in qualitative health research: A comparison between face-to-face and online discussion groups about HIV/AIDS. International Journal of Qualitative Methods, 5(3), 55–76. https://doi.org/10.1177/160940690600500307

29.

Hurworth

Clark

Martin

Thomsen

(2005). The use of photo-interviewing: Three examples from health evaluation and research. Evaluation Journal of Australasia, 4(1-2), 52–62. https://doi.org/10.1177/1035719X05004001-208

30.

Ingram

W. C.

Steger

R. A.

(2015). Comparing the quality and quantity of information received from traditional, webcam and board chat focus groups. Business Studies Journal, 7, 33–45. https://doi.org/10.1177/16094069221142406

31.

James

Busher

(2006). Credibility, authenticity and voice: Dilemmas in online interviewing. Qualitative Research, 6(3), 403–420. https://doi.org/10.1177/1468794106065010

32.

Joinson

A. N.

(2005). Internet behaviour and the design of virtual methods. In Hine

(Ed.), Virtual methods: Issues in social research on the internet. Berg.

33.

Jones

J. E.

Jones

L. L.

Calvert

M. J.

Damery

S. L.

Mathers

J. M.

(2022). A literature review of studies that have compared the use of face-to-face and online focus groups. International Journal of Qualitative Methods, 21, 1–12. https://doi.org/10.1177/16094069221142406

34.

Keen

Lomeli-Rodriguez

Joffe

(2022). From challenge to opportunity: Virtual qualitative research during COVID-19 and beyond. International Journal of Qualitative Methods, 21, 1–11. https://doir.org/10.1177/16094069221105075.

35.

Kitzinger

(1994). The methodology of focus groups: The importance of interaction between research participants. Sociology of Health and Illness, 16(1), 103–121. https://doi.org/10.1111/1467-9566.ep11347023

36.

Kornbluh

(2023). Facilitation strategies for conducting focus groups attending to issues of power. Qualitative Research in Psychology, 20, 1–20. https://doi.org/10.1080/14780887.2022.2066036

37.

Kruger

R. A.

Casey

M. A.

(2000). Focus groups. A practical guide to applied research (3rd ed.). Sage.

38.

LaForge

Gray

Stack

Livingston

C. J.

Hildebran

(2022). Using asynchronous online focus groups to capture healthcare professional opinions. International Journal of Qualitative Methods, 21, Article 160940692210956. https://doi.org/10.1177/16094069221095658

39.

Lathen

Laestadius

(2021). Reflections on online focus group research with low socio-economic status African American adults during COVID-19. International Journal of Qualitative Methods, 20, 1–10. https://doi.org/10.1177/16094069211021713

40.

Liamputtong

(2011). Focus group methodology: Principle and practice. Sage Publications.

41.

Lobe

(2017). Best practices for synchronous online focus groups. In Barbour

R. S.

Morgan

D. L.

(Eds.), A new era in focus group research (pp. 227–250). Palgrave Macmillan. https://doi.org/10.1057/978-1-137-58614-8_11

42.

Lobe

Morgan

Hoffman

K. A.

(2020). Qualitative data collection in an era of social distancing. International Journal of Qualitative Methods, 19, 1–8. https://doi.org/10.1177/1609406920937875

43.

Lijadi

A van Schalkwyk

J (2015). Online Facebook Focus Group Research of Hard-to-Reach Participants. International Journal of Qualitative Methods. https://doi.org/10.1177/1609406915621383

44.

Lo Iacono

Symonds

Brown

D. H. K.

(2016). Skype as a tool for qualitative research interviews. Sociological Research Online, 21(2), 103–117. https://doi.org/10.5153/sro.3952

45.

Mann

Stewart

(2000). Internet communication and qualitative research: A handbook for researching online. Sage.

46.

Marley

Kay Jin Tan

Wang

Byrne

M. E.

Wang

Tang

Ramaswamy

Luo

Sylvia

S. S.

Tucker

J. D.

(2023). Online focus group discussions to engage stigmatized populations in qualitative health research: Lessons learned. International Journal of Qualitative Methods, 22, 16094069231204767. https://doi.org/10.1177/16094069231204767

47.

Moylan

C. A.

Derr

A. S.

Lindhorst

(2015). Increasingly mobile: How new technologies can enhance qualitative research. Qualitative Social Work, 14(1), 36–47. https://doi.org/10.1177/1473325013516988

48.

Murgado-Armenteros

E. M.

Torres-Ruiz

F. J.

Vega-Zamora

(2012). Differences between online and face to face focus groups, viewed through two approaches. Journal of Theoretical and Applied Electronic Commerce Research, 7(2), 15–16. https://doi.org/10.4067/s0718-18762012000200008

49.

Newman

P. A.

Guta

Black

(2021). Ethical considerations for qualitative research methods during the COVID-19 pandemic and other emergency situations: Navigating the virtual field. International Journal of Qualitative Methods, 20, 1–12. https://doi.org/10.1177/16094069211047823

50.

Nicholas

D. B.

Lach

King

Scott

Boydell

Sawatzky

B. J.

Reisman

Schippel

Young

N. L.

(2010). Contrasting internet and face-to-face focus groups for children with chronic health conditions: Outcomes and participant experiences. International Journal of Qualitative Methods, 9(1), 105–121. https://doi.org/10.1177/160940691000900102

51.

O’Connor

Madge

(2003). Focus groups in cyperspace: Using the internet for qualitative research. Qualitative Market Research, 6, 133–143. https://doi.org/10.1108/13522750310470190

52.

O'Connor

Madge

Shaw

Wellens

(2008). Internet-based interviewing. In Fielding

N. G.

Lee

R. M.

Blank

(Eds.), The Sage handbook of online research methods (pp. 271–289). Sage.

53.

Oringdorrf

(2004). “My way”: Piloting an online focus group. International Journal of Qualitative Methods. https://doi.org/10.1177/1609406904003003

54.

Palithorpe

B. C.

(2017). Emergent design. In The international encyclopedia of communication research methods. John Wiley and Sons. https://doi.org/10.1002/9781118901731

55.

Ravitch

M (2020). FLUX Pedagogy: Transforming Teaching and Leading during Coronavirus. Perspectives on Urban Education, 17(4), 18–32.

56.

Reid

J Reid

J M (2005). Online Focus Groups: An In-depth Comparison of Computer-mediated and Conventional Focus Group Discussions. International Journal of Market Research. https://doi.org/10.1177/147078530504700204

57.

Reips

U-D

(2000). The Web experiment method: Advantages, disadvantages, and solutions. In Birnbaum

H (Ed.), Psychological experiments on the Internet. Academic Press.

58.

Roberts

J. K.

Pavlakis

A. E.

Richards

M. P.

(2021). It’s more complicated than it seems: Virtual qualitative research in the COVID-19 era. International Journal of Qualitative Methods, 20, 1–13. https://doi.org/10.1177/16094069211002959

59.

Rupert

D. J.

Poehlman

J. A.

Hayes

J. J.

Ray

S. E.

Moultrie

R. R.

(2017). Virtual versus in-person focus groups: Comparison of costs, recruitment, and participant logistics. Journal of Medical Internet Research, 19(3), Article e80. https://doi.org/10.2196/jmir.6980

60.

Saberi

(2020). Research in the time of coronavirus: Continuing ongoing studies in the midst of the COVID-19 pandemic. AIDS and Behavior, 24(8), 2232–2235. https://doi.org/10.1007/s10461-020-02868-4

61.

Soedirgo

Glas

(2020). Toward active reflexivity: Positionality and practice in the production of knowledge. PS: Political Science and Politics, 53(3), 527–531. https://doi.org/10.1017/S1049096519002233

62.

Stewart

Williams

(2005). Researching online populations: The use of online focus groups for social research. Qualitative Research, 5(4), 395–416. https://doi.org/10.1177/1468794105056916

63.

O'Leary

Nagraj

El-Awaisi

O'Carroll

Xyrichis

(2020). Doing interprofessional research in the COVID-19 era: A discussion paper. Journal of Interprofessional Care, 34(5), Article 1791808. https://doi.org/10.1080/13561820.2020.1791808

64.

Tates

Zwaanswijk

Otten

van Dulmen

Hoogerbrugge

P. M.

Kamps

W. A.

Bensing

J. M.

(2009). Online focus groups as a tool to collect data in hard-to-include populations: Examples from paediatric oncology. BMC Medical Research Methodology, 9, 15. https://doi.org/10.1186/1471-2288-9-15

65.

Terrance

A. L.

Johnson

G. M.

Walther

J. B.

(1993). Understanding the communication process in focus groups. In Morgan

D. L.

(Ed.), Successful focus groups: Advancing the state of the art (pp. 51–64). Sage.

66.

Timmermans

Tavory

(2022). Collecting. In Timmermans

Tavory

(Eds.), Data analysis in qualitative research: Theorizing with abductive analysis (pp. 45–68). University of Chicago Press.

67.

Turner

Rushby

Gauthier

James

Bahaj

Aragon

Sweetnam

Ellis

(2021). An Online ‘Face to Face’ Focus Group Approach for Understanding How Household Energy Use and Green Investment Decisions Differ by Personality Traits. International Journal of Qualitative Methods. https://doi.org/10.1177/16094069211038312

68.

Varma

D. S.

Young

M. E.

Krieder

C. M.

Williams

Vaddiparti

Parisi

Semeah

L. M.

(2021). Practical considerations in qualitative health research during the COVID-19 pandemic. International Journal of Qualitative Methods, 20, 1–5. https://doi.org/10.1177/16094069211043755

69.

Williams

Clausen

M. G.

Robertson

Peacock

McPherson

(2012). Methodological reflections on the use of asynchronous online focus groups in health research. International Journal of Qualitative Methods, 11(4), 368–383. https://doi.org/10.1177/160940691201100405

70.

Zwaanswijk

van Dulmen

(2014). Advantages of asynchronous online focus groups and face-to-face focus groups as perceived by child, adolescent and adult participants: A survey study. BMC Research Notes, 7, 756. https://doi.org/10.1186/1756-0500-7-756