Uncovering the Voices of Marginalized Minorities in Public Policy Research - A Critical Review of the Image and Text-Based Vignette Method

Introduction

Public actors do not make decisions in isolation. They interact with the other stakeholders: the suppliers, policymakers, customers, NGOs, media, and other social interest groups when responding to public policy. These individual actors collectively shape and influence each other’s judgments and decisions (Greenwood & Suddaby, 2006). They can also significantly and purposively modify, reconstruct, and influence society, exposing vulnerable groups' suppressed or marginalized interests (Danneris, 2016; Lawrence et al., 2011). Therefore, using network analysis, including these varied voices, would offer in-depth and all-rounded insights into public policy and social governance research (Brennecke et al., 2021; Diemer & Regan, 2022).

Despite the potential of network analysis, past research mainly focused on examining the dyadic interactions; especially between business firms and policymakers (Ertekin & Atik, 2020; Yao et al., 2022); business firms and associations (Chaney et al., 2016; Humphreys, 2010), and policymakers and NGOs (Nilsson, 2017). The voices of marginalized and vulnerable groups are often neglected (Cheah, 2019; Lewis et al., 2021; Sidanius & Pratto, 1999). Sidanius and Pratto (1999) refer to marginalized or vulnerable groups as those who differ from the prevailing majority and have long been excluded and invisible for failing to conform to conventional gender, age, race, sexuality, ability, health and other arbitrarily imposed distinctions. They include ethnic minorities, the underprivileged, LGBT and victims of abusive/unfair treatment (Pilkington & Acik, 2019). In this study, we extend the definition of vulnerable groups to those who feel anxious, fearful, or restless in expressing their thoughts on a social incident. Consequently, researchers may face difficulties in exposing their thoughts. This is because the potential participants tend to avoid disclosing sensitive information during surveys/interviews, fearing their identities would be exposed.

Tolich (2004) revealed that researchers often consider external confidentiality but neglect internal confidentiality, which may affect the insights of the research context. External confidentiality refers to the researcher’s commitment to protecting the participants' identities in the research via a consent statement signed between the parties before the interview. Internal confidentiality entails the protection of participants’ identities from one another in the report (Tolich, 2004) or protecting the participants’ identities in the final report as the industry insiders could easily recognize it, as there are limited outspoken insiders.

Given internal confidentiality concerns, participants often decline to participate in the interview (Johnson, 2017). Such reluctance is seen when discussing sensitive topics that are related to social issues and public policy, for instance, child abuse in orphanages in East Java, Indonesia (Sutinah & Aminah, 2018), corruption culture in Germany, Austria, and the USA (Campbell & Göritz, 2014), and sexual harassment at work (Baum, 2019). Sutinah and Aminah (2018) showcased the difficulties of encouraging the orphanages involved in the abusive claim. Baum’s (2019) job market research revealed that firms facing allegations of sexual harassment at the workplace might refuse to participate in such interviews. This includes sexual harassment of the opposite-sex and same-sex employees (Bendixen & Kennair, 2017) and sexual minorities, that is, LGBTQ employees (Brassel et al., 2019). These companies may issue a mandate of silence to restrain the staff from disclosing any details. In addition, the high number of undisclosed cases and the extensive use of Confidential Settlement Agreements may contribute to the current prevalence of sexual harassment at work, thereby resulting in the underrepresentation of such cases in policy research (Baum, 2019; Campbell & Göritz, 2014). Moreover, even if the firms are compelled to accept the interview invitations, the participating firms may assign atypically loyal staff to participate. As such, researchers could be misled or receive embellished data about the firm’s actual situation.

In addition, some data collection methods commonly used in the West may not be applicable in the East due to the socio-cultural differences. For instance, Rudolph & Young’s (2021) study on the HIV risk network among men who have sex with men (MSM). The authors interviewed 20 MSM from Kentucky, the USA, who are involved in sex and drug co-usage networks. Researchers in the East may face challenges in recruiting similar potential respondents, especially in the Middle-Eastern and other Muslim-majority Asian countries, as MSM is illegal. Thus, participation in such interviews may indicate they commit the “crime” or have close contact with those who have committed it. Therefore, these participants or their family members may hesitate to participate in the survey or interview; even anonymity and confidentiality are assured. The same was found in Schaefer et al.’s (2021) study on the prison-based therapeutic community and Jasny et al.’s (2021) study on participant engagement in environmentally-focused social science research.

Quantitative and Qualitative Vignette Methods

To access such sensitive and confidential data in policy studies, Bradbury-Jones et al. (2014) suggest adopting the vignette method. A vignette is a short, fictionalized story describing a hypothetical scenario (Sampson & Johannessen, 2020). Some scholars use real cases to collect information from their respondents [see, e.g., Sampson and Johannessen (2019) and Cheah and Koay (2022)]. The vignette can be in the visual form of images, pictures, or drawings to describe a scenario (Khanolainen & Semenova, 2020). It can be in paper or digital form (McInroy & Beer, 2021) to explore the respondents’ opinions on a sensitive situation (Barter & Renold, 2000; Finch, 1987). The quantitative experimental vignette method is commonly adopted in consumer research, using the constant-variable-value-vignette (CVVV) method (Wason et al., 2002). Most of these studies focus on the respondents' behavioral or moral judgments regarding issues such as the immoral selling process to under-age consumers (Bateman & Valentine, 2015, de Vicente Bittar, 2018), buyer revenge behaviors (Zourrig et al., 2009), and unethical fashion consumption (Manchiraju & Sadachar, 2014). When collecting data, the respondents are divided into a control group and an examination group to examine the differences in judgments (Aguinis & Bradley, 2014; Shea et al., 2019). The respondents are then given identical vignettes and must answer forced-choice or closed-ended questions. Wason et al. (2002) shared a few shortcomings of the quantitative CVVV approach. First, the respondents may have different opinions on the given forced choices. They may pick the same answer for multiple rationales (Hunt & Vitell, 2006). Second, although the forced-choice scale can summarise the respondents’ opinions, it fails to uncover the evaluation process (Hyman, 2004). Finally, the respondents may be motivated to respond to the survey by showing socially desirable behavior (Hunt & Vitell, 2006).

Scholars, therefore, suggest the qualitative-vignette method, as it enables the researchers to obtain in-depth insights (Aujla, 2020; Finch, 1987). For instance, the researcher can solicit the study context’s answer to how, why, and to whom. In contrast to the quantitative CVVV method that subscribes to the reductionist notion that ‘belief’ and ‘action’ are binary opposites, the qualitative vignette instead believes in conceiving “interviewees' response to vignette stimuli as social action in their own right” (Jenkins et al., 2010, p. 178). Therefore, a qualitative vignette aims not to conclude a definite prediction of a respondent’s behavior but to gather insights into the respondent’s interpretive framework and perceptual process (Jenkins et al., 2010). For this reason, a vignette is usually incorporated into the interview or focus group discussion (Atzmüller & Steiner, 2010).

The qualitative-vignette method is commonly applied in education, nursing and social work (Bradbury-Jones et al., 2014; Finch, 1987). Bradbury-Jones et al. (2014) emphasized the benefits of using the vignette technique in nursing studies. Specifically, their study suggested that patients who hesitate to reveal their experiences will likely open up their minds when using the vignette technique. Patients are more willing to share using the vignette approach as it allows a space between their own experience and the given scenario. The vignette method is useful for examining sensitive topics involving vulnerable groups. It can uncover the meanings, assumptions, professional opinions, and social judgments on socio-political issues (Barter & Renold, 2000; Keddell & Hyslop, 2018). Aujla (2020) uncovered the police’s perspective regarding ‘honor-based crimes and forced marriages in Canada by employing the vignette approach. Using the vignette-image method, Khanolainen and Semenova (2020) uncovered school bullying by exploring the students’ tension and dilemma.

The qualitative-vignette method allows researchers to explore issues inductively, which is useful when there is a lack of research on similar topics (Cheah & Koay, 2022). It allows the exploration of the meaning-making process of vulnerable groups and how they explain and derive subjective meanings from the vignette by relating to their lived experiences (Aujla, 2020). It also uncovers the decision-making process, where the participants can link the past, present, and future, allowing the sequential development of issues under the study. The greatest benefits of the qualitative-vignette method are that it investigates events, activities, and choices as they emerge and sequence themselves over time (Van de Ven, 1992), which refines the researcher’s data reporting.

Designing Vignette and Recruiting Respondents

A good vignette should meet three criteria. Firstly, it should mirror the real situation (Bryman et al., 2016; Jenkins et al., 2010). Secondly, the respondents must relate their experience to the given situation so that the researcher can elicit the respondents' judgment (Bryman, 2016). Thirdly, the vignette should contain a third-party form of attitude and behavioral questioning to encourage reflective replies. Such an approach is critical to examining the issues of vulnerable groups, as they may be reluctant to answer when asked openly (Finch, 1987). It corresponds with Hughes' (1998) assertion, where he suggested that the questions attached to the vignette should showcase these entities’ issues, allowing a space between the questions and the respondents’ own experiences. Moreover, it provides a more comfortable ambiance during the interview. Barter and Renold (2000) summarised the benefits of the vignette method by saying that it permits behaviors in context to be examined, uncovers one’s judgments, and offers a less personal and, thus, more comfortable method of discussing sensitive issues.

Researchers may adopt multiple approaches to participant recruitment; They could use the theoretical sampling method proposed by Scott (2013) and Cheah (2021), i.e., to include the participants from these categories: the policymakers, business firms, associations, professions, and the general public. Alternatively, researchers also can zoom in on a target group by employing the purposive sampling method, as found in Aujla’s (2020), Lee and Goh’s (2020), and Sampson and Johannessen’s (2020) studies which only include the vulnerable groups as the research target. To motivate the participants of vulnerable groups, they should be informed that the interview questions are based on a ‘case study’. We should use the less-technical term ‘case study’, instead of ‘vignette’. They should be informed that they only connect their experience to the case study if willing to do so. The researcher can also encourage the participants to identify other potential participants with in-depth knowledge of the issues. i.e., the application of snowballing sampling technique.

To ensure the internal confidentiality of the participants, their background information should be limited to age range and professional group. Wherever there is any chance of identifying the participant, details must be disguised at the researcher’s discretion. A few questions about the sensitive issues can be raised for their feedback when commencing the interview. This is the preliminary step to see the interviewees’ openness to the topic of discussion. For participants who hesitate to answer, the vignettes can be presented to create space between their own experiences and the case. Hereafter, we proposed three cases of adopting vignettes to examine vulnerable groups’ sensitive issues.

The subsequent sections showcase three scenarios of vignette designs that meet the proposed three criteria of designing a vignette. Vignette 1 indicates multiple online news taken from Lee (2014) and McCankey (2014) to describe the aircraft crash and missing flight of Malaysia Airlines in 2014. Section 3.2 describes the adoption of images in gathering insights into social issues. Lastly, section 3.3 shows how images and text could be combined to uncover hidden truths.

Case 1: The adoption of a text-based vignette

The two airline tragedies spark discussions and research in different areas, such as corporate communication strategy (Othman & Yusoff, 2020), emotion in social media (Changsong et al., 2017; Yeo et al., 2020), conspiracy theory (Moulding et al., 2016), and motivation to participate in social network sites following the crisis (Wang, 2016). Using vignette one as a reminder of an event long ago, some projective questions can be included when gathering the data (See Table 1). The researchers can then draw insights into the actors involved through the respondents’ illustrations to match the three criteria needed in a text-based vignette (Vignette 1)

OPEN IN VIEWER

Table 1.

Sample questions related to Vignette 1.

No	Projective Questions	Criteria Met
1	Crisis-Management strategy study Based on the vignette presented. What is the Airline’s strategy for handling the incidents? Follow-up questions: a) Are the airline and the Malaysian government effectively handling the tragedies? b) Who/which party should be involved in the crisis management besides the policymakers and the airline?	Criteria 1: Mirror the actual situation Criteria 3: Third-party form of attitude and behavioral questioning to encourage reflective replies
2	Policy on aircraft Maintenance and control study Based on your experience taking the airline, do you think the aircraft is well maintained? Follow-up questions: a) What should the airline do to minimize the chances of such a tragedy? b) Who should take the biggest responsibility for such incidents. c) What should the policymakers do to minimize the risk?	Criteria 1: Mirror the actual situation Criteria 2: Respondents can relate to the actual situation and can extract the respondents' judgments Criteria 3: Third-party form of attitude and behavioral questioning to encourage reflective replies.

OPEN IN VIEWER

Vignette 1.

Malaysia Airlines’ Tragedies [Source: (McCankey, 2014) and (Lee, 2014)].

.

Gathering and Reporting Diverse Views in the Reort

The vignette’s potential in uncovering the voices of vulnerable groups is apparent as it resolves the general concern around insufficient data. However, the data quality depends on the researchers’' expertise and experience, in essence, how they break the ice and unlock the obstruction to collect the confidential thoughts of the respondents. Hughes and Huby (2004) disclosed that the common challenge that researchers face is that the participants may react to vignettes by providing what they perceive others would give instead of taking the chance to reveal their opinion (Finch, 1987), fearing their identity can be easily revealed even the interview is conducted in one-to-one basis, that is, the internal confidentiality concern (Tolich, 2004). That is because limited outspoken individuals have insight and knowledge of the incidents. Additionally, although the vignette creates a distance between the scenario and the respondents’ self-experience, the researcher may need help analyzing data. Some participants relate themselves to the character, while others answer in a third-party account. Hence, the vignette method may not serve the purpose of the researcher’s stance if the objective is to discover what individuals do in reality (O’Dell et al., 2012). This is practical only when adopting the phenomenological or ethnographical approach by inviting participants who have lived experience of the subject matter. Researchers who adopt these approaches need to differentiate the participants’ own live experiences or the third-party experience by raising follow-up questions such as: “Do you face the issue by yourself?” or “How did you handle the situation when you experienced such an obstacle?“.

However, if the researcher adopts a case study approach, whether the interview data is from a first or a third-party account does not matter. It is because the case study method involves an in-depth, detailed examination of a real-world context (Bromley, 1986). Hence, researchers must collect all-rounded voices (Cheah, 2019; Cheah & Lee, 2022) by cross-examining actors of different categories (industry experts, practitioners, and social interest groups) to enrich the research findings. Researchers can then quote these experts’ opinions when interviewing the participants of vulnerable groups. The latter would be more willing to share after knowing the experts have already recognized the issues. Some of the following questions can be included during the interview with the vulnerable groups:

“During our interview with NGO, we noticed that they have recently voiced out these unfair treatments publicly. Do you think the policymakers have done enough to address this issue?"

The above-suggested methodological approach aligns with Hermans' (2002) thought of self-functions as a ‘society of mind’, where the individual opinion and the collective others emerge as the self-space. As also argued by O’Dell et al. (2012, p. 706), ‘the self in this perspective is a dynamic dialogical structure composed of a multiplicity of internal and external identifications'. The I-position is included in the conversations that reveal the study’s dynamics, including tension, disagreement, boycott, etc. Self-contradictory views can exist in the same manner as in reality in society. The dominance of one position over the other, such as ‘I as a victim’ and ‘I as the victim’s family member’, is not fixed and can change across time and space.

Conclusion, Limitations and Some Reflections

As Sampson and Johannessen (2020) and O’Dell et al. (2012) highlighted, qualitative vignettes permit researchers to quickly build credibility, whereas vignettes amplify their effect as stimulus materials. It grants the researchers temporary membership or insider status (Lave & Wenger, 1991). Moreover, it encourages openness and reduces the likelihood of idealized answers, which is critical in policy studies. Furthermore, it promotes trust in the sensitive research environment where the participants may be suspicious and hesitant. In the context of abusive/unfair treatment of marginalized groups, the victims are likely to reveal their problems, deviant practices, and abnormal behavior and reflect on the proscribed if the vignette method is used (Sampson & Johannessen, 2020). Thus, its contribution to policy research is remarkable.

Although a vignette is a useful projecting technique, the concern lies in ‘over-disclosure.’ From the research ethics perspective, researchers may be accused of getting participants to tell things they do not want to. To address this claim, the participants must be assured that they did not need to relate the scenario to themselves unless they were willing to do so. They should also be informed that they can refuse or stop the conversation whenever they feel uncomfortable. Alternatively, the researcher may initiate a participant-led discussion. For example, the researcher could identify active participants or outspoken individuals from a pilot interview that involved a vulnerable group. Then, invite them to lead the discussion. This will encourage more in-depth sharing as participants feel they are talking to like-minded members. As such, they feel less offensive. The researcher, who is perceived as an ‘outsider’, could play the observer’s role during the discussion. To avoid the claim of ‘over disclosure’, the researcher could exercise member checking by sharing the key findings with the participants before concluding the research (Birt et al., 2016). This step indirectly allows vulnerable group participants to endorse or approve the findings and minimize the ‘over-disclosure’ accusations.