The Suitability and Acceptability of the Think-Aloud Method to Aboriginal and Torres Strait Islander Adults

Abstract

Background

There is a long history of research being conducted on Aboriginal and Torres Strait Islander peoples that has offered questionable benefit but occasioned great distress and distrust. Using research methods that are suitable and acceptable to Aboriginal and Torres Strait Islander peoples is a critically important step towards restoring trust and improving the accessibility and relevancy of research that better addresses the needs of Aboriginal and Torres Strait Islander peoples. The current research aims to qualitatively evaluate Aboriginal and Torres Strait Islander peoples’ perceptions of the suitability and acceptability of engaging in a think-aloud interview embedded in an individual yarn; a think-aloud yarn.

Methods

We employed the think-aloud method as part of the larger What Matters 2Adults study. Aboriginal and Torres Strait Islander participants were engaged in a think-aloud yarn, then immediately following, a follow-up yarn to explore the acceptability of the think-aloud method. The follow-up yarns were audio recorded, transcribed verbatim, and analysed utilising reflexive thematic analysis with an Indigenous epistemological lens.

Results

A total of 17 Aboriginal and Torres Strait Islander adults participated in our study. Participants reported that taking part in the think-aloud yarn was acceptable, not difficult and was less onerous than writing down their thoughts. Engaging participants in a social yarn before the think-aloud research yarn ensured they were comfortable verbalising and assured them that the process was confidential. Thinking out loud gave participants the opportunity to reflect and to think critically about their responses. Some found that by thinking out loud they were able to better understand the statements in their own mind and felt they were therefore able to provide a more authentic response. Because of these benefits, participants conveyed that the think-aloud yarn is a vitally important component in testing items and developing measures for Aboriginal and Torres Strait Islander people.

Conclusions

Overall, our study found that the think-aloud yarn is acceptable to Aboriginal and Torres Strait Islander peoples and therefore a suitable method for use in studies that involve them.

Keywords

methods in qualitative inquiry qualitative evaluation phenomenology focus groups community based research

Introduction

It has been well documented that Indigenous peoples are among the most over-researched populations in the world and are rarely the instigators or leaders of the research conducted on them (Smith, 2021; Tsey, 2001; Smith et al., 2020). Such research may offer more harm than benefit to Indigenous peoples. In Australia, identifying research methods that reframe the colonial-centric research paradigm to one that aligns with Indigenous knowledges and empowers and privileges the voices of Aboriginal and Torres Strait Islander peoples is critical to conducting research by and with Aboriginal and Torres Strait Islander peoples (Smith, 2021; Smith et al., 2020; West et al., 2012; Haynes et al., 2022). Dismantling prevailing colonising research practices is challenging (Smith, 2021; Smith et al., 2020; West et al., 2012; Haynes et al., 2022; Rigney, 2001; Absolon, 2011). Western knowledge traditions have historically prevailed, with an accompanying underlying assumption that valid research is only produced using medicalised and positivist research practices (West et al., 2012; Haynes et al., 2022). This calls for the decolonisation of research whereby the historically Western-dominant research space is critically evaluated and research practices are shifted towards Indigenous epistemologies (Smith, 2021). In order to conduct research that is decolonising, it is important that Indigenous understandings of the interconnectedness of the spiritual, physical, emotional and intellectual aspects of being, are emphasised over Eurocentric worldviews that predominantly see these as disparate and fragmented (Smith et al., 2020; Haynes et al., 2022).

Increasingly, Indigenous researchers are working to reshape research methodologies to ensure that Indigenous knowledges and approaches are recognised, valued and used in research that involves Indigenous peoples (Durie, 2004b; Haynes et al., 2022; Ryder et al., 2020). Indeed, leading Māori academic, Sir Mason Durie, conceptualised this new rise of Indigenous researchers as ‘agents at the interface’ (Durie, 2004a, 2004b, 2005), where the ‘interface’ is the liminal space that sits between both knowledge systems – Western biomedical/bioscience approaches and Indigenous knowledges. Through the development of research approaches that challenge conventions and consider the core values of those involved, research can generate Indigenous knowledges that facilitate new understandings and approaches that address systemic health and wellbeing inequalities (Smith et al., 2020; Haynes et al., 2022; Doyle et al., 2017). This can be achieved through the development of Indigenist research and implementation of Indigenous research methods (e.g., Yarning (Bessarab & Ng'Andu, 2010)). To date little consideration and thought has been given to the use of Western research methods and the appropriateness of these methods in research conducted with Aboriginal and Torres Strait Islander peoples. Indeed, some Western research methods may be suitable and acceptable as they are, or with modification, but we need to assess the appropriateness and alignment of these methods with Indigenous ways of being, knowing and doing.

The think-aloud method has grown in use as an effective health research method (Charters, 2003; Ryan et al., 2009; Boateng et al., 2018) and is one method that has the potential to align with Indigenist methods, such as Yarning, that are currently used in Aboriginal and Torres Strait Islander health research (Ryan et al., 2009; Tonkin et al., 2017; Castelain et al., 2016). Grounded in cognitive psychology, modern think-aloud methods were originally used to understand the relationship of thought and words (Charters, 2003). Think-aloud is a type of cognitive interviewing that is an effective approach to gain access to participants’ higher-level cognitive processes involved in working memory (Charters, 2003). The process of thinking out loud involves participants verbalising thoughts they would ordinarily internalise while performing a set task (Charters, 2003). By asking the participants to speak out loud their internal thoughts and processes, the researcher can gain in-depth understandings of the decisions participants are making while performing the task (Charters, 2003). This is an important outcome of cognitive interviewing whereby the interrogation of the questions/tasks can be examined for whether they are eliciting the information intended by the authors (Boateng et al., 2018). This allows for the questions/tasks to be augmented, clarified or modified, to fit their specific objectives (Charters, 2003; Ryan et al., 2009; Boateng et al., 2018). To our knowledge, the think-aloud method has not been assessed for acceptability with Aboriginal and Torres Strait Islander peoples. Understanding the acceptability of the think-aloud method for use in research conducted by and with Aboriginal and Torres Strait Islander peoples is important in extending and augmenting the suite of appropriate research methods for this population.

The aim of the current research is to qualitatively evaluate Aboriginal and Torres Strait Islander peoples’ perceptions of the suitability and acceptability of engaging in a think-aloud interview, embedded in an individual yarn; a think-aloud yarn, based on the findings of individual yarns conducted immediately after the think-aloud yarn. The findings of the current research will be valuable to those looking to use the think-aloud method with Aboriginal and Torres Strait Islander peoples in a culturally appropriate way.

Methods

Research Team

Acknowledging the backgrounds, perspectives and values that each of the research team bring to the study is important, both in research that involves Aboriginal and Torres Strait Islander peoples, and when using analysis methods that acknowledge the role, we as researchers bring, to our interpretations of the data. The first author (AG) is a proud Pakana woman from Lutruwita (Tasmanian Aboriginal) and an early career researcher with experience in Indigenous peoples’ health and wellbeing research globally. Senior author (GG) is a proud Kamilaroi woman from New South Wales with extensive experience in Aboriginal and Torres Strait Islander health and wellbeing research. KH, KA and AD are all non-Indigenous senior researchers with extensive experience in Aboriginal and Torres Strait Islander health and wellbeing research. Our team also brings experience and expertise in Aboriginal and Torres Strait Islander health research, including qualitative research (AG, GG, KA), wellbeing measurement (AG, GG, KA, KH), outcomes measurement, health economics and preferences (KH), health services research (AG, GG, KA, KH, AD) and epidemiology (AD). All authors are Australian.

Design

Yarning is an important research method that our team has employed successfully with Aboriginal and Torres Strait Islander peoples previously (Garvey et al., 2021; Garvey et al., 2021; Anderson et al., 2021). Yarning is culturally appropriate for Aboriginal and Torres Strait Islander peoples, owing to its alignment with Aboriginal and Torres Strait Islander peoples’ ways of knowing, being and doing (Bessarab & Ng'Andu, 2010). Yarning comprises of sharing, listening, interpreting, re-interpreting and making sense of the knowledges’ that participants provide. It is relaxed and informal, following the natural flow of the participant and how much, or how little, they want to share. Yarning in research, as first legitimised by Bessarab and Ng'Andu (2010), typically includes four different aspects of yarning: the social yarn, the research yarn, the collaborative yarn and the therapeutic yarn, that can be applied in a one-on-one situation (individual yarn) or as a group (yarning circle; yarning workshop; yarning session) (Bessarab & Ng'Andu, 2010). Our study is unique in that it embedded think-aloud interview methods into the ‘research yarn’ aspect of an individual yarn, creating the think-aloud yarn.

This project sat within the larger What Matters 2Adults study (WM2A study) (Howard et al., 2020), which employed Collaborative Yarning Methodology across the entire the project (Garvey et al., 2021; Shay, 2021), consistent with the Yarning research process (Bessarab & Ng'Andu, 2010). This included an extensive and iterative process, whereby three different groups of researchers and stakeholders were involved in the co-analysis and interpretation of the data collected in Phase 1 (Garvey et al., 2021; Garvey et al., 2021), including in the drafting of wellbeing statement for the new WM2A wellbeing measure.

We evaluated the suitability and acceptability of think-aloud interview methods for assessing the face and content validity of the draft statements. The current research reports the findings from the yarns we conducted with the participants immediately following the think-aloud yarn (follow-up yarns) (see Figure 1).

Figure 1.

Flow-diagram showing where the current study fits within the larger WM2A study.

Participants and Recruitment

Individuals who identified as Aboriginal and Torres Strait Islander adults, aged 18 years or older were eligible to participate in the WM2A study (Howard et al., 2020). Participants were identified from participant lists from earlier WM2A phases, investigator and collaborator networks, and from participant networks using a snowball sampling strategy. In qualitative research, snowballing is a highly effective and acceptable method of sampling participants from hard-to-reach populations (Naderifar et al., 2017).

We emailed invitations to participate in a think-aloud yarn to previous WM2A study participants who indicated their interest in participating in future research phases. This email included a participant information sheet outlining what the study was about, as well as a consent form that included consent to record the think-aloud yarn. Those who accepted the invitation were contacted by AG to organise a time that suited them to conduct the think-aloud yarn, to minimise participant burden. Other participants identified through investigator and collaborator networks were recruited using the same process.

Procedure

All think-aloud yarns were conducted face-to-face, co-led by AG and KA, at a time and place that suited the participant. Prior to commencement of the think-aloud yarns, the researchers explained the participant information sheet and participants signed the written consent form. The think-aloud yarns incorporated four components: 1) social yarn, 2) self-complete questionnaire, 3) practice think-aloud question, and 4) think-aloud research yarn. A follow-up yarn was then conducted where the researchers yarned with the participants about their experiences with the think-aloud yarn component. All participants were asked at the time of the yarn if they were still happy for the yarns to be audio recorded; all participants consented.

Firstly, an initial social yarn was held to facilitate genuine and culturally safe engagement and build trust and rapport between the researcher and participant. Secondly, participants self-completed a questionnaire to capture socio-demographic characteristics (e.g., age, sex, relationship status, level of education etc.) and health status (e.g., chronic diseases). Thirdly, an introduction to and practice of the think-aloud method was conducted to familiarise participants with the process. Participants then engaged in a think-aloud research yarn while completing the draft wellbeing measure statements (n = 49). Lastly, we asked participants about their experiences with, and cultural acceptability of, the think-aloud yarn, starting off with an initial broad question, “What did you think about the process of ‘thinking out loud?”. Their response then informed the flow and direction of the follow-up yarn, with the researchers asking specific prompting questions of interest (see Table 1). This paper focusses on the acceptability of think-aloud methods and does not report the findings of the face and content validity of the draft statements for the larger WM2A project for which the think-aloud method was employed (results not yet published).

Table 1.

Prompt Questions for the Follow-Up Yarn About Engaging in the Think-Aloud Yarn.

What did You Think About the Process of ‘Thinking out Loud’?
Prompt questions
- Did you find it easy? Hard? Challenging? In what ways?
- Were the practice exercises helpful in understanding what to do? Do you think we need more?
- Were you comfortable saying everything you were thinking out loud?
- Did speaking the statements out loud help you to understand or answer the question?

Analysis

Theoretical Assumptions

Researchers AG and KA utilised Braun and Clarke’s reflexive thematic analysis (RTA) (Braun & Clarke, 2019), a form of thematic analysis that recognises researchers' active role in the interpretation of the data. We employed multiple coders in our RTA process as this is known to produce a richer interpretation of meaning, through sense-making and bringing multiple perspectives to the analysis (Braun & Clarke, 2019; Byrne, 2022). In order for the reader to have an understanding of how to view our findings, it is important that we first address the underlying theoretical assumptions that we have brought to our interpretations of the data and why (Braun & Clarke, 2019; Byrne, 2022). Firstly, we have approached our interpretations of the data from an Indigenous epistemology (Kovach, 2021). This ‘way of knowing’ somewhat aligns with constructivist epistemology, however, it extends beyond the idea that meaning is “socially produced and reproduced via an interplay of subjective and inter-subjective construction” (p.1396) (Byrne, 2022). Indigenous epistemologies encompass one’s family, community, culture, kinship structures, spirituality and Country (land) (Garvey et al., 2021; Garvey et al., 2021; Kovach, 2021; Butler et al., 2019), as well as the ongoing impacts of colonisation, marginalisation, social inequality and racism (Kovach, 2021; Paradies, 2016). Further, we approached our analysis from an experiential standpoint, although, in recognition of structural determinants of knowledge creation, we also took a critical standpoint to enable inferences to be made about the participants' wider social context (Braun & Clarke, 2019; Byrne, 2022; Kovach, 2021). This dual approach acknowledges experiences of colonisation and aligns well with Aboriginal and Torres Strait Islander peoples’ ways of knowledge creation. The qualitative data was coded inductively by using both semantic and latent coding (Braun & Clarke, 2019; Byrne, 2022).

Analysis Procedure

The follow-up yarns, which explored participants’ experiences of participating in the WM2A think-aloud yarn, were audio recorded, transcribed verbatim, and uploaded to NVivo 10 (Castleberry, 2014) for analysis. AG and KA utilised RTA, as outlined by Braun and Clarke (Braun & Clarke, 2019), which is a reflexive process that typically utilises six phases that are iterative and often overlap. Throughout the RTA process, AG and KA listened to the audio recordings while reading the transcripts to re-familiarise themselves with the nuances of data, taking notes of their analytic observations and insights. Three transcripts were initially coded independently and line-by-line by both AG and KA to form their initial codes. The three transcripts were purposively chosen to try and cover the breadth of data across all the follow-up yarns. Once coded, AG and KA met to discuss their initial codes and explore where their coding converged and diverged. At this point potential themes were discussed and added to NVivo. AG then coded the remaining transcripts under these initial themes and new themes, using an inductive coding process. The coding naturally centred around the follow-up yarning questions about the think-aloud yarns. After coding all transcripts, AG revised the coding to ensure both semantic and latent codes of meaning had been derived. During this process, themes were identified, defined and labelled. From this, AG and KA then met a final time to discuss the coding and themes, to then refine and finalise the themes reported here.

Results

Participant Characteristics

All 17 participants identified as Aboriginal, with a minority who identified as Aboriginal and Torres Strait Islander. Participants ranged in age from 19 to 75 years (mean age 40), half identified as female, half resided in Queensland Australia (with the other half from Western Australia and the Northern Territory), and all spoke English as their main language at home. At the time of the think-aloud yarn, half of the participants reported being partnered and less than half reported having dependants living at home (n = 41%, range 1–7 dependents). The sample were largely educated, employed and healthy in general: most had completed Year 12 (highest level of high school in Australia) and half had obtained a university degree, two-thirds were in paid employment, with most of these participants earning between $25,000–80,000 per year, and two-thirds reported that they did not have any existing chronic disease.

Participant Experience of the Think-Aloud Yarn

Participants were asked to reflect on their experience taking part in the think-aloud yarn. During the follow-up yarns they were prompted with questions related to the ease of participating, the practice question, how comfortable they were thinking out loud and comprehension of the statements (see Table 1). Participant responses on each of these questions are described below.

Usability of the Think-Aloud Yarn for Participants

Most participants spoke about how easy the think-aloud yarn was for them. Some found that verbalising their thoughts came naturally to them, while others needed more prompting. Some also spoke about how being able to verbalise their thoughts while doing the survey was much less onerous on them, which allowed them to provide more insights than they ordinarily would in a typical written survey. Indeed, Aboriginal and Torres Strait Islander peoples adhere to oral traditions of passing on knowledge through story-telling, dance, song and other methods, this may in part explain why the participants found the process of thinking out loud to be easy to use and helpful in facilitating the complexity of what they wanted to express. This aspect of ‘thinking out loud’ that is inherent to the think-aloud method appears to align with this cultural practice.

“Easy. I think it’s a lot easier to read it out loud than to answer the questions [in my head]” (16)

“I don’t like writing lots of words. So, I may not have been able to convey the nuance of what I'm trying to say… I'm much better at saying what I feel than trying to write down what I feel.” (13)

Some spoke about how the practice question at the beginning of the interview helped them to understand quite quickly the think-aloud process. They liked the simplicity of the practice question, and found it helped them to understand how we wanted them to think out loud.

“I didn’t quite understand what [the interviewer] wanted me to do, and then we went through [the practice question] that you gave an example, and it was actually quite helpful.” (17)

Comfort Levels Engaging with the Think-Aloud Yarn

Most participants spoke about feeling comfortable verbalising their thoughts. Some said they think out loud on a regular basis, so it felt natural for them. This also speaks to the inherent nature of the think-aloud method aligning with Indigenous epistemologies that are oral in nature. Others said knowing the interview was confidential made them at ease to verbalise their thoughts, which highlights the imperative to engage in a social yarn with the participant before proceeding with the research, to ensure two-way communication and trust is established. Some explained that the nature of the questions was not confronting so felt at ease to speak freely. Indeed, this speaks to the importance of considering the nature of the statements/questions being asked in a think-aloud yarn to ensure they do not inflict harm.

“So, I'm used to certainly speaking my mind without fear or favour, and it's kind of like well if you are, if you create a forum where I'm allowed to speak my mind, don’t be surprised if I speak my mind.” (13)

“Oh yeah, because I know it’s confidential. I don’t have a problem with it…” (05)

“They’re not too kind of deep and disturbing, that you’ll have to think a lot or not want to kind of divulge too much about yourself.” (01)

Some spoke about how thinking out loud could be confronting for some. Although, they did not find it confronting themselves. This is noteworthy for consideration along with the nature of statements/questions being asked. Ensuring the think-aloud yarn is a culturally safe space for the participant is crucial to ensure no harm is inflicted on participants.

“…it’s not confronting to me, but I can see it could be confronting to other people. It’s saying you have to think about more. It’s talking about things you don’t normally talk about, that you normally internalise.” (04)

“…and then it would depend on who you were speaking aloud with… some people might suffer a bit of shame because their speech, like, with me with writing, I don’t go to training for writing because I can’t write, I’ve got to use voice activated software, and so if somebody had a speech impediment, then I probably could see a problem, but for me, not a problem.” (10)

Other participants who said that they do not normally verbalise their thoughts found the experience of the think-aloud yarn helpful and therapeutic, as they had the space to speak openly about their lives and passions. This is congruent with the collectivist nature of Aboriginal and Torres Strait Islander culture, where individuals feel safe to speak about their concerns with others they trust in their family and community. As one participant stated below “I don’t mind thinking aloud and sharing my thoughts with you on what I think”; the key words of note being ‘with you’. This again reflects on the importance of building a safe and trusting environment through employing a social yarn from the outset.

“It was interesting; it made me feel like I was saying, like speaking my whole life… what it reminded me of is when I talk to a psychologist… It’s the only time you speak out loud like that and open up and think about – maybe that’s bad on my part, the fact that I don’t normally get a chance to speak out loud, but it actually allowed me to – it was a positive thing… it actually allowed me to talk about my issues or problems or thoughts about certain aspects of my life, which I don’t normally talk about, I normally internalise.” (04)

“I’m usually a private person. I don’t like thinking out loud. I like to keep things to myself. But I found it helpful in a way because I’m talking to you about how I feel and 'cause I’m pretty passionate about Indigenous health, I don’t mind thinking aloud and sharing my thoughts with you on what I think.” (11)

Some participants spoke explicitly about the importance of the social yarn and language, with one participant who described themselves as being a ‘nervous person’ highlighted that the social yarn before the think-aloud research yarn set them at ease. Another participant elaborated on this, explaining the social yarn is important for setting the tone of the interview.

“I think it would be easy in front of someone you’re more comfortable with... I think [the Social Yarn] definitely helped me. I’m quite a nervous person…” (17)

“…if I was going to do this survey within a community, then I would make myself known to the community and I’d listen to the way they talk, because wherever you go, us mob will speak differently wherever you go… these are sort of things that I would be taking into account...” (10)

Understanding and Reliability of Responses When Engaging in the Think-Aloud Yarn

Most participants spoke about how verbalising their thoughts improved their comprehension of the statements. This was mostly attributed to being given a safe space and time to verbalise their thoughts with the researchers, and then really think about what the statement was asking and how it was relevant to them. It is possible that by thinking out loud any internalised feelings of judgement, which may exist as an imprint of the Western gaze, were alleviated. The language participants used to describe the release of pressure that the think-aloud yarn affords perhaps points to experiences of oppression that have worked to silence Aboriginal and Torres Strait Islander people when they are within Western Spaces.

“…being able to speak out loud about it and be able to actually understand that there is no time pressure, it’s not an exam, it’s not supposed to be like an exam. It’s a good thing, provided that you have the privacy, the understanding and that surrounding that it’s – you can say whatever you want, you're not going to – nothing bad is going to come. If you don’t understand the question, you don’t understand the question. That’s perfectly legitimate and fine. I think it’s a good thing…” (14)

Some spoke of how verbalising their thoughts in this safe environment not only increased their comprehension of the statements, but also enabled them to think more deeply and critically about the statement and provide a more authentic response. This was bolstered by the method encouraging them to fully think through their decision-making before responding, which they identified as leading to a more reliable response.

“Yeah, I probably wouldn’t have picked up on the ‘family and me’ thing if I didn’t say it out loud, like all those other things, I thought, well maybe they need to split that. Because I’d probably just be like, okay, that’s the – just answer like that. You might not have got a more authentic response or a truer response if I didn’t think it out loud.” (05)

“I like it better than just checking boxes. I think it’s a good way to get a better picture and an understanding of – what am I trying to say.” (06)

Lastly, one participant spoke about how they use this process of thinking out loud in their teaching practice to improve student understanding, especially their Aboriginal and Torres Strait Islander students. Interestingly, they specifically employed this method to enhance the students understanding of the “non-Indigenous way of thinking”. This further strengthens the argument that the think-aloud method when combined with yarning, aligns with Indigenous epistemologies where knowledge is taught, passed on and understood, using oral traditions.

“I am a teacher, and I am a tutor and a mentor so, when I teach my students, what I am teaching them is how to think and what to think about... Nursing is a different world and, to become a nurse, to make them a capable nurse in a Western world, particularly for my Indigenous mob, I try to get them to understand the non-Indigenous way of thinking, the nursing way of thinking. So, I talk it out loud and I explain the psychology behind things, so they understand every aspect of it.” (09)

Practicality of Using Think-Aloud Yarns for Research Purposes from Participant Perspective

When reflecting on their experience doing the think-aloud yarn, some participants identified how practical it was for Aboriginal and Torres Strait Islander people, particularly in ensuring the language used in the statements resonated with their Indigenous ways of knowing, being and doing. This was especially important to the face and content validity of the statements (findings not yet published) that found specific words used in the statements had different meanings to different participants. Indeed, Aboriginal and Torres Strait Islander peoples are not one single group, so using the think-aloud yarn that is a “very practical” tool to ensure “these things are vetted” can ensure the statements/questions are “put in language” that makes sense to Aboriginal and Torres Strait Islander peoples at a national level.

“I personally think it's an awesome idea to have these things vetted. Because I'm tired of doing surveys where you read a question that’s so badly written, it's like who wrote this, and… why didn’t the ethics person at least notice this. It's just so poorly set up. And I've seen ones where you could literally answer it immeasurable ways, because it's so, it's not focused enough. So, it's really good that somebody’s trying to think of a way to focus these things down.” (13)

“Yeah. I liked [the think-aloud yarn]. I like that as a tool, because then you’re doing it live and you’re getting feedback live, so you can make those changes right away. It’s very difficult once you put something out there to bring it back in and change it… I think that’s… very practical” (08)

“… there’s too many ambiguous surveys, you know, they just tick a box and none of them really fit, and for us, for our mob, the language is just an intense thing, [the think-aloud yarn] needs to be done because [the measure] needs to be put in language that… most [Aboriginal and Torres Strait Islander] people understand...” (10)

This was especially true for those who felt they provided richer responses owing to the nature of having to explain their thoughts and being afforded the time and space to have their voice heard. One participant highlighted that if they had not been thinking out loud, they would have raced through the questions without thinking more about how these things could be impacting their life, stating they are an “Aboriginal person” and that they are “just resilient, [in] survival mode”. This highlights again how internalised judgements from the Western gaze has forced Aboriginal and Torres Strait Islander peoples to be silenced in settings where they do not see themselves or their culture valued. The think-aloud yarn gave the participant the space to have their thoughts, feelings and opinions heard and valued, by first establishing a safe space through the social yarn, and then combining the think-aloud method with a research yarn, to enable this safe space to continue across the whole yarn. Further, by employing Dadirri (Ungunmerr-Baumann et al., 2022; West et al., 2012), an Indigenist method that focusses on deep listening, instead of the one-way communication that is characteristic of the standard think-aloud method, researchers were able to show deep respect to the participant that Dadirri affords.

“…sometimes you think as an Aboriginal person, I’m just resilient, survival mode, get through this, just answer the questions. You’re not thinking about that in that moment, but when you’re thinking about it out loud, you’re like, oh yeah, that’s right, that’s happening for me.” (05)

Discussion

The think-aloud yarn was highly acceptable to the Aboriginal and Torres Strait Islander participants in this study. Participants saw great value in verbalising their reflections and considerations as they were responding to the statements, while the researchers sat quietly, listening, and offering encouragement and affirming body language without judgement or influence. Participants found that the method gave them the space and time to fully consider and think about their responses, rather than hastily ticking boxes, which they may do in settings where they do not feel valued.

While cognitive interviewing techniques, like the think-aloud method, offer clear benefit for developing robust measures (Boateng et al., 2018), the value in using this approach in combination with accepted Indigenist methods Yarning and Dadirri, in research with Aboriginal and Torres Strait Islander peoples extends beyond this. Think-aloud yarns are synergetic with Indigenist research approaches and offers benefits in breaking down the barriers to Aboriginal and Torres Strait Islander people being involved in research (Smith et al., 2020; West et al., 2012). Indeed, our participants spoke of how engaging in the think-aloud yarn gave them the space, freedom and safety to speak freely about their thoughts, feelings and opinions, without fear of being judged or persecuted. Thus, evidencing this method’s innate ability to break down the barriers that have placed Aboriginal and Torres Strait Islander peoples in “survival mode” as one participant stated. This was achieved through creating a culturally safe space by purposefully embedding the think-aloud interview into an individual yarn, and employing Dadirri in place of the one-way communication that is characteristic of the currently used think-aloud method.

Yarning is increasingly being used in research that involves Aboriginal and Torres Strait Islander peoples, since first being characterised as a research method by Bessarab and Ng’Andu in 2010 (Bessarab & Ng'Andu, 2010). Yarning respects the oral traditions and values of Aboriginal and Torres Strait Islander peoples, ensuring Indigenous knowledges are privileged in a culturally safe manner (Bessarab & Ng'Andu, 2010; Laycock et al., 2011; Faulkhead & Russell, 2006). The research method of yarning encompasses different types of yarning: social yarning, research topic yarning, therapeutic yarning (when needed) and collaborative yarning (Bessarab & Ng'Andu, 2010). Think-aloud methods can be used in a way that aligns with research topic yarning, once embedded in an individual yarn (the think-aloud yarn) to establish a space of cultural safety for Aboriginal and Torres Strait Islander peoples to comfortably participate in research (Smith et al., 2020). In our study, the think-aloud yarns were co-led by an Aboriginal researcher and a non-Indigenous researcher. As evidenced in other research with Aboriginal and Torres Strait Islander participants (Garvey et al., 2021; Butler et al., 2020), the social yarn is an important element in building trust and fostering relationships between researchers and participants. Engaging in the social yarn provided a culturally safe space in which the think-aloud yarn could occur (Bessarab & Ng'Andu, 2010); multiple participants indicated that the social yarn was an important element of the success of the think-aloud yarn. Indeed, this was imperative to breaking down Western narratives of internalised judgement, allowing them to feel safe to speak freely without fear. Further, while one-way communication is the standard in other think-aloud interviews, researchers in the current study found it appropriate to respond in instances where the participant was sharing their stories. These responses were to demonstrate the researcher was engaged and provided encouragement and acknowledgement to participants of their reflections and ruminations throughout the process. This is congruent with research topic yarning, which we combined with the think-aloud interview; the think-aloud research yarn.

The think-aloud yarn also strongly resonates with the traditional approach of Dadirri. In the language of the Ngangikurungkurr people in the Northern Territory, Dadirri refers to a state of deep listening and quiet awareness (Ungunmerr-Baumann et al., 2022). Dadirri has been aligned with reflective research practices, often used by Aboriginal and Torres Strait Islander academics, in a way that respects the knowledge of participants and empowers both the researchers and the participants (West et al., 2012). This is achieved through Dadirri’s innate way of ensuring the voices of Aboriginal and Torres Strait Islander peoples are valued and heard (West et al., 2012). We found when using the think-aloud method with Aboriginal and Torres Strait Islander participants, they naturally fell into storytelling, and we, as researchers, found ourselves naturally following the Dadirri way of listening. This is an important finding of our research, as feeling personally and culturally unvalued as an Aboriginal and Torres Strait Islander person in previous experiences with researchers, was evident within the participants language throughout. We found that by also employing Dadirri in place of the standard one-way communication of other think-aloud studies, that Dadirri’s innate ability to show deep respect further contributed to creating a safe space for the participants. Indeed, as identified by Rigney, bringing together appropriate Western methodologies and Aboriginal and Torres Strait Islander peoples’ ways of knowing, being, and doing, allows researchers to find effective solutions and to break down oppression (Rigney, 2001).

The importance of using research methods that are suitable and acceptable for use by and with Aboriginal and Torres Strait Islander peoples is critical in fostering research that empowers Aboriginal and Torres Strait Islander people and communities, rather than subjugating them. Dominant and accepted forms of medicalised and positivist research often dismiss or negate Indigenous ways of knowing, being and doing, through the application of a ‘settler’ lens (Smith et al., 2020; West et al., 2012). These persistent forms of research only seek to further colonise Aboriginal and Torres Strait Islander peoples, and the outcomes confer little benefit to the individuals and community as a whole (Smith, 2021; Tsey, 2001; Smith et al., 2020; West et al., 2012). In order to fuel methodological reform in this space, researchers need to value and use methods that have been identified by Aboriginal and Torres Strait Islander peoples as suitable and acceptable to them (Smith et al., 2020).

The small sample size and demographics (largely educated, employed and healthy) of our study may limit the generalisability of its findings, and we recommend more studies employ think-aloud yarns to further explore the acceptability and usability of this method with Aboriginal and Torres Strait Islander peoples in larger samples and different contexts. Participants in this study alluded to the content of the statements themselves, used in the think-aloud yarn, as impacting on their comfort levels when verbalising their thoughts. This may have been due to the strengths-based nature of the draft statements being evaluated for the WM2A study and their grounding in the aspects of life that are important to Aboriginal and Torres Strait Islander peoples (Howard et al., 2020). Thus, we recommend researchers carefully consider the content of the statements/questions that they wish to evaluate before they adopt this method. Indeed, ensuring that no harm befalls participants in any research study is of utmost importance and this is especially true for Aboriginal and Torres Strait Islander peoples who have endured harm at the hand of research practices in the past (Smith, 2021; Tsey, 2001; Smith et al., 2020; Hugman et al., 2011). It is pertinent therefore that researchers who intend to use think-aloud yarns with Aboriginal and Torres Strait Islander peoples first consider any potential risks to participants and ensure all aspects of the research is culturally safe. An important strength of our study was the strong Aboriginal and Torres Strait Islander governance and engagement throughout, the grounding in Indigenist methodologies, and the co-leading of the think-aloud yarns by an Aboriginal researcher, ensuring our findings are aligned with Indigenous ways of knowing, being and doing.

Conclusions

Our study confirms that think-aloud yarns are acceptable to Aboriginal and Torres Strait Islander peoples and therefore a suitable method when engaging in research that involves them. The think-aloud method was found to coalesce well with the Indigenist methods of Yarning and Dadirri (West et al., 2012; Bessarab & Ng'Andu, 2010). Combing think-aloud methods with Yarning and Dadirri to create the think-aloud yarn, worked to prevent participants feeling unvalued and unheard, and alleviated the internalised racism and judgements of the Western gaze. Continuing to build on the suite of available research methods that have shown to be suitable and acceptable for use by and with Aboriginal and Torres Strait Islander peoples is critically important in making research accessible and relevant to Aboriginal and Torres Strait Islander peoples to better address their needs.

Footnotes

Author Contributions

Alana Gall: Conceptualisation, Methodology, Formal analysis, Investigation, Data Curation, Writing – Original Draft, Writing – Review and Editing, Project administration. Kate Anderson: Conceptualisation, Methodology, Formal analysis, Investigation, Writing – Review and Editing, Supervision. Kirsten Howard: Conceptualisation, Methodology, Writing – Review and Editing, Supervision. Abbey Diaz: Writing – Review and Editing, Supervision. Gail Garvey: Conceptualisation, Methodology, Formal analysis, Writing – Review and Editing, Supervision. All authors have read and approved the manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The What Matters study is funded by a National Health and Medical Research Council (NHMRC) Project grant (#1125434). This study was also supported by the NHMRC funded Centre of Research Excellence in Targeted Approaches To Improve Cancer Services for Indigenous Australian Australians (TACTICS; #1153027). A.G. was supported by a NHMRC Postgraduate Scholarship (APP1168150) and a TACTICS Postgraduate Scholarship top-up. K.A. salary was supported by the NHMRC funded What Matters study (#1125434). A.D. was funded via the NHMRC Centre of Research Excellence in Targeted Approaches To Improve Cancer Services for Aboriginal and Torres Strait Islander Australians (TACTICS; #1153027) and a University of Queensland Faculty of Medicine Fellowship. G.G. salary was supported by an NHMRC Investigator Grant (#1176651). We acknowledge that this research project was initiated at and supported by the Menzies School of Health Research. The views expressed in this publication are those of the authors and do not necessarily reflect the views of the funding agencies.

Ethical Approval

Ethics approvals were obtained from the Human Research Ethics Committee of the Northern Territory Department of Health and Menzies School of Health Research (Ref: 2019–3333); University of Sydney Human Research Ethics Committee (Ref: 2019/672), and UTS Human Research Ethics Committee (Ref: ETH194460).

ORCID iD

Alana Gall

References

Absolon

K. E.

(2011). Kaandossiwin: How we come to know. Fernwood Publishing.

Anderson

Gall

Butler

Arley

Howard

Cass

Garvey

(2021). Using web conferencing to engage aboriginal and Torres Strait Islander young people in research: A feasibility study. BMC Medical Research Methodology, 21(1), 172. https://doi.org/10.1186/s12874-021-01366-y

Bessarab

Ng'Andu

(2010). Yarning about yarning as a legitimate method in Indigenous research. International Journal of Critical Indigenous Studies, 3(1), 37–50. https://doi.org/10.5204/ijcis.v3i1.57

Boateng

G. O.

Neilands

T. B.

Frongillo

E. A.

Melgar-Quiñonez

H. R.

Young

S. L.

(2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health, 6, 149–149. https://doi.org/10.3389/fpubh.2018.00149

Braun

Clarke

(2019). Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health, 11(4), 589–597. https://doi.org/10.1080/2159676x.2019.1628806

Butler

T. L.

Anderson

Condon

J. R.

Garvey

Brotherton

J. M. L.

Cunningham

Tong

Moore

S. P.

Maher

C. M.

Mein

J. K.

Warren

E. F.

Whop

L. J.

(2020). Indigenous Australian women's experiences of participation in cervical screening. PLOS ONE, 15(6), Article e0234536. https://doi.org/10.1371/journal.pone.0234536

Butler

T. L.

Anderson

Garvey

Cunningham

Ratcliffe

Tong

Whop

L. J.

Cass

Dickson

Howard

(2019). Aboriginal and Torres Strait Islander people’s domains of wellbeing: A comprehensive literature review. Social Science and Medicine, 233, 138–157. https://doi.org/10.1016/j.socscimed.2019.06.004

Byrne

(2022). A worked example of Braun and Clarke’s approach to reflexive thematic analysis. Quality and Quantity, 56(3), 1391–1412. https://doi.org/10.1007/s11135-021-01182-y

Castelain

Girotto

Jamet

Mercier

(2016). Evidence for benefits of argumentation in a Mayan indigenous population. Evolution and Human Behavior, 37(5), 337–342. https://doi.org/10.1016/j.evolhumbehav.2016.02.002

10.

Castleberry

(2014). NVivo 10 [software program]. Version 10. QSR International; 2012. American Journal of Pharmaceutical Education.

11.

Charters

(2003). The use of Think-Aloud methods in qualitative research: An introduction to Think-Aloud methods. Brock Education Journal, 12(2), 68–82. https://doi.org/10.26522/brocked.v12i2.38

12.

Doyle

Cleary

Blanchard

Hungerford

(2017). The yerin dilly bag model of indigenist health research. Qualitative Health Research, 27(9), 1288–1301. https://doi.org/10.1177/1049732317700125

13.

Durie

(2004a). Exploring the interface between science and indigenous knowledge. 5th APEC Research and Development Leaders Forum.

14.

Durie

(2004b). Understanding health and illness: Research at the interface between science and indigenous knowledge. International Journal of Epidemiology, 33(5), 1138–1143. https://doi.org/10.1093/ije/dyh250

15.

Durie

(2005). Indigenous knowledge within a global knowledge system. Higher Education Policy, 18(3), 301–312. https://doi.org/10.1057/palgrave.hep.8300092

16.

Faulkhead

Russell

(2006). What is Australian indigenous oral history? In Proceedings of the international oral history association conference. 12–16 july 2006, University of Technology.

17.

Garvey

Anderson

Gall

Butler

T. L.

Cunningham

Whop

L. J.

Dickson

Ratcliffe

Cass

Tong

Arley

Howard

(2021). What Matters 2 adults (WM2Adults): Understanding the foundations of aboriginal and Torres Strait Islander wellbeing. International Journal of Environmental Research and Public Health, 18(12), 6193. https://doi.org/10.3390/ijerph18126193

18.

Garvey

Anderson

Gall

Butler

T. L.

Whop

L. J.

Arley

Cunningham

Dickson

Cass

Ratcliffe

Tong

Howard

(2021). The fabric of aboriginal and Torres Strait Islander wellbeing: A conceptual model. International Journal of Environmental Research and Public Health, 18(15), 7745. https://doi.org/10.3390/ijerph18157745

19.

Haynes

Marawili

Mitchell

Walker

Katzenellenbogen

Bessarab

(2022). Weaving a mat that we can all sit on": Qualitative research approaches for productive dialogue in the intercultural space. International Journal of Environmental Research and Public Health, 19(6), 3654. https://doi.org/10.3390/ijerph19063654

20.

Howard

Anderson

Cunningham

Cass

Ratcliffe

Whop

L. J.

Dickson

Viney

Mulhern

Tong

Garvey

(2020). What Matters 2 adults: A study protocol to develop a new preference-based wellbeing measure with aboriginal and Torres Strait Islander adults (WM2Adults). BMC Public Health, 20(1), 1739. https://doi.org/10.1186/s12889-020-09821-z

21.

Hugman

Pittaway

Bartolomei

(2011). When ‘do No harm’ is not enough: The ethics of research with refugees and other vulnerable groups. British Journal of Social Work, 41(7), 1271–1287. https://doi.org/10.1093/bjsw/bcr013

22.

Kovach

(2021). Indigenous methodologies: Characteristics, conversations, and contexts (2nd ed.). University of Toronto press.

23.

Laycock

A. F.

Walker

Harrison

Brands

(2011). Researching indigenous health: A practical guide for researchers (p. 279). The Lowitja Institute.

24.

Naderifar

Goli

Ghaljaie

(2017). Snowball sampling: A purposeful method of sampling in qualitative research. Strides in Development of Medical Education, 14(3). https://doi.org/10.5812/sdme.67670

25.

Paradies

(2016). Colonisation, racism and indigenous health. Journal of Population Research, 33(1), 83–96. https://doi.org/10.1007/s12546-016-9159-y

26.

Rigney

L.-I.

(2001). A first perspective of Indigenous Australian participation in science: Framing Indigenous research towards Indigenous Australian intellectual sovereignty. Aboriginal Research Institute, University of South Australia Adelaide.

27.

Ryan

Watson

Entwistle

(2009). Rationalising the ‘irrational’: A think aloud study of discrete choice experiment responses. Health Economics, 18(3), 321–336. https://doi.org/10.1002/hec.1369

28.

Ryder

Mackean

Coombs

Williams

Hunter

Holland

A. J. A.

Ivers

R. Q.

(2020). Indigenous research methodology–weaving a research interface. International Journal of Social Research Methodology, 23(3), 255–267. https://doi.org/10.1080/13645579.2019.1669923

29.

Shay

(2021). Extending the yarning yarn: Collaborative Yarning Methodology for ethical Indigenist education research. The Australian Journal of Indigenous Education, 50(1), 62–70. https://doi.org/10.1017/jie.2018.25

30.

Smith

L. T.

(2021). Decolonizing methodologies: Research and indigenous peoples (3rd ed.). Zed Books Ltd.

31.

Smith

R. L.

Devine

Preston

(2020). Recommended methodologies to determine Australian indigenous community members’ perceptions of their health needs: A literature review. Australian Journal of Primary Health, 26(2), 95–103. https://doi.org/10.1071/PY19078

32.

Tonkin

Jeffs

Wycherley

T. P.

Maher

Smith

Hart

Cubillo

Brimblecombe

(2017). A smartphone app to reduce sugar-sweetened beverage consumption among young adults in Australian remote indigenous communities: Design, formative evaluation and user-testing. JMIR mHealth and uHealth, 5(12), Article e192. https://doi.org/10.2196/mhealth.8651

33.

Tsey

(2001). Making research more relevant to the needs and aspirations of indigenous Australians: The importance of research capacity development. Aboriginal and Islander Health Worker Journal, 25(1), 19–24.

34.

Ungunmerr-Baumann

M.-R.

Groom

R. A.

Schuberg

E. L.

Atkinson

(Carlie) Atkinson

Wallace

Morris

(2022). Dadirri: an Indigenous place-based research methodology. AlterNative: An International Journal of Indigenous Peoples, 18(1), 94–103. https://doi.org/10.1177/11771801221085353

35.

West

Stewart

Foster

Usher

(2012). Through a critical lens:indigenist research and the Dadirri method. Qualitative Health Research, 22(11), 1582–1590. https://doi.org/10.1177/1049732312457596