Reconceptualizing the Member Check Interview

Validity

There have been numerous attempts to create, modify, and synthesize criteria for validity in qualitative research (e.g., Creswell & Miller, 2000; Hammersley & Atkinson, 1995; Lather, 1995; Lincoln & Guba, 1985; Parker, 2004; Stiles, 1993; Whitmore, Chase, & Mandle, 2001). Indeed, Morse, Barrett, Mayan, Olson, and Spiers (2002) lamented the “plethora of terms and criteria introduced for minute variations and situations in which rigor could be applied” (p. 15). The sheer number of proposed validity criteria is enough to leave a seasoned researcher exasperated and a novice researcher dizzy. Cho and Trent (2006) divided this “plethora of terms and criteria” into two primary categories: transactional and transformational validity. The category of transactional validity is defined as “an interactive process between the researcher, the researched, and the collected data that is aimed at achieving a relatively higher level of accuracy and consensus by means of revisiting facts, feelings, experiences, and values or beliefs collected and interpreted” (Cho & Trent, 2006, p. 321). Transactional validity serves as an umbrella term for criteria that do not abandon truth collection as a worthy goal of research. This is not, however, a naïve realist position—truth is not seen as an objective account of social reality, but rather as a coherent understanding of a participant's perception of reality.

The other form, transformative validity, is defined by Cho and Trent (2006) as “a progressive, emancipatory process leading toward social change that is to be achieved by the research endeavor itself” (pp. 321–322). This is similar to Kvale's (1995) notion of “pragmatic validity,” in which one way to determine an interpretation's truthfulness is by its utility. If the interpretation has proved useful to participants in the sense that they are mobilized to action, then there is evidence of transformational validity. Valid research will change rather than mirror truth. Transactional and transformational validity may seem theoretically dichotomous to one another, but they do not necessarily work in opposition. Cho and Trent (2006) have argued that techniques from both of these approaches could be used to gain a holistic view of validity in qualitative research. In this holistic approach, validity is a fluid process that eschews dichotomies such as practical/emancipatory and transactional/transformational. They have argued that no single validation technique can ensure validity. Researchers should be clear about their specific approach to validity, and then consumers of the research can judge the adequacy of these techniques for themselves. Vague and abstract references to various validation strategies will not suffice.

By the end of their article, Cho and Trent (2006) remained sceptical about the ability to capture an objective snapshot of a participant's subjective reality. In many forms of qualitative research, the interview is seen as the time when a researcher should actively solicit a participant's unadulterated viewpoint. From a transactional point of view, researchers should ensure that they remain dispassionate and utilize probes in order to access information (e.g., thoughts, feelings, and memories) that already exists inside a participant's consciousness. From a transformational position, the interview does not capture subjective truth—it assists in the creation of truth. Truth is contingent not only on each participant's subjective viewpoint, but also on each interaction. Thus, since truth representation is not only impossible but also undesirable, the goal of research is not to achieve representation, but rather to change problematic social conditions, institutions, thoughts, behaviours, and so forth. Any attempt to capture subjective truth is folly, because the mere act of asking questions will change truth.

The Member Check

The member check, also known as member validation (e.g., Seale, 1999), can be described as a research phase during which “the provisional report (case) is taken back to the site and subjected to the scrutiny of the persons who provided information” (Lincoln & Guba, 1985, p. 236). During this process, the “persons who provided information” are able to determine if the researcher has accurately reported their stories. Although the member check can be conducted in a number of ways, I will focus on the member check interview in this article. In this process, the participant is provided with relevant sections of a research report and is invited to comment on the accuracy of the report. Typically, the focus is on the content of the participant's experiences, emotions, and thoughts (e.g., what happened, how did she feel, what was he thinking?), although participants can also be asked to comment on the analysis. Lincoln and Guba (1985) described the member check as an optimal means of assessing the validity of a qualitative study because the first step of many qualitative projects is to accurately understand the participant's worldview. Cho and Trent (2006) have argued that the member check could be used when the purpose of a study is to seek truth, and also when the purpose of a study is to produce change. They differentiate between different types of member checks (technical, ongoing, and reflexive), and they argue that the specific type should be appropriate for the research question. Thus, the member check can span the transactional/transformational divide.

Despite the potential utility of the member check, it has been problematized on multiple levels. These problems are glaring if the member check is seen only as a criterion of transactional validity. First, I will present a brief summary of the difficulties faced by researchers who have utilized the member check process. Bloor (1997) described the discomfort experienced by a researcher when a participant endorsed the researcher's interpretation, but on further questioning it became clear that this participant had missed the researcher's central argument and instead expressed enthusiastic endorsement for a very minor point. Buchbinder (2011) interviewed other qualitative researchers about their experiences with conducting member check interviews and found that the researchers struggled with boundaries and power dynamics in the research relationship. For example, Buchbinder's participants reported that they were uncomfortable with the shift from receiving information to providing interpretation, and felt that the research relationship became akin to therapy at this point.

Researchers have also received negative responses to presenting interview transcripts without analysis because participants are often surprised by their false starts and use of fillers (see Forbat & Henderson, 2005, for a summary of multiple problems with providing participants with direct transcripts). Thus, even when participants are presented with the raw data, they can still feel discomfort and distance from their own words. These examples suggest that the member check process can be uncomfortable for the researcher and the participant. In addition, any researcher who hopes to conduct a member check with the sole purpose of confirming or disconfirming the truthfulness of statements will be easily frustrated. Bloor (1997) explained that participants and researchers have different goals when they are part of a research project, and this might impact the member check process. Although it might be the researcher's goal to focus on the accuracy of responses, participants in research projects might have other goals such as gaining sympathy from the researcher, protecting themselves (or others), rationalizing their behaviour, or many other possibilities. Doyle (2007) found that her participants used the member check interview to discuss many different topics, and she interpreted this positively as an indication of their high level of comfort with the research process.

In these examples, the member check had been utilized as a means of verifying the accuracy of a participant's words, but it has also been used as means of equalizing power relationships within the research relationship by enlisting participants as members of the research team. The power dynamics can be shifted by allowing the participants to have partial control over their represented selves. Fine et al. (2003) took this step even further by utilizing their participants as co-researchers and listing them as authors on their publications. Participants can be seen as functioning as the researcher's conscience to assist with researcher reflexivity. By being told that they have told the story incorrectly, researchers are given the opportunity to reflect on their own biases and other sources of misinterpretation.

In this vein, Cho and Trent (2006) have argued that the member check is a reflexive process; however, they primarily focused on the reflexivity of the researcher and did not address issues with representation. They write, “Reflexive member checking seeks to illuminate a better representation of the lived experience [emphasis added] of the participants being studied…. the researcher should openly express how his or her own subjectivity [emphasis added] has progressively been challenged and thus transformed” (p. 332). In this passage, the suggestion is that the researcher's subjectivity will be changed throughout the research process, but the research participant has a fairly fixed “lived experience.” If truth is partially created during the interview and analysis processes, then it should not be assumed that an unchanging copy of this truth should be lodged in the participant's subjectivity. Josselson (2011) framed this issue as a question of ownership. When a participant, who was also a qualitative researcher, became involved in the interpretative process, Josselson insisted that the participant remain an additional interpreter of the text and not the owner of the final narrative. He asserted a distinction between the participant's subjective experiences and the interview text.

This does not mean that member checks are of little use for postmodern research; rather, I propose that the member check is an ideal way to span the transactional/transformational divide. Nevertheless, the member check needs to be reinterpreted in order to adequately address postmodern insights regarding the research process. In other words, the member check can be a reflexive process for both the researcher and the participants. This can be clarified by focusing on one specific type of transformational validity, that is, catalytic validity, or the ability of the project to serve as a catalyst for its participants. For Lather (1986), in order to achieve a high degree of catalytic validity, the research project must assist participants in “knowing reality in order to better transform it” (p. 67). Thus, validity is not measured by a study's correspondence to an approximated objective or subjective reality, but rather by the impact of the research on the participant. Simply, if a research project has empowered the participants to action, then the participants were able to know reality enough to engage and change it.

I agree with Cho and Trent (2006) that these forms of validity are not necessarily held in opposition. In the following sections, I present an example of the member check used in a way that spans the transactional/transformational divide. These member check interviews were completed in order to aid researcher reflexivity and assess the catalytic validity of the project. I allowed for and expected changes to occur both in the researcher's (my) subjectivity and in the participants' subjectivities. I specifically sought out their responses to participating in the interviews and to reading my write-up. Thus, while the accuracy of interpretation was not entirely dismissed, the focus of this project was on the impact of interpretation.

Rationale

The idea to systematically incorporate the member check as a form of data collection arose out of a prior research project (see Koelsch, 2008). When I first utilized a member check, it had taken over a year to complete data collection and the initial analysis. Fortunately, I was still able to locate the majority of my participants and receive their feedback on my initial write-up. For this prior study, my goal was to achieve transactional validity; in other words, I wanted to know if my participants agreed that I had told their stories correctly. As a feminist researcher, I also wanted to empower the participants to read and comment on my interpretations. I was not quite sure what I would do if we drastically disagreed, but I was prepared to face that situation if it presented itself.

A year is a long time, especially for my college-aged participants. I did not take this into account at the time, but was alerted to this by one of my participants, Kira (all names of participants in this article are pseudonyms). Kira reported, “unfamiliarity” with her voice as re-presented throughout my analysis. She could not recall actually having said these things to me, and was surprised to see her experience presented in this way. However, Kira trusted me as a researcher—she was not questioning my transcription or suggesting that I had manipulated her words; rather, she was noting how much she had changed. I imagine the situation is akin to finding an old high school diary and feeling alienated from the writer. Of course, there must have been some level of recognition. Kira was able to imagine having stated these things a year ago; they were not so inconsistent as to arouse suspicion regarding my credibility as a researcher. Goldblatt, Karnieli-Miller, and Neumann (2011) encountered a similar phenomenon in their research. They described a situation in which a participant noted that the transcripts no longer reflected her experience. The authors write, “Which of the two perspectives was the ‘truth,’ reflecting the experience during that time? Which should the researcher use?” (p. 392). They presented this as a challenge for researchers, which indeed it is. But, the assumption here is that the research participant's view should not have changed between the time of their participation in the original interview and the time of the member check, and furthermore it was a complication that the participant's view had changed. Why would this be the assumption?

Method

With this prior study in mind, I designed a follow-up study that included a member check, but the member check was not included with the sole purpose of verifying information. Rather, I wanted to ask participants about their reactions to the write-up. This was not the only purpose of the study, but it was purposefully included based on my prior research experiences. Using semi-structured interviews, I met with participants up to two times in order to gather my initial data. I used The Listening Guide (Gilligan, Spencer, Weinberg, & Bertsch, 2003) to analyze the data, and I completed separate write-ups for each participant. After distributing the write-ups to participants, I met with each one for a member check interview. Following this, participants were invited to participate in a focus group. All stages of this project were approved by the university's institutional review board, and the focus group and interviews were audio recorded.

Prior to the member check interviews, participants were instructed to read my write-up in order to check it for accuracy. They were informed that the write-up could be altered as a result of the responses from the member check interview. The member check interviews were semi-structured and began with a general inquiry into the accuracy of the write-up. Specifically, I asked if it seemed that I had portrayed their stories accurately. Next, I asked if they felt that the write-up treated them fairly and respectfully. Then, I asked if they had any comments or objections to my interpretations or any other part of the write-up. Finally, I asked if they had anything else they would like to tell me.

The focus of this project was young women's heterosexual experiences, particularly those that were nonconsensual, confusing, or otherwise problematic. During the initial semi-structured interviews, I asked women to describe these sexual experiences. The participants described experiences ranging from regretted kisses to nonconsensual intercourse while intoxicated. During the interviews, I asked each participant if she would label her experience as sexual assault or rape and the reasons why she would or would not. For the purposes of this article, I will describe reactions during the member check interview and only include content when it is relevant. Specifically, I will share interview data related to the catalytic validity of the project. When relevant, data collected during the member check interview was included in any write-up or presentation of this project.

The Member Check Interview

The analysis was presented as an individual case write-up, so each participant received her own information. I had disguised personal details and occasionally edited participant quotations if they were deemed unreadable due to multiple false starts or excessive use of fillers, but otherwise I had not altered the participant's words. Each participant had an individualized response to the research project during the member check interview. Some participants responded specifically to the write-up and others responded broadly to their participation in the research project. Because participants often responded not only to the write-up but also to their general participation, in this article I also provide a brief summary of any general reaction to the project. I interviewed a total of five college-aged women in their freshman year at a midwestern university, and each response is presented separately below.

Jodi was an outspoken young woman who seemed to ask as many questions as she answered during the interviews. In general, she stated that she liked the write-up and shared it with her parents. During the member check interview, Jodi explained that she had recently spoken with a friend who stated that 1 in 5 women in her (the friend's) sorority disclosed an unwanted sexual experience while intoxicated, which they did not report to the authorities. Jodi said that she reflected on her participation in my study during this conversation. During the member check interview, Jodi also commented on the research process. She described my write-up as “different” and stated “if I ever go into research I want to do something fun like this.” She also reported that she was able to discern empathy in my write-up of her behaviour. She stated, “what if you were like ‘well these girls are pathetic’ well then you'd see it … even if you were trying to be neutral … it would come out in your research.” By reading the write-up, Jodi was able to see her situation presented without perceived negative judgment. Her disbelief in researcher neutrality suggested that she anticipated some type of judgment regarding her sexual behaviour and perhaps of college women's sexual behaviour in general. My write-up also provided a venue with which to share her experiences with her parents. In summary, Jodi's responses during the member check interview suggested that her participation in the study provided her with a sense of validation from both me and her parents, and an increased sensitization toward sexual violence against women.

Trish reported that the write-up was “interesting” and that the text “clicked” when she read the conclusion. During the initial interviews, Trish described a situation in which she passed out due to alcohol intoxication and awoke to a male acquaintance having sex with her. While initially discussing this event, Trish focused on her prior attraction to this young man, her choice to become intoxicated, and her decision to remain sexually involved with him following the incident. During the member-check she stated, “After reading that, it was really shady of him [the man involved] to do and really inappropriate, like I was obviously passed out drunk … he doesn't get dibs on me just because he bought me all the alcohol.” By reading my write-up, Trish was able to reflect on her situation from the man's perspective. She was able to see that even though she perhaps did not make the healthiest or safest choices, the young man was an active agent as well, one who chose to have intercourse with an unconscious woman. During the initial interview, Trish told the story from her own subjective viewpoint, with a focus on her own perceived agency. By seeing the story presented on paper, abstracted from the interview data, she was able to see the incident from a third person perspective, with her and the male involved as “characters” in a larger story. In summary, Trish was able to move from a position of exclusive self-blame to one in which the man assumed some responsibility for the situation.

Despite my reassurances that I was not specifically seeking stories about rape or sexual assault, Madison remained apologetic and unsure about her story. During our final interview, she reported that she felt that she was “wasting my time” because she was not providing me with “hard data.” Madison worried that her experience of an unwanted and regretted kiss did not fit the focus of my study. After reading my write-up, she stated that if other women read it they would agree, “yeah that is kinda a problem,” and she described her experience of an unwanted kiss as an experience that many women might have encountered. Madison was able to see how her story fit into a larger societal discourse of heterosexual relationships and problematic sexual experiences. In a similar manner to Jodi, she also commented on the nature of the research: “I've read so many psychology experiences and things like that and I'm used to just numbers being thrown at me but I think you approached [it] in a really interesting way.” Madison appreciated the qualitative nature of the project. Perhaps her initial concern regarding the importance of her story was related to the idea that it would not have been “counted” in a quantitative study. In summary, by seeing the write-up, Madison was able to see how her experience was important and informative, even though it was not classified as rape or sexual assault.

During her interview, Caitlyn described an internal phenomenon that fit the description of dissociation. In the write-up, I provided a definition of dissociation and suggested that it fit Caitlyn's experience. When Caitlyn read the write-up and found this term, she said “It kinda made me feel like I wasn't losing my mind. Like whatever happened there was a word for it.” By reading my analysis, Caitlyn found a useful label for an uncomfortable sensation. Instead of potentially “losing her mind,” Caitlyn was able to conceptualize her experience as a natural reaction to a troubling event. In summary, the write-up might have assisted in de-pathologizing Caitlyn's natural reaction.

Stephanie reported that the write-up “brought up a lot of ideas.” She also provided one of the most powerful examples of the transformational validity of the research process. She stated that shortly after our initial interview her friend was sexually assaulted. Stephanie stated that her friend was worried that she was “the only one” and that “no one would believe [her].” Stephanie informed this friend, “there's a psychology study going on, lots of people have been talking about their experiences.” Stephanie further reported that her friend believed “that [sexual assault] doesn't happen here” and through her participation in the study Stephanie was able to respond, “yes it does.” Stephanie concluded by stating that her friend reported the incident to the authorities. By referring to her participation in the study, Stephanie was able to assure her friend that sexual assaults do “happen here” and that people do talk about them. During the study I asked participants about the topic of sexual assault. The act of asking gave participants the message that sexual assault is a phenomenon worthy of study.

In summary, participants described changes that they directly related to their participation in the study. These changes can be broadly divided into three categories: intrapersonal, interpersonal, and political. These categories are presented as a means of conceptualizing and organizing the effects of the research process on these participants, but it is assumed that there is overlap between categories. Trish and Caitlyn provided examples of intrapersonal change; Caitlyn experienced normalization when I applied the word “dissociation” to her internal response and Trish was able to see that she was not the only one to blame for her unwanted sexual experience. Jodi and Stephanie experienced changes in the interpersonal field, because both were able to engage in supportive dialogues with their friends about sexual assault. Jodi was also able to use the study as a means to engage her father in a dialogue about her dating life. Finally, Madison evidenced political change. She was able to see her experience as part of a larger societal problem. These responses were all noted in the member check interview and are part of each woman's evolving view of her own sexual experiences.

Conclusions

Although I cannot conclusively state that the research project produced change in these participants (indeed, from a positivist perspective, threats to internal and external validity abound), these examples have been provided to suggest that member check interviews can be utilized to bridge a gap between transactional and transformational validity. The member check interview has traditionally been used as a means of assessing transactional validity, and it can still be used for these purposes. Had I completely misunderstood the story, the participants could have told me this. However, the utility of the member check interview has been expanded to include transformational validity. In addition to suggesting whether or not I “got it right,” participants were able to reflect on how participation in the study affected their thoughts and/or behaviours. The researcher can utilize, in powerful ways, these portions of the member check interviews, when participants stray from the traditional task of “checking” the data.

Each participant described above was able to both check her story, in order to increase the level of transactional validity (i.e., inform me if she found any errors in her story), and contribute to the transformational validity of the project. After seeing their stories, Trish, Madison, and Caitlyn were able to place their own reactions within a larger societal discourse on heterosexual relationships and the consequences that occur when these interactions become hurtful. After participating in the initial interviews, Jodi and Stephanie engaged in dialogues with other women about sexual assault. It is possible that the process of answering questions about their own experiences allowed them to contribute to these discussions in new and useful ways. These responses can artificially be dichotomized into intrapersonal and interpersonal change, but it is assumed that neither change can happen without the other. Trish's story, in particular, highlights the effect of the analysis on an individual participant. Her subjective experience of the events she described in the interview has likely changed, but that does not make the initial interview data less valid or truthful. Murray (2003) also found that his participants' perceptions changed during the interview process, and interpreted this as a sign of the potential for therapeutic change in qualitative interviews.

In this way, qualitative research interviews can be seen as similar to therapy. While different from a therapy relationship in many fundamental aspects (e.g., the goal, any payment arrangement, and the stated roles of participant and interviewer), the research interview might have a therapeutic impact. In his study of the wives of professional athletes, Ortiz (2001) found that his participants reported positive therapeutic effects as a result of the research interviews. He attributed this to the cathartic process of discussing their inner feelings and gaining self-knowledge, which could be used for personal growth. Similarly, Shamai (2003) has argued that interview participants can benefit from both telling their stories and reflecting on their experiences. Similar to the therapist, the researcher often provides support, care, and empathy (Colbourne & Sque, 2005). Though the role of the researcher is explicitly different from that of a therapist, in some situations this difference can be beneficial (Gale, 1992). For example, some patients expect therapists to “fix” their problems, while interviewers are expected to simply understand. In the latter scenario, the burden is on participants to understand and convey their perspectives. Finally, Stuhlmiller (2001) noted that the very act of asking a participant for a story conveys the message that the individual's story is important and part of a larger social narrative.

Cho and Trent (2006) have suggested that the member check can be used to seek the truth and create social change; however, in both instances the participant's viewpoint is seen as somewhat fixed. Although Cho and Trent allow for the researcher's subjectivity to be challenged and suggest that the participant's viewpoint can be changed by the report, this is not accounted for in the member check interview itself. Thus, although participants might be changed by the research process, Cho and Trent do not suggest how this can be taken into account during the process itself. The participants in my study reported changes in the intrapersonal, interpersonal, and political realms. These changes support the validity of the overall research project. It has been suggested that researchers encourage participants to reflect on their own interpretations of their stories in order to see if these experiences have changed.

This article suggested a method of bridging the transactional/transformational divide (Cho & Trent, 2006). By viewing the member check as a means of assessing change within the researcher's subjectivity and the participant data, many of the problematic features of the member check (e.g., those presented by Buchbinder, 2011) can be seen as strengths. Divergence in opinion regarding who has the correct story (the participant or the researcher) can be viewed in a way other than as a battle of interpretations. While it may never be entirely comfortable as a researcher to share your interpretations with participants, it is important to remember that subjectivities are not static and the act of participating in research may impact each participant. Thus, the member check is a useful tool for both seeking accuracy and assessing change throughout the interview process. Because it can be utilized for both of these goals, the member check interview is a powerful means with which to assess validity, and it is consistent with Cho and Trent's (2006) holistic approach to validity in qualitative research. This article provides an example of the member check interview as used in this manner, and it is hoped that other qualitative researchers will employ it for similar purposes. While the member check may not be sufficient to ensure the validity of a study, if employed carefully it can provide both an assessment of validity and additional data.

As a psychologist, I believe that asking and answering questions can have a powerful and transformational effect. Typically, we imagine this happening in a purposefully therapeutic context, but it is naïve to imagine that this might not also happen in a research context. We have ethics committees and institutional review boards to ensure that minimal or no harm will be brought on research participants, but as qualitative researchers influenced by postmodernism, we do not believe that the interview process is neutral and that information received from participants has been retrieved from a fixed isolated mind. Therefore, it is important to think about the effects of the research interview on participants in creative and constructive ways, even if we are not explicitly conducting research aimed at direct social change. If our projects are important and worthwhile, then we should expect that our participants might be changed by their participation. One way to examine this change is to use a member check interview in which participants are given the space to reflect on their participation in and reaction to the interview process.