Understanding and Reporting Qualitative Research

The Ebb and Flow of Qualitative Research in Rehabilitation Counseling

Interest in the use of qualitative research methods to inform practice in rehabilitation counseling and related fields has increased significantly over the past three decades, as selectively exemplified by an early (Murphy & Salomone, 1983) and a recent (Mizock & Russinova, 2013) research report in Rehabilitation Counseling Bulletin (RCB). These articles exemplify the evolution not only of qualitative research methodology but also of the topical issues and prominent consumer populations of their respective periods in rehabilitation services. Qualitative rehabilitation research has its historical roots in anthropology and sociology. Some of the classical qualitative works in these fields dealt focally with disability (e.g., Edgerton, 1967; Scott, 1969; Zola, 1982) and are the progenitors of today’s qualitative scholarship in disability studies. Among the earliest prototypes of qualitative research (although it was not called that at the time), the work of some prominent shapers of early-to-mid 20th century psychology should be included, notably that of Freud, Piaget, and their less popularized peer, Kurt Lewin. We would argue that the latter’s legacy became the most influential to the field of contemporary rehabilitation counseling/psychology and to today’s concepts and methods of qualitative research.

Lewin’s influence occurred through the significant contributions—via distinct programs of qualitative research—by three of his students: Tamara Dembo, Beatrice Wright, and Roger Barker. All of their qualitative research was theoretically grounded and highly attuned to the perspective of the research participants. One program used participant observation to understand the experience of people newly admitted to a large public mental hospital (Dembo & Hanfmann, 1934, 1935). In terms of the authors’ sensitivity to the experience of the research participants and the effect of environmental forces on the perception of mental illness, parts of these articles are as progressive as those written by today’s psychosocial rehabilitation scholars. The second program of Lewinian-influenced early qualitative research was the series of exploratory semistructured interviews with World War II soldiers in their early adjustment to life with visible injuries (Dembo, Leviton, & Wright, 1956, 1975; Ladieu, Adler, & Dembo, 1948; Ladieu, Hanfmann, & Dembo, 1947; White, Wright, & Dembo, 1948). This program generated important findings and implications for understanding the differences in perspectives and values between the “insider” (the person living with an impactful experience, in this case the disability) and the “outsider” (anyone without that direct experience). Barker’s conceptual contributions to rehabilitation were made early in his career when he directed a research group at Stanford University that produced seminal publications in our field (Barker, 1948; Barker, Wright, & Gonick, 1946). Thereafter, the majority of his career was devoted to developing a genre of qualitative research involving extensive naturalistic observation that he called ecological psychology (Barker, 1968, 1978). Although his own studies using this genre were focused on everyday life in small communities, this methodology has also been applied to rehabilitation settings (e.g., Keith, 1988; Mackey, Ada, Heard, & Adams, 1996).

The modern world of scientific research has been skeptical and slow to recognize the place and value of studies that are based on qualitative research ideology and methodology. In contrast, the postmodern world of multiple realities has embraced qualitative research perspectives and approaches with an open and intrigued mind. Likewise, rehabilitation counseling is moving beyond the “discourse of justification” (i.e., the defense of qualitative research as a legitimate form of inquiry; Devers, 1999, p. 1156) to the contemporary view of qualitative research as a rather misunderstood and underutilized option for creating useful knowledge (Bellini & Rumrill, 2009; Berrios & Lucca, 2006; Chwalisz, Shah, & Hand, 2008; Hagner & Helm, 1994; Hanley-Maxwell, Al Hano, & Skivington, 2007; Koch, Schultz, Kontosh, & Conyers, 2006; Ohman, 2005). As Kline (2003) duly noted, “Quantitative research, because of its fundamental assumptions, is not designed to describe such uniquely human and unquantifiable phenomena as social construction processes, consciousness, and emotional experiences; rather its purpose is to isolate facts, causes, and ‘truth’” (p. 82). Qualitative research facilitates description and analysis of social processes, practices, and phenomena, as well as nuanced understandings of how participants view those processes, practices, and phenomena in the contexts of their lives and social environments.

In the field of rehabilitation counseling, researchers and practitioners have also gradually come to the realization that qualitative research and its philosophical openness to discovery and faithful representation of research participants’ voices are well-matched to understanding the multifaceted needs and complex challenges experienced by its clients (e.g., people with disabilities searching for employment, adolescents with disabilities struggling in schools, parents trying to raise children with disabilities). This appreciation of and willingness to explore and work within (rather than eliminate or control) naturalistic complexity was delineated by McCarthy and Leierer (1999) as one of the seven values and principles that qualitative research and the practice of rehabilitation counseling share. Other shared values and principles noted by these authors include investment in building partnership, respect for both technique and intuition, self-examination, and constructing meaning. What qualitative research uniquely contributes to professional knowledge is a phenomenological understanding of the lived experiences of the research participants, and that perspective can be crucial to addressing social issues and problems more fairly and effectively. In this sense, qualitative research readily lends itself to participatory action research (PAR) strategies and provides a mechanism for people with disabilities to voice their concerns, engage in inquiry, and foster social change. “Participants contribute to making public their understandings and experiences in ways that can lead to powerful changes in their worlds—through both greater public or professional understanding of their lives and even, perhaps, through their own greater interpersonal understandings” (Niesz, Koch, & Rumrill, 2008, p. 120). Qualitative research also provides a means for including the voices of individuals who are unable to complete quantitative instruments in the discourse about the life experiences of people with disabilities.

In considering future directions in rehabilitation research, Bellini and Rumrill (2009) noted that qualitative research will play a vital role in building our profession’s knowledge base. However, they advocated for the application of more rigorous qualitative research methods to investigate phenomena that have not previously been examined, identify variables for theory-building purposes, and better understand the challenges confronting people with disabilities in contemporary society. Along these same lines, journal editors in a variety of applied social science disciplines have pointed to the need for clearer criteria to evaluate the quality of qualitative research manuscripts. These criteria should serve the purpose of “advancing more appropriate and improved use of qualitative methods, creating a greater willingness to fund and publish qualitative research, and developing informed consumers of qualitative research results” (Devers, 1999, p. 1153).

The question of how to evaluate qualitative manuscripts has engendered considerable debate in applied social science disciplines, and consensus has yet to be achieved regarding the identification of a specific set of criteria that could be applied across qualitative research designs, paradigms, and professional disciplines (Devers, 1999; Whittemore, Chase, & Mandle, 2001). Some have argued that searching for a single set of criteria is misguided because it falsely assumes that qualitative inquiry is a unified field rather than one that embraces a diversity of theoretical frameworks and methodological procedures (Cohen & Crabtree, 2008). Others have maintained that “cogent criteria for evaluating qualitative research are needed. Without well-defined agreed upon, and appropriate standards, qualitative research risks being evaluated by quantitative standards, and rejection of more radical methods that do not conform to quantitative criteria” (Cohen & Crabtree, 2008, p. 335). Furthermore, as Devers (1999) noted,

Nothing provides more opportunity to strengthen a discipline than an explicit examination and clarification of the criteria by which research is assessed. Such criteria represent a discipline’s collective perspective on fundamental issues, including the kinds of questions that are worth asking; the ways of answering questions that are legitimate; the appropriate role of the researcher; and the ways in which results should be presented and used. (p. 1154)

The purpose of this article is to propose general guidelines for authors to consider in writing qualitative research manuscripts to be submitted to RCB. The recommended guidelines were derived from a review of the methodological literature and criteria proposed by qualitative researchers and journal editors in rehabilitation counseling and related disciplines (e.g., rehabilitation psychology, counseling, medicine, nursing, occupational therapy). Although qualitative research encompasses a variety of approaches, each with its own standards for what constitutes rigorous inquiry, these guidelines reflect common characteristics of methodologically sound research that are shared by most qualitative approaches. These guidelines are offered with the understanding that as qualitative rehabilitation research continues to evolve, incorporating more innovative designs and methodologies, criteria for evaluating what constitutes quality must also evolve. Because our primary target audience is the person writing a qualitative research report for journal submission, the remainder of this article provides guidelines organized in the sequence of the typical sections of such a manuscript. These guidelines will also be of use to editorial reviewers and readers of qualitative research reports.

Introducing the Qualitative Research Study

The introduction section of a qualitative research report orients the reader to the problem being studied, summarizes the literature related to the research problem, identifies gaps in the literature, states the research purpose, poses the research questions that guided the inquiry, and introduces the research design used in the investigation (Creswell, 2012). In introducing the study, authors of qualitative manuscripts should demonstrate that their “entire research project uses a consistent epistemological perspective” (Kline, 2008, p. 212), and this begins with ensuring that the problem to be studied calls for investigation using a qualitative approach, and that the entire study has coherence across the research purpose, questions, and methods (see also Maxwell, 2005).

Although there are some variations among qualitative approaches in terms of the role and scope of the literature review, there is good consensus that the literature review is of secondary importance to the primary focus on the experiences, views, and meanings of the research participants (Creswell, 2007). While the literature review in quantitative research provides a major direction for the research questions, its primary purpose in qualitative research is to identify gaps in what is known about the phenomenon being investigated and in the methodological procedures used to examine it. In this sense, the literature review sets the stage for establishing how qualitative research will close those gaps (Kline, 2008). For example, Hunt, Milsom, and Matthews (2009) provided a review of the literature on the counseling and mental health needs of lesbians and lesbian couples and then concluded that “despite a comprehensive search, we found only limited research that addressed the experiences of lesbians with physical disabilities” (p. 169). The authors then went on to describe how their qualitative study would address this void:

Given that we were interested in women’s lived experience as lesbians with physical disabilities, we used phenomenological inquiry, namely, because it provides a framework for describing the women’s individual experiences with the phenomenon of interest. Our intention was to have readers gain an understanding of the experiences of the women whom we interviewed—to have them think, “I understand better what it is like for someone to experience that phenomenon.” (Creswell, 1998, p. 46)

Theory also has more varied uses in qualitative research than it does in quantitative research. Whether theory is used as the basis for a conceptual framework (as in the use of critical theory in critical ethnography), as a theoretical end point (as in the development of grounded theory), as a set of epistemological premises or a guiding philosophy of knowledge and meaning (as in some types of phenomenology), or not at all (as in other types of phenomenology), is largely determined by the qualitative approach the researchers use (Creswell, 2007). When existing theories are used in qualitative research, they are used as lenses to promote understanding as opposed to propositions to confirm or deny. Qualitative data are analyzed inductively for what they tell us about participants’ experiences, practices, views, and meanings, and, as such, they should not be made to conform to the preexisting theories. Therefore, introductory sections of qualitative research manuscripts may present theoretical frameworks, but these frameworks should articulate a way of asking research questions or understanding data. They should not serve as hypotheses.

Purpose statements and corresponding research questions in qualitative inquiry can be classified as exploratory, explanatory, descriptive, or emancipatory (Marshall & Rossman, 2011). Exploratory questions are posed when the phenomenon being studied is not well-understood or when researchers want to generate hypotheses for further research. Explanatory questions seek to discern and comprehend patterns related to the phenomenon of interest or identify probable relationships shaping the phenomenon. Perhaps the most common purpose for qualitative research is description; descriptive questions require the documentation and portrayal of social processes and/or the participants’ experiences, views, practices, and meanings as richly as possible. Finally, emancipatory questions are posed when researchers want to create opportunities for research participants to engage in research for social critique and action.

Given the typical purposes of qualitative research, research questions should be broad, open-ended, and asking “how,” “what,” or “why.” Research questions that are closed-ended (“do” questions), oriented toward confirming hypotheses, or focused on understanding the relationships among variables are less appropriate for most qualitative research. Maxwell (2005) refers to this distinction as that between variance questions that “focus on difference and correlation” (p. 74) and call for quantitative research methods, and process questions, which ask about social processes, meanings, and context and call for qualitative research approaches. Qualitative researchers generally limit their research to one to three broad questions. Keeping questions few in number, broad, and open-ended enables researchers to obtain rich, vivid accounts of the research participants’ experiences and guards against leading the participants in a direction that supports the researchers’ preconceptions about the phenomenon being studied. Authors should be consistent in referring to the same research purpose and questions throughout all sections of the manuscript. Conyers (2004) wrote an exemplary purpose statement and research questions in her investigation of the employment concerns of individuals living with HIV and AIDS:

The purpose of this study was to better understand and conceptualize employment issues for persons living with HIV/AIDS at varied stages of considering employment by giving voice to participants from diverse backgrounds who represent the emerging demographics of the illness. The following research questions were explored:

After clearly stating the purpose of the investigation and the associated research questions, authors should then provide a plausible justification for studying the problem through a qualitative research lens and for choosing a specific qualitative approach (Rubel & Villalba, 2009). Qualitative research generally, and specific approaches (e.g., ethnography, phenomenology, grounded theory, etc.) in particular, have purposes and theoretical assumptions that must be aligned with the goals of the particular study. Purpose statements and research questions must be congruent with the qualitative approach used in the inquiry. Some manuscripts submitted to RCB have stated the use of approaches that are inconsistent with the focus of the study. For example, authors have posed a research purpose and questions that focus on developing an understanding of the essence of participants’ lived experiences (i.e., phenomenology) and then, in their data analysis and interpretation, have described the development of a theory grounded in the research participants’ lived experiences (i.e., grounded theory).

Just as quantitative research encompasses a variety of designs and methods, so too does qualitative research. Often, qualitative researchers find that established approaches to qualitative inquiry, such as grounded theory or phenomenology, are inappropriate for their study’s purpose. In these cases, qualitative researchers develop their research designs by drawing on various methods to generate the data to answer their research questions. Denzin and Lincoln (1998) use the French term, bricolage, in describing how qualitative researchers must select the best methods for their study. “The qualitative researcher-as-bricoleur uses the tools of his or her methodological trade, deploying whatever strategies, methods, or empirical materials are at hand” (Denzin & Lincoln, 1998, p. 3). Pragmatic-yet-unnamed approaches to qualitative inquiry are thus often the best way to answer a particular set of research questions.

Caelli, Ray, and Mill (2003) refer to studies that fall outside an established approach as generic qualitative designs, thereby explaining,

From our perspective, generic qualitative studies are those that exhibit some or all of the characteristics of the qualitative endeavor but rather than focusing the study through the lens of a known methodology, they seek to do one of two things: either they combine several methodologies or approaches, or claim no particular methodological viewpoint at all. (p. 2)

The authors go on to note some problems with generic approaches to qualitative research, including a lack of a theoretical position, incongruent methodological assumptions, or incommensurable methods. To avoid these problems in generic research, authors should explicitly address the following: (a) the theoretical positioning of the researcher, (b) the congruence between methodology and methods, (c) the strategies to establish rigor, and (d) the analytic lens through which the data are examined (Caelli et al., 2003, p. 5).

Several authors of qualitative research studies published in RCB have employed generic approaches to understanding the cognitions of a targeted group about a specific experience, social role, or label. Their purpose is to capture what members of the group freely think about the experience or role or label in their own words, unfettered by the preconceived stimulus dimensions or the structured and limited response options established by the researcher. Rather than conduct interviews, many of these researchers have participants write their free responses to open-ended question(s) and give them the time and space to express multifaceted cognitive schema. Koch (2001) mailed a survey to applicants of a public vocational rehabilitation agency. The open-ended survey questions asked (a) what they would like (preference) and (b) what they thought would be (anticipation) with regard to the six aspects of the agency’s services. Martz, Strohmer, Fitzgerald, Daniel, and Arm (2009) asked their participants to list 10 characteristics of four disabilities (i.e., spinal cord injury, mental illness, hearing impairment, and AIDS). McCarthy and Leierer (2001) requested that their group of 40 former clients write up to 10 statements each about the “ideal” and the “minimally qualified” rehabilitation counselor.

The study’s design must be logical, coherent, and aligned with its purposes and methods. Readers and reviewers of RCB are likely to want to know why a generic approach was developed instead of an established approach. Similarly, in generic designs, qualitative researchers may borrow data collection or other methods from grounded theory or phenomenology, for example, but they should make clear to readers why they have done so. In the end, authors must establish that their research design is sound and is the best approach to answer their research questions.

Authors demonstrate to readers (and reviewers) their proficiency in using qualitative methods by identifying the use of a specific or generic qualitative approach coherent with their research purpose and questions, providing a strong rationale for selection of that approach with supportive citations from the authoritative methodological literature, and then using research procedures that are coherent with their selected approach (Kline, 2008). We will again use an excerpt from Conyers’ (2004) article as an example; posing the three research questions listed above, she then introduced her research design as follows:

This study used grounded theory methods (Strauss & Corbin, 1990) to help broaden our theoretical understanding of employment and HIV/AIDS. Qualitative research can help researchers explore phenomena that are not clearly defined (Denzin & Lincoln, 1994). In light of the recent advancements in medical outcomes for many people with HIV, little is known about the specific employment-related issues and concerns faced by this population. Therefore, it is important to draw upon the lived experiences and perceptions of people with HIV to develop a theoretical foundation that will help us better understand and asses this phenomenon. (p. 6)

Gilbride, Stensrud, Vandergoot, and Golden (2003) also demonstrated methodological coherence in introducing the research design they used in their investigation of the characteristics of employers open to hiring workers with disabilities:

A grounded theory qualitative design was chosen for this study. Although some research has been conducted on employer openness, it is in a formative stage and lacks specificity and broad empirical validation. Grounded theory is an effective method for developing a theory from the bottom up; that is rather than beginning with a theory about employer openness and testing it, this strategy allows the theory to emerge from the data. (p. 131)

In summary, introductory sections of qualitative research manuscripts for RCB should show a logical fit among the research problem, literature reviewed, theoretical framework, research purpose, research questions, and the specific approach to qualitative research used in the study. The relationships among these research design components must be clearly visible to readers in the introductory sections of the manuscripts.

Describing the Research Methods

In the research methods section, it is incumbent on authors to demonstrate the consonance of the research methods with the research purpose, questions, selected qualitative approach, and the specific procedures for carrying out the study (Kline, 2008; Maxwell, 2005). An additional criterion is to provide evidence that the authors implemented strategies to faithfully document and represent the lived experiences of the research participants. This criterion is achieved by the authors’ transparency in describing all procedures, using terminology from the methodological literature to support their methods, articulating how their role as the instrument of data collection influenced the research process, and describing strategies they used to ensure the trustworthiness of their findings and interpretations. This goal of transparency requires authors to provide enough detail that readers have a clear understanding of all of the methods used to generate and analyze data, as readers cannot assess the quality, rigor, and trustworthiness of the study if they do not have access to the details of the research processes.

Technical language associated with their particular approach should be used by authors to describe their research procedures and methodological decisions, which should have been derived from one or more respected primary resources on the relevant qualitative approach. Citations to support their methodological decisions should also be provided from primary methodological sources as opposed to secondary sources such as qualitative textbooks. For example, in describing procedures for qualitative case studies, authors may cite sources such as Merriam (1988), Yin (2003), or Stake (1995). Authors using grounded theory approaches would likely cite sources such as Glaser and Strauss (1967), Strauss and Corbin (1990), Corbin and Strauss (2008), or Charmaz (2006) if conducting constructivist grounded theory research. Ethnographers might cite Agar (1996), Hammersley and Atkinson (2007), Spradley (1979, 1980), or many others. Those using a phenomenological approach may cite Colaizzi (1978), Moustakas (1994), Giorgi (2009), or Van Manen (1990). Finally, those using a narrative life history or narrative inquiry approach could cite Chase (2008), Denzin (1989), or Clandinin and Connelly (2004).

In qualitative research, purposeful sampling procedures are used to select and recruit participants or the research site itself (as in ethnographies or some organization-based case studies). When individual participants are selected, it is not for the participants’ representativeness of a larger population but for their personal experiences with the phenomenon being explored. In purposeful sampling, researchers intentionally select participants who are most capable of providing provide rich, thick descriptions of their experiences, practices, and/or their perspectives as related to the phenomenon of interest. In research methodology sections of manuscripts, authors should identify the specific purposeful sampling procedure they used (e.g., maximum variation, typical case sampling, etc.; Patton, 2002), as well as their rationale for using that procedure. If specific inclusion criteria were used to identify potential participants, those criteria should be provided. In terms of number of participants included in their samples, authors should provide authoritative citations from the methodological literature for their decisions regarding sample size; however, Hunt (2011) cautioned that rather than being overly concerned about the number of participants selected for inclusion in the study, authors should focus on (a) providing a strong rationale for how and why participants were selected and (b) demonstrating that they collected detailed and thorough data from the participants.

Descriptive information provided about the research participants can include demographic characteristics (e.g., disability type, gender, race, age, employment status), number of participants who participated in the study, how participants were recruited and contacted, the nature of interactions the researcher had with participants (e.g., how informed consent was obtained, confidentiality issues, amount of time spent in the field with participants, length of interviews, whether they engaged with participants as a participant observer or acted solely as an interviewer or observer), and participants’ relationships with each other if they are members of the same group (i.e., in focus groups and case studies). Authors should also disclose any relational dynamics (e.g., cultural differences, socially determined inequities in power) that may have affected the collection of data (Rubel & Villalba, 2009).

Well-written and detailed descriptions of research methods provide cogent evidence to readers that the researchers spent sufficient time with participants to ensure that they thoroughly and accurately captured the participants’ nuanced experiences with the research topic. How much time did the researchers spend in the field? How many interviews were conducted with each research participant and how long did each last? What forms of observation did the researchers engage in? How many hours of observation were logged? What prior experience did the researchers have with people, processes, and settings similar to those specifically studied in the reported research? These are the types of questions that authors should address to provide evidence of what Lincoln and Guba (1985) refer to as prolonged engagement with the research participants.

Because interviews with the research participants are often the primary source of data used in qualitative rehabilitation research, authors should inform readers whether these were group interviews (e.g., focus groups) or individual interviews; whether they were structured, semistructured, unstructured, or conversational. They should also indicate the approach to or style of interviewing using the methodological literature that informed their decisions. For example, Rubin and Rubin (2012) offer a different approach to qualitative research interviewing (called a conversational partnership model) than does Weiss (1995; see also Kvale & Brinkmann, 2008). In addition, some approaches to qualitative research have particular approaches to interviewing; phenomenological interviews, for example, are different from ethnographic interviews (Spradley, 1979) and grounded theory interviews (Marshall & Rossman, 2011).

When interview protocols are used, authors should provide details regarding how these protocols were developed to “convey their intentionality in creating questions that open up informative conversations rather than direct conversations to predictable ends” (Rubel & Villalba, 2009, p. 301). Were the questions developed by the research participants? Did the questions emerge from the authors’ review of the literature or their personal and professional experiences? Were the questions developed in consultation with experts? In their article reporting research findings from a multiple method, multiple site case study, Lindstrom, Flannery, Benz, Olszewski, and Slovic (2009, p. 192) described the development of interview questions for their semistructured interviews as follows: “Interview questions and topics were developed based on a review of relevant education and rehabilitation literature, our previous work with developing and implementing collaborative service delivery models . . . as well as our specific research questions.” Interview questions should be open-ended and non-leading to avoid steering interviewees in directions that support the researchers’ assumptions about the phenomenon being investigated. Authors can include interview questions in appendices or tables or describe the focus of questions in the methods section narrative so that readers can judge for themselves the fit of the interview questions to the research purpose and qualitative approach.

Although interviews are often the primary source of data collected, any other data (e.g., from observations, documents, audiovisual materials, etc.) collected and analyzed by the researchers should be described as well (Creswell, 2007). In qualitative research, many researchers triangulate their analyses and interpretations by collecting and analyzing data from these alternative sources. Again, we shall use the case study article by Lindstrom et al. (2009, pp. 192–193) as an illustrative example. The authors indicated that, in addition to data collected in semistructured interviews with the participants, “secondary methods and sources included a review of existing program documents (e.g., grant applications, performance reports, and college policies) and a supplemental written survey completed by site personnel to describe changes in service delivery.”

Inductive data analysis is a defining characteristic of qualitative research. It involves answering open-ended research questions by reasoning from parts to a whole, from particulars to generals, or from the individual to the universal. Data collection and analyses are done in successive iterations, with analysis guiding further data collection. In writing about their data analysis approach and procedures, authors must explain how they engaged in systematic and rigorous data-analytic processes to represent thoroughly the participants’ described experiences and to guard against inadvertently focusing on the data that they found most intriguing while neglecting other salient data because they found it less interesting. Rigorous and systematic data analysis is evidenced when authors demonstrate to readers that they adhered to analytic methods associated with the specific approach they used. For example, the analytic processes used in grounded theory research differ fairly significantly from those in phenomenological research. To bolster the credibility of the research findings produced, supportive citations from the authoritative literature on the qualitative approach used should be included in describing methodological procedures and decisions (Kline, 2008). As illustrated in the following excerpt from their article, Hunt et al. (2009) provided a detailed step-by-step description of their data analysis process using terminology from the approach of phenomenology and citations from the authoritative methodological literature on phenomenology:

We continued to review the transcripts as we moved into the horizontalization phase (Moustakas, 1994), where we independently identified significant statements that reflected the participants’ interpretations of their experiences with the phenomenon, deleting any repetitive or nonessential statements (Creswell, 1998; Hein & Austin, 2001). From there we independently began to define themes based on the horizontalized statement, thus creating a structural description (i.e., imaginative variation) of the participants’ experiences (Creswell, 1998; Moustakas, 1994; Polkinghorne, 1989). (p. 169)

Authors should also keep in mind that data analysis does not stop at simple categorization of the data by research question. They must demonstrate that they moved beyond surface observations and explanations to understand the phenomena in as much depth as possible. Transparent, detailed explanations of each step in the data analysis process should be clearly described, and authors should report on (a) how the analysis changed based on the inductive and emergent process and (b) how they determined that data saturation was reached. In addition, if multiple researchers conducted the study, details should be provided regarding how differences in the coding and analysis of the data were resolved between research team members. This information enables readers to judge whether the data analysis process was democratic, ethical, and collaborative.

In qualitative research, the researcher is the instrument of data collection, meaning that, unlike other forms of research in the social sciences in which instruments such as surveys, tests, or scales are used, the researchers themselves collect, analyze, and interpret the data. Because understandings are developed in interaction with the research participants, qualitative researchers must be cognizant of their own influence on all aspects of the research, from their relationships with research participants to how their own thinking and life experiences predispose them to viewing and interpreting qualitative data in certain ways. This awareness is increased when researchers practice reflexivity. Reflexivity refers to critically self-reflecting on one’s biases, theoretical predispositions, and professional and personal orientations to the phenomenon one is studying and how these may influence data collection and analyses. Reflexivity in qualitative research requires (a) an openness to critically examining how one’s own subjectivity influences the research; (b) a willingness to engage in reflexive processing throughout the research process (i.e., keeping a reflexivity journal, writing “identity memos” (Maxwell, 2005), bracketing in phenomenology, etc.); and (c) honesty in reporting in research manuscripts the researchers’ “biases and assumptions that shape or maybe even compete with the purposes of the study” (Rubel & Villalba, 2009, p. 299). In their qualitative article titled The Benefits Trap: Barriers to Employment Experienced by SSA Beneficiaries, Olney and Lyle (2011) included the subheading Researchers as Instrument in their research methods section and illustrated their researcher reflexivity by describing the primary investigator’s perceptions regarding the phenomenon studied as well as her training and experience using a phenomenological framework.

A second example is provided by Hunt et al. (2009). In their phenomenological investigation of partner-related experiences of lesbians with physical disabilities, they explicitly stated who they were in relation to the research topic: “ . . . we are three White women, one who identifies as a lesbian and two who identify as allies (straight women who are affirming of lesbians, gay men, and bisexuals)” (p. 169). They then described the potential biases they brought to their investigation and strategies they used to address their biases:

Because of our professional and personal experiences, we were aware that we had a vested interest in serving as advocates for lesbians and people with disabilities. As such, we discussed possible biases (Moustakas, 1994) at every stage of the research, including when we generated interview questions, before we analyzed the interview transcripts (but after our first read-through) and throughout the data analysis process, to ensure the resulting themes and subthemes truly emerged from the responses. (p. 170)

In this way, the authors demonstrate their reflexivity and relate the reflexive processes that they used throughout the study.

In addition to demonstrating to readers that they were reflexive in collecting, analyzing, and interpreting data, authors of qualitative manuscripts must document that they used quality standards and rigorous procedures “to prevent the disregard for and distortion of the experiences and perspectives of the research participants” (Niesz et al., 2008, p. 119). Authors must keep in mind that evaluation standards in quantitative research (i.e., internal validity, external validity, reliability, and objectivity) do not apply to the evaluation of qualitative research because of their inconsistency with the philosophies underpinning qualitative designs and methodologies (Lincoln & Guba, 1985). Yet, qualitative researchers must demonstrate in their research manuscripts that they employed appropriate mechanisms to ensure that their analyses and interpretations relate sound representations of the participants’ perspectives as opposed to those of the researchers.

Qualitative researchers have given serious consideration to what constitutes rigorous and high-quality qualitative research, engendering a good deal of debate (Whittemore et al., 2001). Indeed, the methodological literature is rife with varied quality criteria, within and across specific approaches to qualitative research. Lincoln and Guba (1985), for example, proposed an oft-cited set of criteria used by authors of qualitative research articles published in RCB that assesses the “trustworthiness” of the interpretations and findings of qualitative research. Lincoln and Guba (1985) suggested that

the basic issue in relation to trustworthiness is simple: How can an inquirer persuade his or her audiences (including self) that the findings of an inquiry are worth paying attention to, worth taking account of? What arguments can be mounted, what criteria invoked, what questions asked, that would be persuasive on this issue? (p. 290)

Strategies proposed to build a study’s trustworthiness are aimed at improving the fidelity of data collection and analyses to the research participants’ practices, perspectives, and understandings. Other authors, such as Morrow (2005) and Rolfe (2006), provide alternate sets of criteria for judging the quality and rigor of qualitative research. Ultimately, it is less important that authors ascribe to a particular set of criteria and more important that authors ascribe to at least one set of criteria, providing an appropriate rationale for the criteria they selected that demonstrates consistency with their research approach (Rubel & Villalba, 2009).

Specific methods, reflective of the criteria used by the researchers to judge the quality and rigor of their research, should then be sufficiently but succinctly described. Among the strategies most frequently referenced in qualitative manuscripts published in RCB are triangulation, member checks, peer debriefing, and negative case analysis. Triangulation is certainly one of the most common strategies that qualitative researchers use to ensure the rigor, thoroughness, and credibility of their findings. Triangulation refers to the use of multiple data sources, data collection strategies, or researchers to provide a check on data, ultimately building a more thorough and robust corpus of data and fostering credibility with readers.

Member checks (also referred to as respondent validation) refer to the process of the researcher’s sharing of interview transcripts and the data analyses with the research participants to generate feedback on the soundness of the analyses and interpretations; provide opportunities for participants to make corrections in data sources or disagree with interpretations; gather additional data; and generate alternative explanations. Involving participants in the data analysis process through the use of strategies such as member checks, “provides an avenue for challenges to researchers’ interpretations and for the generation of data developed at the intersection of the researchers’ and participants’ understandings” (Niesz et al., 2008, p. 119).

Peer debriefing provides a similar feedback loop, thereby testing researchers’ interpretations and seeking alternatives but with outsiders to the study rather than participants. In peer debriefing, qualitative researchers consult with knowledgeable colleagues who are not directly involved in the research about various aspects of the research process such as research procedures and logistics, ethical dilemmas encountered in the field, and, especially, emerging analyses. Peer debriefers can question and critique the researcher’s analyses and interpretations and raise new possibilities in the form of alternative explanations or additional questions to pursue. Qualitative researchers may also leave an audit or decision trail so that others may review a detailed overview of each step in a study. Audit trails recount “the rationale underpinning the research decisions taken en route, and the actual course of the research process rather than the idealized version that the reader is usually presented with” (Rolfe, 2006, p. 309).

Finally, in negative case analysis, the researcher deliberately and systematically searches the data corpus for data that contradict emerging findings. Conducting a negative case analysis ensures that the multiple and varied perspectives of all participants are incorporated into the data analysis process. Taken together, these strategies (negative case analysis, peer debriefing, member checks, and triangulation) all serve to add rigor and promote the researcher’s reflexivity. The use of these strategies should be shared with readers, albeit briefly, to add to the transparency of the study, thereby building credibility.

Strategies specific to the qualitative approach may also be used. For example, in phenomenology, bracketing refers to the researcher’s efforts to understand and then “bracket out” his or her own views on the phenomena under investigation. Some phenomenologists engage in bracketing interviews (i.e., having colleagues interview them just as they will interview their participants), with the goal being to raise awareness of and then “set aside” their biases and prejudgments.

In summary, qualitative manuscripts submitted to RCB should have transparent methods sections that indicate the use of appropriate and well-informed sampling, data collection, and analysis methods. Sufficient information about sampling and data collection procedures enables readers to evaluate for themselves the transferability of the finding to their own settings (Kline, 2008). In addition, manuscripts should make clear how the research methods were characterized by rigor, reflexivity, and the goal of capturing participants’ experiences, practices, and meanings. Finally, because the researcher is the instrument of data collection and analysis in qualitative research, manuscripts should include a brief description of what the researcher has done to ensure rigor and quality. This suggests an orientation to the fidelity of data and analyses to participants’ lived experiences.

Reporting and Discussing the Research Findings

In reporting the research findings, qualitative researchers strive to demonstrate precision and depth in documenting and representing the participants’ worlds. Rubel and Villalba (2009) noted that

where an ANOVA or SEM path analysis, or Pearson coefficient “speaks” for a sample in a quantitative study and manuscript, the qualitative researcher uses quotations or illustrations from interviews, focus groups, field notes and observations, or their [sic] own journal entries to convey, illustrate, and support important interpretations or concepts. (p. 303)

Because qualitative data analysis requires a rich, detailed, and nuanced description of textual data (e.g., interview transcripts, field notes), findings cannot be succinctly reported. As Creswell (2007) has indicated, authors must provide enough detail in reporting the findings that their work “comes alive . . . transporting the reader directly into the world of the study” (p. 182). At the same time, authors must be conscientious of page limits set by journal publishers. Authors of qualitative research manuscripts are thus challenged with the task of maintaining an appropriate balance between detailed description and brevity.

The manner and format for presenting qualitative research findings varies depending on the qualitative approach used by the researchers. Again, authors should be familiar with the authoritative methodological literature relevant to the qualitative approach they are using so that they can organize and report results accordingly. For example, while reporting findings from a grounded theory study, the findings section should culminate in a description of the grounded theory or model developed in the analysis. A presentation of themes of participants’ experience does not fulfill the promise of grounded theory research, which is pursued to develop theory. Conversely, while reporting findings from a case study, a good deal of description of the case is required. Phenomenological studies should include a well-developed analysis of the nature of the experience (or, in some forms of phenomenology, the “structure” or “essence” of the experience) that was at the center of the study.

In presenting the findings, authors use participant quotations or vignettes and descriptions from observations to explain the concepts and themes that emerged in the data analysis (Kline, 2008). Quotations, the primary form of evidence presented in most qualitative studies in RCB, are written verbatim to aid in “telling the story” and providing the reader with evidence to illustrate that research findings reflect the participants’ experiences, meanings, and contexts as opposed to the researcher’s assumptions. Using quotes to elaborate themes also illuminates the understanding developed by the researchers, distinguishes the voices of the research participants from those of the researchers, and allows readers to make their own interpretations of meaning (Elliot, Fischer, & Rennie, 1999). Quotations should be judiciously interwoven throughout the text to illustrate findings that emerged in the data analysis. Rubel and Villalba (2009) used the term “referential adequacy” to describe how quotations should be used to support research findings: “ . . . interpretations, concepts, or connections between concepts presented as part of the results should be supported by participant quotations that clearly fit the conclusions drawn by researchers and provide a certain degree of context by which to judge that fit” (Rubel & Villalba, 2009, p. 304). Common errors to avoid include overusing quotes, using quotes too sparsely, providing only lists or tables of quotes, stringing quotes together without explaining how they support the findings, and ineffective use of quotes (i.e., quotes do not aid in “telling the story” or illustrating a theme).

Ultimately, reporting qualitative research findings presents the difficult challenge of translating the extensive study of lived experience into a linear, textual form. Diverse qualitative research approaches have specific expectations for reporting findings, but all must find ways to draw the reader into the worlds of research participants while, at the same time, providing evidence (in the form of qualitative data) to support the findings that are being presented. The success of a qualitative manuscript often hinges on the interweaving of participant data and the researcher’s assertion of the findings from the study.

Like quantitative manuscripts, the discussion section of qualitative manuscripts synthesizes the research findings, compares and contrasts the findings with existing theories and research evidence, describes limitations of the research, and examines implications for practice and further inquiry. As previously noted, however, qualitative research differs from quantitative research in that when linking the research findings to existing theories, qualitative researchers must be cautious not to fit their findings into preestablished theoretical categories. Indeed, the role of theory in qualitative research is to illuminate research findings and not to test a priori theories. Moreover, the inductive, open-ended nature of qualitative research reflects the assumption that participants’ lived experiences, views, and practices can be used to inform theories themselves (Niesz et al., 2008). Conyers (2004), for example, illustrated how the data in her study provided evidence for revising an established theory, and devoted a significant portion of her discussion to elaborating how her research findings “expand on the ecological model of career development” and “lay a theoretical foundation for understanding the employment-related issues and concerns of people with HIV/AIDS” (p. 14).

In discussing the research limitations, authors are again cautioned not to use quantitative research criteria to evaluate the quality and rigor of their research or to describe research limitations. Many manuscripts submitted for review, and even published qualitative articles, have cited small sample size and lack of generalizability as limitations of their research. Such characteristics of qualitative research generally should not be viewed as limitations of specific studies. Because qualitative research is situated in specific contexts or focused on a relatively small group of individuals, there is no assumption that the participants in qualitative studies reflect the broader population. Instead, the interest is in how local contexts situate or influence the phenomena under investigation rather than in larger-scale trends in rehabilitation counseling. Limitations in qualitative studies may include problems in data collection, unanswered questions by participants, unexplored topics during the period of data collection, or a need for a better sampling of participants or sites for inclusion in the study (Creswell, 2012). An example is how Lindstrom et al. (2009) described limitations of their multiple-method, multiple site case study as “ . . . the primary interview participants in the case studies were community college and rehabilitation professionals. Significant insights could have been gained by including additional consumers, employers, and training-site supervisors in the case study interviews” (p. 199).

Finally, in discussing implications for practice and research, authors must keep in mind that “qualitative research is not designed to be conclusive; rather it is a stimulus for ongoing conversation” (Kline, 2008, p. 216). In the following excerpt, Gilbride et al. (2003) demonstrate this understanding in their use of tentative language to discuss implications and link their implications to the research findings:

Most of the employers in this study had received ongoing support and assistance from rehabilitation professionals. They were consistent in their appreciation for that support and the role that it played in their ability to successfully hire and accommodated people with disabilities. These results suggest that more opportunities for people with disabilities could be generated if rehabilitation professionals expanded their support to more employers. (p. 136)

In summary, discussions of qualitative research findings should proceed with solid understandings of the assumptions and purposes of qualitative research. This means that findings regarding participants’ experiences, practices, and meanings should not be overgeneralized but should be used to inform theory and practice, however tentatively. In other words, readers of qualitative research should be reminded of the specificity of the contexts of the research while also encouraged to take the worlds of the research participants seriously.

Conclusions

The purpose of this article was to propose general guidelines for writing and evaluating qualitative research manuscripts to be submitted to RCB. In considering these guidelines, contributing authors, readers, and editorial reviewers must keep in mind that qualitative research approaches have developed across a range of disciplines and philosophical stances (paradigms). Accordingly, there are diverse ways in which to conduct worthwhile qualitative research. Across these, perhaps the two most important criteria for qualitative researchers are the need for (a) coherence and congruence among all the research design elements and in the conduct of the research and (b) thorough and accurate representations of the participants’ experiences, practices, and meanings in all the aspects of the research. This entails foundational understanding of the nature, purpose, and appropriate use of qualitative research in general and of specific approaches in particular.

Ultimately, generating clear but flexible recommendations for writing qualitative research manuscripts has the potential of initiating more dialogue about research quality, fostering more valuable research, and serving an educative function for researchers. We want to, again, emphasize that these guidelines are not static and should evolve as the field of qualitative research evolves and as rehabilitation researchers become increasingly adept at using qualitative procedures to better understand the complex worlds of the primary beneficiaries of our research—people with disabilities functioning in multidimensional community contexts. Toward our mutual goal to improve meaningful knowledge, we invite authors, reviewers, and readers of qualitative rehabilitation research to critically assess its many (and sometimes unorthodox) applications while embracing its spirit of openness to collaborative questioning and deep discovery.