Abstract
Doctoral education is the primary time in which scholars learn about research methodologies and begin to develop their own research agendas and skills. Yet, to date, few research studies have examined graduate students’ perceived value of, and access to, training in multiple research methodologies. The purpose of this study was to explore special education doctoral students’ experiences at research-intensive universities in relation to mixed methods, quantitative, and qualitative methodologies. Using a mixed-methods research design, we explore the extent to which research judgments, skills, paradigmatic values, and methodological identities are diverse and how those features interact when doctoral students judge research. First, doctoral students were invited to participate in a survey (replicating McKim, 2017). Then, students who volunteered during the survey were individually interviewed about their methodological training and identity. We present the results and discuss how they can inform personnel preparation for the next generation of research scholars and consumers.
Social scientists have long pursued understanding social phenomena through a variety of methods. Methodological traditions in social inquiry are often categorized as: (a) quantitative (QUANT); (b) qualitative (QUAL), and (c) mixed methods (MM) (Creswell, 2014), despite the somewhat arbitrary nature of these categories (Sandelowski, 2014). Special education research has historically been dominated by QUANT methodological traditions (e.g., single-case experimental research; randomized control trials) and associated epistemologies (Klingner & Boardman, 2011; Skrtic, 1995). Systematic literature reviews have found that MM research, in particular, is still rarely utilized in special education and the MM research that is published in the field often does not align with quality standards (Corr et al., 2020; Onwuegbuzie & Corrigan, 2018). The lack of training opportunities may contribute to a less diverse methodological landscape, particularly the underutilization of MM research (Corr et al., 2020; McKim, 2017). Intensive methods training from the National Center for Special Education Research (NCSER) remains limited to QUANT methods, including only single case experimental designs and sequential, multiple assignment, randomized trial experimental designs (NCSER, n.d.). Future special education researchers must be taught to consume, use, and critique varied methodologies.
Researchers and commentaries from past decades have espoused the importance of rigorous methodological training for special education doctoral students (e.g., Prehm, 1980) and described programs grounded in particular methodological and paradigmatic perspectives (e.g., Heward et al., 1995). Yet, few researchers have examined graduate students’ access to training in different methodological traditions and the ways doctoral preparation shapes their methodological perceptions and abilities. The research that has examined the impact of special education doctoral programs has focused on participants’ overall satisfaction with doctoral training, including general preparation in research, teaching in higher education, and other aspects of academic job preparation (Tyler et al., 2003; Wasburn-Moses, 2008); career choices after graduation (e.g., Pion et al., 2003; Tyler et al., 2003, 2012); and post-graduation publication rates (Troup-Leasure et al., 1992). In their study of education doctoral students, which included special education doctoral students, Lambie and colleagues (2014) found that students who conducted and published research reported greater research self-efficacy, which was associated with greater interest in research.
Outside of special education, McKim (2017) examined how graduate students enrolled in education or psychology programs perceived the value of QUANT, QUAL, and MM methodologies. Using vignettes formatted to reflect each methodological tradition, participants responded to questions about the value of the methods and results of the passage. She found that participants rated the MM study as the most valuable, even when controlling for prior experience.
Purpose
Researchers have not yet examined the methodological training provided in special education doctoral programs. Therefore, the extent to which doctoral training programs prepare future researchers to understand, apply, and critique diverse methodological applications is largely unknown. The purpose of this study was to examine how special education doctoral students are trained methodologically. We started with the premise that doctoral programs are a key influence on scholars’ developing research skills and values. We focused on doctoral students’ understanding of—and thereby, to some extent, special education’s socialization around—research methods and identity. Our research questions were the following:
Method
We collected data sequentially from an online survey via Qualtrics™ and subsequent interviews conducted via Zoom™. Methods were mixed for the purposes of development, complementarity, and initiation (Greene, 2007). Both data sources were used to fully understand the phenomena and to identify divergent findings, initiating new, more nuanced understandings.
Participants
We recruited doctoral students enrolled in the top 10 research-intensive special education doctoral programs, according to the U.S. News and World Report (2018) rankings. We contacted the special education department heads of each Insitute of Higher Education (IHE) and requested that they distribute an email invitation to participate in the survey to their doctoral students and send follow-up reminder emails to their students; 248 students received an invitation to participate in the survey (per institution range: 9–54 students). See Tables 1 and 2 for the demographic information.
Demographics Information.
Methodological Coursework and Peer Review Experience.
Note. SD = standard deviation; QUAL = qualitative; QUANT = quantitative; MM = mixed methods.
Data Sources
In Phase 1, we surveyed doctoral students in special education programs. In Phase 2, we interviewed doctoral students who self-nominated to be interviewed after completing the survey.
Phase 1: Survey
The survey was a partial replication of McKim’s (2017) questionnaire. Participants were asked to read each of the three research study reports (QUANT, QUAL, and MM), delivered in random order. Then, they responded to the same questions as McKim’s about the quality and clarity of each report and their perceptions of its value (see Table 3). Participants completed the survey online via Qualtrics (20–40 minutes to complete). Of the 248 students invited to participate in the survey, 66 doctoral students completed it, a response rate of 26.6% (see Table 1). At least one doctoral student participated from each institution (response rate range by institution: 11%–55%). Participants received a US$10 gift card for their participation.
Means and Standard Deviations for Survey Responses by Passage Block.
Note. All items used the following Likert-type scale: (1) strongly disagree; (2) somewhat disagree; (3) neither agree nor disagree; (4) somewhat agree; (5) strongly agree. SD = standard deviation.
This item is reverse coded. bThis question was inadvertently omitted from the survey in the mixed methods block.
Phase 2: Interview
Twenty-two survey participants opted to participate in an audio-recorded interview in which they were asked to respond to questions about their methodological training, values, and identity (see Appendix A and other appendices in online supplemental material). A graduate student transcribed each interview for use during data analysis. Then, a special education postdoctoral scholar created a one-page summary of each individual interview and emailed it to the participant as a first-level member check.
Analysis
Initially, we analyzed data from the survey and interviews separately. Findings were then integrated to develop assertions and final inferences (Miles et al., 2014).
Initial survey analysis
Survey data were checked for missing data, which ranged from 3% to 14%, for an average of 10% missing responses. Those who had missing responses tended to skip the questions in blocks for an entire passage, rather than skipping questions within a block. During this initial check, we found that one item (“I have a clear understanding of what the researcher did.”) was mistakenly left out of the MM block. We dealt with missingness by listwise deletion. Analyses were run using the 32 (out of 33) items that were present for all three vignettes. Each survey item was assigned to a construct: (a) judgment, (b) knowledge and skills, or (c) values. Judgment questions were those that asked respondents to determine a methodology’s rigor, ability to address the research questions, and appropriateness based on the research purpose. Questions were coded under the knowledge and skills construct if they asked respondents to rate their understanding of the study’s methods, interpretations, and findings. Finally, values survey items were those that asked respondents to rate the extent to which the study findings were useful, reflected participant voices, and provided deeper knowledge. Some survey items were only included in the overall analysis (see Table 3). Reliability analyses were performed on the three constructs and the overall score. Coefficient alpha estimates were .83, .94, and .78 for the judgment, knowledge, and values scores, respectively, and .96 for the overall score.
We conducted a descriptive analysis comparing participants’ self-selected researcher identity (i.e., identified as a QUANT, QUAL, MM, or Other researcher, collected on a demographic question in the survey) and their self-rated confidence in interpreting the results presented in each vignette. We also compared the mean rating for survey items across the three vignettes by researcher identity groups. Linear mixed-effects regression models treating person as a random factor were constructed to assess participant score variability. Four models were constructed, one for each of four dependent variables: the overall score, the judgment score, the knowledge score, and the values score. The dataset contained several potential predictors, including participants’ self-identified racial/ethnic identity, researcher identity (QUANT, QUAL, MM, Other), status before PhD program (student or employed), and two-way interactions of the factors. A model was considered viable only if the included interactions consisted of factors whose main effects were also included in the model. In every case, the passage type was included as a predictor. The models selected for interpretation were the models that had the lowest Akaike’s information criterion for each dependent variable; see online supplemental Appendix B. Appendix C (online supplemental) gives the results for each chosen model.
Initial interview analysis
Interview transcripts were reviewed and coded by a subgroup of the research team led by the second author. This was an iterative process of deductive and inductive analysis (Miles et al., 2014). First, ten codes were determined that reflected the four primary constructs represented in the research questions and their potential intersections. These codes were based on the study’s conceptual framework and matched the constructs used for initial survey analysis (i.e., researcher identity, judgment, knowledge and skills, and values, and their intersections), reflecting deductive analysis grounded in an existing framework (Bazeley, 2017). Next, the first three authors engaged in an inductive, open-coding process in which they individually reviewed a subsection of transcripts and identified meaningful units or excerpts of data (Saldana, 2015). Then, they met and discussed the transcripts to define additional codes (Saldana, 2015).
Two graduate students used the resultant codebook to individually code each transcript and regularly met with the second author to discuss disagreements and come to consensus. Meetings supported reliability by ensuring codes were applied consistently and facilitating discussion about code definition revisions (Miles et al., 2014). Meeting notes served as an audit trail documenting which transcripts were coded, code applications to support future consistency, and code definition adjustments. The use of multiple researchers to collaboratively identify meaningful data excerpts and to categorize, define, and apply codes supported trustworthiness (Brantlinger et al., 2005).
Mixing initial findings
After initial analysis of the survey and interviews, the second and fifth authors created a joint data display that combined the survey and interview data from participants who completed both in a single table (Guetterman et al., 2015). The data display organized (a) survey questions by the construct the survey question was most relevant to (i.e., judgment, values, methodological knowledge and skills, researcher identity; see Table 3) and recorded the participants’ numerical response to the question, and (b) representative interview excerpts based on applied codes (i.e., excerpts coded as “judgment” or “judgment being influenced by [e.g., values, pre-doctoral experiences, or training]” were added in a column after the participant’s numerical scores for “judgment”-related survey questions). Excerpts were chosen that reflected “critical” examples of data representing the breadth and depth of participants’ responses (Bazeley, 2017), including all identified influences on their judgment of research methods, research values, researcher identity, and methodological knowledge and skills.
Finally, three members of the team (Authors 2, 3, and 5) made assertions and propositions (Miles et al., 2014) in response to each research question using the findings from the survey analysis and interview analysis separately and together. A total of 11 initial assertions were determined and a detailed record of the evidence used to build each assertion was made. Then, the remaining team members reviewed the assertions and evidence, including raw data and the initial analyses of survey and interview data, to confirm and/or disconfirm the initial assertions. Finally, the team met as a whole to review and finalize the assertions presented.
Results
The assertions are represented in Figures 1 and 2 and are described in detail here. We begin with assertions related to the influences on and diversity of participants’ researcher identity, knowledge and skills in research methodologies, and their values. Then, we describe the assertions related to participants’ methodological judgments and what influenced those judgments. Below, we use a numerical (participant number) and alphabetical (school) code to delineate participants (e.g., 03A, 19C, 21E) to maintain confidentiality of our participants and also show the depth and breadth of the data collected.

As a result of our mixing, this figure represents assertions related to the influences on and diversity of participants’ researcher identity, knowledge and skills in research methodologies, and their values (Research Question 1).

As a result of our mixing, these assertions are related to participants’ methodological judgments and what influenced those judgments (Research Question 2).
Influences on and Diversity of Researcher Identity, Knowledge and Skills, and Values
In the survey, participants reported if they were a “(a) QUAL, (b) QUANT, (c) MM, or (d) Other” researcher. These responses constituted their “researcher identity” for the analysis.
Diversity of and influences on choice of researcher identity
Diversity of researcher identity
Overall, there was significant diversity in the researcher identity participants selected in the survey, with 29 participants self-identifying as QUANT researchers, 20 as MM researchers, and 15 as QUAL researchers. One participant selected “Other” and explained that they did not identify as a researcher (see Figure 1). Interview participants reflected this diversity, with 9 participants self-identifying as QUANT, 7 as MM, 5 as QUAL, and 1 as “Other.”
Influences on choice of researcher identity
QUANT-identifying participants’ identities were heavily influenced by their coursework and research experiences (including their success in QUANT coursework and research experiences [see Table 2]) and their perceptions of QUANT as the dominant methodology in special education (see Figure 1). One QUANT-identifying participant stated, “Because I’ve taken so many quantitative classes and I’ve taken so few qualitative, those have sort of shaped my identity because those are just the opportunities I’ve had” (03A). Participants also recognized that, as a field, special education typically focuses on numeric data and QUANT research, and therefore valued that as an important contribution to the field’s knowledge base. One QUANT-identifying participant remarked, “I was a special education teacher for years and years and when we think about data in school, we think about quantitative data and when the state wants data on students, they want quantitative data” (21E). The existing focus on numeric data and quantitative analysis in special education, both within research and PreK-12 education, influenced some doctoral students to identify as a QUANT researcher.
QUAL- and MM-identifying participants’ identities were comparatively more diversely influenced by pre-doctoral experiences, their own values, and the freedom to choose coursework and experiences within their doctoral program (see Figure 1). For example, one QUAL-identifying participant acknowledged, “In special ed, there is a strong push for quantitative,” but when it came to their identity, they explained that “years of teaching have affected my researcher identity in always thinking, ‘Why are teachers doing certain things they’re doing?’ . . .And then looking at how people interact with each other and how can they. . .improve instruction” (23E). The participant’s practical experience led them to QUAL research despite the observed focus on QUANT research in the field and doctoral program.
QUAL- and MM-identifying participants’ research values also influenced their researcher identities. A MM-identifying participant similarly explained, You need objective, large-scale data, rigorous designs to really know if things work. But as we’re getting going and we’re doing pilots in groups who maybe are underrepresented, we need to be using a different design because otherwise, the decisions we’re making could be false. . . From an equity standpoint, I feel very much. . . an ethical responsibility . . . to make sure that to the best of my ability I’m helping to amplify all voices. And a lot of the time when you use only one type of design, you’re not able to do that. (19C)
QUAL and MM participants also cited opportunities to make their own educational decisions as important for their developing identities. One MM-identifying participant said their second advisor was very helpful because of the freedom he gave the student. The participant explained, “. . .he was more interested in what I wanted to do. . . So I think having the opportunity to say. . .’ this is what I want to do,’ then to. . .get his feedback and ideas. . .I think was great because I was actually able to do what I was actually interested in doing. . .” (25D). For both QUAL- and MM-identifying participants, options to pursue their own research interests contributed to them developing non-QUANT identities.
Influences on and diversity of methodological knowledge and skills
Influences on knowledge and skills
Regardless of their researcher identity, participants reported that their methodological knowledge and skills were primarily influenced by their access to and the quality of research training and experiences, which were dominated by QUANT opportunities across all programs (see Figure 1). In the survey, participants reported the research coursework they had access to across their years of education (not exclusively in their doctoral program; see Table 2). Only 9 of 66 survey participants (13.6%) reported having more QUAL courses than QUANT or MM; the rest of the participants had taken more QUANT courses (86.4%). Participants described the disproportionate availability of QUANT courses and research experiences in interviews. For example, a QUANT-identifying participant reported, So everyone has to take —there’s two stats courses. . . so it’s kind of ANOVAs and regression. And then beyond that, it’s sort of your decision of which courses you want to take. I think we only offer . . . one qualitative course. (10F)
Students reported that QUANT coursework and experiences were more commonly required and offered during their doctoral programs in alignment with survey responses.
Participants also noted other program structures that contributed to them accessing more QUANT coursework and experiences, and therefore having greater access to QUANT research knowledge and skills. These features included programs requiring students to choose a methodological sequence or track and a greater number of faculty who primarily conducted, and could therefore advise students on, QUANT research. For example, one QUANT-identifying participant who reported that their department required students to choose a methodological track remarked, “I probably didn’t realize that if I continue down this quantitative track. . .that’s going to affect the questions that I can answer” (03A). In interviews, some participants shared that they were expected to choose a methodological track or sequence early in their program, before understanding the implications of such a decision, and a QUANT track was often the default because more QUANT coursework was required and available. In terms of faculty support and advisor-offered research experiences, one QUAL-identifying participant explained, “There aren’t many qualitative methodologists in our department. . .I would need a postdoc experience.” Availability of QUAL-oriented faculty affected participants’ access to QUAL research knowledge and skills by affecting available coursework, advising, and research experiences.
Diversity of knowledge and skills
Although participants agreed that access to and quality of training and research experiences were the biggest influence on their methodological knowledge and skills, participants’ perceptions of their knowledge and skills varied based on their researcher identity (Figure 1). Initially, survey analysis indicated no differences in knowledge and skills related to researcher identity. For the scores on the survey items in the knowledge subdomain (see Table 3), using a Tukey adjustment for pairwise comparisons, we found no significant differences between the three researcher identities on survey questions asking about their knowledge of the vignette methodology: QUAL-QUANT, t(54) = 1.56, p = .271; MM-QUANT, t(54) = 2.29, p = .07; and MM-QUAL, t(52) = 0.52, p = .86.
Interview data analysis, however, revealed interesting differences in perceived knowledge and skills for participants that were aligned with their researcher identity. QUANT-identifying participants reported higher knowledge and skills for QUANT methods and QUAL-identifying participants reported higher knowledge and skills for QUAL methods while MM-identifying participants varied in their perceptions of the methodology in which their knowledge and skills were higher (see Figure 1). This divergence between the survey and interview findings could have been because participants were oriented toward the knowledge and skills needed to understand and consume research while completing the survey; when responding to interview questions, participants more often spoke to the knowledge and skills needed to conduct research. In general, QUANT-identifying participants, and some MM-identifying participants, reported higher perceived knowledge and skills in QUANT research given the greater access to QUANT training. When asked what had contributed to their methodological knowledge in an interview, one MM-identifying participant described, “I think it really comes down to how many courses I have taken and what my experience has been working on projects with my advisor. I haven’t done a lot of qualitative so I’m just more comfortable doing quantitative” (25D). Although QUAL-identifying participants took QUANT coursework, often as required by their program, it did not have the same influence on their knowledge and skills because they did not access the QUANT research experiences that others did. For example, one QUAL-identifying student said, . . . some people were working on stats courses with their advisor or other faculty members and I never was, and so all of my data was made up or used from different sets of data that the professor supplied to me, so my quantitative training just always feels really theoretical. . . (26D)
Conversely, QUAL-identifying participants still felt more prepared to conduct QUAL research compared to other methodologies, despite their desire to have more programmatic support for their preferred methodology. They often expressed a greater preparation to conduct QUAL research because they did not believe they possessed necessary statistical skills for QUANT research and/or because they possessed skills or beliefs they perceived as necessary for QUAL research. For example, one QUAL-identifying participant explained, My skills lie in connecting with people and understanding people and my curiosity in where people are coming from and why they’ve done the things they’ve done . . . I connect with them fairly easy so for me sitting behind a computer running R code all day to try and understand people is not how I do that anyways. (26D)
Other QUAL-identifying participants similarly expressed challenges or a lack of desire to understand or analyze statistical information. As another QUAL-identifying participant expressed, “Qualitative research is easier for me number one. . .I guess math’s always been hard for me” (23E). QUAL-identifying participants felt they needed more skills in QUAL research, but they still felt more prepared to conduct it than QUANT research.
MM-identifying participants reported more variety in which methodology they felt their knowledge and skills were higher, but all reported that they lacked desired knowledge and skills in MM. In the absence of MM courses and experiences, MM-identifying participants reported trying to, as one participant termed it, “mix and match” QUANT and QUAL research knowledge. They, however, recognized that that did not amount to the skills to do MM research. One participant remarked that there is “nothing particularly for mixed methods. And that’s one disadvantage [of my program]” (05A). Instead, MM-identifying participants’ perceptions of their knowledge and skills varied based on coursework and research experiences such that most expressed greater perceived knowledge and skills in conducting QUANT research and some felt more prepared to conduct QUAL research. None of the MM-identifying participants expressed that they had the most knowledge and skills in MM research. One MM-identifying participant described, “The one I use in my own study, like t-test, is more simple stat analysis and I’m currently going to use structural equation modeling. So, I think I’m viewing my skill set as that part. And at the same time my dissertation is a mixed methods study” (11A). Another MM-identifying participant said, “In conducting a study itself, I would be much more comfortable with qualitative studies. I don’t have a high confidence in my method skills regarding. . . statistics” (06A). Although MM-identifying participants valued MM research and planned to conduct it, they felt primarily prepared to implement single-method research.
Influences on and diversity of values
Influences on values
Influences on participants’ research values, including their beliefs about the purpose of research and the priorities that undergird their scholarship, differed across participants based on their identity (see Figure 1). Although some values were expressed across participants (e.g., conducting research that has a positive impact on the field, not doing harm to research participants), diverse influences contributed to some differences in research values.
The most frequently cited influence on values for QUANT-identifying participants was doctoral coursework and research experiences with advisors (see Figure 1). For example, one QUANT-identifying participant reported that research reproducibility was an important value for them because, “I took a meta-analysis class where. . .I saw most of the people that tried to reproduce couldn’t get the same results or even calculate the same effect sizes to run the meta-analyses. . .” (20E). Some participants discussed their coursework and research experiences broadly, while others named specific courses that influenced their values, including courses on meta-analysis, hierarchical regression, and measurement theory.
In contrast, QUAL- and MM-identifying participants cited pre-doctoral program experiences and aspects of their non-researcher identities as the dominant influencers on their research values, including their previous experiences as practitioners and membership in a marginalized group (see Figure 1). Although several QUANT-identifying participants did name teaching experiences as an influence on their values, pre-doctoral experiences were more consistently cited as influential by QUAL- and MM-identifying participants. For example, one QUAL-identifying participant reported, As a teacher I’m like, “I could’ve told you that was what was going to be the outcome. . .” That’s the stuff that got me there as a teacher because it’s like you’re [researchers] not listening to me as a teacher. . . So definitely my years of teaching have impacted my research values. (23E)
Similarly, a MM-identifying participant expressed that supporting equity was important based on their time working in communities. After saying that “[the] equity piece, that needs to be foundational to everything,” the participant explained that their views were from, “just being out in the real world and talking to people. . .being a home visitor since I was 18 and doing a lot of work in communities” (19C). Overall, pre-doctoral experiences more often influenced QUAL- and MM-identifying participants’ perceptions of the purpose of research and how it should be conducted and reported.
Diversity of values
Participants reported a variety of values that guide their methodological thinking, both when conducting and consuming research. Many of these differences were aligned with their researcher identity (see Figure 1).
QUANT-identifying participants frequently cited replicability and rigor as values, where rigor was defined based on QUANT analysis procedures or quality guidelines (e.g., including all appropriate dependent variables in statistical models, accounting for error in statistical models). For instance, one QUANT-identifying participant said that they wanted to make sure all of their research was “designed in a way that is logical and replicable” (10F). Regarding the importance of rigorous research, another QUANT-identifying participant explained, “From a one-variable model look at the difference between two groups to a multivariate mixed-effects model where you’re looking at a million different things at once, know where the error is and know what story you can tell. . .” (09F). Thus, a focus on QUANT analyses and QUANT research training appeared to influence the research values QUANT-identifying students named.
MM-identifying and QUAL-identifying doctoral students both emphasized the need to conduct research that supports equity and represents participants’ perspectives as partners in the research. For these participants, the need for research to support equity, as illustrated by the above quote, was primarily addressed by research highlighting the perspectives of participants, particularly those from marginalized groups, and positioning participants as partners. For example, a QUAL-identifying student reported, “Like that’s really, really important to hear the students and the families’ voice in this research. I had to remind myself that. . .that was one of my concerns when I was a teacher” (14J). Pre-doctoral experiences led them to value student/family perspectives by ensuring participant representation in research. Sometimes, doctoral students explicitly discussed the representation of participants from marginalized groups. A MM-identifying participant explained, “It’s something that really bothers me when I look at a study [and] the sample is all white, all-female, [all] this SES” (25D). MM- and QUAL-identifying students valued research that included participant perspectives as a way of better representing diverse participant experiences and needs.
As an extension of representing participant perspectives in research, MM- and QUAL-identifying doctoral students also valued partnership with participants. A QUAL-identifying student who said they took an “equity-based approach” recalled, All of the studies that I have designed have been created so that choice and a range of choices is available to participants. There’s always flexibility . . . the students I work[ed] with, I learned early on that they felt powerless in their lives. When they acted out. . . they just wanted a choice in what was happening to them, they needed control. . . (04A)
Providing participants with choice was an opportunity for them to have more control over their representation and participation. Similarly, a MM-identifying student stated, “I just want to make sure that whatever study I conduct, I put participants as a partner instead of an object under a microscope. I want them to have as much agency as possible in the study itself” (06A). MM- and QUAL-identifying students consistently discussed the importance of participants having a voice and/or active role in research.
Methodological Judgments and Influences on Judgment
Figure 2 displays the assertions we made regarding participants’ methodological judgments in response to Research Question 2. Participants were asked to judge three separate vignettes in the survey, represented in the columns in Figure 2, and then were asked follow-up questions about these vignettes and their judgment processes and influences during the interview. Intersections between knowledge and skills, values, and researcher identity (represented in the rows in Figure 2) and participants’ judgment about the vignettes provide interesting information about the ways these constructs intersected as participants engaged in the survey task.
Influence of knowledge and skills on judgment
Access to and quality of research experiences that support knowledge and skill development influence judgment of vignettes across methodologies (see Figure 2). Participants across researcher identities repeatedly described how circumstances of their program and the quality of their coursework and research experiences made them more or less prepared to judge a certain methodology. For example, one QUANT-identifying participant explained, “I would say I feel least confident with the qualitative just because we don’t cover qualitative research” (21E). Similarly, one QUAL-identifying participant reported, “I just finished taking another stats class and I just need more practice with it to be able to read something and say, ‘Okay definitely I know what they’re saying, what’s right, what’s wrong’” (23E). Coursework and research experiences were consistently described as influential for participants’ perceived ability to judge each survey passage, regardless of researcher identity.
Interactions between knowledge and skills and values when judging
All participants made largely positive judgments about the MM passage and valued it highly, even though most (if not all) of the participants had limited knowledge about, or experience with, MM research. Participants’ rating and perception of the value of the vignettes were consistently higher for the MM passage than the QUAL or QUANT vignettes (see Figure 2). As previously described, participants in both the survey and interviews had limited opportunities to develop knowledge and skills in MM. Based on the linear mixed-effects regression model examining overall passage ratings across survey questions given in Table 3, however, the MM passage had the highest mean score (M = 3.77, SE = 0.08), followed by the QUAL passage (M = 3.29, SE = 0.08) and the QUANT passage (M = 2.53, SE = 0.08). Post hoc tests of differences between passage blocks were conducted using a Tukey adjustment for pairwise comparisons. The means of all three blocks differed significantly from each other: QUAL–QUANT, t(108) = 8.21, p < .001, 95% CI = [0.54, 0.98]; MM–QUANT, t(108) = 13.41, p < .001, 95% CI = [1.02, 1.46]; and MM–QUAL, t(108) = 5.20, p < .001, 95% CI = [0.26, 0.70].
Based on the linear mixed-effects regression model for the scores on the survey items in the Knowledge and Skills subdomain given in Appendix B, we investigated the differences between the vignettes using a Tukey adjustment for pairwise comparisons. The means of all three vignettes differed significantly from each other: QUAL–QUANT, t(108) = 7.58, p < .001, 95% CI = [0.62, 1.18]; MM–QUANT, t(108) = 10.58, p < .001, 95% CI = [0.97, 1.54]; and MM–QUAL, t(108) = 3.00, p < .001, 95% CI = [0.07, 0.64]. Thus, the vignette with the highest mean Knowledge and Skills score was the MM vignette (M = 4.05, SE = 0.10), followed by the QUAL (M = 3.69, SE = 0.10) and then QUANT (M = 2.79, SE = 0.10) vignettes.
This pattern continued when post hoc tests of differences between passage blocks were conducted using a Tukey adjustment for pairwise comparisons based on the linear mixed-effects regression model for the Values subdomain given in Appendix C. The means of all three blocks differed significantly from each other: QUAL–QUANT, t(108) = 12.36, p < .001, 95% CI = [1.06, 1.56]; MM–QUANT, t(108) = 15.50, p < .001, 95% CI = [1.39, 1.90]; and MM–QUAL, t(108) = 3.14, p = .006, 95% CI = [0.08, 0.59]. Thus, the vignette with the highest mean Values score was the MM block (M = 3.97, SE = 0.08), followed by the QUAL (M = 3.64, SE = 0.08) and then the QUANT passages (M = 2.33, SE = 0.08).
Furthermore, on specific survey questions, regardless of identity, participants generally rated the MM passage higher than the QUANT and QUAL vignettes on questions related to strength (i.e., “This is a strong methodological study.”), clarity (i.e., “I have a clear understanding of what the researcher found.”), and utility (e.g., “The results are useful.”; see Table 3).
One reason that MM was seemingly perceived as a valuable methodology despite limited opportunities to develop knowledge and skills in MM was that it was seen as a way to combine the strengths of QUANT and QUAL research. Many participants across identities described MM as combining QUANT and QUAL in a way that was beneficial. For example, one QUANT-identifying participant said, “. . .I got to the mixed methods [passage], I was like yes! . . . It seemed like a more complete picture to really understand the problem and the research question. . .” (10F). Thus, students may have valued the MM passage highly and rated it as particularly useful because they viewed it as combining the benefits and contributions of QUANT and QUAL research.
Interactions between values and researcher identity when judging
When discussing how they judge research, QUANT- and MM-identifying participants valued MM research as a compromise or bridge between expectations that QUANT research be present in special education research and the representation of their values in research (see Figure 2). For example, one MM-identifying participant explained, “Mixed methods are the smartest because you get the qualitative part, you get a great statement, and then you mix it with quantitative testing those things out” (05A). Although QUANT research did not need an explicit reason for its inclusion, participants named specific justifications for using QUAL research that often positioned it as secondary, such as illustrating the “story” of the phenomena captured by the QUANT research.
While QUAL-identifying participants expressed seeing the value of MM research, however, QUAL-identifying participants described unique values that led them to defend the contributions of QUAL research in its own right while judging both the QUAL and MM vignettes as useful and reliable (see Figure 2). That is, they resisted what they described as external pressure to include QUANT data on its own or as a focal point of MM research. For example, one QUAL-identifying participant explained, I’m a woman of color and my story’s pretty complex. There’s a lot of different oppressions that I experience that I think a predominately White academic body doesn’t capture in many of their methodologies, particularly quantitative methodologies. . .And I don’t think it’s malicious or intentional on their part, but my story is not captured with what they are doing. So I choose qualitative to capture the complexity. . . Those stories of nuance and complexity is where research should continue to be pursued. (04A)
Another QUAL-identifying participant explained, “So to me, I guess one of my values being that research be accessible to everyone lends itself to my draw to qualitative research” (02I). QUAL-identifying participants were unique in both their understanding of QUAL vignettes and their defense of the methodology when discussing their judgments and research values. Notably, QUAL students’ interview responses contrasted with the survey finding that students generally rated the MM vignette as more valuable, regardless of identity.
Influence of researcher identity on judgment
Participants’ identity as a researcher interacted with their judgment in two distinct ways: (a) the survey passage they rated as clearest and (b) the survey passage they found easiest to judge (see Figure 2). Participants across researcher identities rated the QUANT passage as the least clear passage (i.e., survey items “I have a clear understanding of the methodology the researcher chose.” and “I have a clear understanding of what the researcher found.”; see Figure 2). The MM passage was clearest for QUANT- and MM-identifying participants and the QUAL passage was the clearest for QUAL-identifying participants (see Figure 2). During interviews, however, participants usually indicated that the passage aligned with their methodological identity was easiest to judge (see Figure 2). QUANT- and QUAL- identifying participants referenced their preparation and interests when explaining which passage they viewed as easiest to judge. One QUANT-identifying participant explained, “I would say the easiest [to judge] would be quantitative because that’s where most of my preparation has been in and where my research is in” (21E). Similarly, a QUAL-identifying participant said the QUAL passage was easiest, explaining, “Just cause it’s [the] one that is the type of research that I’m most interested in and I feel more comfortable with” (02I). Meanwhile, MM-identifying participants varied in which passage they believed was easiest and hardest to judge for a variety of reasons, including doctoral training. One MM-identifying participant who said the QUANT passage was the easiest to judge and the QUAL passage was hardest explained, “Because the majority of my training is in quantitative research, I think that made it a little easier” (25D). Thus, students’ ratings of the passages’ clarity and their perceived ability to judge the passages somewhat differed, which largely related back to their methodological research training and experiences. Potentially, students drew more on their perceived ability to implement research when considering ease of judgment while they rated passage clarity independent of their skills to conduct the research.
Discussion
Doctoral preparation programs remain the primary source of methodological training for doctoral students in special education (Hutchinson & Lovell, 2004; Saunders et al., 2016). Special education departments in IHEs bear the responsibility of preparing future special education researchers who can ethically, responsibly, and effectively conduct and consume research (Ford et al., 2016; Page, 2001). Moreover, doctoral programs are increasingly preparing graduates who will enter the workforce and will train future generations of teachers; faculty at doctoral-granting institutions hold the majority of personnel preparation grants (Smith & Montrosse, 2012). The findings have implications for doctoral programs and how we prepare students to consume and conduct research.
Addressing Access to Expansive Methodological Training
First, participants in this research study indicated that features of their program, including course offerings, requirements, and available faculty for advising, strongly influenced their methodological knowledge and skills, researcher identity, and ability to judge and consume research. Many participants reported few opportunities to explore coursework and projects that were not required or related to the research experiences offered by their advisors. Although doctoral programs must be focused, the field is largely focused on QUANT methods, which meant that program structures more consistently reflect QUANT skills and epistemologies. Both implicit and explicit practices and processes that are intended to make programs more efficient (e.g., creating research tracks, designating coursework, primarily working with an advisor) affected students’ access to diverse methodological training opportunities. Extending the length of the doctoral program is not recommended (Pion et al., 2003). Programs can, however, consider how to balance requirements across methodologies and epistemologies. Our findings indicate the need for program flexibility, advising and mentorship practices, diversifying faculty, and embracing MM, described subsequently.
Program flexibility
MM- and QUAL-identifying students reported few training opportunities that aligned with their methodological interests, desired skills, and values. Students across researcher identities recognized shortcomings in their ability to judge research that did not align with their researcher identity or the skills supported by their preparation program. When programs allowed flexibility, students were more likely to be exposed to new perspectives and access desired research experiences. Although previous research has noted the importance of providing funded research assistantships (e.g., Tyler et al., 2003, 2012) and opportunities for publishing research (Lambie et al., 2014), the present findings emphasize the need for such opportunities to be intentionally flexible and diverse in the methodologies represented. Faculty may consider funding opportunities that allow students to approach critical issues in special education from a variety of paradigms and epistemologies.
Advising and mentorship practices
To that end, “support” from advisors looked different for different students in the present study. Participants reported a more direct type of support, such as an advisor’s invitation to work on active research projects that reinforced coursework material, and therefore advanced their methodological knowledge and skills. Participants also described an indirect type of support from their advisors, such as when an advisor facilitated their exploration of coursework or research ideas outside of traditional training opportunities or their advisor’s area of expertise. Notably, programs were diverse in terms of participants’ researcher identities and values, yet program structures and advising did not always reflect that same diversity.
Diversifying faculty
Others have noted the need to increase racial and cultural diversity of faculty (Tyler et al., 2012), and the present analysis suggests that increasing methodological diversity should also be a consideration to diversify the field’s knowledge base. Increasing methodological diversity can improve through activities such as interdisciplinary collaborations, learning and using novel research methods and methodologies, and publishing in—and receiving peer review from—a broader array of journals.
Embracing MM
Findings from both the survey and interviews replicated McKim’s (2017) finding that doctoral students value MM research. Yet, few participants in the current study had taken a MM course or felt prepared to conduct MM research, including those who self-identified as a MM researcher. As MM research is increasingly recognized as contributing unique knowledge and calls to increase its rigorous use in special education persist (Christ, 2018; Corr et al., 2020; Onwuegbuzie & Corrigan, 2018), faculty in doctoral programs may particularly consider how to prepare students to understand and rigorously implement MM research.
Addressing Reviewing and Conducting Research
Participants’ responses revealed that the process of reviewing research and the process of conducting research are two distinct yet mutually reinforcing skills. Participants reported needing more knowledge and skills to implement a method than to understand and judge it. Interestingly, researcher identity did not predict differences in knowledge ratings oriented toward understanding the survey vignette methodologies and rating their clarity. Participants with different researcher identities, however, did differ in their perceived knowledge when oriented toward their ability to implement research methodologies and their confidence in their methodological judgments. This finding complicates Lambie and colleagues’ (2014) finding that education doctoral students’ research knowledge was associated with their research self-efficacy. Given that doctoral programs are preparing students to both consume and critique various research methods as well as implement a specialized area of research, future studies could further explore the relationship between learning to understand research and learning to conduct research as well as the unique and common influences on both.
Limitations
We identified three main limitations to this research study. First, the survey sample size was relatively small and, therefore, we could not apply a more sophisticated statistical approach, which could have taken into account the measurement error involved in creating the perceived value scores. Furthermore, the number of students in programs may fluctuate annually due to funding, faculty availability, and acceptance rates. Due to the nature of this sample, we recommend replicating this research with additional populations. Finally, participation in the survey and interview was voluntary and varied between programs. Despite these limitations, this research reveals patterns across the programs and addresses important gaps in knowledge about the methodological training of special education doctoral students.
Conclusion
Today’s doctoral students will become the researchers, journal reviewers, and teacher educators of tomorrow, the field must ensure these future leaders are able to critically consume research from a diverse array of research methods. Although it is not feasible to train all doctoral students in all methodological possibilities, doctoral programs must recognize how program structures and practices create critical gaps in research and, as a result, special education practice. It is critical for special education doctoral programs to take action to intentionally support a more diverse array of consumers and producers of research.
Supplemental Material
sj-docx-1-tes-10.1177_08884064221103902 – Supplemental material for Methodological Training in Special Education Doctoral Programs: A Mixed-Methods Exploration
Supplemental material, sj-docx-1-tes-10.1177_08884064221103902 for Methodological Training in Special Education Doctoral Programs: A Mixed-Methods Exploration by Catherine Corr, Hailey Love, Melinda R. Snodgrass, Justin L. Kern and Mia Chudzik in Teacher Education and Special Education
Supplemental Material
sj-docx-2-tes-10.1177_08884064221103902 – Supplemental material for Methodological Training in Special Education Doctoral Programs: A Mixed-Methods Exploration
Supplemental material, sj-docx-2-tes-10.1177_08884064221103902 for Methodological Training in Special Education Doctoral Programs: A Mixed-Methods Exploration by Catherine Corr, Hailey Love, Melinda R. Snodgrass, Justin L. Kern and Mia Chudzik in Teacher Education and Special Education
Supplemental Material
sj-docx-3-tes-10.1177_08884064221103902 – Supplemental material for Methodological Training in Special Education Doctoral Programs: A Mixed-Methods Exploration
Supplemental material, sj-docx-3-tes-10.1177_08884064221103902 for Methodological Training in Special Education Doctoral Programs: A Mixed-Methods Exploration by Catherine Corr, Hailey Love, Melinda R. Snodgrass, Justin L. Kern and Mia Chudzik in Teacher Education and Special Education
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
Author Biography
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
