Design Patterns in Mixed Methods Research: Embedding Mixed Methods Matched Comparisons Sampling in Previous Design Decisions

Phase 1: Setting Targets (What and Who)

Phase 2: Determining the Qualitative Sample

Phase 3: Selecting the Qualitative Sample Using Quantitative Data

Mixed Methods Matched Comparisons Sampling as a Design Pattern

Turning now to mixed methods matched comparisons sampling as a design pattern, I note that it can be classified as “qualitatively driven” (Hesse-Biber, 2010; Morse & Niehaus, 2009). In mixed methods matched comparisons sampling, quantitative data are subordinate; they serve the qualitative research strand, not the other way around. The distinction between qualitatively and quantitatively driven studies is well established in the mixed methods literature (Tashakkori et al., 2021) and has been made in various ways. According to Morse and Niehaus (2009), for instance, a qualitatively driven mixed methods study is inductive. I argue that the design pattern for mixed methods matched comparisons sampling is qualitatively driven for at least three related reasons: (1) The quantitative data are not analyzed to draw conclusions, (2) the groupings they create are preliminary and may change as a result of qualitative analysis, and (3) the sampling of quantitative data is determined by the requirements for the qualitative research strand. To my knowledge, this last argument is new in the discussion on mixed methods sampling.

In the design pattern for mixed methods matched comparisons sampling, quantitative data are used to identify members of different subpopulations, but they are not analyzed statistically. A design with this characteristic would be classified as the case-selection variant of an explanatory sequential design (Creswell & Plano Clark, 2018) or a preliminary quantitative input design (Morgan, 2014). Accordingly, Visser et al. (2018) did not determine the overall level of procrastination in their first-year students, nor did they correlate procrastination scores with other relevant variables. Each individual’s score on the APSI test was classified as either high, intermediate, or low. From each of the three groups, eight students were invited for an interview. Hence, the quantitative data were used for diagnostic purposes only, making them subordinate to the qualitative data. The qualitative data were analyzed and used to draw conclusions. Although not unimportant (Corrigan & Onwuegbuzie, 2023), such quantitative data are not the carriers of conclusions in this design pattern, but instead diagnostic tools for identifying members of different subpopulations.

The subordinate character of quantitative data in the design pattern for mixed methods matched comparisons sampling is further demonstrated by the fact that quantitative grouping into subpopulations is always preliminary and may change based on qualitative findings; something that did not occur, but might have occurred in Visser et al. (2018), in which the qualitative outcomes aligned perfectly with the quantitative subpopulation divisions. In this design pattern, the quantitative diagnostic scores, such as those of the APSI, should not be interpreted as proof that individuals with essentially different characteristics, such as different procrastination levels, exist outside the context of the study. Instead, such grouping is viewed as a provisional and temporary distinction made for the purpose of the study’s qualitative research strand only, which again shows the subordinate role of quantitative data in the design pattern.

Finally, in the design pattern for mixed methods matched comparisons sampling, quantitative data are also subordinate because the qualitative research strand determines how the quantitative sample is drawn. The aim of the design pattern is to design a sampling plan for a qualitative research strand in which the experiences of a subpopulation are compared to those of other subpopulations within the same population, with the aim of acquiring insights into the factors that differentiate one subpopulation from another. Not unlike a quantitative experiment, such a comparative qualitative research strand requires a population acting in a relatively homogeneous environment. Only then can identified differences be attributed to differences between subpopulations and not to differences between environments. As a result, the population inhabiting this relatively homogeneous environment will often be small. Visser et al. (2018) chose as their relatively homogeneous environment the first year of one teacher education program at one specific university. They studied the reactions of different subpopulations (students with high, intermediate, and low levels of procrastination) to this environment. Within such a small population, data are, to the extent possible, collected from all members, so that the heterogeneous qualitative sample can be drawn from a big enough pool. Accordingly, Visser et al. (2018) administered the APSI questionnaire to all 215 members of their population (186 responded). In a quantitative-only study, the researchers might have drawn a random sample of students from several different clusters (i.e., universities) to make their findings more generalizable. Instead, Visser et al.’s (2018) choice to survey all members of their small population is based on the requirements of their comparative qualitative research strand.

In summary, the qualitative research strand of the design pattern for mixed methods matched comparisons sampling determines how the quantitative sample is drawn. If quantitative considerations had played a role, then the quantitative sample would have been a random sample from a relatedly heterogeneous population to maximize generalizability. Instead, the quantitative sample in the design pattern is drawn from a relatively homogeneous environment to optimize comparison in the qualitative research strand. Ideally, the complete target population would be sampled—an ideal that is almost never attained.

In this article, I have analyzed the design decisions related to one mixed methods sampling design pattern. My conclusion is that the design pattern for mixed methods matched comparisons sampling comes into being when researchers make specific design choices: they study experiences, use qualitative methods of data collection, they study an identifiable subpopulation and compare it to other subpopulations of the same population. Given these choices, I recommended collecting quantitative data from a population in a relatively homogeneous environment, and use these quantitative data to draw a sample for the qualitative research strand. Furthermore, I advised including as many members of the population as possible in the quantitative sample, ideally approaching a sample of the complete target population (Patton, 2015).

Contribution to Mixed Methods Research

To better understand the benefits and limitations of focusing on only one design pattern, the design pattern for mixed methods matched comparisons sampling can be compared with other authors’ descriptions of mixed methods sampling in four ways. First, mixed methods sampling is often presented in the mixed methods literature as a combination of quantitative probability sampling and qualitative purposeful sampling (Creamer, 2018; Tashakkori et al., 2021). However, in the design pattern for mixed methods matched comparisons sampling, quantitative sampling is not random. Instead, it is a sample that approximates the complete, usually small, target population (Patton, 2015)—in the exemplar, 186 of the 215 population members. Random sampling is preferable if quantitative data are going to be analyzed statistically, which is not the case in this design pattern. Furthermore, the mixed methods literature describes quantitative and qualitative samples as relatively independent, each determined by its own logic. In contrast, in this design pattern, quantitative sampling is not independent of the qualitative research strand but rather is determined by it. A sample that approximates the complete target population of one homogeneous environment is sought to optimize comparison in the qualitative research strand.

Second, this article locates the selection of different groups in a different phase of the research process than is common in the mixed methods literature. In the mixed methods literature, selecting different groups is seen as a decision within sampling, for instance, “selecting settings, groups and/or individuals that represent a sample” (Corrigan & Onwuegbuzie, 2023, p. 139) or “identif[ying] the stratification variable of interest and the subgroups on that variable (as in QUAN stratified sampling)” (Tashakkori et al., 2021, p. 169) [emphasis added]. In contrast, in Table 1, the selection of different groups is seen as a selection of subpopulations, which happens prior to sampling. As a result, this design pattern recognizes that the different subgroups in a sample represent different subpopulations, instead of being simply different groups within a sample.

Third, the preceding design decisions that I identified as most relevant to mixed methods matched comparisons sampling are different from those mentioned in the mixed methods literature. The preceding decisions in Table 1 refer to the needs of the qualitative research strand, such as what experiences to study, the subpopulation of interest, whether to include comparison, and the methods of qualitative data collection. The design decisions relevant to mixed methods sampling mentioned in the mixed methods literature include the purpose of the study, its purpose of mixing, its theoretical drive, the research question, prior knowledge, and the study’s research paradigm (Corrigan & Onwuegbuzie, 2023; Creamer, 2018; Morse & Niehaus, 2009; Tashakkori et al., 2021).

Lastly, mixed methods design patterns change how one views mixed methods research design. “Fully integrated mixed methods research design” (Creamer, 2018; Teddlie & Tashakkori, 2009) involves integration of qualitative and quantitative components at all stages of the research process separately, such as formulating the research question, sampling, data collection, data analysis, and drawing inferences. In contrast, I view a mixed methods study as a collection of design patterns that each spans various research components. Design patterns are combined in a study and overlap. It is possible that some of their requirements conflict with each other, and such conflicts should then be resolved. For example, if quantitative data are not only used to draw a qualitative sample, but are also analyzed statistically, the optimal sample for inferential statistical analysis (a quantitative random sample of the population at large) may differ from the optimal quantitative sample for the qualitative research strand (an approximation of the complete target population of people sharing a specific environment).

Fetters et al.’s (2013) mixed methods integration framework uses construction metaphors: building, connecting, merging, and embedding. In this article, I have turned to a different metaphor, inherited from architecture: that of design patterns (Alexander et al., 1977). I have shown how one decision in research design, that of mixed methods matched comparisons sampling, is embedded in a larger whole, a design pattern, which is itself one of various design patterns within a mixed methods design. I have thereby provided mixed methods researchers with more detailed support related to mixed methods matched comparisons sampling than would otherwise be possible.