Designing Mixed Methods Research by Mixing and Merging Methodologies: A 13-Step Model

Research Questions, Research Components, and Research Strands

The research question is an essential building block of all research (Step 5 in Table 1), and plays an important role in making design decisions (Blaikie, 2010; Bryman, 2007; Teddlie & Tashakkori, 2009). The research question is a research component, which we define as any entity that is the object of research design, such as the research question, research purpose, method of data collection, or method of data analysis. In this article, we define a research strand (Steps 7, 8, and 9) as the set of research components within one study, whose joint application leads to one set of findings. In most mixed methods research, more than one set of findings is generated, which are subsequently integrated. Therefore, it is important to distinguish between the separate research strands of a mixed methods study.

The research components of one research strand should be coherent. Coherence can be achieved by selecting the research components based on an existing methodology, which describes which research components should be used in combination, and how. This view of cohering components is based on Joseph Maxwell’s (2013) interactive research model. In his model, Maxwell refers to coherence between the five research components: goals, conceptual framework, research questions, methods, and validity. We extend the concept of coherence to include any research component that is part of a research strand.

Methodologies

A methodology (Steps 9 and 12) is a coherent framework that is based on specific philosophical assumptions and sociopolitical commitments and that involves specific methods and guidelines for practice (Greene, 2008, 2015). Examples are grounded theory, the case study, and the survey. In this article, we view a methodology as an established configuration of research components whose combined action achieves a specific research purpose in a specific way. Thus, rather than making separate design decisions completely anew for each separate research component, a methodology shows the researcher which decisions, selections, and arrangements are possible for each research component, and thus tells a researcher how coherence between research components can be established.

Different methodologies have different configurations of research components. Take for example the case study and grounded theory. A case study is an empirical inquiry in which a social phenomenon is investigated and described in-depth within its real-life context (Yin, 2013). The social phenomenon in its real-life context constitutes the case, or, when more contexts are investigated, the multiple cases, of the study. A case study comprises various data sources and typically takes a longer period of time compared with, for example, an experiment. In contrast, a grounded theory study is aimed at generating middle-range theory (Glaser & Strauss, 1967; Merton, 1968). This theory is grounded in, yet abstracted from, the data. The theory is derived from repeated analysis, often of textual data, using constant comparison. New data are collected and analyzed until the theory is saturated (theoretical sampling). In terms of their research components, a case study and grounded theory have different research purposes, aimed at developing detailed insight and description versus a middle-range theory. They have different scopes, namely a scope of only one specific case or several specific cases versus a case-transcending scope. They also differ in the data analysis techniques that are used: various types of data analysis and sampling in the case study, and constant comparison and theoretical sampling in grounded theory.

We define a method as a technique for the sampling, data collection, or data analysis, through which methodologies are implemented (Sandelowski, 2003). It is often not easy to distinguish between a methodology and a method, partly because the same label can be used to describe both a methodology and a method (Sandelowski, 2003). For example, the term narrative interview can refer to a specific method of data collection, in which participants are asked to tell in detail about their experiences with a specific phenomenon, and how these experiences came about. Alternatively, the term narrative interview could refer to a methodology, which, in addition to the narrative interview as a method, includes a specific type of analysis aimed at reconstructing a narrative story, and at discovering causation in singular events. In this article, we will usually use terms such as narrative interview, grounded theory, and survey to refer to methodologies, unless otherwise stated.

One and the same research question can be addressed using various methodologies, and it is up to the researcher to decide which methodology to use. Thus, to answer the question “Which circumstances lead to procrastination among university students?” researchers could conduct a case study, a survey study, or an experiment. To obtain detailed insight into the process of procrastination, researchers could interview a procrastinator, observe a procrastinator in action, theoretically reflect on a report about a procrastinator, or place themselves in a situation in which they are likely to procrastinate.

Different methodologies impose different requirements on their research components. One important requirement for a survey is that the sample size be sufficiently large. In contrast, the requirement for a large sample does not apply to the narrative interview methodology. In addition to differing in their requirements, methodologies may also differ in their assumptions. The research components of specific methodologies may have specific assumptions concerning ontology, epistemology, and/or axiology, such as specific personal values, a specific view on causation (Johnson, Russo, & Schoonenboom, 2017), a specific view on trustworthiness or objectivity (Schoonenboom, 2017a), or a specific view on what counts as knowledge (Lincoln, 2010). One assumption, for example, underlying in-depth analysis of one case, is that causation is already visible in one case, in the paths through which one unique event leads to another unique event. This type of causation is different from the probabilistic causation that is investigated in survey research, which assumes that causation becomes visible by considering many instances (Johnson et al., 2017).

Research Purposes and Research Purpose Types

A researcher chooses the methodology for his or her study based on his or her research purpose (Steps 1 and 3). We define a research purpose as something a researcher wishes to achieve by conducting a research study. Research purposes can be formulated in various domains, such as intellectual, personal, and practical (Maxwell, 2013), and at various levels of abstraction. In this article, we distinguish between immediate research purposes (Step 3) and remote research purposes (Step 1). The immediate research purpose is what a researcher aims to achieve by answering the research question. The remote research purpose is the overall, larger, further-reaching purpose, which the study should contribute to achieving. For example, an immediate research purpose “Understand the role of perceived ease of use and perceived usefulness in university teachers’ decisions to use an e-learning tool” may be pursued to contribute to the remote research purpose “Develop guidelines to support university teachers in choosing and working with e-learning tools.” The immediate research purpose “Obtain an in-depth description and understanding of procrastination among students at this university” may be pursued to contribute to the remote research purpose “Understand how procrastination develops through students’ actions.”

Research purposes come in various research purpose types (Steps 2, 4, and 7). In addition to the remote research purpose type (Step 2) and the immediate research purpose type (Step 4), we distinguish the research strand purpose types of individual research strands (Step 7). Common research purpose types include exploration, description, explanation, prediction, evaluation, understanding, change, and assessing impacts (Blaikie, 2010). Several research purpose types can be combined within one study. A case study, for example, could be used to obtain both an in-depth description and understanding.

The immediate research purpose and purpose type and the research strand purpose type play a role in choosing a methodology. For example, for an immediate research purpose that is aimed at an in-depth description and understanding of a specific setting, case study methodology would be a reasonable choice; and for a research purpose that is aimed at understanding a specific phenomenon, grounded theory would be a reasonable choice.

Scope

Scope refers to the reach of the conclusions that are drawn from research findings. We use the term scope of the study (Step 6) to refer to the reach of the conclusions that are drawn at the level of a study as a whole, and the term research strand scope (Step 9) to refer to the scope of the conclusions that are drawn on the basis of the findings of one individual research strand.

Methodologies differ in scope. The scope of a case study, for example, is typically narrow. Its purpose is first and foremost to provide a detailed description and understanding of a specific setting. Additionally, a case study may, but need not be used for generalization purposes in the form of analytic generalization (Yin, 2013). On the other hand, grounded theory is conducted to derive a so-called “middle-range theory” (Merton, 1968), a theory that goes beyond the cases that have been observed. Its methodology of theoretical sampling is aimed at reaching the point where adding new cases will not lead to changes in the theory; the implication is that the theory will also apply to new cases and can be generalized beyond the observed cases. In that case, the point of theoretical saturation is said to be reached.

The research strands of a mixed methods study often have different scopes. In a first research strand, for example, a researcher could derive themes from narrative interviews among a small sample of participants; the scope of the finding of this first research strand will usually be that small sample. In a second research strand, the researcher could take a random sample of an entire population, and administer to this random sample a questionnaire that is based on the themes of the first research strand. The scope of this second research strand will be the entire population.

Mixed Methodologies and Merged Methodologies

We define a mixed methodology as a methodology with more than one research strand, each of which is conducted according to a different methodology. Examples of mixed methodologies are the case study and ethnography (Bazeley, 2016; Morse & Niehaus, 2009). In addition to such established mixed methodologies, a mixed methodology comes into being whenever a researcher combines two or more research strands, each belonging to a different methodology, within one study. Such emerging mixed methodologies can be described and given labels, for example, “Using a survey to generalize pathways that emerge from narrative interviews” (Example 1).

We define a merged methodology as a methodology with only one research strand, which is created by merging elements from two or more existing methodologies, comparable to Gobo’s (2015) “merged methods” (see also Bazeley, 2016). In Example 2, the researcher collected short utterances from many participants. This is a merger of a data type that belongs to a qualitative methodology (utterances) and a sample size that belongs to a quantitative methodology (large sample size). The researcher gave this merged methodology the label realist survey (Schoonenboom, 2017b), a variation on the realistic interview (Pawson & Tilley, 1997).

Purpose of Mixing and Purpose of Merging

Combining methods in mixed methods research is done by a researcher for a specific purpose, which we call a purpose of mixing, which is different from the research purpose (Bryman, 2006; Greene, Caracelli, & Graham, 1989; Schoonenboom, Johnson, & Froehlich, in press). In this article, we extend the purpose of mixing to refer to the rationale for combining methodologies. We will present examples of mixed methods research in which methodologies, rather than methods, are combined (Flick, 2017; Greene, 2008, 2015).

A purpose of mixing describes how the two or more research strands of a mixed methods study are connected. For example, to understand the pathways through which a specific phenomenon develops, a researcher may decide to first explore these pathways, using narrative interviews, and then to investigate whether these pathways are valid for the population at large, using a questionnaire. In that case, the purpose of mixing is “generalization to the same population” (Schoonenboom et al., in press). The aim is to generalize the findings of the first research strand to the population by conducting a second research strand (see Example 1 for a detailed explanation).

We define a purpose of merging as the purpose for merging components from more than one methodology into one new merged methodology. An example of a purpose of merging is “to obtain a conclusion based on qualitative data from a large sample.” A researcher can pursue this purpose of merging by collecting short statements (a data type borrowed from a qualitative methodology) from many participants (a large sample size borrowed from a quantitative methodology; see Example 2 for a detailed explanation).

Example 1: Pathways From Abuse to Sexually Transmitted Infections

The first example in which two methodologies are mixed within one study is Mitchell Fuentes’ (2008) study on the relation between women’s experiences of abuse and a heightened risk for sexually transmitted infections (STIs). Example 1 is described in Figure 1; its 13 steps and the building blocks involved are listed in Table 2.

OPEN IN VIEWER

Figure 1.

Example 1: Summary of Mitchell Fuentes (2008) study, discussed by Hesse-Biber (2010).

OPEN IN VIEWER

Table 2.

Example 1: A Real-Life Example of Mixing Methodologies: Mitchell Fuentes (2008).

Step	Description
Remote
1. Remote research purposes	The development of STI and HIV prevention strategies and the empowerment of women (Mitchell Fuentes, 2008)
2. Remote research purposes type	Change
This study
3. Immediate research purpose	Understanding pathways through which experiences of abuse among women lead to an increased risk of STIs
4. Immediate research purpose type	Understand
5. Research question	What are the pathways through which experiences of abuse among women lead to an increased risk of STIs?
6. Scope of the study	A population of abused women
Research strands
	Research Strand 1	Research Strand 2
7. Research strand purpose type	Explore	Describe
8. Research strand scope	A sample of 28 women	A population of abused women
9. Research strand methodologies	Narrative interviews	Survey
Research strand combination
10. Purpose of mixing	Generalize the pathways to a population
11. Adaptations required for mixing	The 28 women should represent the population as well as possible; maximum variation is required. The constructs of the questionnaire should match the themes discovered in the narrative interview
12. Mixed methodology label	Using a survey to generalize pathways that emerge from narrative interviews
13. Iteration	Not visible in Example 1

Note. STI = sexually transmitted infection.

Example 1 involves a mixed methodology with two research strands, and the purpose of mixing is to use a survey in Research Strand 2 to generalize pathways that emerge from the narrative interviews in Research Strand 1. Table 2 describes the 13 steps and the building blocks involved. Note that these 13 steps are not necessarily the steps that were followed, either consciously or unconsciously, by the researchers in any of the examples.

Example 1 starts with a remote research purpose, “the development of STIs and HIV prevention strategies and the empowerment of women.” The remote research purpose type is change: The remote research purpose is to empower women and to decrease their risk of STI and HIV. To successfully develop the required strategies, more knowledge is needed about the causes of STI and HIV among abused women. Therefore, the immediate research purpose of this study is “Understanding pathways through which experiences of abuse among women lead to an increased risk of STIs.” The immediate research purpose type is understand. The research question whose answer achieves the immediate research purpose is “What are the pathways through which experiences of abuse among women lead to an increased risk of STIs?”

Now the question arises about the scope the researcher should choose for the study as a whole. The researcher could choose a narrow scope, focusing on a detailed understanding of how experiences of abuse lead to an increased risk of STI among a small sample of women. However, the scope of the remote research purpose of developing prevention strategies encompasses an entire population of abused women. Therefore, choosing a population of abused women as the scope of the study would bring the researcher closer to her remote research purpose than would focusing on a small sample. This explains why a population of abused women was chosen as the scope of the study.

The choice of a population of abused women brings some practical problems with it, as the pathways were unknown at the beginning of the study. In terms of research purpose types, there is a need both to explore the related factors and their pathways and to describe the factors and their pathways that can be considered valid for the population. As always, there are various ways to obtain valid conclusions, but there are some ways that would not work. For example, as the factors were unknown at the beginning of the study, a monomethod survey study using a closed questionnaire would not work, as the researcher would not know what to ask. A monomethod qualitative study using narrative interviews would probably not work either, as it is unlikely that the researcher would have the resources to interview a sample that could be considered large enough to be representative of the population.

One possible solution, which was practiced in this study, is to have an exploratory strand first, followed by a descriptive strand. The exploratory strand can reveal the factors and their pathways. Its scope can be narrow, and in Example 1 the scope consisted of a sample of 28 women. In a second research strand, the factors and their pathways were translated into a questionnaire, which was administered to a larger sample of 215 women. The second research strand enabled themes that were found in the exploratory strand to be generalized to the population of abused women. Each research strand had its own methodology: the narrative interview for the exploratory strand and the survey for the descriptive strand. The research strands were connected by the purpose of mixing “generalization to the same population”: The results of the first research strand (the factors and their pathways) were generalized in Research Strand 2 to the population of abused women, which was the scope of the study.

Combining the two methodologies was not a simple addition. Combining the two methodologies brought with it specific requirements and the need for adaptations that would not have been necessary if these methodologies had been used on their own. Restrictions applied to the sample of Research Strand 1 and to the survey question topics of Research Strand 2. Although a sample of 28 women cannot be representative of a whole population, it is nevertheless important, that chances are high that factors and pathways that are found in the population of interest would also show up in the small sample. Therefore, a maximum variation sample was required. Thus, the scope of the study as a whole restricted the kind of sample that would be useful for Research Strand 1. Similarly, the factors and pathways measured in Research Strand 2 had to match the factors and pathways found in Research Strand 1. Otherwise, it would be incorrect to say that the factors and pathways of Research Strand 1 were generalized in Research Strand 2 (which does not mean that the results of Research Strand 2 should match those of Research Strand 1).

Example 2: University Teachers’ Decisions to Use e-Learning Tools

In Example 2, a new methodology is created using elements of existing methodologies. Schoonenboom (2017b) investigated the opinions of university teachers on the role that ease of use and usefulness play in their intention to use e-learning tools. Example 3 is described in Figure 2 and its 13 steps and the building blocks involved are listed in Table 3.

OPEN IN VIEWER

Figure 2.

Example 2: A real-life example about factors that influence university teachers’ intention to use e-learning tools (Schoonenboom, 2017b).

OPEN IN VIEWER

Table 3.

Example 2: A Real-Life Example of Merging Methodologies: Schoonenboom (2017b).

Step	Description
Remote
1. Remote research purposes	Develop guidelines to support university teachers in choosing and working with e-learning tools, and achieve a more nuanced understanding of technology acceptance
2. Remote research purposes type	Change and understand
This study
3. Immediate research purpose	Understand the role of perceived ease of use and perceived usefulness in university teachers’ decisions to use an e-learning tool
4. Immediate research purpose type	Understand
5. Research question	What role do ease of use and perceived usefulness play for university teachers when they consider using an e-learning tool to perform a specific instructional task?
6. Scope of the study	Teaching staff at a few universities in the Netherlands
Research strand
	Research Strand 1
7. Research strand purpose type	Explore and Understand
8. Research strand scope	Teaching staff at a few universities in the Netherlands
9. Research strand methodologies	Merger of survey and interview
Research strand combination
10. Purpose of merging	Obtain a conclusion based on qualitative data from a large sample
11. Adaptations	Blend of elements of existing methodologies: (1) a combination of positioning and explanation from the realistic interview (Pawson & Tilley, 1997); (2) utterances about own experiences from the interview; (3) large sample size and use of descriptive statistics from the survey. Main adaptations: use of “thin descriptions.”
12. Merged methodology label	Realist survey
13. Iteration	Not visible in Example 2

Example 2 involves a merged methodology with one research strand, and the purpose of merging is to obtain a conclusion based on qualitative data from a large sample. Table 3 describes the 13 steps of Example 2 with the building blocks involved.

Schoonenboom’s (2017b) study had two remote purposes, one related to practice, namely to develop guidelines to support university teachers in choosing and working with e-learning tools, and one scientific purpose. The study had partly arisen out of a belief that the technology acceptance model did not match university teachers’ experiences of working with e-learning tools. The purpose type of the practical remote purpose was change; the researcher wanted to contribute to a change for the better in the support of e-learning at universities. The purpose type of the scientific remote purpose was understand; the researcher strove for a more nuanced understanding of the reasons for using e-learning tools.

In order to be able to support university teachers to use e-learning tools, it is necessary to know which factors play a role in their decision to use an e-learning tool. Therefore, the immediate research purpose was “Understand the role of perceived ease of use and perceived usefulness in university teachers’ decisions to use an e-learning tool,” and its purpose type was understand. The research question corresponding to this immediate research purpose was “What role do ease of use and perceived usefulness play for university teachers when they consider using an e-learning tool to perform a specific instructional task?” This research question partly built on the existing technology acceptance model and included this model’s constructs ease of use and usefulness, while it left open what the connection between these constructs would be.

As a scope, the researcher chose the university teaching staff at a few universities in the Netherlands. Her choice was similar to the choice of scope in Example 1: By selecting a relatively broad scope, she would come closer to the remote goal of supporting teaching staff, as the findings within this scope, in contrast to findings in a small sample, would provide a good basis for deriving guidelines for the populations of these teachers.

The research purpose type of the study’s research strand could be described as explore and understand. In the technology acceptance model, the relationships between ease of use, usefulness, and intention were always determined by the researcher on the basis of respondents’ answers to separate questions about ease of use, usefulness, and intention. As a result, it was not known how university teachers themselves viewed the relationships between these three constructs, and thus exploration into these views was necessary.

In Example 1, where the researchers had a comparable challenge of combining an initial exploration with a study scope at the level of the population, the researcher chose a small scope for an initial exploration, followed by a generalization to a population. In Example 2, the situation was different. In addition to modelling the relationships between ease of use, usefulness, and intention, the researcher wanted to obtain a rich collection of opinions about these relationships from a large sample of teachers that would be representative of the population. Therefore, rather than combining two methodologies—narrative interviews with a small sample and a quantitative survey with a large sample—the researcher used elements of both approaches to create a qualitative methodology that would nevertheless cast a wide net on a large sample. She asked teachers whether they agreed with a number of statements about the relationships between ease of use, usefulness, and intention, and to elaborate their positions. The result was a large number of one-paragraph elaborations (“thin descriptions” according to Schoonenboom [2017b]), which were then categorized as showing one of the various possible relationships between the three constructs. The researcher gave this methodology its own name: the realist survey.