Multi-Resolution Design: Using Qualitative and Quantitative Analyses to Recursively Zoom in and out of the Same Dataset

Abstract

A recent challenge is how to mix qualitative interpretation with computational techniques to analyze big qualitative data. To this end, we propose “multi-resolution design” for mixed method analysis of the same data: qualitative analysis zooms-in to provide in-depth contextual insight and quantitative analysis zooms-out to provide measures, associations, and statistical models. The raw qualitative data is transformed between excerpts, counts, and measures; with each having unique gains and losses. Multi-resolution designs entail transforming the data back-and-forth between these data types, recursively quantitizing and qualitizing the data. Two empirical studies illustrate how multi-resolution design can support abductive inference and increase validity. This contributes to mixed methods literature a conceptualization of how mixed analysis of the same big qualitative dataset can create tightly integrated synergies.

Keywords

multi-resolution design mixed analysis quantitizing qualitizing data transformation

Technological advances are expanding the potential of mixed methods research (Chang et al., 2021; Sripathi et al., 2024). Digitization is creating an exponential growth in qualitative data, such as social media posts, videos, documents, and transcripts. Simultaneously, algorithms are increasingly able to analyze qualitative data at scale, including categorizing entities, assessing semantic similarity, and visualizing data. These advances create opportunities and challenges.

The opportunity is to obtain frictionless method integration during analysis. Anchoring both human-qualitative and computational-quantitative analyses in the same data can create tightly integrated findings. This could circumvent the traditional tradeoff between depth (prioritized in qualitative analysis) and breadth (prioritized in quantitative analysis), yielding an analysis that is simultaneously grounded in particulars and robustly general (Shahin, 2016).

The challenge is to conceptualize how a single big qualitative dataset can be analyzed qualitatively and quantitatively simultaneously. Traditionally, mixed methods research focused on mixing separate silos of qualitative and quantitative data, either in parallel or in sequence. But, big unstructured qualitative datasets (e.g., heterogeneous collections of text, images, videos), can be recursively restructured for both qualitative and quantitative analyses.

We propose and develop “multi-resolution design” to conceptualize using qualitative analysis to “zoom in” on contextualized particulars and quantitative analysis to “zoom out” revealing statistical patterns (Gillespie et al., 2024). Multi-resolution designs are exploratory, recursive, and transformational. As with transformation designs, qualities are converted into quantities, but, this process is bidirectional and occurs throughout the analysis. As with recursive designs, the analysis moves back-and-forth between qualitative and quantitative analyses, but, crucially the qualitative and quantitative methods are analyzing the same data.

Our aim is to conceptualize how qualitative interpretation and quantitative-computational analyses can be combined to analyze the same qualitative dataset. First, we review the growing number of studies that combine methods to analyze the same data, arguing that these emerging studies need more conceptualization. Second, we use the mixed methods literature to conceptualize multi-resolution designs as recursive transformation (qualitizing and quantitizing) during the analysis phase. Third, we analyze the gains and losses of structuring raw data for qualitative and quantitative analyses, and argue that recursively restructuring the data can accrue gains, offset losses, and harness synergies. Finally, we use two research studies, each with an interactive visualization that enables zooming in and out, to illustrate how a multi-resolution design can support abductive inference (i.e., theory creation) and increase the validity of research.

Mixed Analyses of Big Qualitative Data

Analyzing the same data quantitatively and qualitatively, termed mixed analysis, is increasing (Onwuegbuzie & Johnson, 2021). While there is a long history of mixed analysis with interview (Fakis et al., 2014) and focus group (Vogl, 2019) transcripts, it has recently burgeoned in big data studies of social media (Andreotta et al., 2019; Rodriguez & Storer, 2020). This increase is being driven by technological advances in storing, quantifying, and visualizing big qualitative datasets (O’Halloran et al., 2021).

Mixing qualitative and quantitative methods is particularly suited to big qualitative data. Indeed, qualitative interpretation is already routinely used within computational techniques (Chen et al., 2018) to identify features to measure and model, create human-labeled datasets for training algorithms, interpret clustered output from unsupervised techniques (e.g., LDA/Bert clustering), and create textual measures (Boyd & Schwartz, 2021).

Qualitative and computational analyses have complementary strengths. Comparisons consistently find that computational analysis is more efficient and reliable, while manual analysis is more subtle and contextual (Chang et al., 2021; Ho et al., 2021; Kermani et al., 2024). Automated techniques rely on lexical features while manual analysis can interpret the context (Rodriguez & Storer, 2020)—although this is changing with the advent of large language models (Ziems et al., 2024). Simply put, manual qualitative analysis is a valuable limited resource that should be targeted at phenomena that are subtle, contextual, or not easily defined.

There have been several proposals to formalize mixing qualitative interpretation with computational analysis. Shahin (2016) advocates two steps: (1) using natural language processing algorithms to provide a statistical overview; and (2) using in-depth manual analysis to examine interesting sub-samples. Refinements of this basic quantitative-to-qualitative sequence include adding computational steps for obtaining and cleaning data (Andreotta et al., 2019) and adding a final step for integrating the analyses (Chang et al., 2021).

These proposals provide useful step-by-step procedures for obtaining, cleaning, summarizing, sampling, and interpreting big qualitative datasets. However, they still conceptualize the qualitative and quantitative analyses separately, overlooking the potential for deeper integration of analysis and findings, such as by using qualitative-quantitative visualizations (Guetterman et al., 2015; O’Halloran et al., 2021). Also, more theorization is required (Onwuegbuzie & Johnson, 2021): How can the same raw data be structured to enable both qualitative and quantitative analyses? What are the tradeoffs of each type of data structure? And, what are the potential synergies?

Recursive-Transformative Mixed Analyses

Mixed methods theory conceptualizes how qualitative and quantitative approaches are integrated to yield findings that are more than the sum of the parts (Fetters & Freshwater, 2015). Fetters and Molina-Azorin (2017b) identify 15 dimensions of possible integration, including, theoretical, researcher, literature, sampling, design, research aims, data collection, analysis, and interpretation. Integration usually occurs during the final interpretation phase, when analyses, based on siloed qualitative and quantitative datasets, are synthesized. In contrast, multi-resolution designs pertain to integration during the analysis phase.

Integration during analysis has been rare (Bryman, 2006; Greene et al., 1989), but there is growing interest (Onwuegbuzie & Johnson, 2021). There are three ways that integration can occur during analysis (Love & Corr, 2022; Vogl, 2019). First, one form of data can be used to inform another. This can be either exploratory (e.g., using interviews to develop survey items) or explanatory (e.g., surprising experimental results being followed up with interviews or observations). Second, one form of data can be used to organize another. This could include grouping qualitative data in terms of quantitative variables (e.g., demographics, assessment scores, outcomes) or vice versa (e.g., grouping scores in terms of emergent qualitative types). Third, one form of data can be transformed into another, such as, converting text or images into numbers by counting or scoring. Multi-resolution designs are of this third type, namely, integration through data transformation.

There are two types of data transformation, or conversion: quantitizing and qualitizing (Teddlie & Tashakkori, 2009). Quantitizing entails converting qualities into quantities (Sandelowski et al., 2009). For example, Onwuegbuzie and Leech (2005) analyzed qualitative data thematically and then converted it into binary variables that were subsequently analyzed quantitatively. Qualitizing is less frequent and less clearly conceptualized (Nzabonimpa, 2018). It entails converting quantities into qualities, such as: creating narrative summaries of quantitative findings, defining qualitative types based on scores, and also examining the qualitative particulars that underpin quantities (Creamer, 2017; Vogl, 2019). While quantitizing and qualitizing are usually conceptualized as unidirectional, multi-resolution designs use both bidirectionally as data is transformed back-and-forth between qualitative and quantitative forms.

This bidirectional quantitizing and qualitizing can be conceptualized as a recursive mixed methods research design. Recursive research designs are rare (Teddlie & Tashakkori, 2009) and have focused on moving between separate qualitative and quantitative datasets (e.g., Christ, 2007; Kerrigan, 2014; Nzabonimpa, 2018). For example, Nastasi et al. (2007) report on a mental health improvement project in Sri Lanka where the intervention was iteratively modified over several years based on recursively analyzing qualitative (participant observation, interviews, focus groups) and quantitative (psychological measures, outcomes) datasets. Multi-resolution research, however, entails tighter recursion because it is based on the same dataset; meanings are transformed into numbers and the numbers are reverted to meanings.

A multi-resolution design, as a recursive transforming design, leverages aspects of both exploratory and sequential designs (Creswell & Creswell, 2018). Exploratory sequence designs begin with a qualitative analysis that feeds forward into a quantitative analysis that is grounded in the qualitative analysis. Explanatory sequence designs begin with a quantitative analysis that feeds forward into a qualitative analysis that aims to generate plausible explanations for the quantitative findings. Recursively moving back-and-forth between the qualitative and quantitative analyses creates a synergistic loop of grounding the quantitative analysis in qualitative particulars and generating plausible explanations for the quantitative findings.

In summary, multi-resolution research entails recursively transforming qualities into quantities and reverting quantities back into qualities. The qualities add validity and insight to the quantities, and the quantities add context and rigor to the qualities. Moreover, discrepancies between the qualitative and quantitative methods become more productive because they can confidently be attributed to the analyses rather than being an artefact of different datasets (e.g., time, context, or method of data collection).

Quantitizing and Qualitizing Tradeoffs

Qualitative and quantitative data are often presented as fundamentally different (e.g., Creswell & Creswell, 2018; Morgan, 2007), and even belonging to incommensurable paradigms (Guba & Lincoln, 1994). Qualitative data, it is argued, pertains to qualia, experiences, and meanings while quantitative data pertains to quanta, frequencies, and statistical associations (Shweder, 1996). While these are important differences, the distinction is often over-stated and has been criticized as misleading and unhelpful (Creamer, 2017). Moreover, in research practice the distinction often becomes fuzzy (Bazeley, 2017; Knappertsbusch, 2020).

Quantification can occur in qualitative research. For example, frequencies are routinely relevant to the claims being made. Phrases such as: “the majority of cases,” “most respondents,” “some participants,” “frequently following the same pattern,” and “more often than not” occur in qualitative research (Hammersley, 1996; Morgan, 2018). Such phrasing reveals that quantitative statements provide a valued background for interpreting qualitative data and making qualitative contributions.

Qualitative judgment is widespread in quantitative research. Most quantitative research entails the quantification of qualitative phenomena; turning observations, behaviors, and experiences into numbers (Berka, 1983; Trochim & Donnelly, 2021). Survey respondents quantify experiences using verbal anchors such as “sometimes” and “rarely” (French et al., 2007; Wagoner & Valsiner, 2005). The qualitative element also lurks in questions of validity, when, for example, a measure is evaluated through qualitative assessment (e.g., expert raters, concept sorting, face-validity checks, exit interviews; Gobo et al., 2022).

Given these overlaps between qualitative and quantitative methods, there have been arguments to abandon the dichotomy (Knappertsbusch, 2020). However, abandoning the distinction would be rash (Morgan, 2018). Instead, we propose that qualitative and quantitative data are not different ontological types, but instead merely different transformations of human events into data.

All data originates in events, namely, unstructured human activity. Data is created by structuring records of these events. Although all data is conceivably revertible to the originating events, this is challenging with traditional quantitative methods (e.g., surveys, experiments) where the output is usually data with a predefined quantitative structure. Thus, when choosing a traditional method of data collection, one is often also choosing a type of data, and thus also making a commitment to a certain type of analysis. However, big qualitative data does not have this predefined structure. Instead, it offers a choice: how should the raw data be structured?

Table 1. Conceptualizes How Events are Transformed Into three Broad data Types: excerpts (e.g., Quotes, Observations, Video clips), Categories (e.g., Counts of excerpts), and measures (e.g., survey Scores, Reaction Times, Textual Similarity). Because each data Type has Complimentary Gains and Losses, there is Added Value, or Synergy, in recursively Moving Between the data Types (Gillespie et al., 2024).

Table 1.

Data Types Created From Human Activity.

	Events
	Human activity that is naturally occurring (talking, photographing, posting on social media, purchasing) or created by research (interview, survey, task, experiment)
	Raw data
	Traces of events that are potentially analyzable. These can be unstructured (documents, images, transcripts, social media posts) or pre-structured (survey responses, reaction times, experimental outcomes)
Data type	Excerpts	Categorizations	Measures
Transformation	Selection	Categorizing	Scaling
Definition	Excerpts are selections of records that illustrate a concept (e.g., qualitative coding)	Categorizing uses clearly defined inclusion and exclusion criteria to determine in/out membership	Measures convert raw data into ordinal (e.g., ranking), interval (e.g., 5-point scale), or ratio (e.g., amount) variables
Example	A quote from a social media post	The topic of a social media post	The sentiment of a social media post
Gains	Provides high-validity contextual understanding (excerpts can be viewed in the context of the raw data)	Enables statements of equality (= and ≠), and thus counts and modes that can reveal differences and changes	Enables mathematical operations (<and > for ordinal; + and – for interval; and * and/for ratio) and thus statistical modeling and generalization
Losses	Excerpts are selective and can have an unclear relation to the broader data (e.g., unclear sampling, risk of cherry picking)	Loses some particularity and context. Homogenizes within categories and accentuates differences between categories	Scores are generic (scores risk being disconnected from raw data) and decontextualized (difficult to retrieve the context for the score)
Illustrative analyses	Case studies, conversation analysis, grounded theory	Counts, frequencies, mode, crosstabulations, chi-square	Exploratory data analysis, confirmatory hypothesis testing, statistical modeling
Multi-resolution design	Zooming in (qualitizing)	Qualitized or quantitized	zooming out (quantitizing)

Events refer to what actually happened (people talking, shopping, playing, working, posting, writing, etc.). Each event is an abundance of infinite potential and could be the basis for numerous analyses (Feyerabend, 2001). These “pre-data” events are maximally rich, contextual, and particular. The first in research is creating records of these events. For example, conversations are audio recorded and then transcribed and social media activity leaves a digital footprint of text, images, and metadata.

Raw data are the collected traces or records of events. Traditionally they have been pre-structured during data collection as either qualitative (e.g., interviews) or quantitative (e.g., survey ratings). Naturally occurring data, on the other hand, is unstructured. It is valuable because it is part of society reproducing itself (e.g., not artificial), but, because it was not engineered for research, it can be messy (unclear sampling, missing or bot-generated data). Accordingly, naturally occurring raw data requires wrangling and cleaning to become suitable for research. Moreover, because it is not pre-structured, naturally occurring data often offers a choice of data structures. For example, should social media posts be interpreted as excerpts? Or categorized as positive or negative? Or scaled using an algorithm for measuring sentiment?

Excerpts refer to a selection of raw data that forms the basis for an in-depth interpretative analysis. Typically, the researcher identifies bits of text or image that illustrate a theme in the data or a concept in the literature. Examples of theory-driven interpretation of excerpts include identifying third-turn repairs (Schegloff, 1992) and analyzing multivoicedness (Aveling et al., 2015). Analyzing excerpts is particularly useful for ambiguous, contextual, or multidimensional concepts—termed ballung concepts (the German word for cluster or congestion; Cartwright & Bradburn, 2011). These concepts are characterized by family resemblance, with multifaceted meanings that vary depending on the context and audience. For example, concepts such as “culture,” “power,” or “practices” help to sensitize researchers to aspects of the data. One limitation of excerpts is that it can be difficult to justify the selection or explain how it relates to the population.

Categorizations are more formally operationalized than excerpts. They have clear definitions with inclusion and exclusion criteria. Categorization is essentially a judgment of equivalence (= and ≠) because everything within a category is equal and everything outside the category is different. Examples include categorizing comments as toxic, themes in news articles, or people in pictures. These categorizations cannot be ranked or ordered on a scale. However, instances of a category can be counted. Categorizations tend to obscure within-category differences (e.g., one can count people and treat them as equivalent, but no two people are the same).

Measures entail systematically scaling empirical qualities to a concept being measured (Zeller et al., 1980). Phenomena with zero values (e.g., reaction time, number of sweets eaten) are measured using ratio scales. Phenomena without zero values but with equal intervals (e.g., date, location) are measured using an interval scale. Phenomena without a zero value and with unequal intervals (e.g., preferences, ordered categories) are measured using rank orders. Transforming raw data into measures gains powerful mathematical operations, but there is always a loss of specificity (subtle differences between units and intervals are suppressed). But, there are phenomena that are not suited to measurement (Berka, 1983; Zeller et al., 1980), such as, phenomenological experience (e.g., qualia), categorical phenomena (group membership), and ballung concepts (e.g., heuristics, culture), and forcing measurement of such phenomena leads in invalid measurement.

A multi-resolution design entails recursively transforming and restructuring raw data so as to enable both quantitative and qualitative analyses. Qualitizing provides particularity and context, but, the relation between the excerpts and the larger dataset is often unclear. Quantitizing gains statistical operations and can reveal subtle patterns only evident in aggregated data, but, it can obscure subtleties. By recursively transforming raw data into excerpts, categories, and measures it is possible to accrue gains and offset losses.

Illustrating the Added Value of Multi-Resolution Designs

Forcing a choice between transforming raw data into excerpts, categories, or measures limits the potential of the raw data to add value through multiple transformations. Recursive data re-structuring (i.e., moving back-and-forth between excerpts, categories, and measures) leverages the full breadth and depth of the data. Some of the resultant synergies are already clearly conceptualized in the mixed methods literature (Bryman, 2006; Greene et al., 1989).

Zooming-in, moving from measures towards the excerpts of the raw data, enables rich description (what is the phenomenon being measured?), conceptual framing (can the phenomena be conceptualized using various concepts?), and generating explanations (why was X not associated with Y?). Such zooming-in can offset the weaknesses of quantitative analysis (i.e., adding particularity and context), illustrate results, challenge results, explore the processes underlying statistical trends, and generate explanations for unexpected quantitative results.

Zooming-out, by structuring the raw data in terms of categories and measures, enables counting and measurement (how frequent is the observed phenomenon?), testing relationships (is Y associated with X?), and exploring associations (what might be associated with the phenomena?). Such zooming-out can offset the weaknesses of qualitative analysis (e.g., situating excerpts within the larger dataset), add completeness to the analysis (e.g., frequencies, differences, changes), and increase robustness and generalizability.

In addition to these established synergies, we argue that back-and-forth zooming-in and zooming-out can (1) facilitate abductive inference, and (2) increase the validity of findings. The following sections illustrate these contributions using two empirical studies.

Facilitating Abduction

The creative potential of mixed methods research has long been emphasized (Greene et al., 1989). The recurring idea is that combining qualitative and quantitative methods can yield novel findings that are irreducible to either method (Fetters & Freshwater, 2015). But how does this occur? And how can it be supported?

Compared to the numerous volumes on how to test theories, there is little conceptualization of how to create theories worth testing. Methodology texts emphasize either induction (bottom-up, often qualitative) or deduction (top-down, often quantitative), to the neglect of abduction (creating theories). Abduction entails going beyond data and existing theories to generate explanations for empirical and theoretical anomalies (Peirce, 1955; Tavory & Timmermans, 2014). Although abduction is central to most scientific breakthroughs, it has been difficult to formalize.

For abductive theory building, McGuire (1997) advises obtaining an in-depth grounded understanding of the phenomena and attending to discrepancies, logical tensions, and empirical surprises. Multi-resolution research supports both processes. Zooming into the particulars, what went on (i.e., what was actually said, what was done in the experiment, and the contextual details of an outlying datapoint) enables saturating oneself in the subtle and messy details of the phenomenon. Also, moving back-and-forth between the qualitative particulars and general findings supports discovering tensions, and simultaneously provides the means for probing and investigating them (i.e., by zooming-in or out on the phenomena).

We will illustrate how a multi-resolution design can facilitate abduction using data and findings from a study examining sense-making in online diaries (Zittoun et al., 2023). The aim of this project was to understand how people navigate personal and societal crises and specifically the material, social, and symbolic resources that aid their sense-making and adaptation in situation of vulnerability.

Data

The data were publicly available diaries. For over 20 years, people have been regularly posting public diary entries about their lives to websites (e.g., Open Diary, My Diary). These websites now contain many thousands of diaries some spanning 20 years, making them a valuable data source for studying sensemaking longitudinally. For the present illustration, we focus upon one diary from an unmarried man, without children, who began keeping a diary at age 48 and has over one thousand entries over 22 years (737,461 words across 1206 diary entries). This diary was selected from a sample of 324 diaries based on the following quantitative criteria: the diary spanned over 20 years, had relatively complete coverage, had a substantial drop in sentiment indicating a rupture or crisis, and the diarist consented to our publishing openly on his diary.

Analysis

The text from the diary was obtained and processed in Python. SpaCy (Honnibal et al., 2022) was used to extract entities (people, places, events, books, films), resolve coreferences (e.g., identify that “Mary,” “she,” and “sister” are the same), and create visual overviews of the diary to aid in-depth reading. Sentiment was analyzed using VADER (Hutto & Gilbert, 2014). Plots were created using Plotly (Plotly Technologies Inc, 2022), to track the changing profile of entities, themes, and sentiment over time. Many plots were created and interactively refined in conjunction with a close qualitative reading of the diary.

Figure 1 is a static screenshot of an interactive visualization that plots the diarists’ sentiment in four domains of the diarists’ life. These domains were identified based on a qualitative reading of the diary and then identified computationally using keywords (in terms of data type, these are categorizations). In the plot, each dot represents a diary entry, with the size of the dot indicating the number of sentences that pertain to the domain. The plot has two measures: the Y axis is the yearly rolling average sentiment score for the theme (−1 is most negative and 1 is most positive; Hutto & Gilbert, 2014); and the X axis is the date of the diary entry (from 1999 to 2021).

Figure 1.

A screenshot of the interactive visualization showing measures (sentiment score, date), categorizations (domains of life), and excerpts (sentences categorized by domain and scored for sentiment).

The interactive visualization (https://473x.github.io/mrd/diary.html) has added functionality: hovering over each dot produces a pop-up revealing the sentences represented by the dot (i.e., the excerpts; with bold text revealing the keywords used to categorize the sentence into the given domain), clicking on the center of each dot hyperlinks to the original online diary post (i.e., the raw data; currently disabled to preserve anonymity), the domains can be turned on/off using the legend on the left, and one can zoom in on parts of the figure by drawing a box on the figure. Accordingly, this figure can be used to recursively zoom out (to reveal macro patterns) and zoom in (to reveal the scored excerpts, or sentences, and even the raw data, or original posts). Thus, the interactive visualization preserves the links between the raw data, excerpts, categories, and scores.

Findings

The diarist had suffered from depression (indicated by low sentiment from 2000 to 2010), then he became quite fulfilled in his work (higher sentiment from 2010 to 2017), but then experienced a drop in his wellbeing (lower sentiment after 2017). Figure 1 shows how the sentiment for all four domains drops in 2017, with the lowest sentiment relating to his work. Qualitative analysis revealed that he retired in 2017 to care for his mother who died in 2020.

Zooming in on the posts and sentences talking about work revealed a puzzle; the experience of retirement was unstable. Three years before retirement, it is romanticized: “I want so badly the leisure time and peace that comes with being able to sit on the porch all morning” and “work seems to be less necessary for me now to be the person I have become and am meant to be.” But, in the moment of retirement, with the loss of social contact, is experienced as disruptive and traumatic: “The past two weeks have been an incredible transition period, really difficult psychologically as I’ve struggled with second thoughts” and “I’m feeling a bit lonely – and fighting off occasional, rather intense but brief bouts of depression where I ask myself if I’ve made a horrible mistake in retiring.” But, the imagination of retirement adjusts to become the necessary condition for post-retirement life: “It’s been 2 1/2 years since I retired, and that momentous milestone was one of the best things that ever happened to me” and “I’ve never looked back with the slightest regret about whether it was the right decision to retire.” This re-interpretation of retirement in positive terms is in stark contrast to the zoomed-out view in Figure 1, showing that the overall sentiment of the diary never recovers to the pre-retirement level, due to regular reports about feeling lonely and depressed.

The abductive insight is that singular events (i.e., retirement) can have multiple imaginations (Zittoun & Gillespie, 2015). Retirement shifts from being a reverie, to a crisis, to a resource for sense-making. This adds nuance to the established idea that retirement occurs in stages, from planning, through implementation, to adjustment (Wang & Shi, 2016). Moreover, it reveals potential tensions between the life narrative that adjusts to the transition and the routines of daily life that don’t adjust (i.e., entrenched social isolation).

Discussion

A purely quantitative analysis might have averaged the sentiment about retirement, missing the nuance between phases of retirement, or might have failed to see that although the sentiment does not adjust to retirement, the life narrative does adjust. Equally, a purely qualitative analysis might have failed to see that despite the adjustment of the life narrative, the overall pattern of life and associated sentiment fails to adjust. Moreover, it is the movement between both analyses that reveals tensions: the contrast between the zoomed out analysis (the decline in the sentiment measure across domains) with zoomed in analysis of excerpts (e.g., “one of the best things that ever happened”). Such transformations between measures and excerpts yield productive tensions that spur theory generation.

Enhancing Validity

Multi-resolution designs can increase the validity of findings by adding constraints to interpretation. Both qualitative and quantitative analyses aim to constrain interpretation with empirical data, yet both have also been criticized for having too many degrees of freedom, thus weakening the legitimacy of findings.

Qualitative research has been criticized for “cherry picking,” that is, selectively presenting quotations to suit a given interpretation (Morse, 2010), having poor reliability (Lee et al., 2020) and small samples with unclear generalizability (Seawright, 2016). Multi-resolution designs addresses this by revealing how a given excerpt relates to the larger dataset (e.g., frequency, sampling).

Quantitative research has been criticized for lacking validity (i.e., measures disconnected from the phenomena; Eronen & Bringmann, 2021), over-interpreting spurious correlations (Calude & Longo, 2017), and potentially encoding biases (O’Neil, 2016). A multi-resolution design helps address these concerns by reverting measurements to the underlying qualitative data, thus enabling a continuous checking for validity, causal patterns, and biases.

Multi-resolution research has extra rigor because it is doubly constrained; it must meet the quality criteria for both quantitative and qualitative research. Quantitative research aims for reliability, validity, and generalizability. Qualitative research aims to be credible, contextual, and richly described. Interpretations within a multi-resolution design must satisfy quality criteria both at a statistical level and also at the level of individual excerpts and cases (Seawright, 2016). This double constraint uses the full breadth and depth of raw qualitative data to limit the degrees of freedom. This can enhance overall validity beyond what can be achieved with a thin slice of the data (i.e., only excerpts, categories, or measures).

We illustrate this double-constraint in an experiment reported by Noort and colleagues (2021) about people voicing safety concerns. The literature on people speaking up has been dominated by survey and vignette methods (Noort et al., 2019a). Accordingly, the research team developed an experimental scenario to examine what people actually said when observing the experimenter being unsafe (Noort et al., 2019b).

Data

Participants (n = 404) took part in a brainstorming task on creative use of a plank (maximum weight 30 kg). Once they had completed this scene-setting task, participants evaluated ideas ostensibly suggested by the previous participant, while the research assistant demonstrated the ideas. All participants evaluated the same ideas including the risky suggestion to use the plank as a footbridge between two chairs. When the research assistant (weighing about 60 kg) started to assemble the wobbly footbridge and walk across it, would participants speak up? Would they say the footbridge was unsafe?

Analysis

In addition to several survey measures and traditional statistics, the study analyzed participants’ talk. This talk was measured and analyzed qualitatively in Python using Scattertext (Kessler, 2017; Noort et al., 2021). Figure 2 is a static screenshot of an interactive visualization of participants talk during the experiment. The figure combines excerpts (word-use in context), categorizations (voice/silence, concerned/unconcerned), and measures (probability of each word being in each category). The figure displays the words most spoken by participants who were unconcerned and silent (lower-left), who were unconcerned and yet still voiced (top-left), who were concerned and voiced (top-right), and who were concerned but remained silent (lower-right). In the interactive visualization (https://473x.github.io/mrd/experiment.html) clicking on a word (or searching for it) displays the word-use in context (split by the voice/no voice categorization).

Figure 2.

A screenshot of the interactive visualization that enables moving between macro patterns across experimental conditions and micro word-use in-context for each outcome (illustrated with the utterance “uhum”).

Findings

Over half of the participants who reported being concerned in the post-experimental survey did not speak up about their concerns. However, these participants were not silent; they spoke with hesitation (“oh,” “ah,” “uhum”; lower-right quadrant). In contrast, participants who were concerned and voiced (top-right quadrant) mentioned safety critical information (“thirty kg,” “kilos,” “maximum”) and were assertive (“be careful,” “but”). However, this plausible pattern was also deceptively simple. Recursively moving back-and-forth revealed that concerned-voicing participants often hesitated just prior to speaking up. Atypical speak-up utterance was: “Ah yes, I can… uhm before […] ... Or. Oh, ee, yes… And then how would you... (oof) Uhm. Like that's already a, but maybe, how can you, it's only, it can only take thirty kilos, so.” This yields the abductive insight that instead of hesitation being the inverse of speaking up it might actually be the early stage of speaking up, indicating cognitive load or communicating discomfort. This finding about “muted voice” gains legitimacy because it is doubly-constrained; evident quantitatively (with between participant measures) and qualitatively (with extracts positioned within the measures).

Discussion

The findings have added validity because the interpretation is constrained by both the qualitative and quantitative analyses, and the interactive visualization enables readers to recursively browse excerpts of participants’ talk organized in terms of the categories and measures used in the experiment. The open science movement has focused on sharing numeric data (Shrout & Rodgers, 2018), but, it is increasingly possible to share the raw data which underpin the numbers (e.g., observations and transcripts from an experiment; Glăveanu & Gillespie, 2021). Being open about the underlying raw data can enhance validity by re-connecting qualitative particulars with quantitative patterns, and enabling each to reinforce and challenge the other.

Data Requirements

The guiding principle of a multi-resolution design is that all quantitizing and qualitizing is reversible; the raw data should always be retrievable and analyzable. This principle, however, imposes two data requirements.

First, multi-resolution research data that can be analyzed both qualitatively and quantitatively. Table 2 illustrates suitable datasets. The simplest case is when the same raw data is used, for example, analyzing a corpus of text both qualitatively and quantitatively (e.g., the diary study reported above). Less clear cases entail pairing qualitative and quantitative data in terms of events or people. For example, quantitative meta-data about time or location could be paired with social media posts (same event) or a post-experiment interview could be paired with outcomes from the experiment (same people). In these cases, it is still possible to zoom-out to view overall patterns (e.g., geographic location, experimental outcomes) and zoom-in to analyze specific excerpts (e.g., posts, utterances). However, because the qualitative and quantitative analyses are applied to different (albeit paired) data, it is not the same data that is being recursively restructured. This limits the power of the multi-resolution design. For example, it would not be possible to leverage the synergy of using the qualitative interpretation to check and challenge the validity of the numeric results. Additionally, discrepancies between the qualitative and quantitative analyses cannot be attributed exclusively to the analyses, but, could be a byproduct of the different datasets (mode of collection, interviewer style, etc.). Qualitative and quantitative data paired merely by topic (not data, events, or people) is not suitable for multi-resolution research because it does not enable recursive quantitizing and qualitizing.

Table 2.

Types of data and Suitability for Multi-resolution Designs.

Illustrative datasets	Same data	Same event	Same people	Same topic	Suitability for multi-resolution research
Text, images, or observations that can be analyzed both qualitatively and quantitatively	x	x	x	x	Highly suitable
Social media posts paired with meta-data, survey scores paired with open-ended responses, what participants said in an experiment paired with outcomes		x	x	x	Somewhat suitable
Experimental outcomes paired with post-hoc interviews with participants or document analysis paired with a survey of people involved			x	x	Least suitable
Interviews or focus-groups with one group of people and then surveys with another group of people on the same topic				x	Not suitable

Second, to enable the double-constraint of multi-resolution research, the data should meet the quality criteria for both qualitative and quantitative research (Buckley, 2018). For the qualitative analysis, the data should comprise contextualized raw data (e.g., quotes, videos) that are credible, dependable, and afford thick description. For the quantitative analysis, the data should be sufficiently sampled and standardized to enable robust measurement. The sample size should be sufficiently powered from the focal analysis. Big qualitative data often easily meets the sample size requirements (sometimes the sample is the population), but, it can have problems with missing data and data quality. Examples of big qualitative data include: videos (e.g., YouTube, CCTV footage, on-the-scene reporting), social media posts (e.g., Reddit, X), online reviews (e.g., Fishbowl, Glassdoor, Trustpilot), and documents (e.g., diaries, formal complaints, parliamentary debates, political speeches, earnings call transcripts, company reports). Alternatively, one can create suitable data using traditional methods. Qualitative data collection methods (e.g., interviews, focus groups, observations, video recordings) produce data that can be quantified (Fakis et al., 2014; Vogl, 2019)—provided it is of sufficient volume. Equally, one can add qualitative data to quantitative processes for data collection (e.g., talk-aloud during survey completion, open-ended survey questions, video recording experiments, post-experiment interviews) to create tightly paired data (Moore & Gillespie, 2014; Niculae et al., 2015).

Discussion

Traditionally qualitative data was analyzed qualitatively, and quantitative data was analyzed quantitatively. But, this overly neat compartmentalization is breaking down. Big qualitative datasets, when combined with new computational techniques, offer a choice of qualitative, quantitative or mixed analysis. We argue that it is only by recursively transforming the raw data into multiple formats that the full depth and breadth of the data can be leveraged.

Expectations for research transparency are increasing. The cost of recording, transcribing, and storing large volumes of qualitative data are reducing. Advances in computing enable analyzing, visualizing, and disseminating qualitative and quantitative data side-by-side. The open science movement has created platforms to share both numeric and qualitative data. Thus, the choice between depth (qualitive) or breadth (quantitative) is anachronistic; we should expect research to be transparent in both depth and breadth.

Seizing this opportunity, there is a growing literature on how to analyze qualitative data using both human interpretation and computational processes (Andreotta et al., 2019; Buckley, 2018; Chang et al., 2021; Shahin, 2016). However, this literature has focused on procedural quantitative-to-qualitative sequences, with limited theorizing of the potential for synergies or recursive designs. Our contribution has been to utilize mixed methods designs to theorize multi-resolution design as recursive transformation; zooming in with qualitative analysis and zooming out with quantitative analysis. This makes contributions to the literatures on mixed, qualitative, and quantitative research.

Contributions to Mixed Methods Research

The challenge of big qualitative data requires a mixed methods approach. Computational analyses of qualitative data cannot supplant manual validation, interpretation, and abductive leaps of theory-building (Bazeley, 2017; Bennett, 2015). Algorithms can reliably identify objects, actors, actions, and quotations, score sentiment and more subtle phenomena, and inductively cluster themes. However, human interpretation is required to frame questions, identify features, ensure validity, interpret findings, and understand limitations. In short, the time is ripe to combine computation and interpretation within a mixed methods framework (Bazeley, 2017; Ho et al., 2021).

Multi-resolution designs replace the idea of qualitative and quantitative methods being conducted side by side (e.g., the metaphor of triangulation; Fetters & Molina-Azorin, 2017a) with a new metaphor of zooming-in and zooming-out on the same qualitative data. This conceptual shift is possible because raw data is conceptualized as transformable into excerpts, categories, and measures. Each data type has gains and losses, but, by moving between them, it is possible to accrue gains, ameliorate losses, and harness synergies.

Typically mixed methods research is conducted on separate datasets, with only loose integration at the interpretation phase (Åkerblad et al., 2021). Integrating findings across datasets is challenging (Fetters & Molina-Azorin, 2017b). Potentially important tensions can be byproducts of the different methods of data collection (i.e., different procedures, participants, social contexts, temporal timepoints). Multi-resolution research removes these uncertainties: any emerging tensions are in the data. Tensions between qualitative and quantitative analyses become more productive and any interpretation is doubly constrained, having to account for both the zoomed-in and zoomed-out analyses.

The two studies presented show how novel visualization technologies can be used to enhance mixed analyses of the same datasets. Joint displays are a powerful means to create tight integration, with qualitative and quantitative data presented side-by-side (Guetterman et al., 2015; McCrudden et al., 2021). A multi-resolution design does not require joint displays, but, such displays vividly operationalize the metaphor of zooming-in and out of the data. Our two interactive visualizations enable readers to zoom-in and out of our data; making both the quantitative and qualitative data simultaneously browsable, and providing added rigor and transparency to the analyses.

Across these contributions, multi-resolution designs can deepen the integration between qualitative and quantitative approaches thus enabling mixed-methods synergies (Fetters & Freshwater, 2015). In the illustrations above, neither the abductive insight (about retirement having different meanings before, during, and after) nor the validity challenge (about the voice-silence binary being over-simplistic) would have been possible with only quantitative or qualitative analysis.

One limitation is that, although multi-resolution designs allow for the tightest possible integration between qualitative and quantitative analyses, the fact that only one dataset is used, precludes synergies arising from multiple datasets. For, example different datasets might reveal very different aspects of the target phenomenon, thus creating bigger and more challenging tensions. Accordingly, it is still necessary to conceptualize potential integrations on many dimensions (Fetters & Molina-Azorin, 2017b).

Augmenting Quantitative Research

Multi-resolution research can contribute to the validity of quantitative research, which, at present, is struggling with the replication crisis (Shrout & Rodgers, 2018). This is the problem of experimental research, especially in psychology, failing to replicate (Open Science Collaboration, 2015). Broadly it is agreed that experimental research has had too many degrees of freedom (Wicherts et al., 2016). To date the focus has been on reducing the degrees of freedom by preregistering research and making data open access (Shrout & Rodgers, 2018). We suggest that these degrees of freedom can be further constrained by incorporating the qualitative data that underpins quantitative data.

All quantitative research entails quantifying qualities, and the associated risk that quantities do not reflect the underlying qualities. Accordingly, best practice is for researchers to return to the qualitative data to check the validity of their measures (Berka, 1983; Zeller et al., 1980). Multi-resolution research takes this idea further, arguing that it is increasingly possible for quantitized data to be revertible to its qualitative form throughout the analysis. Technological advances in algorithms and visualizations means that statistical analyses, even with large volumes of data, can be frictionlessly reverted to qualitative data. Recursively returning to the underlying qualitative data during the analysis, and making this underlying data available for secondary research, would increase the validity and transparency of experimental research.

As the costs of recording, storing, and analyzing data reduce, it could become routine to have richer raw data that is closer to the events being studied. Surveys could include open ended questions, talk-along interviews (Moore & Gillespie, 2014), and talk-aloud protocols (French et al., 2007; Wagoner & Valsiner, 2005). Experiments could include recordings or transcripts of what actually happened in the experiment (e.g., participants interacting; Psaltis & Duveen, 2007). Broadening the raw data would enabling in-depth analysis of surprising findings and support secondary analyses (Glăveanu & Gillespie, 2021).

One limitation is that a multi-resolution design should not be used simultaneously with confirmatory hypothesis testing. Multi-resolution research is exploratory, focused upon generating theory, finding explanations, and ensuring validity. Exploratory research is a necessary, but often neglected part of social science, that is fundamentally different from confirmatory research (Stebbins, 2001; Tukey, 1980). Confirmatory research entails stating a hypothesis before the data analysis; while exploratory research entails creating hypotheses through the analysis. Multi-resolution research should be conducted before confirmatory hypothesis testing on a training dataset (entirely separate from the test dataset to avoid overfitting) or after confirmatory hypothesis testing on the test data as a post-hoc validity check or post-hoc generation of possible explanations for the findings.

Augmenting Qualitative Research

A multi-resolution design can make qualitative research more rigorous, efficient, and transparent. Qualitative research has been criticized for being based on small samples (Chang et al., 2021), “cherry picked” excerpts (Morse, 2010), and lack of clarity about how selected excerpts relate to the population. Addressing these concerns by obtaining larger samples has been challenging because manual analysis is labor intensive.

Computational analysis can contribute to these challenges by augmenting qualitative analysis (Leeson et al., 2019). First, automated techniques are efficient and viable for simple thematic analysis (Chang et al., 2021; Ho et al., 2021; Kermani et al., 2024). Second, text data can be searched for linguistic features, specific themes or subtle patterns. These search algorithms can use distributed language models that score semantic and even conceptual similarity (Neuman, 2016). These two advances, in topic modeling and semantic search, create the potential for AI augmented qualitative research.

Augmenting qualitative research, by automating basic analyses, would enable more ambitious research and provide a rationale for choosing excerpts to analyze in detail. Such automation would “free-up” human interpretation so that it can be directed to where it adds most value, namely, in making subtle and contextual judgments. The algorithms would be akin to diligent research assistants, working tirelessly in the background, processing huge volumes of data, to provide accessible intermediary overviews that guide more in-depth qualitative analysis (Janasik et al., 2009; Wiedemann, 2013). These computational assistants could implement hunches, search for patterns, and create visualizations that enable and empower human interpretation. This guided human interpretation would, in turn, generate further insights and suggestions that would give direction to the computational assistants.

Reconnecting the Micro and the Macro

The social world is not stratified into different ontological levels (e.g., with sounds, utterances, and language being incommensurable). Rather, the social world is holistic, and any apparent levels are a byproduct of our conceptualization, data, and analysis. By analyzing the same data both qualitatively and quantitatively, and by zooming-in and zooming-out, we can study the social world as both a macro and micro phenomenon simultaneously (e.g., to analyze words within discourses and discourses as comprised of words).

Multi-resolution designs conceptualize the micro and macro as not merely “connected” but as the same phenomenon viewed at different resolutions. This deeper insight is obscured if raw data is thinly sliced into only excerpts, categories, or measures; potentially creating problems for conceptualizing how micro and macro phenomenon are related (Cornish, 2004). Using qualitative methods and small samples to study phenomena up-close and using quantitative methods with entirely different but larger samples to study the same phenomenon as a statistical abstraction has sliced phenomena in unnatural ways, creating a disconnect between concrete particulars and statistical patterns. In contrast, multi-resolution designs aim to preserve the ontological links between the micro and the macro through analyzing datasets that afford both zooming-in and zooming-out, thus revealing the micro in the macro and the macro in the micro.

Footnotes

Acknowledgments

We would like to thank Alex Goddard for helpful comments.

Author Contributions

AG, CdSL and VG developed the initial idea of multi-resolution design. AG wrote the first draft of the manuscript and CdSL, VG, TZ, and MB contributed additions and comments.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

Swiss National Centre of Competence in Research LIVES.

ORCID iDs

Alex Gillespie

Marcos José Bernal Marcos

References

Åkerblad

Seppänen-Järvelä

Haapakoski

(2021). Integrative strategies in mixed methods research. Journal of Mixed Methods Research, 15(2), 152–170. https://doi.org/10.1177/1558689820957125

Andreotta

Nugroho

Hurlstone

M. J.

Boschetti

Farrell

Walker

Paris

(2019). Analyzing social media data: A mixed-methods framework combining computational and qualitative text analysis. Behavior Research Methods, 51(4), 1766–1781. https://doi.org/10.3758/s13428-019-01202-8

Aveling

E.-L.

Gillespie

Cornish

(2015). A qualitative method for analysing multivoicedness. Qualitative Research, 15(6), 670–687. https://doi.org/10.1177/1468794114557991

Bazeley

(2017). Integrating analyses in mixed methods research. Sage.

Bennett

(2015). Found in translation: Combining discourse analysis with computer assisted content analysis. Millennium, 43(3), 984–997. https://doi.org/10.1177/0305829815581535

Berka

(1983). Measurement. Springer.

Boyd

R. L.

Schwartz

H. A.

(2021). Natural language analysis and the psychology of verbal behavior: The past, present, and future states of the field. Journal of Language and Social Psychology, 40(1), 21–41. https://doi.org/10.1177/0261927X20967028

Bryman

(2006). Integrating quantitative and qualitative research: How is it done? Qualitative Research, 6(1), 97–113. https://doi.org/10.1177/1468794106058877

Buckley

(2018). Simultaneous analysis of qualitative and quantitative social science data in conservation. Society & Natural Resources, 31(7), 865–870. https://doi.org/10.1080/08941920.2018.1446232

10.

Calude

C. S.

Longo

(2017). The deluge of spurious correlations in big data. Foundations of Science, 22(3), 595–612. https://doi.org/10.1007/s10699-016-9489-4

11.

Cartwright

Bradburn

(2011). A theory of measurement. In The importance of common metrics for advancing social science theory and research (pp. 53–56). National Academies Press.

12.

Chang

DeJonckheere

Vydiswaran

V. G. V.

Buis

L. R.

Guetterman

T. C.

(2021). Accelerating mixed methods research with natural language processing of big text data. Journal of Mixed Methods Research, 15(3), 398–412. https://doi.org/10.1177/15586898211021196

13.

Chen

N.-C.

Drouhard

Kocielnik

Suh

Aragon

C. R.

(2018). Using machine learning to support qualitative coding in social science: Shifting the focus to ambiguity. ACM Transactions on Interactive Intelligent Systems (TiiS), 8(2), 1–20. https://doi.org/10.1145/3185515

14.

Christ

T. W.

(2007). A recursive approach to mixed methods research in a longitudinal study of postsecondary education disability support services. Journal of Mixed Methods Research, 1(3), 226–241. https://doi.org/10.1177/1558689807301101

15.

Cornish

(2004). Making “context” concrete: A dialogical approach to the society-health relation. Journal of Health Psychology, 9(2), 281–294. https://doi.org/10.1177/1359105304040894

16.

Creamer

E. G.

(2017). An introduction to fully integrated mixed methods research. Sage.

17.

Creswell

J. W.

Creswell

J. D.

(2018). Research design: Qualitative, quantitative, and mixed methods approaches. Sage.

18.

Eronen

M. I.

Bringmann

L. F.

(2021). The theory crisis in psychology: How to move forward. Perspectives on Psychological Science, 16(4), 779–788. https://doi.org/10.1177/1745691620970586

19.

Fakis

Hilliam

Stoneley

Townend

(2014). Quantitative analysis of qualitative information from interviews: A systematic literature review. Journal of Mixed Methods Research, 8(2), 139–161. https://doi.org/10.1177/1558689813495111

20.

Fetters

M. D.

Freshwater

(2015). The 1 + 1 = 3 integration challenge. Journal of Mixed Methods Research, 9(2), 115–117. https://doi.org/10.1177/1558689815581222

21.

Fetters

M. D.

Molina-Azorin

J. F.

(2017). The Journal of Mixed Methods Research starts a new decade: Principles for bringing in the new and divesting of the old language of the field. Journal of Mixed Methods Research, 11(1), 3–10. https://doi.org/10.1177/1558689816682092

22.

Fetters

M. D.

Molina-Azorin

J. F.

(2017). The journal of mixed methods research starts a new decade: The mixed methods research integration trilogy and its dimensions. Journal of Mixed Methods Research, 11(3), 291–307. https://doi.org/10.1177/1558689817714066

23.

Feyerabend

(2001). Conquest of abundance: A tale of abstraction versus the richness of being. University of Chicago Press.

24.

French

D. P.

Cooke

McLean

Williams

Sutton

(2007). What do people think about when they answer theory of planned behaviour questionnaires? A “think aloud” study. Journal of Health Psychology, 12(4), 672–687. https://doi.org/10.1177/1359105307078174

25.

Gillespie

Glăveanu

de Saint Laurent

(2024). Pragmatism and methodology: Doing research that matters with mixed methods. Cambridge University Press. https://doi.org/10.1017/9781009031066

26.

Glăveanu

V. P.

Gillespie

(2021). Cognition stays wild: A commentary on Ross and Vallée-Tourangeau’s rewilding cognition. Journal of Trial & Error, 2(1), 40–44. https://doi.org/10.36850/r4

27.

Gobo

Fielding

N. G.

La Rocca

van der Vaart

(2022). Merged methods: A rationale for full integration. Sage.

28.

Greene

J. C.

Caracelli

V. J.

Graham

W. F.

(1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255–274. https://doi.org/10.3102/01623737011003255

29.

Guba

E. G.

Lincoln

Y. S.

(1994). Competing paradigms in qualitative research. Handbook of Qualitative Research, 2(163–194), 105.

30.

Guetterman

T. C.

Fetters

M. D.

Creswell

J. W.

(2015). Integrating quantitative and qualitative results in health science mixed methods research through joint displays. The Annals of Family Medicine, 13(6), 554–561. https://doi.org/10.1370/afm.1865

31.

Hammersley

(1996). The relationship between qualitative and quantitative research: Paradigm loyalty versus methodological eclecticism. In Richardson

J. T. E.

(Ed.), Handbook of qualitative research methods for psychology and the social sciences (pp. 159–174). BPS Blackwell.

32.

Chen

Shao

Bao

Tarfa

Brossard

Brown

Brauer

(2021). A mixed methods study of public perception of social distancing: Integrating qualitative and computational analyses for text data. Journal of Mixed Methods Research, 15(3), 374–397. https://doi.org/10.1177/15586898211020862

33.

Honnibal

Montani

Van Landeghem

Boyd

(2022). spaCy: Industrial-strength natural language processing in Python. Explosion. [Computer software].

34.

Hutto

C. J.

Gilbert

(2014). Vader: A parsimonious rule-based model for sentiment analysis of social media text. Proceedings of the International AAAI Conference on Web and Social Media, 8(1), 216–225. https://doi.org/10.1609/icwsm.v8i1.14550

35.

Janasik

Honkela

Bruun

(2009). Text mining in qualitative research: Application of an unsupervised learning method. Organizational Research Methods, 12(3), 436–460. https://doi.org/10.1177/1094428108317202

36.

Kermani

Makou

A. B.

Tafreshi

Ghodsi

A. M.

Atashzar

Nojoumi

(2024). Computational vs. qualitative: Analyzing different approaches in identifying networked frames during the Covid-19 crisis. International Journal of Social Research Methodology, 27(4), 401–415. https://doi.org/10.1080/13645579.2023.2186566

37.

Kerrigan

M. R.

(2014). A framework for understanding community colleges’ organizational capacity for data use: A convergent parallel mixed methods study. Journal of Mixed Methods Research, 8(4), 341–362. https://doi.org/10.1370/afm.1865

38.

Kessler

J. S.

(2017). Scattertext: A browser-based tool for visualizing how corpora differ. arXiv Preprint arXiv:1703.00565.

39.

Knappertsbusch

(2020). “Fractal heuristics” for mixed methods research: Applying Abbott’s “fractal distinctions” as a conceptual metaphor for method integration. Journal of Mixed Methods Research, 14(4), 456–472. https://doi.org/10.1177/1558689819893573

40.

Lee

L. W.

Dabirian

McCarthy

I. P.

Kietzmann

(2020). Making sense of text: Artificial intelligence-enabled content analysis. European Journal of Marketing, 54(3), 615–644. https://doi.org/10.1108/EJM-02-2019-0219

41.

Leeson

Resnick

Alexander

Rovers

(2019). Natural language processing (NLP) in qualitative public health research: A proof of concept study. International Journal of Qualitative Methods, 18(3), 1609406919887021. https://doi.org/10.1177/1609406919887021

42.

Love

H. R.

Corr

(2022). Integrating without quantitizing: Two examples of deductive analysis strategies within qualitatively driven mixed methods research. Journal of Mixed Methods Research, 16(1), 64–87. https://doi.org/10.1177/1558689821989833

43.

McCrudden

M. T.

Marchand

Schutz

P. A.

(2021). Joint displays for mixed methods research in psychology. Methods in Psychology, 5(1), 100067. https://doi.org/10.1016/j.metip.2021.100067

44.

McGuire

W. J.

(1997). Creative hypothesis generating in psychology: Some useful heuristics. Annual Review of Psychology, 48(1), 1–30. https://doi.org/10.1146/annurev.psych.48.1.1

45.

Moore

Gillespie

(2014). The caregiving bind: Concealing the demands of informal care can undermine the caregiving identity. Social Science & Medicine, 116(1), 102–109. https://doi.org/10.1016/j.socscimed.2014.06.038

46.

Morgan

D. L.

(2007). Paradigms lost and pragmatism regained: Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1(1), 48–76. https://doi.org/10.1177/2345678906292462

47.

Morgan

D. L.

(2018). Living within blurry boundaries: The value of distinguishing between qualitative and quantitative research. Journal of Mixed Methods Research, 12(3), 268–279. https://doi.org/10.1177/1558689816686433

48.

Morse

J. M.

(2010). “Cherry picking”: Writing from thin data. Qualitative Health Research, 20(1), 3. https://doi.org/10.1177/1049732309354285

49.

Nastasi

B. K.

Hitchcock

Sarkar

Burkholder

Varjas

Jayasena

(2007). Mixed methods in intervention research: Theory to adaptation. Journal of Mixed Methods Research, 1(2), 164–182. https://doi.org/10.1177/1558689806298181

50.

Neuman

(2016). Computational personality analysis: Introduction, practical applications and novel directions. Springer International Publishing. https://doi.org/10.1007/978-3-319-42460-6

51.

Niculae

Kumar

Boyd-Graber

Danescu-Niculescu-Mizil

(2015). Linguistic harbingers of betrayal: A case study on an online strategy game. arXiv Preprint arXiv:1506.04744.

52.

Noort

M. C.

Reader

T. W.

Gillespie

(2019a). Speaking up to prevent harm: A systematic review of the safety voice literature. Safety Science, 117(1), 375–387. https://doi.org/10.1016/j.ssci.2019.04.039

53.

Noort

M. C.

Reader

T. W.

Gillespie

(2019b). Walking the plank: An experimental paradigm to investigate safety voice. Frontiers in Psychology, 10(1), 7. https://doi.org/10.3389/fpsyg.2019.00668

54.

Noort

M. C.

Reader

T. W.

Gillespie

(2021). The sounds of safety silence: Interventions and temporal patterns unmute unique safety voice content in speech. Safety Science, 140(1), 105289. https://doi.org/10.1016/j.ssci.2021.105289

55.

Nzabonimpa

J. P.

(2018). Quantitizing and qualitizing (im-) possibilities in mixed methods research. Methodological Innovations, 11(2), 1–16. https://doi.org/10.1177/2059799118789021

56.

O’Halloran

K. L.

Pal

Jin

(2021). Multimodal approach to analysing big social and news media data. Discourse, Context & Media, 40(1), 100467. https://doi.org/10.1016/j.dcm.2021.100467

57.

O’Neil

(2016). Weapons of math destruction. Crown.

58.

Onwuegbuzie

A. J.

Johnson

R. B.

(2021). The Routledge reviewer’s guide to mixed methods analysis. Routledge.

59.

Onwuegbuzie

A. J.

Leech

N. L.

(2005). On becoming a pragmatic researcher: The importance of combining quantitative and qualitative research methodologies. International Journal of Social Research Methodology, 8(5), 375–387. https://doi.org/10.1080/13645570500402447

60.

Open Science Collaboration . (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. https://doi.org/10.1126/science.aac4716

61.

Peirce

C. S.

(1955). Philosophical writings of Peirce. Dover Publications, Inc.

62.

Plotly Technologies Inc . (2022). Collaborative data science. [Computer software]. https://plotly.com/

63.

Psaltis

Duveen

(2007). Conservation and conversation types: Forms of recognition and cognitive development. British Journal of Developmental Psychology, 25(1), 79–102. https://doi.org/10.1348/026151005x91415

64.

Rodriguez

M. Y.

Storer

(2020). A computational social science perspective on qualitative data exploration: Using topic models for the descriptive analysis of social media data. Journal of Technology in Human Services, 38(1), 54–86. https://doi.org/10.1080/15228835.2019.1616350

65.

Sandelowski

Voils

C. I.

Knafl

(2009). On quantitizing. Journal of Mixed Methods Research, 3(3), 208–222. https://doi.org/10.1177/1558689809334210

66.

Schegloff

E. A.

(1992). Repair after next turn: The last structurally provided defence of intersubjectivity in conversation. American Journal of Sociology, 97(5), 1295–1345. https://doi.org/10.1086/229903

67.

Seawright

(2016). Multi-method social science: Combining qualitative and quantitative tools. Cambridge University Press.

68.

Shahin

(2016). When scale meets depth: Integrating natural language processing and textual analysis for studying digital corpora. Communication Methods and Measures, 10(1), 28–50. https://doi.org/10.1080/19312458.2015.1118447

69.

Shrout

P. E.

Rodgers

J. L.

(2018). Psychology, science, and knowledge construction: Broadening perspectives from the replication crisis. Annual Review of Psychology, 69(1), 487–510. https://doi.org/10.1146/annurev-psych-122216-011845

70.

Shweder

R. A.

(1996). Quanta and qualia: What is the “object” of ethnographic method. In Jessor

Colby

Shweder

R. A.

(Eds.), Ethnography and human development: Context and meaning in social inquiry (pp. 175–182). University of Chicago Press.

71.

Sripathi

K. N.

Moscarella

R. A.

Steele

Yoho

You

Prevost

L. B.

Urban-Lurain

Merrill

Haudek

K. C.

(2024). Machine learning mixed methods text analysis: An illustration from automated scoring models of student writing in biology education. Journal of Mixed Methods Research, 18(1), 48–70. https://doi.org/10.1177/15586898231153946

72.

Stebbins

R. A.

(2001). Exploratory research in the social sciences. Sage.

73.

Tavory

Timmermans

(2014). Abductive analysis: Theorizing qualitative research. University of Chicago Press.

74.

Teddlie

Tashakkori

(2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Sage.

75.

Trochim

W. M.

Donnelly

J. P.

(2021). Research methods knowledge base. https://conjointly.com/kb/

76.

Tukey

J. W.

(1980). We need both exploratory and confirmatory. The American Statistician, 34(1), 23–25. https://doi.org/10.1080/00031305.1980.10482706

77.

Vogl

(2019). Integrating and consolidating data in mixed methods data analysis: Examples from focus group data with children. Journal of Mixed Methods Research, 13(4), 536–554. https://doi.org/10.1177/1558689818796364

78.

Wagoner

Valsiner

(2005). Rating tasks in psychology: From static ontology to dialogical synthesis of meaning. In Gulerce

Hofmeister

Staeuble

Saunders

Kaye

(Eds.), Contemporary theorizing in psychology (pp. 197–213). Captus Press.

79.

Wang

Shi

(2016). Work, retirement and aging. In Warner Schaie

Willis

S. L.

(Eds.), Handbook of the psychology of aging (pp. 339–359). Elsevier.

80.

Wicherts

J. M.

Veldkamp

C. L. S.

Augusteijn

H. E. M.

Bakker

van Aert

R. C. M.

van Assen

M. A. L. M.

(2016). Degrees of freedom in planning, running, analyzing, and reporting psychological studies: A checklist to avoid p-hacking. Frontiers in Psychology, 7(16), 32. https://doi.org/10.3389/fpsyg.2016.01832

81.

Wiedemann

(2013). Opening up to big data: Computer-assisted analysis of textual data in social sciences. Historical Social Research/Historische Sozialforschung, 38(4146), 332–357.

82.

Zeller

R. A.

Zeller

R. A.

Carmines

E. G.

(1980). Measurement in the social sciences: The link between theory and data. Cambridge University Press.

83.

Ziems

Held

Shaikh

Chen

Zhang

Yang

(2024). Can large language models transform computational social science? Computational Linguistics, 50(1), 237–291. https://doi.org/10.1162/coli_a_00502

84.

Zittoun

Gillespie

(2015). Imagination in human and cultural development. Routledge.

85.

Zittoun

Gillespie

Bernal Marcos

M. J.

(2023). Development and vulnerability across the lifecourse. Culture & Psychology, 5(6), 1354067X231201384. https://doi.org/10.1177/1354067X231201384