Abstract
Taking a sociological view, we can investigate the empirical consequences of variations in the rhetoric of sociological methodology. The standards advocated in Qualitative Literacy divide communities of qualitative researchers, as they are not explicitly connected to an understanding of social ontology, unlike previous qualitative methodologies; they continue the long-growing segregation of the rhetorical worlds of qualitative and quantitative research methodology; and they draw attention to the personal competencies of the researcher. I compare a rhetoric of qualitative methodology that: derives evaluation criteria from perspectives on social ontology that have been developing progressively since the early twentieth century; applies the discipline-wide evaluation criteria of reactivity, reliability, representativeness, and replicability; and asks evaluators to focus on the adequacy of the textual depiction of research subjects.
Keywords
In Qualitative Literacy, Small and Calarco (2022) (hereafter, “the authors”, “S and C”; the book as QL) demonstrate how, by applying several newly phrased evaluation standards, readers can discriminate between better and less meritorious reports of qualitative research. In an appendix, the authors explain that their motivation was in significant part to aid grant reviewers. The need is real. At a conference in which NSF officials invited qualitative researchers to help improve their evaluation process, I recall expressions of frustration with proposals that used the same, non-discriminating boilerplate, in particular “grounded theory,” to certify methodological quality. On the rear cover of QL, top officials of two private foundations endorse the volume. Grantors are an unusual but, in power dynamics, a strategic focus for methodological writing. If research money will be allocated based on this book's standards, the authors’ evaluation criteria will echo throughout qualitative research.
It is, then, striking that QL's evaluation criteria mostly apply to texts with substantial substantive writing, which qualitative research proposals systematically lack. As the authors acknowledge, qualitative researchers usually seek funding before they have much data to report. The authors then try to save the utility of the book for grant officials. But the admission draws our attention to the sociology of methodology writing in academic sociology. How will QL's recommendations be used? In what social settings, through which interactions, by and with whom, and with what effects?
S and C might have started by laying out the contexts in which qualitative research is reviewed. The variety is great: mentors personally guiding students, whether collectively in methodology courses or one-on-one; deliberations of committees charged with authorizing PhD research; departmental meetings to choose from a pool of mixed qualitative and quantitative job candidates; university administrators, intra- and interdepartmental committees reviewing promotion cases; professional reviewers assisting specialized qualitative journals and reviewers reporting to discipline-spanning journals; agents and book editors looking for manuscripts with “trade” potential; reviewers of proposals for public and private funding; grant officials deciding how to digest reviewers’ reports; reviewers and editors for university presses; and the small community of writers on qualitative methodology.
The bulk of the book consists of analyses of texts. If reviewers of qualitative research projects read this book, they will find a discriminating methodological perspective, one informed by an unusually sophisticated sensitivity to how quantitative researchers see and deal with methodological issues. They will also come across tidbits that, in my reading, elicited marginal notations of “nice”: the seemingly unmotivated “bread crumbs” that fall in interviews, which astute researchers will pursue to a data banquet; the different methodological requirements when trying to explain variations in individual versus group behavior; how reporting percentages can be less accurate than not counting at all.
But while the authors are insightful, reliable evaluators, it is less clear that the evaluation criteria they advocate will be picked up and used with their discernment and discretion. The standards they propose come not in technical language but in universally accessible colloquialism. It is likely that the life of the rubrics proposed in the book will impact qualitative research far beyond any impact that would come from thoroughly reading QL.
Evaluators do not necessarily read the texts they evaluate. Texts based on participant observation and free-form interviews are longer and less standardized than the journal articles reporting most quantitative sociology. Reading them has especially great opportunity costs. And yet, as the authors effectively illustrate, the marvels as well as the devils are in the details. Vaughan's (2021) book on air traffic controllers took twenty years to write, runs over 600 pages, and, I can attest, reveals treasures all along the way. But what percentage of a committee charged with hiring, promoting, or distributing awards would actually read the monster? There is no automatic penalty for remaining silent and casting an unexplained vote. The temptation is to defer to a specialist, not just out of respect but as a quid pro quo for having done the reading on behalf of the group. It would carry heavy interpersonal, inter-club, and personal reputational risks for nonspecialists to challenge the judgments of a supporter who had read to the end. Across the various social contexts in which qualitative texts are evaluated, there is a strong market for buzzwords.
Most of QL puts long-articulated insights into a new vocabulary, as the authors implicitly acknowledge through their generous references. As a result, the empirical value of the book hinges on the differences that S and C's rhetoric will make when compared to other ways of making the same points. Cutting across the various situations in which evaluation is done, I see three likely effects, none of which the authors may intend or desire.
Small and Calarco's evaluation vocabulary is likely to be taken up in ways that abandon historical continuity. The newly framed criteria do not attempt to build on the criteria previously used in the history of qualitative methodology. The new standards are not explicated as specifications of prior concepts.
Second, QL will probably promote methodological segregation within sociology, as the new rubrics are not likely to be used outside of qualitative research. Quantitative researchers may wonder why it is important for descriptions to be “palpable,” one of the recommended evaluation standards. If a study explains a significant part of the variation in important behavior, isn’t that enough?
Another likely effect of QL's new rhetoric is to reorient evaluators away from assessing how the claims made in a text relate to the area of social life studied, and toward a focus on the competency and moral character of the researcher as a person. QL gives evaluators a nontechnical rhetoric to shortcut extensive textual examination by just characterizing researchers. The risk is to reinforce differences in institutional and mentor prestige.
One way to make sociological methodology more sociological is to ask how given methodological criteria impact the triadic interaction among readers, researchers, and research subjects. A methodological rhetoric in sociology can beckon readers to focus primarily on researchers as persons (How good are they? What are their sensibilities? What moral competencies are indicated by their writing?) or primarily to focus on how well texts represent subjects and subjects’ social worlds.
The difference may appear to be slight but rhetoric is powerful. Consider the difference between a methodology that asks whether the researcher “followed up” on initial sketches by questioning and testing hypotheses, and so demonstrated curiosity, a lively self-criticism, and personal growth beyond naivete and prejudice; versus a methodology, like analytic induction (Katz 2001a), that encourages the writing of texts as a report of revisions of hypotheses after confrontations with negative cases, which can be done formally as an intellectual biography of the research project (see the classic example of Lindesmith 1947) or, more commonly, through nuanced descriptions of variations on a theme, which is a wholly impersonal way of making a causal argument by demonstrating that one has progressively refined the definition of the explanans and explanandum through successive confrontations with negative cases. 1
Exposure v. Pointillism
That the authors are not focused on the empirical applications of their recommendations is apparent early on. Under the banner of “exposure,” they offer a seemingly obvious standard that invites mechanical application: “The greater the contact (with subjects), the better the data … more time in the field produces better data … ceteris paribus, 480 h of data are far more than 120 … 1560 h of data are more than 800.” (There is surprising redundancy in this short book. Compare the extraordinary compactness and clarity in a foundational eighty-four-page booklet on quasi-experimental design, Campbell, Stanley, and Gage 1966).
In fact, one of the major challenges for PhD mentors is that students often spend too much time in the field, whether because they are insecure about their ability to write and/or because they acquire commitments that make it increasingly difficult to leave. Because qualitative field research is so personal, “ceteris paribus” is magical thinking. Both mentors and students know the researcher only goes through life once, and the exposure one student will require is not what others need.
Arguably, less is more. Every fieldworker has experienced the perspicacity of Simmel's stranger. Stay around too long and it gets harder to ask the dumb questions that novices are permitted without loss of face. But the fundamental problem with the “more exposure better data” is not Simmel's implication of a law of diminishing return. It is the unethnographic character of the recommendation.
As a practical matter, the issue is often artificial. In writing proposals, applicants usually don’t have the freedom to define how long they will be in the field. PhD students pitch proposals to fit the temporal framework that works for their patrons. If more exposure is better, should granting agencies give half as many awards to fund twice as long a standardized immersion?
S and C know and admire but don’t work through the implications of a recurrent pattern in research biographies. Often a qualitative sociologist's initial contribution—which for some will establish the personal reputation that becomes a foundation for a multi-decade career in academia—comes not so clearly from greater exposure to a field, in comparison to graduate student peers, but from an exposure to sociology that distinguishes them from peers in activities or places they knew before becoming students. Anderson (1923) as a hobo; Becker (1953, 1963) as smoking marijuana and playing gigs with other musicians; Irwin (1970), once a felon, writing about felons; and Rios (2011) writing autobiographically about the experience of being a subject to police power. S and C might say these examples make their point, but the sample is biased. By definition, we don’t know about those with long exposure who produce nothing significant. The practical value of methodological writing does not escape the same methodological questions that substantive works confront.
Timmermans and Tavory (2022) advocate for “abductive” reasoning, the “surprise” moment when inductive familiarity blooms into a generalized conception. This brings us back to Simmel's observation about the perspicacity of the “stranger.” Ethnographers should take great care with initial fieldnotes, not only so that they develop a craft sense of possible detail but so that they will preserve observations that may soon become invisible (as emphasized in a leading fieldwork guide, Emerson, Fretz, and Shaw 2012). The essence of good sociological analysis is comparison, which for some will be elicited through brief exposures to many sites.
Perhaps, a quantitative measure of “exposure” is appealing because it substitutes for a too-personal negative judgment. A judgment of inadequate exposure implies a denigrating view of the researcher. The antonyms of sufficient exposure are dilettantism, superficiality, and flakiness.
As an alternative, we can use an evaluation standard like pointillism, which is about the structure of the text. At least to the extent that all sociological contributions hinge on their improvement of causal explanation (which, were there space, I would argue is always the case, however masked by “description” and “discovery” as claims of purpose; see note 1), we always need data showing variation in cause and effect. Research projects that produce descriptions of a mass of behaviors varying densely around a theme set up more powerful tests of explanation than do scattered observations. The fieldworker labors close up to describe each data point, often from a gut sense of being drawn to another variation of something previously seen. When writing, the researcher stands back and finds patterns sequentially related in ways that are compatible with causal understanding.
The initially appealing formula, “the more exposure the better,” does not take into account the interaction realities of qualitative research and misleads the evaluator to focus on the researcher's biography. The core issue is more fundamentally and universally expressed as the density of data, which can support the author's substantive claims by enabling the ruling out of reasonable alternatives: my claimed pattern holds in all these kinds of situations, for all these kinds of people, in all these phases of interaction, across all of these historical and geographic divisions. When readers appreciate qualitative data as colorful, quantitative researchers may scorn the reaction as sentimental. But when the “richness” or “color” in a researcher's descriptions is an upshot of variations that reflect personality, time, and place differences, the reader's superficially emotional reaction has good methodological justification.
Cognitive Empathy and Palpability v. Ontologically Justified Description
That the authors are shifting away from continuity with prior methodological insights, and moving away from proposing universal methodological standards, in favor of focusing on the person of the researcher, becomes clear when they present their five evaluation criteria. A researcher's cognitive empathy depends on how well the researcher understands how members view their world and understand themselves in it. Without specifying the interconnections, S and C develop a new language for what Glaser and Strauss, in a fifty-year-old book that now has over 15,000 citations, called “grounded theory.”
When in Chicago, Strauss edited a volume which, in its most difficult and unique section, on the experience of time, went back to student notes from George Herbert Mead's early twentieth-century classes (Mead and Strauss 1956). In mid-century, early twentieth century ferment over the ontology of social life became an empirically researched intellectual stew containing the influences of Mead's student, Blumer, Park's student, Hughes, and their participant observing students, Strauss, Goffman, and Becker, setting the background for the book that Strauss wrote with the Columbia-trained, Lazersfeld-influenced/resisting Glaser.
Grounded Theory was conceived as a bridge connecting the generations of qualitative researchers to each other and to Mead's philosophy, but not in a way that would remain in dialogue with quantitative researchers. Glaser and Strauss immediately (1967, p. viii) declare their intention “to develop canons” separate from those governing quantitative research: their subtitle is “Strategies for Qualitative Research.” When S and C substitute cognitive empathy for “grounded theory,” they do not repair this questionable segregating move in the history of sociology's methodological thought.
We can get at “cognitive empathy” and what makes data “palpable” by building on instead of departing from previous advances. And we can do so in a way that connects directly to the concerns of quantitative researchers to test hypotheses about the effects of the sequential placement of behavior and events. The recommendation here is that evaluators ask how well a text's descriptions of social life fit the elements of social ontology, or what is true about any instance of social behavior (Katz 2002).
Blumer was essentially a social ontologist. He would always ask, does the text show social interaction, i.e., how the subject is shaping behavior to influence the perceptions and thus the responses of others? After Blumer, qualitative researchers built up work on two sequential dimensions of social ontology, which can be summarized as the orientations of a Janus-faced “nextness.” Does the description show how the person is shaping his/her behavior to be, and to be seen by self and others as being, subsequent to a prior action? The conversation analysts became exceptionally sensitive to retrospective nextness. In writing a transcript of a conversation, one cannot hear an utterance as “Oh!” versus as “O(‘Brian)” versus as the inadvertent escape of a guttural “O” sound without hypothesizing how the speaker is relating the production of that sound to what the speaker and correspondents understand happened just before. In this case, “Oh!” marks (exhibits to both speakers) a “change of state” (Heritage 1984).
Instead of asking whether the author shows empathy, evaluators can ask whether the text shows subjects engaged in interaction and relating their behavior to what happened before. The retrospective nextness of all behavior applies at all levels of analysis. Qualitative researchers can achieve palpability through various forms of sequence analysis which aim to show that if a certain pattern has occurred, another pattern will have occurred before. In micro interaction studies, the text should show subjects shaping their behavior to follow from the immediately prior constellation of interaction. In career studies, the past to which current behavior is anchored, and the reason why a subject acts as observed to act now, might be a never-repeated mistake discovered years ago during the first day on the job.
When using social ontology as a guide for evaluating qualitative data-based texts, we do not make one-off or specifically evaluative judgments but we connect to fundamental requirements for a sound substantive explanation. Note that for transcribers to describe a subject as saying “Oh!” as opposed to O(‘Brian), they must hypothesize a causal relation, that the subject was exhibiting, as opposed to contemplating and privately aborting, a shift in the conversation. The writer describes a subject-made, subject-perceived, and subject-exhibited causal relation that a given action is provoked by a prior action, which the given action continues, diverges from, purposively ignores, etc. Good description shows retrospective nextness, which already is a causal claim that the actor pushed off from a specific prior action.
Likewise for the authors’ “palpability”: if we focus on social ontology, we can find a more theory-based, universally applicable, graspable alternative for qualitative researchers that connect with common concerns of quantitative researchers. We can empathize with what S and C are trying to do in touting “palpability”: they want to avoid artificial abstraction in texts. But their favored synonym, “concreteness,” indicates something has gone awry. The term is common in sociological writing but, if we take the metaphor seriously in its indication of hardness, ever-progressive dryness, inflexibility, and permanence, the application to the soft, juicy, and fluid character of social life should be recognized as weird. Sociologists seem to love the term, but it is at best a place holder. When sociologists praise the “concreteness” of a description they only make clear something negative, that it does not fit the antonym, abstraction. But what is the positive meaning of the term?
“Palpability” as a guide to evaluating a text can be compared to a social ontological guidance which urges readers to see if the text shows actors attending to the forward-oriented nextness that everyone always shapes into their behavior. We might instead speak of “luminous” moments in social life, which are emotionally resonant, intensely meaningful, and poignant to subjects as well as to readers (Katz 2001b). These are the embarrassing mistakes; the great escapes from imminent disaster; the privately registered understandings of the future fatefulness of what seems to all others present to be banal moments; the moments when, in Goffman's analysis of “action,” an act becomes a deed. A personal favorite is Eli Anderson's description and explanatory use of “going for cousins.” One of his low-status research subjects secured Eli's consent to be presented as kin when they went to a high status event. Anderson (2003) was but a graduate student, but he was a graduate student at the University of Chicago, a prestigious association for a janitor. The episode reverberated deeply for the researcher and subject as well, and is a critical lynchpin in the book's narrative.
S and C's compelling chapter on “palpable” text passages may well do the job of socializing evaluators to become more sensitive and demanding of qualitative texts. Another way to get the same points across is to look for indications that authors are indicating the forward-looking nextness of the behavior they describe. The social ontological claim, which has been honored in practice if not discussed explicitly in the century of work that has been inspired by pragmatism, is that every action is a project, a means of going beyond the moment, a way of connecting the present situation to others that come later and/or to ongoing relationships that the actors currently understand they are sustaining in other places. The qualitative researcher's discovery of the trans-situational, “there” and “then” relevancies to people of what they are doing overtly in the here and now can be a bridge to the background conditions—demographic, ecological, or historical—that quantitative researchers typically offer as explanations.
To portray behavior as problem-solving, whether the problem is personal dignity, such as where to take a leak if you are unhoused (Duneier and Carter 1999) or physical survival, such as how to live with the perils of being evicted (Desmond 2016), is to write a palpable text. When researchers grasp trans-situational orientations that some but not other subjects in a scene grasp, they have achieved empathy in a way that will make an especially powerful contribution.
S and C propose standards to be used in evaluation that by themselves give little practical guidance to researchers. How does one make data palpable? What are the techniques for achieving cognitive empathy? But if we take social ontology as a touchstone, we find clear directions for crafting our data gathering. Participant observation and interactive interviewing are techniques specifically adept at getting at the trans-situational relevancies of social behavior. By “hanging out,” the participant observer can discover that actions witnessed earlier had subsequent consequences, which everyone but the researcher realized in the first place.
In open-ended interviews, the researcher can discover what is not accessible to micro analysts whose data consist of recordings of social life or field observations. Interviews give the researcher access to longer stretches of people's lives than field trips can cover or that recordings can practically accommodate. (For an inspiring/discouraging example of what just one workday's recording requires for description and analysis, see Streeck 2017). But it should not be enough for a researcher to hold up the banner of “in-depth interviewing” and criticize the limits of survey questionnaires. Evaluators should look for textual passages in which the author shows the transcending interaction implications of situationally isolated moments, or how the subjects understand the implications of what is happening at home, say in a family and neighborhood where unemployment is high, for what is happening at the observational field site, where workers never complain about bad pay or supervisors’ disrespect; and how subjects understand the implications of what is happening now, for example, the moving blips seen on computer screens at this moment in the air traffic control center, for what will happen in 15 minutes if those planes continue their mutually blind trajectories (Vaughan 2021).
Heterogeneity and Self-Awareness v. the 4 R’s
If it is not already obvious, I have a horse in this race. When writing my dissertation, I was uncomfortable with the defensive methodological position of ethnographers who, unable to find a way to justify the evidentiary basis for their substantive claims, retreated to claim no more then “discovery” and a subservient role relative to quantitative researchers who test hypotheses. I tried to point out the resources that could be built into ethnographic texts to enhance the ability to rule out rival substantive hypotheses, and so meet the most common methodological standards that all scientific research honors (Katz 1982, 2015).
In my theory of qualitative methodology, I summarized most of the methodological questions facing all scientific research as reactivity, representativeness, replicability, and reliability. The uniting themes were that the 4 R’s identify ways to consider and rule out reasonable rival explanations; and that these evaluation standards are as tractable for qualitative as they are for quantitative researchers. A focus on the 4 R’s promotes a triadic interaction in which readers assess an author's text by comparing it with what they can learn of subjects independently of the author's theorized wishes.
For example, a good ethnography will have many tests of the reliability of the author's application of theory through interpreting or coding cases. It will have data that allow what the author wrote about subjects on one page to be compared critically to the portrait of subjects on other pages. Because Desmond's (2016) ethnography is rich in a pointillist sense, focusing narrowly and in fine detail on the experience of eviction in a broad set of cases, we can ask whether his interpretation of a case in which an eviction process led to orphaned children being discovered by child welfare officials, which presumably was for the children a beneficial turn of events, is consistent with his interpretations of other cases showing how eviction makes life worse off for the evicted. Pointillism in ethnography is not simply an esthetic quality. It is methodologically useful because it facilitates the close comparison of multiple cases. For similar reasons, quantitative researchers honor a version of pointillism when they look for a cloud of data mapped onto a graph.
“Representativeness” encompasses what S and C term heterogeneity, “people and places depicted as diverse.” But an inquiry into the representativeness of a set of qualitative data takes into account not diversity per se but whether the data set's range of demographic, place, interaction phase, social positional, historical, and other variation allow all reasonably competing rival hypotheses to be tested. When evaluating representativeness, the reader focuses on the relationship between presented data and a counterfactual, the reasonably relevant data that were not gathered or otherwise do not appear in the text.
With the examples they use to show weak work, S and C convey that the antonym of “heterogeneity” is a parochialism that reflects poorly on the person of the researcher. Their standard of “self-awareness” is yet more directly personalizing: evaluators who charge a lack of self-awareness make a humiliating indictment. The difference in how S and C present works they consider good and bad is revealing. They describe real studies to show the good. To show the bad they create hypotheticals, which sometimes illustrate researchers’ stereotypes and prejudgments. S and C seem to acknowledge this personalizing thrust by refusing publicly to name bad authors. Their courtesy will not restrain evaluators working in private spaces.
Used as evaluative standards, the 4R’s keep qualitative researchers talking the language of their quantitative colleagues, in sociology and science more broadly; while QL's standards turn toward methodological self-segregation. “Reactivity” encompasses what S and C aim at with “self-awareness,” that the researcher unwittingly evokes behavior that otherwise would not occur. Will survey researchers, who for generations have been eager to detect interviewer effects, and physicists who since Heisenberg have been coping with the uncertainty principle, now begin to reflect on whether they are sufficiently self-aware? Would it not better promote cross-fertilization within sociology to use an evaluation rhetoric that made clear that quantitative and qualitative researchers are fighting the same methodological enemies?
Conceiving self-awareness as a problem implies a cure through self-examination. But if anything of value is to be taken from twentieth-century depth, interpersonal, and clinical psychology, it is that we should not assume that researchers can through force of will awaken to recognize what had been self-deception. “Reflexivity,” a term that over the past thirty years has often been used by ethnographers to call for more self-awareness, is arguably little more than a justification for an author to declare allegiance with contemporary moral and political sensibilities through a humbling histrionics of self-criticism. A less psychological, less personalizing fix was outlined over sixty years ago by Becker (1958), who recommended that participant observers give higher evidentiary value to observations of subjects when subjects were interacting with others in consequential situations. In what was a move to reach out to hostile quantifying colleagues (done with a touch that I suspect was tongue-in-cheek), he actually quantified a greater weight to be given observations of behavior in multiparty, versus isolated researcher/research subject, interaction situations.
For interviews, which are almost by definition one-on-one, there is another guidance to minimize problems of reactivity. Evaluators can ask whether respondents are answering “why” something occurred as opposed to describing “how” it happened. Answers to “why” questions often lead to self-aggrandizing declarations of motives. A “why?” question often alerts respondents to the prospect that the interviewer will be judging what their response indicates about their character. There are usually “good reasons” why a person mated with a given person, came to do their current job, moved to a new neighborhood, or eats a certain kind of food. Any experienced qualitative researcher knows that “how” questions often elicit answers that reveal that “why?” answers were misleading products of the interview situation. It is not because respondents, when asked “how?,” are any less concerned about the impressions they may be making on the researcher. But the brush strokes of self-portraiture are not as free. Answers to “how” questions point to places, events, and ways of doing things that can be investigated subsequently. “Why” questions often invite the respondent to enter a world of personal fantasies that it would be disrespectful to question. Explanations of “how” are generally more palpable because they empower the reader and the researcher to look through the respondent and get in touch with social life independent of the subject's wishes.
Replicability v. Verisimilitude
Another indicator of QL's turn toward the person of the researcher and away from a methodology applicable to all scientific research is the neglect of the replicability issue. In recent years, ethnography has been challenged by suspicions that data are made up or interpreted deceptively. Related challenges have come to psychology, economics, and even the hard sciences. Replicability in the human sciences does not mean replicating a researcher's operations on the same data by returning to the research scene or interviewing the same people again. That is impossible. The first research intervention may have changed what it studied and in any case, the subjects will have aged. The pragmatic meaning of replicability is being able to test the researcher's claims on presumably similar, independently inspected data. Studies satisfy replicability when they effectively invite the reader to look to the area of social life studied and join the hunt for an explanation. That may lead to devastating critical evaluations of the researcher but the criticism can be delivered in a formally impersonal way. Instead of a personalized moral accusation, the critic can insist on the production of confirming evidence.
Consider a passage that S and C appreciate from Thorne's (1993) book on gender relations. Thorne describes children on a playground wearing coats on a warm day. The value of noting the contrast between their apparel and the weather might be logically justified were there a reasonable rival hypothesis that the patterns found don’t apply in all seasons, or that her subjects related differently by gender because they were overheated. But that is not the case here. So why do we need to read these trivial details?
There is a good reason: mnemonics. Setting the scene helps the reader recall the interaction. By aiding memory, the landscape imagery promotes discussion of the text and so, the possibility that others will point to independently acquired confirming or disconfirming evidence. That would be a form of replicability in a pragmatic sense.
The risk is that writing qualitative details that are not proposed as tests of reasonable rival hypotheses are manipulative. QL could be read as a guide to how to produce verisimilitude. Even if you don’t give the reader a basis for ruling out reasonable rival explanations, you’ll get a good evaluation if you present a morally attractive research self as a sociologist who is empathetic, self-aware, doggedly curious, down-to-earth, and comfortable interacting with a variety of people in a variety of places. That's the kind of person we can trust to find and report the truth. The more scientifically compelling ideal is trust with verifiability.
Beyond a Hierarchy of Methodological Strategies
Whatever evaluators do with this book, Small and Calarco should be recognized as doing an exceptionally good job of showing qualitative writers working well on compelling subjects. If evaluators read through the book, they will find that participant observation and free-form interview studies are enriching sociological knowledge on a great range of current, important social issues. At a minimum, all of us in the business should be grateful to the authors for taking this detour from their own distinguished research paths to incisively display, to specialist and general readers alike, the strengths of our community's contributions.
But if we can work out a methodological rhetoric that spans rather than segregates quantitative and qualitative research strategies, we would improve both. Such a perspective would be nothing new. See Polya (1954). In the current demand to “define the estimand” (Lundberg, Johnson and Stewart 2021), there is a revived awareness of the need to develop the inevitably qualitative analytic assumptions of quantitative sociology. 2 If ethnographers would mount a fulsome response to this call, they might reverse their position in the hierarchy of sociological research, which often has been not as sources of evidence but as suppliers of illustrations that carry more charm than do statistical demonstrations. In contrast, ethnographies, histories, and other forms of qualitative research, along with prior findings in quantitative studies, are the sources for the plausible reasoning—what Polya discussed as good reasons for hunches, guesses, or bets—that serve as the assumptions for mathematical proofs. That long-budding recognition would place qualitative research at the foundation of our collective enterprise.
Footnotes
Acknowledgments
The author would like to thank Felix Elwert and Colin Jerolmack for helpful comments.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
