Abstract
Anderson’s (1990) Streetwise, Duneier’s (1992) Slim’s Table and (1999) Sidewalk, and Goffman’s (2015) On the Run epitomize the high level of understanding that carefully researched and written sociological urban ethnographies bring to studying particular social processes, structures, and institutions. They provide the in-depth and detailed type of “thick” description that Geertz (1973) argued provided the context necessary to understanding the people being studied within the culture in which they are embedded. While ethnographic research continues to be a mainstay of anthropologists, it is far less favored among sociologists for whom the validity, reliability, and generalizability of more quantitatively focused research—such as secondary data analysis, survey research, and (less frequently) experimental research—is far more appealing and scientific.
The underlying assumption is that numerical representations of social life are a more unbiased and objective form of evidence while qualitative forms of evidence are viewed as being more biased and less objective. It is this issue of evidence that Lubet examines in Interrogating Ethnography. A Northwestern University law professor, he uses Duneier’s (2011) idea of an “ethnographic trial,” as framed by the standards for evidence in courts of law, to examine sociological urban ethnographies. The project on which this book is based was stimulated when, on the advice of a friend, Lubet (2018), a scholar whose expertise is in the area of proof and “the quality of acts and assertions in diverse settings” (p. ix), read Goffman’s (2015) book On the Run as well as a number of strongly positive reviews that praised it. However, his reaction to the book was significantly different from those of sociologists and professional book reviewers. Lubet (2018) found that “almost from the opening pages, there were vignettes that struck me as implausible at best” (p. xi), and he asked whether “this was an aberration in ethnography, or the norm” (p. xiii). His goal is to ascertain the degree to which sociological ethnographers “test the accuracy of their own work, and the validity of their conclusions” (p. 1).
To examine the norms of evidence for urban ethnographies, Lubet read “over fifty ethnography monographs and a like number of journal articles . . . and consulted hundreds more essays, reviews, textbook chapters, and other sources” (p. xiii). He persuasively argues that “lawyers’ methods of analysis, inquiry, and fact-testing can be used to scrutinize works of scholarship, while also addressing ethics issues and related problems raised by ethnographic studies” (p. 6). After introducing and explaining the idea of an “ethnographic trial,” chapters one and two discuss the main forms of evidence found in trials (testimony, hearsay, opinion, and documentation) and how they are relevant when considering informants as the source of evidence in ethnographies. Chapters three to eight then examine what Lubet considers to be specific issues that arise in ethnographies, “unreliability, credulity, selectivity, rumor, anonymity, and criminality” (p. 7), and the concluding chapter discusses the criteria for an “evidence-based ethnography” in sociology.
In an interesting analogy presented in chapter one that is carried throughout the book, Lubet argues that the witness in a trial is similar to an ethnographer in that both are presenting information and evidence that they are asking others to accept as being truthful and accurate. He carries this analogy further when stating that an ethnographer reporting on “directly observed phenomenon” presents “first-order evidence, because of its nearly universal acceptability” compared to what the ethnographer “has heard from others, which raises the hearsay rule” (p. 11), where it is far more difficult to ascertain the accuracy and truthfulness of the information conveyed. He states that it is important for ethnographers to clearly and explicitly distinguish between the types of information and evidence presented to the reader.
Chapter two discusses two other forms of evidence, in addition to direct observations or hearsay, that the ethnographer (as “witness”) can bring to bear in his or her description: opinions that present the witness’s/ethnographer’s “interpretation of facts and events” and documentation that Lubet describes as “facts that have been frozen in time” (p. 15). He carefully distinguishes between different types of opinion and documentation found in urban ethnographies and provides specific illustrations of how these different forms of evidence are used. The majority of the text (chapters three through eight) focuses on the six specific issues where Lubet first provides a brief overview of the particular issue associated with ethnographic evidence, examines different aspects of the issue, and then offers suggestions concerning how to improve the reliability and validity of such evidence. Throughout the entire text, he provides detailed examples of more highly reliable and valid presentation of ethnographic evidence as well as problematic presentation of such evidence. Finally, the concluding chapter explicitly identifies three key standards “needed to bring ethnography into line with other evidence-based disciplines,” accuracy, candor, and documentation, by engaging in consistent fact-checking, revisiting and reinterviewing informants and communities, and sharing field notes (as quantitative data are shared among scholars) (p. 136).
Interrogating Ethnography is well written and logically organized, and Lubet provides extensive illustrations for each point drawn from among the over 100 ethnographies and articles he reviewed as part of this project. He highlights ethnographies that epitomize the best practices as well as those that represent weakly constructed ethnographic views. The reason this type of analysis is both interesting and compelling for sociologists is that it serves as an overarching epistemological review and critique concerning the standards for reliable and valid ethnographic evidence. This type of argument is important because it transcends our discipline-bound tensions and debates concerning qualitative versus quantitative forms of evidence. The issues addressed, clarity of writing, and breadth of illustrations provided make this an ideal text for a variety of courses from the sophomore to graduate level, including the required research methods class and a field research methods course. Lubet addresses the purpose and need for rigorous ethnographic data collection and representation and provides the framework within which such work can be evaluated. His arguments can be used to present a counterpoint to the typical discussion of reliability and validity found in the general methods class that tends to focus on quantitatively focused research. The instructor could create an assignment (article analysis, class debate, issues evaluation) where students evaluate the validity and reliability of a specific article or compare how validity and reliability are assessed for quantitative studies versus qualitative studies. Additionally, the idea of an “ethnographic trial” is invaluable for framing a field research methods class where the instructor asks students to use the framework for developing their own research projects, evaluating their own data collection, shaping their own writing, and evaluating ethnographies they may be reading in the class.
Furthermore, because of the focus on urban ethnographies, Interrogating Ethnography could provide a broad overview of relevant ethnographies and issues associated with understanding urban settings for more substantive courses such as criminology or inequality. In discussing each of the six issues associated with presenting ethnographic evidence within the framework of an ethnographic trial, Lubet actually addresses key social processes (e.g., why informants embellish their experiences and the impact of systems of stratification on social relations) and social structures (e.g., racism in law enforcement) unique to urban environments. This book could serve as a framing device for a number of different substantive undergraduate courses, from criminology to inequality/stratification to urban sociology, whereby, for example, each of the issue chapters introduces a key aspect of the substantive topic under consideration. A sample assignment could ask students to read reviews of the ethnographies critiqued by Lubet and discuss how it impacts understanding the specific topic being discussed.
While the book is certainly thought-provoking and provides a compelling argument concerning how to improve the validity and reliability of ethnographic evidence, there are two concerns that instructors would need to address if using it in their classes. First, as stated earlier, he suggests specific ways that sociological ethnographies can raise their standards to meet those of other “evidence-based disciplines” (p. 136). It appears as though the disciplines to which he refers are journalism and the law since he does not actually reference the standards for ethnographic evidence in other social sciences such as anthropology or communication. He references the rare discussions of ethnographic evidence in sociology and compares them to how the legal profession addresses similar types of issues. Yet, anthropologists have a long and intense history of debating the exact same issues as raised in Interrogating Ethnography. The instructor will need to reference additional sources to more fully explore the issue of reliability and validity of ethnographic evidence. For example, Wolf (1992) presents the exact same situation that occurred in a rural Chinese village three different ways: as described in her field notes, as described in an article she published in an academic journal, and as a fictionalized tale. She discusses the purpose of ethnographic evidence as well as the different forms that allow it to be reliable and valid. Behar (1997) examines how the ethnographer’s presence impacts the community in which she is embedded, thus shaping and changing what happens during field work. These shifting social dynamics then impact the ability of the original researcher or those interested in further exploring that field site to confirm the accuracy of the ethnographic evidence presented. Ryerson (2018) highlights three 2018 publications that also examine the degree to which ethnographers and ethnographies fully capture the “truth” about any given situation, social process, or social structure. In other words, while ethnographic evidence can be made to be more reliable and valid, the degree to which any research presents a completely accurate and “true” representation of people, situations, and events is always limited.
The second concern that detracts from the strength of Lubet’s discussion centers on the original motivation for the project: whether sociological ethnographies do a better job working with ethnographic evidence than Goffman (2015). It quickly becomes clear that he identifies Mitchell Duneier’s ethnographies as the “gold standard” and Goffman’s work to epitomize the worse possible example of ethnographic evidence, with all the other ethnographies falling somewhere in between the two. Each chapter references what Lubet argues are significant problems with Goffman’s work compared to all other ethnographies. When reading the footnotes, it becomes obvious that he is obsessed with dissecting Goffman (2015) simply to show her limitations as an ethnographer. Instructors can actually use this overt bias to have students consider the degree to which any research, ethnographic or quantitative, is capable of uncovering “truth.” More to the point, Lubet’s discussion can serve as a counterpoint to class discussions and assignments that critique all types of research. Despite these two concerns, Interrogating Ethnography makes an important contribution to developing a more rigorous framework within which ethnographic evidence is collected, analyzed, and presented.