Abstract
The authors (Jerolmack and Khan [J&K]) argue that sociologists who use surveys and interviews to understand behavior ignore the situated nature of action, fail to theorize surveys or interviews as situations, and consequently draw incorrect conclusions from their data. Surveys, they argue, are good at ensuring the representativeness of individuals, but terrible at sampling variation in situations. In making a trenchant case for the situated nature of both behavior and the expression of cultural dispositions, and explicating the relevant virtues of ethnography, J&K constructively revive a conversation that sociologists neglect at great risk.
Although I am sympathetic, I suspect J&K underestimate the ability of researchers to address the attitude–behavior consistency problem, which is really several problems, most of which are soluble. I shall describe five reasons that survey responses may not predict behavior and offer solutions to four of them.
Part of my optimism reflects my view of culture. To be sure, meanings are negotiated in social and physical environments and situations constrain the expression of individual dispositions. But actors often enter into these negotiations with stable and consequential cultural dispositions. Culture, in this view, constitutes an ecology of representations, with social contexts selectively reinforcing or suppressing items in individually variable cultural repertoires (DiMaggio 1997; Semin and Smith 2008). Thus, although collectivities are the proper unit of analysis for the study of culture, persons are often appropriate units of observation.
So why might surveys and interviews fail to elicit useful indicators of behavior?
People may not tell what they know—they may misrepresent their beliefs due to social desirability bias or a related desire for self-verification (Burke and Stets 1999), or because they understand a domain so superficially that they have no underlying attitudes to present. 1 Extensive literatures describe methods for detecting social desirability bias (King and Bruner 2000), attitude strength, and issue salience (Krosnik and Abelson 1992). 2 When respondents don’t care about an issue, we can expect that their behavior will be driven by their environments, including the perceived preferences of peers, which in some instances predict actors’ behavior better than their own preferences (Miller, Monin, and Prentice 2000).
People may not know what they know—they may lack insight into deeply held attitudes, schemes of perception, or cultural repertoires because these are taken for granted (i.e., manifest in automatic rather than deliberative cognition; DiMaggio 1997:269-71). Many methods enable survey researchers to elicit meanings not readily accessible to their subjects, including factorial surveys (also useful against desirability bias; Wallander 2009), Implicit Association Tests (Shepherd 2011), and measures of response latency (Bassili 1995). Even interviewers may benefit from protocols carefully designed to illuminate relatively inaccessible beliefs or feelings (Haidt, Koller, and Dias 1993).
Researchers may incorrectly interpret honest responses. Meaning emerges from relationships among words and representations, yet researchers too often act as if item responses mean the same thing to everyone. Two people may revere the flag or like bluegrass music for quite different reasons or draw different policy implications from support for free markets. Interviewers have the advantage here, as they can probe for meanings. But survey analysts have tools as well: They can compare patterns of association across subsamples or use Relational Class Analysis, a new method expressly designed to identify such within-sample schematic heterogeneity (Goldberg 2011).
People may hold competing views simultaneously. People “know more culture than they use” (Swidler 1986:277): the activation and expression of competing attitudes, the cognitive structures to which they are linked, and the behaviors that follow from them are elicited by social contexts. In such cases, understanding the interview or survey as an action context is especially critical (Eliasoph 1990). Survey researchers can also build environmental variation into surveys through experiments that prime items in different ways (Mutz 2011). Interviewers may draw on sociolinguistic studies of code-switching (Gal 1979) to exploit indexical references to external context within the interview itself.
Environmental effects may swamp accurately reported attitudes and intentions. A power outage or hurricane on Election Day may make it harder to predict voting based on political attitudes. Attitude variation might have little effect on behaviors that are so highly institutionalized that deviance is difficult (e.g., picking a side of the road to drive on). How common are such cases? We really don’t know, but I suspect they are relatively rare, if only because surveys tend not to ask about them.
Does this mean that I greet J&K’s alarm with complacency? Not at all. Relatively few investigators use the methods available to them; fewer and less satisfactory solutions exist for interview studies; and as J&K explain, ethnography remains an indispensible complement to individual-focused methods (and vice versa). Nonetheless, I believe that better methods can yield better behavioral predictions in many cases and that well-executed ethnographies can help us understand the cases in which they do not.