Abstract
This article examines an important and thorny problem in interview research: How to assess whether what people say motivated their actions actually did so? We ask three questions: What specific challenges are at play? How have researchers addressed them? And how should those strategies be evaluated? We argue that such research faces at least five challenges—deception, recall error, reasonableness bias, intentionality bias, and single-motive bias—that more than a dozen strategies have been deployed to address them; that the strategies have been external, internal, or interactional in nature; and that each class of strategies demands distinct evaluation criteria. Researchers will likely fail to uncover motivation if they ignore the possibility of each challenge, conflate one challenge with another, or deploy strategies unmatched to the challenge at hand. Our work helps systematize the evaluation of interview-based studies of motivated action and strengthen the scientific foundations of in-depth interview research.
Keywords
In recent years, sociologists have debated whether and under what conditions qualitative research should be deemed scientific, believable, reliable, or rigorous. They have discussed whether standards for quantitative methods are appropriate to qualitative research, criticized long-standing practices such as anonymizing field sites and destroying field notes, and brought greater scrutiny to the methodological decisions of qualitative researchers (Becker and Geer 1958; Duneier and Carter 2001; Goertz and Mahoney 2012; Jerolmack and Murphy 2019; Lamont and White 2008; Lieberson and Lynn 2002; Manning, Jammal, and Shimola 2016). These debates are happening at a time when the social sciences as a whole are undergoing a reckoning, as long-accepted findings have failed replication tests and repeated retractions by major journals have inspired calls for greater transparency, rigor, and accountability in science.
Among qualitative researchers, an especially heated debate has centered on the scientific value of in-depth interviews—particularly, it has probed whether they should be used to capture anything other than people’s subjective accounts of their lives and circumstances. Researchers have proposed that, since what people say cannot be trusted, interviews are not reliable sources for the study of much more than people’s words and what those words mean to them (on the debate, see DiMaggio 2014; Jerolmack and Khan 2014a, 2014b; Khan and Jerolmack 2013; Lamont and Swidler 2014; Pugh 2013; Tavory 2020). What people say and what they do, the argument goes, are different things. This proposition is not new (Becker and Geer 1957; L. R. Dean 1958; J. P. Dean and Whyte 1958; Deutscher 1966; Deutscher, Pestello, and Pestello 1993; LaPiere 1934; Wicker 1969). Still, many sociologists in recent years have indeed documented differences between people’s words and actions (Pager and Quillian 2005; Raudenbush 2016; Small 2017). And given the efforts across the social sciences to improve the reliability and quality of empirical research, the proposition may deserve renewed attention.
In spite of how heated the debates have been, whether “what people say” is actually “what they do” is not, from a methodological perspective, a single question. It is many. For example, it can refer, among other things, to the distinction between what people say they believe about an issue and what they actually believe; between what they say they have done in the past and what they have actually done; between what they state they would do in a given circumstance and what they would actually do; and between what they say motivated a past action and what actually motivated it. While some very general problems may be common across these questions, many of the most important challenges are likely to be particular to each distinction (see Deutscher 1966; Deutscher et al. 1993; LaPiere 1934). Therefore, any discussion of these issues hoping to be useful at this juncture must be much more precise about the particular distinction at stake.
In what follows, we focus on one of these distinctions, among the most important ones facing in-depth interview researchers today: the relationship between what people say motivated an action they took and what actually motivated it. This distinction involves several thorny issues we discuss below; nevertheless, an example illustrates the core question we ask. An adolescent student reflects on the state of her education, weighs pros and cons, and subsequently decides to drop out of school. An interviewer asks the student what motivated her departure and gets an answer. Should the interviewer believe her? In this article, we make a case that the issue at the heart of this question is essential to sociology today, and we identify the specific challenges the researcher would need to address to arrive at an answer.
Motivated Action
Before we continue, an important class of objections must be addressed. Some scholars have criticized any attempts to uncover the motives behind action—researchers, they say, should try not to uncover motives but to examine what people say in its own right, to study either the vocabulary people use to express their motives (Mills 1940) or the meaning they assign to whatever they express as motives (Holstein and Gubrium 1995:9; Lamont and Swidler 2014; Luker 2008; Riessman 1993). Although perspectives informing this critique vary, the ultimate reason has to do with the presumed inaccessibility of actual motives. The idea is that understanding the “true” motives behind an action is either highly difficult or ultimately impossible, given the complexity human behavior, the limits of individuals’ self-understanding, and the inescapable fact that the interview is an interaction, one that shapes any report of the individual’s past behavior. In contrast, the language people use to describe their motives is itself instructive and should be both the foundation and the sole aim of the analysis.
We do not disagree on the importance of studying language in its own right. In fact, as we show below, understanding language will be important for capturing motives effectively. In addition, as we also discuss below, we concur with the idea that human behavior is both complex and heterogeneous across situations in ways that make the search for motives a thorny endeavor—for example, a researcher may attempt to capture a “motive” when no such motive was there. Finally, we agree with the general idea, often documented by psychologists, that people have only limited insight into their own thought processes, at least when asked directly to describe these processes (Nisbett and Wilson 1977). Asking “why did you do what you did?” may elicit a host of unreliable answers (Tilly 2006). At this juncture in the development of the social sciences, naivety on any of these issues is indefensible.
Nevertheless, we posit that recommending that in-depth interviewers disregard motives, or else restrict the study of motivated action to the question of language, is problematic—for three reasons. First, for better or worse, understanding the motives behind action has and continues to be one of the core preoccupations of sociology (Kadushin 1968; Weber 1978). Why did an individual drop out of school? Opt out of the formal labor market? Emigrate from their country of birth? Give someone a job? Join a protest movement? Move to a given neighborhood? Vote for a political candidate? All of these are motive questions. (They are also causal questions, which is something different—see below.) Regardless of the methods employed in answering them, such questions have and will remain central to the discipline. They are the bread and butter of sociology and other social sciences and are probably inescapable.
Second, in practice, sociologists have repeatedly answered questions of motive specifically by turning to in-depth interviews. We examined all interview-based papers published in the American Journal of Sociology, American Sociological Review, Qualitative Sociology, and Social Forces during a five-year period from 2015 to 2019. Many of these studies were expressly concerned with capturing motives. The studies asked questions such as the following: Why do women freeze their eggs (Brown and Patrick 2018)? Why do community leaders avoid party politics (Pacewicz 2015)? Why do people volunteer for hospice (Baugher 2015)? Why do teachers promote color-blind ideologies (Teeger 2015)? What motivates migrants to return home to Mexico (Chávez, Edelblute, and Korver-Glenn 2016)? Table 1 presents all papers we identified, along with the questions they asked. Each of the questions on the table involves understanding what motivated actors to pursue an action they have undertaken—not merely how they talk about things—and each of them is answered based wholly or in part based on interviews. The practice is no less prevalent in recent interview-based books (e.g., Calarco 2018; Small 2017; Watkins-Hayes 2019).
Studies Using In-Depth Interviews to Capture Motives Behind Action: American Sociological Review, American Journal of Sociology, Social Forces, and Qualitative Sociology, 2015–2019.
Third, for many empirical questions about what motivated an action, there is little choice but to turn to interviews (or surveys), since the phenomena at hand are either difficult or impossible to observe ethnographically. One class of actions of this kind is those involving no outward change in behavior, as when people decide not to do something. Many recent studies in sociology have tried to answer questions of this kind: Why do teachers not teach students to attend to the effects of apartheid on their society (Teeger 2015)? When asked to provide a friend or family loan, why do people not comply (Wherry et al. 2019)? Why do people stay in poor neighborhoods (Rosen 2017)? Why do people not tell their close friends about job opportunities they know about (Smith 2005)? Why do people not turn to those they are close to when they need someone to talk to (Small 2017)? In such circumstances, the researcher’s ability to merely observe what is happening is limited at best. What an external party can usually detect is the absence of action rather than the act or decision to not perform an outwardly observable behavior. Interviews are required to detect whether these nonactions were, in fact, decisions not to act, and, if so, what motivated them.
A different class of actions involves power. When people are subject to the power of others—as employees before their bosses, prisoners before their corrections officers, students before their professors, spouses before abusive partners, adolescents before their parents, and many others—they often deliberately hide their attitudes and motives by altering their behavior (Blair-Loy 2003; Gibson-Light 2018; Kanter 1977; Sweet 2019). In such circumstances, inferring their true motives from their behavior alone is thus exceedingly difficult, and researchers must turn to interviews, at a minimum on a supplementary basis (Rinaldo and Guhin 2019; see Tavory 2020). Thus, to the extent sociologists care to understand motivated action, they will likely continue to need to use in-depth interviews to elicit motives.
Challenges
For studies such as those in Table 1, it is indispensable that what interviewees report as having motivated their action actually motivated their action. Yet among researchers studying motivated action there are surprisingly few systematic assessments of either the specific challenges at play in making that determination or the particular strategies researchers have deployed to address them. In-depth interview handbooks have traditionally focused more on general strategies for interviewing, such as sampling, coding, managing power, and establishing rapport, than on the specific problem of ensuring that an interviewee’s reports about their motives actually match their motives (Briggs 1986; Compton, Meadow, and Schilt 2018; Deterding and Waters 2018; Gerson and Damaske 2020; Gorden 1972; Gubrium et al. 2012; Hesse-Biber and Leavy 2004; Holstein and Gubrium 1995; Kahn and Cannell 1957; Merton 1952; Mishler 1986; Royal and Schutt 1976; Rubin and Rubin 2005; Weiss 1994; Yin 2011).
In addition, while methodological or epistemological debates have examined the relationship between what people say and do, the vast majority of this work has involved the relationship between expressed attitudes and behavior. For example, in perhaps the most important set of works on this topic, Deustcher (1966; Deutscher et al. 1993) has assessed the theoretical and operational issues involved in determining whether people’s attitudes and behavior are inconsistent, as when people’s expressed attitudes about how they would interact with a racial group may differ from what they do in practice (see LaPiere 1934). Indeed, much of the recent debate on the question has been about this particular issue (DiMaggio 2014; Jerolmack and Khan 2014a; Lamont and Swidler 2014; Vaisey 2014; see Tavory 2020).
But our concern in this article is not whether people’s expressed attitudes match their behavior; it is whether their expressed motivation for a past action reflects their actual motives. It is specifically on the class of questions listed in Table 1. It thus represents an entirely different question, one that as we have seen above, lies at the heart of much of how sociology uses in-depth interviews today. The epistemological foundations justifying the claims in studies such as those in Table 1 remain underexplored.
Thus, in what follows, we answer three questions: First, what specific challenges do researchers face when using interviews to capture the motives behind action? Second, what strategies, if any, have sociologists deployed to address these challenges? Third, how should we assess the effectiveness of such strategies? We argue that interviewers seeking to unearth the motives behind action face at least five core challenges—deception, recall error, reasonableness bias, intentionality bias, and single-motive bias—that researchers have used more than a dozen different strategies to address them, and that the strategies have been of one of three kinds: external (relying on data beyond the interview), internal (relying on question wording to address a cognitive process), or interactional (relying on responsiveness during the interview process). We argue that the effectiveness of the strategies will depend on the degree to which, respectively, the two data sources are commensurable, the evidence for the cognitive process is robust, and the interview or interviews evolve in ways relevant to the challenge. The effectiveness of the researcher will depend on the degree to which the scholar identifies the particular challenges at play, avoids conflating one challenge for another, and deploys strategies ideal for the specific challenge at hand. We suggest that our work provides a guideline for evaluating interview-based studies on motives and charts a path toward future progress on the scientific foundations of in-depth interview research. We begin by defining our terms and scope.
Background
Scope
Given what we believe is recoverable through an interview, our conception of motivated action is purposely narrow. We define a “motivated act” as one aimed at fulfilling a motive or set of motives which the actor reflected on before effecting the action (see Dewey 1922). We focus on the interviewer’s ability to capture that motive or set of motives accurately. Given our conception, three issues are worth noting.
First, we note that many and probably most actions are not, in fact, motivated as described above. As Schutz (1964) writes, “It is erroneous to assume that consciousness of […] alternatives and therefore choice is necessarily given before every human action, and that in consequence all acting involves deliberation and preference” (p. 78). He offers an example: “When I walk through a garden discussing a problem with a friend and I turn left or right, I do not choose to do so. I have no alternative in mind” (Schutz 1964:78). Sociologists, psychologists, and philosophers have written a great deal about the differences between action that is motivated as we have described it and action that is not—examples are action that is habitual (Dewey 1922; Schutz 1964), dispositional (Bourdieu 1977, 1990), and intuitive (Kahneman 2003a, 2003b, 2011; Moore 2017; Vaisey 2009). Many actions are simply not motivated, in that they do not involve reflection of any kind, and critics are right to point that researchers risk imputing motives to nonmotivated action (Small 2017). We focus only on actions that are reflectively motivated (see also Weber 1978:8ff).
Second, we note that many actions could in theory be described as “motivated” in a manner different from that on which we focus. Many of the factors that drive an action are either subconscious or unconscious—that is, they shape an action even when the actor has not thought about the motives before acting (e.g., see Bargh et al. 2001). For example, experimental research on discrimination has shown that, in the aggregate, populations such as employers and others are often driven to act by implicit bias or prejudice (Banaji, Blair, and Schwarz 1996; Greenwald et al. 2009; Vila-Henninger 2015). We do not include this kind of “motive” in our discussion, as it involves a large and thorny set of issues—there is a reason much of this work is based on experiments, not interviews—and the study of motives that actors have expressly reflected on before acting already, as we shall see, involves a number of difficult challenges.
Third, we do not presuppose that any class of actions is always motivated. Stated differently, we assume both that there is heterogeneity across situations in whether actions are motivated and that this heterogeneity does not reflect types of actions that can be determined a priori. People reflect on their motives for both important and unimportant decisions; and sometimes reflect and sometimes do not. Some drop out of school after great reflection; others, on a whim. Thus, any action under study may or may not have been motivated, and part of the interviewer’s challenge, as we discuss below, is to determine whether it was. 1
In spite of these caveats, it is clear—as much of the interview-based research in practice assumes—that many actions are deliberately motivated in the particular way we have described. It is also clear that interviews in theory should be capable of recovering some of those motives. Indeed, every paper on Table 1 assumes that interviews can recover motives.
That assumption is not especially radical. For example, it is the foundation of the entire scientific enterprise, since the journal review process is based on the notion that true motives can be elicited. When journal reviewers (actors) make a recommendation for a paper (action), they are asked to justify their recommendation (express their motives) in a reader’s report. Editors take the report as accurately describing the motives behind the action, even if they do so judiciously. Many other actions are similarly motivated—that is, they are perpetrated by people who reflected on their motives—and interviewers must find their own means of eliciting those motives.
Naturally, the observer need not take those expressed motives at face value, for many reasons. For example, in the case of journal reviewing, the reader’s report may mask the reviewer’s true motives (e.g., a theoretical axe to grind) and may exclude some motives (professional envy) in favor of others (flaws in the paper). Moreover, it may even be the case that writing the report itself (expressing the motive) was the way the reviewer arrived at a recommendation, such that the process played a role in the expression. Challenges of this kind make clear that what people express as their motives should not naively be taken to represent true motives. However, they do not counter the fact that an act could have been reflectively motivated and that the motives may be recoverable. The extent to which researchers can recover those motives accurately is our focus.
Motives Versus Causes
Interviewers continue to elicit motivations from their respondents because finding explanations to processes of this kind is indispensable to sociology. Yet the motivations behind an action are not synonymous with the causes of the action—the motives an actor reflected on are at best proximate causes often shaped by structural or other fundamental causes 2 (Freese and Kevern 2013; Lutfey and Freese 2005; Morgan and Winship 2008). Nonetheless, in-depth interviews are indispensable to full explanations of such phenomena, even if the interviews are supplemented by other methods (such as controlled experiments to adjudicate among causal explanations or national surveys to assess the representativeness of uncovered answers). In-depth interviews have unique and well-documented strengths, including the capacity—at least in theory—to uncover motives that the researcher had not thought of and to capture the full array of ways a given motivation may manifest itself.
Still, the core problem remains. Assuming that an action was indeed motivated as we have described—that is, that an actor reflected on the action and was moved to pursue it on the basis of a motive—will the set of motives expressed retrospectively by the interviewee accurately reflect those that motivated the action? Our concern in the pages that follow is to examine, systematize, and assess the specific challenges involved in addressing this problem. There is a philosophical debate to be had over the assumptions involved in many aspects of this question (e.g., see Briggs 1986; Cicourel 1964; Deutscher et al. 1993). Our focus, instead, is largely with practice and our position is ultimately an epistemologically pragmatist one. In what follows, we identify the specific challenges interviewers face, examine what researchers have done in recent years to address them, and assess the effectiveness of these strategies.
Challenges
We have examined the recent empirical work on these questions and identified five challenges central to the attempt to use interviews to uncover motivation. We deliberately set aside challenges that are general to all interview research—such as establishing rapport, reactivity, power, intersectionality, and so on—as these have been discussed at length. We refer readers to Cicourel (1964), Weiss (1994), Deustcher et al. (1993), and Gerson and Damaske (2020) for those issues. The five core challenges are deception, recall error, reasonableness bias, intentionality bias, and single-motive bias.
Deception
Any interviewer faces the possibility that interviewees may lie about the true motives behind their action, whether due to lack of rapport, social desirability bias, or other factors (see Gerson and Damaske 2020; Tavory 2020). Many researchers have discussed this challenge. Gorden (1972) writes that interviewees choose not to share because doing so would threaten their self-esteem, a phenomenon he refers to as “ego threat.” At the most extreme, the interviewee refuses to admit information to himself and to the interviewer, but more commonly, the interviewee fears that the interviewer may disapprove. Pryor’s (1996) study of pedophiles addresses this challenge directly. Pryor writes, the greatest concern I had was whether or not the offenders I interviewed were telling me the truth as they knew it. […] The stigma attached to sex crimes suggests extensive efforts to avoid detection and to save face. During my early fieldwork in this research, a few clinicians repeatedly advised me not to expect a complete or reliable accounting from any respondent. One therapist in particular, who referred cases in the research, commented that the offenders he knew were “like electricity”: “they follow the path of least resistance. They’ll reveal only as much as they have to.” The question, then, is whether any researcher can ever honestly capture the reality of this particular group of people. (P. 24)
Recall Error
Even honest interviewees may not remember their motives or else remember their motives inaccurately. Omission, addition, or distortion of information are all fairly common possibilities for many of the empirical questions that researchers examine. As Rubin and Rubin (2005) write: “Unfortunately, though people often try to tell the truth, sometimes their memories have faded or they have blurred two or more events or characters together, or not remembered exactly what was said” (p. 76; see also Schwarz 1999, 2019; Sudman, Bradburn, and Schwarz 1996).
A common point made by qualitative researchers about recall error is that, in multiple ways, the present alters recollection of the past. Becker and Geer (1957) conceive of the problem as the product of changes that individuals unavoidably experience: Changes in the social environment and in the self inevitably produce transformations of perspective, and it is characteristic of such transformations that the person finds it difficult or impossible to remember his former actions, outlook, or feelings. Reinterpreting things from his new perspective, he cannot give an accurate account of the past, for the concepts in which he thinks about it have changed and with them his perceptions and memories. (P. 32) This study, like most analyses of past social processes in developing nations, depends on people’s recollections about prior experiences. Analyzing retrospective data requires some caution because present-day reports of the past can be malleable. In addition to basic recall difficulties, respondents’ accounts of the past may be influenced by their present-day identities, especially given that [social] movement participation itself may cause identity shifts. (P. 13) the fact that a person’s views and interpretations change over time; past behavior is reinterpreted in light of new information and experiences. This is especially likely when people are forced to undergo mental health treatment or spend time in jail or prison. In such circumstances, the accounts people formulate about their lives are likely to be shaped by the institutional context in which they are embedded. The longer the time between the offense and the interview, the worse the potential overall contamination as well. (P. 300)
Reasonableness Bias
A third challenge, which we call reasonableness bias, is less obvious but equally important. People have good reason to both understand themselves and represent themselves to an interviewer as reasonable. As a result, they may tend to report motivations to their interviewers that appeal to common sense but are in fact post hoc. As Becker (1998) notes, “why” questions may communicate to the interviewee that the researcher is looking for a “good” answer, one that made sense and could be defended…. The answer should express one of the motives conventionally accepted as adequate in that world…. “Why are you late for work?” clearly asks for a “good” reason; “I felt like sleeping late today” isn’t an answer, even though true, because it conveys an illegitimate intention. “The trains broke down” might be a good answer, since it suggests that the-intentions were good and the fault lay elsewhere…. (P. 59)
People may feel compelled to produce answers that “sound good” or “make sense” given the context and constraints—even if these were not, in fact, motives at the time of action.
Psychologists have pointed to several mechanisms behind this effect, including a tendency for people to infer their prior attitudes from behavior that has already occurred (Nisbett and Valins 1972). In an extensive discussion of these issues, Nisbett and Wilson (1977) find that when participants in experiments report on their behavior, their errors are systematically biased toward explanations that seem reasonable and aligned with common heuristics: “They may resort […] to a pool of culturally supplied explanations for behavior of the sort in question or, failing in that, begin to search through a network of connotative relations until they find an explanation that may be adduced as psychologically implying the behavior” (Nisbett and Wilson 1977:249). 3 Furthermore, psychologists have found that as the time between an event and recall of that event increases, people rely more and more on stereotypical, “generalized beliefs” to fill in the narrative and explain their feelings and behavior (Robinson and Clore 2002:938).
For example, in a study of divorce, Hopper (1993) found that people reconstruct their motives postdivorce such that the retrospective motives given by divorcees are inconsistent with what actually happened during the divorce. Hopper described the contrast between descriptions of the experience and descriptions of motives: People did not seem to change their stories about their experiences substantially over time nor did they seem to selectively reveal or conceal information during lengthy interviews. They described failings on both sides, along with a fundamental uncertainty and indeterminacy in their marriages. In contrast, when they described their motives, they seemed to communicate a remarkably unidimensional stance toward those multifacted experiences. (Hopper 1993:804–5)
Intentionality Bias
A fourth challenge is easy to confuse with the third. While people have motive to represent themselves as reasonable, they may also be inclined to represent themselves as intentional—as individuals whose actions are willful and deliberate. An interviewee may incorrectly describe an action that was unexpected, spontaneous, or otherwise unreflective as the product of careful thought. Psychologists have described this as the difference between system 1 (intuitive) and system 2 (reflective) actions (see also Dewey 1922; Schutz 1964). As Shafir, Simonson, and Tversky (1993) write, individuals can be “unaware of the precise factors that determine their choices, and [may] generate spurious explanations when asked to account for their decisions” (p. 13). Tavory (2020:451) has recently referred to a similar idea as “agency bias.”
For example, Edin and Kefala’s (2005) study of why women gave birth before marriage found that some interviewees portrayed themselves as intentional (they wanted a baby), yet expressed ambiguity about the pregnancy (they were not “trying” for a baby): Sam, a 21-year-old mother of a 4-year-old, recalled, “If I was pregnant, I wanted to be pregnant, but if I wasn’t it didn’t matter. It’s like I wasn’t planning to have a kid [but] I wasn’t doing nothing to stop it from happening neither. I was ready if I wound up getting pregnant” (Edin and Kefalas 2005:39–41, italics original). Likewise, when asked whether she had planned to get pregnant, Violet, a 16-year-old mother of a 5-month-old answered, “No, not really. In a way I did, in a way I didn’t. I was confused. I wanted to be a mom and I did not want to be. It was back and forth. I don’t know, I just wanted a baby, I guess’” (Edin and Kefalas 2005:41).
Single-motive Bias
Single-motive bias refers to the tendency of people to report a single motive behind motivated actions even where multiple motivations were at play. As Weber (1978) has argued, many social phenomena result from multiple causes rather than a single cause, and the same is true at the micro level with respect to motivated action. Psychologists have long noted that people often “satisfice” (Simon 1976): Instead of searching through the mind for all of the possible answers, interviewees may “truncate the search process” as soon as a sufficient answer has come to mind—and information that has been encountered more recently and frequently comes to mind first (Sudman et al. 1996:71). Kadushin (1968) notes the implications for interviewers: Unless respondents are queried about each of the relevant factors, the multiple causation of an act is not apparent. Mixing several factors belonging to different elements and then setting their total equal to 100 percent is a typical error […] The fault lies, not in the reasons, but rather in the way they are collected and tabulated. (P. 340)
Indeed, both respondents and interviewers may fall into the trap of letting the mind rest because a satisfactory explanation has been produced, even if it is incomplete. For example, a study of why emigrants fled a war-torn country may easily rest on the expressed motive about fear of war, without probing the fact that other countryfolk with equal motive did not leave. It is possible that isolation, poverty, domestic abuse, or other factors may be as or even more important motives as the single one reported. Researchers have revealed such problems. In a study asking why parents in East and South Asia were motivated to select for sons, Das Gupta and colleagues (2003) found that parents first reported single motives, such as “only men are strong enough to do the really hard work in the fields” and “sons are needed for old age support,” but further probing revealed additional—and the authors argue, more central—ones, such as kinship systems in which daughters cut ties with their natal families after marriage and beliefs about ancestor worship (Das Gupta et al. 2003:168).
Strategies
How have researchers addressed these challenges?
Deception
Researchers have employed six strategies to address the challenge of deception. One is to triangulate interview data with ethnographic observation or with supplementary interviews. An example is Hocschild and Machung’s (1989) study of household labor. After an interviewee, Dorothy Sims, describes at an initial interview that she and her husband share all the housework and childcare duties, the researchers observe a different reality while at dinner with the family. Hochschild and Machung write, Dorothy had handed Timothy to her husband while she served us a chicken dinner. Gradually, the baby began to doze on his father’s lap. “When do you want me to put Timmy to bed?” Dan asked. A long silence followed during which it occurred to Dorothy—then, I think, to her husband—that this seemingly insignificant question hinted to me that it was she, not he or “they,” who usually decided such matters. Dorothy slipped me a glance, put her elbows on the table, and said to her husband in a slow, deliberate voice, “So, what do we think?” (Hocschild and Machung 1989:19-20, italics original)
Another example is Liebow (1967), which sought to understand what motivated the men in his study to marry. In interviews, the men tended to report coercion: One man said he married his wife because his grandmother promised him 50 dollars to do so; another, because his girlfriend and her social worker pressured him. But extended observation and additional conversations with wives and girlfriends, dispelled these stories. Liebow concludes that men presented the “public fiction of coercion” to protect them from embarrassment if the marriage failed (i.e., “I didn’t really want to get married anyway,” Liebow 1967:115).
A second strategy against deception has been to seek contradictions in the interviewee’s narrative and bring them to respondents (Gudjonsson 2003; Royal and Schutt 1976). Such strategies may be aimed not to prove to interviewees they were being deceptive but to prompt a reevaluation of an earlier viewpoint. While researchers rarely describe this as a strategy, they often employ it in practice. This strategy was adopted by Brenner (2017) in his study of religious service attendance. After using alternative question-wording to uncover that interviewees tended to overreport religious service attendance, Brenner asked them to clarify the discrepancy. They admitted to counting bible study, weddings, funerals, and other ceremonies as “religious services” and to choosing an earlier reference period in order to claim that they are “the kind of person” who goes to religious services (Brenner 2017:560).
A third strategy is to reduce the odds of deception by increasing the interviewee’s cognitive load (Anders Granhag and Vrij 2010; Vrij et al. 2006, 2008; Vrij 2017; Vrij, Granhag, and Porter 2010; Vrij, Mann, and Leal 2013). As psychologists have shown, lying is more cognitively demanding than truth telling because of the need to not only create a plausible story but also keep it consistent, and monitor the interviewer’s reactions for doubts or suspicions, and suppress the actual truth (Burgoon and Buller 1996; Christ et al. 2009; Kassin, Appleby, and Perillo 2010; Spence et al. 2001; Vrij 2017; Vrij et al. 2008). In an interrogation or otherwise confrontational context, an interviewer can raise demands on cognitive load by asking the interviewees to multitask, to tell their stories in reverse order, or to maintain constant eye contact with the interviewer (Doherty-Sneddon and Phelps 2005; Vrij et al. 2008, 2013, 2010). High cognitive load has been associated with increased stuttering, slower speech, longer pauses, inconsistent answers, reduced blinking, and decreased movements (Vrij 2017; Vrij et al. 2006, 2009). In research contexts, interviewers have used tamer versions of these strategies. For example, ethnographers have shown the value of increasing cognitive load by, as proposed by psychologists, “requiring interviewees to perform a concurrent secondary task (‘time-sharing’) while being interviewed” (Vrij et al. 2006:142); they have probed respondents while they are in action in various contexts—for example, while preparing to or engaging in exercise (Wacquant 2004) or while driving in the car (Hochschild 2016; Lareau and Rao 2020).
A fourth strategy has been to ask questions about motivation multiple times over the course of one or across multiple interviews. Over time, rapport increases, as does truthfulness, and trustworthiness. Weiss (1994) has written about the insights gained from repeated interviews: “Respondents also may shade their responses to present a positive picture of themselves. This seems to me most likely in a first interview; in later interviews a respondent, more confident of acceptance, may provide corrective information” (Weiss 1994:149). Laslett and Rapoport (1975) describe something similar: “In the first interview, the data offered are characteristically of the kind that puts the ‘best face’ on the family (or the respondent’s) situation. This is not necessarily the result of dissimulating or attempting to mislead the interviewing. It is the culturally normative pattern” (p. 973). As the value of multiple interviews for reducing social desirability bias and deception is widely acknowledged, a number of researchers wrote in their methodological appendixes that they made efforts to interview respondents multiple times to see whether their answers varied (Edin and Kefalas 2005; Pryor 1996).
A fifth strategy to reduce deception has been to word questions in a way that reduces what psychologists have termed “ego threat” (Gorden 1972). If the researcher and interviewee have not yet established sufficient rapport, the interviewee may choose not to share something sensitive if they believe the interviewer may disapprove. Gordon (1972) writes: Questions can be worded to include a face-saving preface to reduce the ego threat. For example, “I know people nowadays do not have time to keep up on everything that is going on in the world, but would you tell me if you have read anything about the Near East crisis?” Several studies on the validity of people’s statements regarding current events have demonstrated that they exaggerated the amount of reading that they have done and tend to feel guilty about how little they know about current world issues. A preface of this type will reduce the urge to overestimate one’s reading and reduce the ego threat involved in admission of ignorance. (P. 282) Perhaps the most frequently used technique for making a delicate question acceptable is to incorporate in it a brief statement intended to “educate” the respondent about the nonjudgmental character of our interest in him. In a survey among adolescent boys, the researcher wanted to find out whether there were differences of opinion between the boys and their parents about what time the boys should come home at night. The researcher felt that it was important to make it acceptable for respondents to admit that there might be family disagreement on this point. The question was worded, therefore: “In talking with young people all over the country, we find that many have disagreements with their families on what time they should be home at night. Do you have disagreements with your parents on this?” This wording did two things; it informed the respondent that he was not unusual if he had such disagreements, and it let him know that the interviewer considered such a response acceptable and normal. (P. 126) When I asked them what they would say to convince readers that they were being honest, the men emphasized that there was really no way to convince people they were telling the truth. They admitted that some offenders do lie, and that the image of dishonesty was not unfounded. They sometimes acknowledged having lied to their counselors themselves because of the fear of legal repercussions. Often they spoke about having nothing to gain by lying since they had already confessed to being guilty. (P. 27)
Recall Error
Researchers have used at least four different strategies to address the challenge of recall error. The first, as above, is to triangulate by obtaining supplementary interviews with others. One example is Scheper-Hughes’s (1992) study of infant death in Brazil. Concerned that respondents might forget parts of their complicated reproductive histories, the author interviewed mothers along with their own mothers, adult daughters, siblings, or female friends: “Together they corroborated the particular woman’s history, serving as one another’s memory checks and guides” (p. 306). Likewise, as Sudman and collaborators (1996) write, Even when events seem to a respondent to be entirely forgotten, additional information can produce new recall. Wagenaar (1986) took some events that his subjects had given up trying to recall and interviewed other participants about them to get additional information that could be used as cues. When these cues were used with his subjects, additional events were recalled. (Pp. 177-78)
A third, related, strategy has been to encourage the interviewee to recall as many details as possible about the time and place in which the motivated action occurred—including previous moods, physical sensations, sights, sounds, tastes, and other contextual details (Bower 1981; Godden and Baddeley 1975; Schwarz 2019; Strube 1987; Tulving 1983). Vrij and colleagues (2013) write, “Experiencing an event should allow various perceptual qualities (e.g., sights and sounds) to be encoded along with other, salient, a-perceptive qualities (e.g., emotions and duration) into a single memory trace” (p. 121). Bracketing questions about motivation within specific questions about sensory perception and context can aid in information retrieval (Sudman et al. 1996). For example, Viterna (2009) interviewed Salvadorian women about why they became guerillas in the civil war by placing “why” questions within a series of factual recall questions about concrete events: After re-creating the environment under which a woman had joined the guerillas in the past (where she lived, who she lived with, how many children she had, what activities she was involved with, etc.), I would then ask the more subjective question of why a woman decided to join the FMLN. I continued this rotation of asking a series of concrete, life experience-rooted questions followed by more narrative questions. […] I hoped that by mixing questions about concrete events with open-ended questions that sought subjective memories, I could reduce memory inaccuracies. (P. 284, italics original) People tend to respond in terms of an ideal image of themselves as integrated, rational beings, and to hide from themselves and the interrogator the often irrational, inconsistent truth. Concretely, when the cross-pressured respondent is asked, “How often do you attend union meetings?” he does not really answer this question; he makes an immediate, largely unconscious translation, and answers, instead, the question, “How often do you picture yourself attending union meetings?” or “In terms of your general believes about unions, how often is it appropriate for you to attend union meetings?” (P. 40, italics original)
Reasonableness Bias
Researchers have used at least two different strategies to address the challenge of reasonableness bias. One strategy requires full knowledge of the decision situation, as it provides an opportunity to probe whether an answer merely reflects a bias toward reasonableness. Gorden (1972) describes a study of interviewees being asked why they ran from a house during an explosion: Some said that they wanted to get out of the house before it exploded. Actually, there was no way for people to suspect that the initial explosion was to be the first in a series; the causal connection between the first and succeeding blasts could not have been known immediately. But once the facts were known, running out of the house appeared the rational thing to have done. These respondents, because of chronological confusion, did not realize that they were explaining their behavior at one point in time as being motivated by an idea not gained until after they had acted. (P. 101, italics original) Women who are cost accounting are doing so before they decided to take a risk. Once they have taken the risk, and particularly once that risk has had negative consequences, they must of necessity reexamine their cost accounting in light of the actual and actualized present costs of the unwanted pregnancy and subsequent abortion. They are in the position of automobile drivers noted in the present study who postpone having repairs made to their cars for financial reasons and who subsequently have an accident. Once the accident has occurred, the previous postponing of repairs seems extravagantly foolish, no matter how sensible it seemed before the accident. (P. 167) Would [they] still have approached person P if they were not motivated by m (e.g., would they have still talked to their lab partner if they were not motivated by desperation?) [Would] they still have approached [person] P if the topic were different (e.g. would they still have confided in their mother if the topic were work, rather than relationship troubles?) [Would motivation] m still have motivated them to approach P if the topic were other than t (e.g., would their desperation still have mattered if the issue were a failed course instead of a failed relationship?) [Would] topic t still have motivated them to approach [person] P if they were not motivated by m (e.g., would they have still talked to a professor they did not know about a given topic if they were not desperate?) Why did they not approach person Q, rather than [person] P, whenever Q would have seemed equally appropriate (e.g., why not approach a different particular graduate student to talk about their second-year paper?) (P. 191)
Gorden (1972) suggests something similar when he writes, “We are more aware of specific actions which would constitute a violation of a norm than we are of the norm itself” (p. 285, italics original). He recommends that researchers set up a hypothetical case, suggest possible actions or alternative actions, and see how the interviewee responds. Gorden writes that a strength of the hypothetical case is that it “avoid[s] inferential confusion by not asking the respondent to abstract the rule logically from the specific examples, but by letting the interviewer handle the inference process” (p. 286). In a related vein, D’Andrade (2005) writes, “In general I have found it is better not to ask informants directly about their models, but rather to ask something that will bring the model into play; that is, something that will make the person use the model” (p. 90, italics original).
Intentionality Bias
There has been comparatively less discussion in the literature about intentionality bias and possible strategies. A compound strategy has been noted—to leave room for the action to be unintentional by asking ask how, rather than why, something came about (Becker 1998; DeLuca, Clampet-Lundquist, and Edin 2016; Small 2017). In an extended discussion of this strategy, Becker (1998) recalls, When I interviewed people, asking them why they did something inevitably provoked a defensive response. If I asked someone why he or she had done some particular thing I was interested in…they answered my “Why?” questions briefly, guardedly, pugnaciously, as if to say, “OK, buddy, that good enough for you?” When, on the other hand, I asked how something had happened, [they] gave accounts that included not only their reasons for whatever they had done, but also the actions of others that had contributed to the outcome I was inquiring about…. They understood the [“Why?”] question to be asking for a cause, […] And not just any old cause, but the cause contained; in the victim’s intentions. If you did it, you did it for a reason. […] “How?” questions…didn’t telegraph the form the answer had to take (in the case of “why,” a reason contained in an intention). As a result, they invited people to include what they thought was important to the story, whether I had thought of it or not. (Pp. 58–59)
Single-motive Bias
Researchers have used at least two different strategies to address the challenge of single-motive bias. One is to expressly ask interviewees whether multiple motivations were at play. For example, Menjívar (2000) examined why one of her respondents, immigrants from El Salvador, chose to stay in San Francisco instead of moving to Washington D.C. with her husband. When asked why, the interviewee first reported that it was because she heard it snows in D.C. and prefers to avoid the weather. Menjívar (2000) reflected: Though not completely surprised, I still found this reason a bit peculiar. The next time I saw her I pressed a bit more to find out why she had stayed and the consequences this would have for her. Without discarding her fear of the snow, she expressed a more powerful reason. She explained that she had been able to obtain baby-sitting jobs through an employer she had met at a community organization, and she felt she could support herself and be able to remit a little if she kept working at about the same pace after her baby was born. (P. 185)
A second strategy has been to ask interviewees to talk about people in similar situations whom they know and their motivations. Rubin and Rubin (2005) write, Normally, you would only ask people about their own experiences, but if the self-reports are suspect, to encourage your interviewees to explain more fully the reasons for dropping out, you might take a different tack and ask the interviewees to provide examples from other people they know. In most answers, the interviewee will be drawing on his or her own experiences as well as those of friends and acquaintances, and may point out which examples are personal. (P. 74)
Assessing Effectiveness
Effectiveness of the Strategies
Each of the numerous strategies we have identified is implemented differently by different authors, some inevitably with greater effectiveness than others. While it is therefore impossible, within our space constraints, to assess how effective each application was, we can identify the general principles needed to determine whether they are likely to be effective. Those general principles depend on what kind of strategy these are. The strategies we have described are of three kinds: external, internal, and interactional.
External
Many of the strategies have relied on data external to the interview to confirm the reports from the interviewee about the motives behind action. Of these, as we have seen, some include interviewing other people; others, observing behavior; still others, assessing whether the reports are consistent with externally known facts of the case. The key is triangulation, which has been used as a strategy to mitigate the effects of deception, recall error, and reasonableness bias. We have not uncovered researchers using triangulation to address the intentionality bias, but the strategy is relevant to that challenge as well. We suggest its effectiveness will depend fundamentally on commensurability, on whether the external types of data are in fact commensurable with the in-depth interviews—that is, whether the external approach does in fact provide insight into the same issue, the interviewer’s motives, as opposed to capturing some other aspect of the phenomenon.
Ensuring such compatibility is not always easy in practice. For example, suppose the research question is what motivated a woman to bear a child before marriage, her reported motive was the love of children, and the potential concern is intentionality bias, the notion that she might have reported this motive while the pregnancy might have been accidental. Suppose the researcher seeks external validation by speaking to the woman’s partner, and the partner expressed that it was not, in fact, intentional. It is entirely possible that he is correct and she is not; but it is also possible that she intended the pregnancy while he was more ambivalent. In this case, the problem is that the partner does not have direct access to the motives of the woman, and for something as sensitive as giving birth intentionally, this lack of direct access may be an issue. Stated differently, the degree to which her expressed motives were intentional can only be addressed in a limited way by asking him. If so, a different strategy might be more effective. An alternative external strategy might be to access—through the interviewee or any other ethical means—information about her contraception. If she was still using contraception when she became pregnant, then the notion that it was intentional is more suspect.
Internal
Many other strategies have relied on how questions are asked to tap into the cognitive processes underlying the challenge at play. Of the strategies listed above, the following are internal in nature: increasing cognitive load, providing diverse recall cues, asking about the most recent occurrence as opposed to a typical occurrence, counterfactual questioning, asking how something came about as opposed to why, and asking about additional motives or the motives of similarly situated others. The key to this set of strategies is question wording. There are enormous literatures on issues such as uncovering deception through psychological means, and it is well beyond scope for us to cover them here (Anders Granhag and Vrij 2010; Granhag and Strömwall 2004; Otgaar and Howe 2017; Vrij 2017). Still, we suggest that the effectiveness of such strategies will depend on how robust the empirical evidence for the cognitive mechanisms at play is. For example, psychologists have found that in an experimental setting, participants assigned cognitively taxing tasks (memorizing long strings of letters) were more likely to be honest than those assigned easy tasks (memorizing short strings; van ’t Veer, Stel, and van Beest 2014). In general, the degree to which the strategy is effective will depend both on how well established the psychological evidence in support of it is and how applicable the extant evidence is to the particular interviewees, conditions, or context.
Interactional
Other strategies have relied on shaping the ongoing interview interaction, or interactions, to mitigate the impact of the potential bias, error, or problem. Of the strategies listed above, listening to and responding to instances of interviewee self-correction, asking a question multiple times within or across interviews, and encouraging interviewees to reflect on and engage directly with the question of honesty are interactional in nature. Talk is inevitably a part of such strategies. But if the key to internal strategies is question wording, the key to interactional ones is responsiveness. Interactional strategies rely on responding to the unfolding interview, or set of interviews, with the aims of building trust and improving honesty, accuracy, and self-reflection. Such strategies are quite common among experienced interviewers, not merely in the study of motivation but in the study of other issues as well. The in-depth interview is, above all, an interaction, and entire fields of thought are focused on the implications of this proposition (Cicourel 1964; Garbarski, Schaeffer, and Dykema 2016; Laslett and Rapoport 1975; Maynard and Schaeffer 2006; Miller and Glassner 1997; Rinaldo and Guhin 2019; Silverman 1993). Because they rely on responsiveness, the effectiveness of such strategies can be assessed by whether the interviewee’s responses shift in direction consistent with the challenge at play over the course of the interview or set of interviews. 4 For example, in the context of deception, the respondent admits to have previously lied; in the context of recall error, they come to remember more; in the context of reasonableness bias, they express nonreasonable explanations; in the context of intentionality, they communicate a lack of intention or spontaneity in their decisions.
Effectiveness of the Researcher
Regardless of the effectiveness of the strategies themselves, the researcher may or may not adopt them effectively. We see three potential threats. One is the failure to identify the challenge at play. Many of the problems in the reliability of empirical research can be traced to a researcher’s failure to note that an issue may be a threat at all. For example, it is common for a researcher to simply ignore that multiple motives may be operating, reporting single motives as true motives. Indeed, with the exception of deception and recall, very few studies have even articulated the challenges as explicitly or systematically as we have here, with most of the work on them implicit or ad hoc. The majority of studies on Table 1 did not explicitly articulate the challenges at play.
A second potential threat is conflating one challenge with another. Different challenges require different strategies, and being unclear on the challenge at play muddles the researcher’s thinking and, in turn, the approach to the interview. A researcher on the lookout for deception may believe that a respondent who reported a motive is lying while the respondent may merely, and honestly, have reported one of multiple motives behind an action. A researcher concerned with reasonableness bias may believe the core problem to be an interviewee trying to paint her actions as aligning with common sense when, in fact, the actions described were not intentional at all. In such contexts, the odds of a successful interview are low.
A third, related threat, is to deploy the wrong strategy for a given challenge. To follow one aforementioned example, if an interviewee is merely reporting one of multiple motives, deploying techniques to avoid deception is unlikely to be helpful, and may even induce people to report answers that were not in fact at play. This is a particularly important issue. Our analysis has been based on the proposition that, when actions were reflectively motivated, an effective interviewer can get closer to uncovering the motives behind the action than an ineffective one, and we have discussed multiple cases where effective interviewers clearly overcame important challenges at play. Nonetheless, full and completely accurate understanding of motives is impossible, and pushing strategies outside of their purview of effectiveness may result in longer responses but greater distance from reality.
Conclusion
We have identified the study of motivated action as one of the most important concerns of in-depth interviewers in sociology today. While some thinkers have proposed that sociologists eschew the use of interviews to study motives, the discipline in practice has done the opposite, repeatedly asking what motivated individuals to effect socially important actions and using in-depth interviews to elicit those motives. That practice is likely to continue, as understanding what motivates people to act—to marry, bear children, migrate, commit crimes, join protest movements, pursue schooling, hire workers, leave jobs, divorce, file law suits, come out, pursue an assault claim, donate, vote, support a political candidate, and much more—will likely remain indispensable to both sociological knowledge and the public interest. Simply refusing to try to uncover motives through interviews is not a viable option for the discipline. Nevertheless, the difficulties identified by critics are real, and, we argue, have called for a much more systematic detailing of the challenges at play, and the strategies to address them, than we have seen.
We have presented at least five challenges—deception, recall error, reasonableness bias, intentionality bias, and single-motive bias—that interviewers hoping to elicit motives face, introduced the multiple strategies researchers have used to address them, and proposed that such strategies rest on one of three general foundations, external, internal, and interactional. We have argued that the effectiveness of any strategy depends on its general foundation: if external, on the degree to which the two data sources are commensurable; if internal, on that to which the evidence for the cognitive process is robust; if interactional, on that to which the interview or interviews shift in ways relevant to the challenge. And we have argued that the effectiveness of the researcher will suffer to the extent the scholar ignores the possibility of each challenge, conflates one challenge with another, or deploys strategies unmatched to the challenge at hand. Our discussion is thus a starting point for researchers to evaluate the effectiveness of interview studies of motivated action.
An implication of our discussion is that an interviewer seeking to uncover motivation would likely follow certain steps: First, anticipate which challenges are likely to be an issue. For example, if the topic is highly private, then deception may be an issue; if the context is morally charged, then reasonableness bias and possibly deception; if the event happened long before the interview, then recall error; and so on. Additional challenges we have not identified may be at play. Second, the interviewer would address the challenges directly, identifying the relevant strategies. Our discussion provided an extensive list, along with numerous examples, and citations to additional work. While the interviewer would study and prepare many strategies in advance, they may also develop and refine new ones over the course of the interviews themselves. Third, the interviewer would acknowledge, in the finished work, the challenges at play, the strategies used to address them, and the degree to which the strategies were likely to be effective. Indeed, among the most important problems we see among weaker interview projects in the field is a failure to acknowledge the issues we have identified explicitly.
We hope our study helps provide a clear path forward. The fact that assessing the effectiveness of different strategies requires different kinds of knowledge, approach, and perspective makes clear that in-depth interview methods will be unavoidably pluralistic, at least to the extent they are concerned with empirically capturing motivated action. While we have only scratched the surface of the problems involved, we believe that continuing to address the issues we describe is necessary to cement the foundations of qualitative research in an era where social science as a whole has faced calls for greater clarity, transparency, and accountability.
Footnotes
Acknowledgments
We thank Jessica Calarco for comments and Harvard University for financial support that made this research possible. We thank participants of the Causality Events December 2019 conference at Harvard for comments.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
