Abstract
How do highly cited scientists account for their success? A number of approaches have been used to explain scientific success, but none incorporates scientists’ own understandings, which are critical to a complete, process-oriented explanation. We remedy this oversight by incorporating scientists’ own descriptions of the value of their work, as reflected in essays written by authors of highly cited articles (‘Citation Classics’). As cultural objects, these essays reveal not only factors perceived to be associated with success but also reflect narrative conventions, and thereby elucidate the culture surrounding success. We enlist Charles Ragin’s Qualitative Comparative Analysis to analyze how factors mentioned in these accounts work in conjunction. Our results show that three ingredients – relationships, usefulness to others, and overcoming challenges – are found in a large majority of scientific success stories.
The opportunity afforded one in writing a commentary on a paper that has achieved Citation Classic status is most unusual. It allows one, by specific invitation from the editor, to reflect personally on what was a well-nigh unique aspect of one’s scientific career.
An occupational disease of authors is a morbid fear that their imperishable words will never be read, so it is pleasant to find that our 1964 paper has been widely cited.
How do we make sense of scientific success? Three sociological approaches have been used to answer this question. Sociologists of science in the Mertonian tradition identify correlates and determinants of scientific success (Fox, 2005; Leahey, 2007; Long et al., 1993; Ragin, 1987; Reskin, 1979) – such as dense coauthorship networks (Moody, 2004; Wuchty et al., 2007) – but their reliance on easily observable bibliometric data precludes a detailed account of activities and dispositions and neglects scientists’ own perspectives. Scientific ethnographies fill this gap by detailing the constituent activities surrounding scientific success and integrating scientists’ own perspectives (e.g. Cetina, 1999; Charlesworth et al., 1989; Fujimura, 1988; Latour, 1987). However, by privileging current and recent activities over more distant events, ethnographic approaches relinquish opportunities for scientists themselves to assess the ‘social context of reception’ (Griswold, 1987a: 1078). Scholars employing a historical and comparative approach present a highly detailed analysis that typically focuses on one or two determinants of scholarly success. For example, in her study of Derrida’s success, Lamont (1987) emphasizes the importance of adaptability, and in his study of philosophical greats, Collins (2000) highlights mentor/mentee lineages and a scholar’s position in the broader field.
Our understanding of scientific success is also informed by folklore: cultural ideas that circulate through scholarly and lay communities. For instance, modern science is widely viewed as methodically mechanical and impersonal (Shapin, 2008). The term ‘scientist’ conjures up images of an individual working in solitude. Scientific ‘discovery’ invokes images of stunning, paradigm-shifting findings (such as the revelation that the earth is not flat), or findings that are immediately accepted as truth. ‘Breakthroughs’ are thought to require an unflappable commitment to science, to the neglect of other aspects of scientists’ lives (e.g. their families and friends). These cultural ideas are powerful: they inform scientific aspirations, shape assumptions about scientific success, alter career strategies, and influence our evaluation of scientific work. They are reproduced through storytelling and informal advice to young scientists.
Even in conjunction, sociological findings and folklore do not provide a complete understanding of scientific success. Scientific communicators should be heeded as much as scientific communication itself (Shapin, 2008: 8). Thus, in this article, we rely on scholars’ own understandings and analytic capacities, which are critical to a complete explanation (Haydu, 1998: 355). Our data comprise essays written by authors of Citation Classics (articles that have surpassed a discipline-specific threshold for number of citations) years after the publication of their article. As reflective and personal accounts of the production, dissemination, and reception of a scientific article, these essays complement extant research whose explanations rest on readily observable factors (e.g. characteristics of the author or the article itself). As Merton (1948 [1968]: 4) notes, publicly available data on scientific products represent few of the ‘intuitive leaps, false starts, mistakes, loose ends, and happy accidents that actually clutter up the inquiry’. Our reliance on scientists’ accounts begins to remedy this deficiency by revealing the rarely observable ‘contingent discourse’ (Gilbert and Mulkay, 1984) that scientists use to account for their success.
Our main goal is to understand how scientists themselves recount and interpret their success. Do their stories support findings from extant sociological research (e.g. the adaptability of the work is critical)? Do they reflect folklore about scientific success (e.g. the best science is planned and solitary)? Scientists’ reflections – what Hermanowicz (2007) calls the ‘subjective career’ – may embellish what we know from both sociological explanations and folklore narratives in two ways. First, they can elucidate the components of factors identified by these sources. For example, beyond just knowing that relationships, especially readily visible coauthor relationships (Moody, 2004), are integral to scientific success, we can assess the influence of more personal relationships, like those (colleagues, spouses, and friends) that are not revealed in either folklore or bibliometric data and thus obscured from most analyses of scientific influence. Second, scientists’ reflections can identify otherwise hidden factors that are only recognizable by the scientists themselves – such as the ubiquity of frustration and the helpfulness of friends. We recognize that such essays are culturally powerful and culturally conditioned. Colleagues, for example, are mentioned in success stories not only because scientists view them as relevant but also because scientists follow narrative conventions associated with success stories. This ‘dual nature’ of accounts (Griswold, 1987b) makes their content all the more illuminating: it gives us a better sense of not just scientific success but the cultural context surrounding it.
Our secondary goals are to assess which factors work in conjunction and to identify multiple possible routes to scientific success. Our goal is to not simply gather ‘all the possible cultural and social influences’ but to ‘understand the interactions of such influences’ (Griswold, 1987a: 1115). We do this by supplementing our content analysis with Ragin’s (2008) Qualitative Comparative Analysis (QCA), which is ideal for understanding whether and how factors work in conjunction. Whereas most literature on scientific impact and intellectual greatness tends to focus on one ingredient, or a ‘best fitting’ set of ingredients that works additively, we aim to assess how such ‘ingredients’ come together to produce one or more ‘recipes’ that characterize scientific success stories. Collaboration combined with novelty may be more prominent in stories of success than mentions of collaboration alone; for example, coauthors may help scientists brainstorm about new solutions to problems.
Explanations of scientific success
Unlike folklore, which typically portrays successful scientists as loners, the extant sociological research highlights the role of networks, social capital, and relationships – at least professional relationships. To help understand what makes some intellectuals and their ideas particularly legendary, Collins (2000) argues that we need to understand scholars’ relationships, particularly sources of alliance and position in various networks. A rich body of quantitative research on collaborative networks documents the importance of coauthors: for example, collaborative scholars are more central in their field (Moody, 2004), and collaborative articles are cited more heavily (Wuchty et al., 2007) compared to sole-authored counterparts. Kuhn (1962) also noted the critical role that supporters play in the development of new modes of research paradigms. Shapin’s (2008) historical analysis demonstrates that the trust and familiarity developed through relationships have become, ironically, more important in the era of modern, high-technology science. In all these explanations, relationships with colleagues and coauthors take precedence over personal relationships with partners and friends.
The novelty (or originality) of science is also shown to be critical to later success. The value of scientific information depends on its originality (Latour and Woolgar, 1986: 219), so scholars try to distinguish themselves by ‘innovating by whatever degree [is] possible within the accepted standards of knowledge’ (Collins, 1968: 1139). Collins’s picture differs markedly from folklore, which depicts influential breakthroughs as ideas that completely diverge from what came before. A superficial reading of Kuhn (1962) is similar, highlighting theoretical paradigm shifts to the neglect of the continual production of new, contradictory results. By relying on citation counts and patents awarded, bibliometric studies of science tend to conflate new results and new theories; however, both aspects of novelty are unlikely to be present in a given scientific work. Qualitative research like ours is likely best suited to uncovering the various aspects of novelty (Dirk, 1999; Guetzkow et al., 2004), including new results and new theory.
However, novelty itself is insufficient. When scientists present new claims and results, they must also ‘persuade others to take the work sufficiently seriously even to bother to check it’ and try to disprove it (Collins and Pinch, 1993: 95). Although every experiment ‘differs from every other in countless ways’ (Collins and Pinch, 1993: 100), questions about what counts as real, and what counts as a correct outcome, can be partly addressed by a ‘critical mass of experimental reports’ (Collins and Pinch, 1993: 105). Thus, while folklore depicts new ideas as immediately embraced by others, the scientific gold standard involves results taken to be replications, validations, and confirmations. While some of this can be accomplished by the authors themselves (e.g. by conducting sensitivity tests and refuting alternative explanations with evidence), much of it is in the hands of subsequent scholars. With time between their publication and later account, and as witnesses to the reception of their work, authors of Citation Classics can assess whether their articles withstood the test of time and further scrutiny – and how they contributed to their scientific success. Scientific fame is built not on solving problems but solving them correctly.
While folklore idealizes the commercial utility of science, sociological studies demonstrate that usefulness to other scientists is also key. As Latour (1987: 25–29) makes clear, the fate of an article depends heavily on whether and how readers and subsequent scientists use the work. Lamont (1987) found that Derrida’s ability to make his scholarship adaptable to others’ scholarship contributed to his emergence as a dominant French philosopher. Moreover, among American physicists, particularly older cohorts, ‘being adaptable’ – one aspect of usefulness – is considered an important quality for academic success (Hermanowicz, 2006).
What do successful scientists themselves have to say? Their attributions may reinforce what we know from folklore and academic investigations; indeed, scientists are neither wholly ignorant of social science nor immune from cultural conventions. Some scientists may very well describe their work as solitary (in accord with folklore); other scientists may attribute their success to mentors and subsequent validation of their work (in accord with academic research). But we also expect that they will reveal much more.
The value added of accounts
Accounts are ‘story-like constructions that contain individuals’ recollections of events’ (Orbuch, 1997: 459). The accounts we analyze herein are solicited, semiautobiographical essays written by highly cited scientists, years and sometimes decades after the publication of their original article. From such accounts, we can glean the origin story behind research projects, but also the context in which it was developed, and the meaning it held for the scientist. The retrospective nature of these accounts allows scientists to recount not only the story behind the production of their article but also its reception in the scientific community (Griswold, 1987b). Because Citation Classic essays encourage scientists to ‘frame their own contributions’ and thereby ‘produce narratives of intellectual selfhood’ (Gross, 2008: 269), they are packed with sociological information that is otherwise unattainable (Franzosi, 1998).
For our purposes, accounts are beneficial because they can elucidate ingredients of the conventional success story (derived from folklore and/or social scientific scholarship) and they can identify concealed ingredients. Accounts allow us to access subjective careers in context (Hermanowicz, 2007). Accounts reveal scholars’ interpretations of influential forces – which often revolve around details of motivations and expectations that a large-scale study would miss (Ewick and Silbey, 1995: 207). With the benefit of hindsight, scientists can tell us why they think their article was received so well: the role of supportive relationships, how obstacles were overcome, and whether the research was useful to others in the scientific community.
Dual nature of accounts
When constructing an account for public consumption, scientists do not simply present the ‘facts’ as they see them, but also consider their own self-presentation, audience expectations, and acceptable vocabularies (Hopper, 1993). This is what Griswold (1987b) refers to as the ‘dual nature’ of accounts. They provide rich data about authors’ intentions and perceptions about their work’s impact while reflecting cultural expectations about what ‘should’ be mentioned when recounting a scientific success (Yearley, 1988), thereby allowing the narrator to demonstrate understanding of, and conformity to, social expectations (Orbuch, 1997: 460). Scientists’ accounts of their highly cited work contain both elements (Mukerji, 1996: 275).
While it is impossible (and perhaps unwise (Shapin, 2008: 18)) to analytically distinguish between ‘reality’ and ‘narrative convention’, acknowledging convention adds complexity, richness, and nuance to our analyses. Specifically, the dual nature of accounts allows us to embellish traditional explanations of scientific success by illuminating the context surrounding scientific success and not just success itself.
Take, for example, the importance of relationships, which the literature reviewed above documents. Surely, scientists who viewed relationships (with mentors, coauthors, and the like) as beneficial to their work were likely to mention them in their essays. However, it is also the case that reference to one’s relationships is expected and indeed conventional in scientific accounts. Mukerji (1996: 263) shows that scientists are particularly cognizant of scientific genealogies and draw upon them heavily when trying to understand and place a scientific work. Even more clearly, Mulkay (1986) documents award-winning scientists’ tendency to engage in what he calls ‘credit reassignment’ when accounting for the scientific work that brought them a Nobel Laureate award. Instead of taking all the credit themselves, scientists conventionally mention and express gratitude to all the individuals who contributed to their success. Thus, a scientist who discusses the importance of relationships is likely recounting his lived experience while recognizing, affirming, and employing the ‘credit reassignment’ convention of this genre of scientific writing.
Data and methods
We capitalize on a unique and under-utilized data source. Instead of examining scientific products (e.g. articles) themselves (Campanario, 1993; Cano and Lind, 1991; Long, 1990; Ratnatunga and Romano, 1997), we analyze essays the authors later wrote about them, thereby accessing scientists’ own interpretations of relevant factors. No study to date has systematically examined these essays, even though they were made available to the public in electronic form years ago. 1 Between the mid-1970s and mid-1990s, the essays were solicited by Eugene Garfield, the developer of Institute for Scientific Information’s (ISI) Web of Science, who asked authors of ‘Citation Classics’ to recount their experiences and discuss why they thought their work had become so important. Only a small subset of published articles receives the field-specific number of citations required to be designated as a Citation Classic. 2 Years after the publication of their original articles, 3 Citation Classic authors were asked to write about ‘how the project was initiated, whether any obstacles were encountered, and why the work was highly cited’ 4 and thus provide a glimpse into the human side of science that is rarely seen in journals. These single-page essays were published in Current Contents from 1977 to 1993 and are publicly and electronically available at the Garfield Library at the University of Pennsylvania.
Sample
We benefit from the public availability of over 4000 ‘Citation Classic’ commentaries spanning 17 years and all major areas of science. 5 These 4000 essays constitute the population of interest from which we have taken a stratified sample of essays representing four diverse time points and seven disciplinary areas. 6 In all, we sampled 367 essays. The disciplines we chose are well represented in the set of Citation Classic commentaries and differ in theoretically interesting ways. 7 While we do not focus on disciplinary differences here, the diverse and representative set of scientific disciplines allows us to confidently make generalizations across fields. 8 We thus focus on general tendencies and variations apparent in the entire sample.
Much can be gained from our focus on highly influential research. The omission of less-influential articles precludes us from drawing causal inferences; instead, our goals are to delineate perceived (and heretofore obscured) components of successful science, elucidate their multiple forms, recognize the influence of cultural conventions on understandings of success, and assess how such components work in conjunction. Narratives about research that achieve Citation Classic status hold immense sway in the scientific community – reinforcing and reworking extant ‘macro-narratives’ that float around the scientific community – largely because of their status. The practice of exclusively studying outliers is justified by the ‘qualitative break that exists between extreme values and lesser values … and also by the cultural importance and historical significance of these extreme cases’ (Ragin, 1987: 11). By focusing on highly influential scholarship, we also avoid the inevitably small subsample size that plagues random samples of events as rare as having profound intellectual influence (Collins, 2000: 57).
Analytic strategy
We use a mixed-methods approach. Our primary analysis is a traditional content analysis in which we coded the text of the essays for themes found in folklore and/or previous sociological literature (e.g., relationships). We also coded themes that emerged inductively, like ‘confronting challenges’ and ‘good timing’. This allows us to assess the relative prominence of many individual themes as well as to elucidate the various ways that a given theme is manifested. For the second part of our analysis, we turn to Ragin’s Qualitative Comparative Analysis (QCA), using the coded data to assess how the various themes work in conjunction. We describe these two stages of our analysis below.
Stage 1
Our first step was to develop a comprehensive coding scheme. We began with a list of factors drawn from previous sociological literature, such as ‘relationships’ and ‘utility’. To these, we added codes that emerged iteratively from our reading, reviewing, and discussion of the accounts. This resulted in a set of 8 main themes (plus 28 subthemes) that were manifestly or latently present in the essays (see Table 1). We carefully defined each code and compiled examples of each from the essays to guide our coding effort. Validity was enhanced by carefully drafting, revising, and finalizing a clear coding scheme that contains multiple examples of each theme as it appeared. Moreover, we applied codes repeatedly when a theme was reiterated in an essay, so we have a sense of how frequently an author touched upon each theme.
Combining themes into 8 sets for content analysis and QCA.
Our coding effort was intensive and designed to ensure reliability. With the final coding scheme in hand, the first author coded all 367 essays with the help of two research assistants. The first author and one research assistant independently coded each essay and met in person weekly (from October 2008 to July 2009) to reconcile their efforts and create a hard copy of each essay indicating the final coding decisions. These hard copies were then given to the second author who reviewed them carefully and noted questions and points of divergence. The two authors reconciled all differences during weekly meetings after which the second author coded all essays in Atlas.ti, a qualitative analysis computer program. We chose Atlas.ti version 6.1 over other qualitative programs (Barry, 1998) because it could handle the format (PDF) of the essays we procured. Atlas.ti is flexible enough to allow several types of codes, comments, and characteristics (e.g. discipline, year, number of coauthors) to be attributed to each essay without forcing an analytical structure upon the data.
Stage 2
The coded data serve as the basis for the second stage of our analysis, which employs Ragin’s (2000, 2008) QCA. QCA is a unique, case-centered approach to data analysis that uses the logic of set theory (i.e. union, intersection), Boolean algebra, and truth tables (which list the logically possible combinations of factors and the empirical outcome associated with each combination (Ragin, 2008: 23)). QCA culls patterns from data in a way that captures the complexity of factors working in conjunction (rather than individual net effects) and identifies multiple recipes associated with a given outcome (rather than imposing a single, best-fitting model). Completing either of these tasks via content analysis alone would be burdensome, inaccurate, and likely infeasible. A software program to assist with analysis is freely available at www.fsqca.com (Ragin et al., 2010).
In QCA, the focus is on the cases (here, essays), which are classified according to their membership in a limited number (ideally ≤ 10) of analyst-delineated sets. Examples of sets include the set of essays that mentions relationships and the set of essays that discusses challenges. The eight sets used in our analysis correspond to the main themes presented in Table 1: RELATIONSHIPS, NEW THEORY, NEW RESULT, CORRECT, UTILITY, CHALLENGES, UNSETTLED, and TIMING. Sets can be crisp or fuzzy. Crisp sets indicate whether each essay is a member of each set, noting membership with a value of 1.0 and nonmembership with a value of 0. Fuzzy sets indicate the degree to which each essay is a member of each set, so membership can be partial (i.e. somewhere between ‘fully in the set’ and ‘fully out of the set’). When more fine-grained information is available to construct fuzzy sets, fuzzy sets are preferred over crisp sets. When fuzzy sets are used, the technique is referred to as fuzzy-set QCA (fsQCA).
To capitalize on the detailed information we coded at the essay level, we use fsQCA rather than QCA. We did not merely code whether a given theme was present in an essay, but also how frequently it was mentioned (e.g. never, once, twice, etc.), permitting a more fine-grained analysis of the salience of each theme to the overall narrative of success. These frequencies are used to develop membership scores in the fuzzy sets. Indeed, fsQCA requires calibration that is ideally theoretically informed. For our purposes, we modify the typical six-value fuzzy set (0, 0.2, 0.4, 0.6, 0.8, 1) by omitting values 0.2 and 0.4 (Ragin, 2008: 31):
0 mentions = 0 fully out of the set;
1 mention = 0.6 more in than out of the set;
2 mentions = 0.8 mostly but not fully in the set;
3+ mentions = 1.0 fully in the set.
Essays that never mention a given theme received a score of ‘0’ for the respective fuzzy set. For example, an essay that never mentioned ‘relationships’ (or one of its sub-themes) was ‘fully out’ of the set of essays that deems relationships important to scholarly success. Essays that mentioned relationships even once jumped to a fuzzy-set score of 0.6, indicating (as does any value greater than 0.5) that the essay is more ‘in’ than ‘out’ of the set. Because no mention constitutes nonmembership (with a corresponding score of 0) and even one mention was enough, in our view, to constitute membership in the respective fuzzy set (a score greater than 0.5, namely, 0.6), the values 0.2 (‘mostly out’) and 0.4 (‘more or less out’) are not used. Essays that mentioned a theme twice were assigned a score of 0.8, and essays that mentioned a theme three or more times were considered ‘fully in’ the set and thus received a score of 1. 9
Although fsQCA is usually used for analysis of cases that vary on the outcome of interest, Ragin has tailored his method for cases like ours: we have many instances of the same thing (each case is a ‘Citation Classic’) and want to identify patterns across them. In essence, our aim is to assess whether and to what extent these instances share features, such as exhibiting a specific bundle (or specific bundles) of constitutive ‘ingredients’. This aim required two modifications to a standard fsQCA application, which we detail in Appendix 1.
Remaining tasks corresponded to standard fsQCA steps. We chose a frequency threshold for the truth table below which combinations were excluded from the analysis (Ragin, 2008: 133). This ensures that we treat combinations with low frequencies as ‘the same as those lacking strong empirical instances’ and allows us to focus on combinations that have enough instances to warrant investigation (Ragin, 2008: 133). Given our relatively large sample size (n = 367) and the large number of sets (8), we decided that combinations (rows of the truth table) that do not characterize at least four essays would be considered ‘remainders’ and excluded from analysis; the included rows represent 62 percent of the essays. 10 After generating output using fsQCA 2.0 (Ragin et al., 2010), we assessed the recipes based on their coverage, unique coverage, and their degree of overlap to get a sense of the most dominant recipes that characterize the accounts. Within fsQCA, the coverage score gauges the degree to which a recipe ‘accounts for’ instances of an outcome (Ragin, 2008: 44) and thus gives a sense of a recipe’s relevance or empirical importance. 11 Unique coverage is a numerical value that denotes the unique contribution of each recipe.
Results
Use of multiple methods helps us fulfill our analytic goals. From our content analysis of the essays, we discuss the relative prevalence of themes and document diversity in the ways in which authors used these themes. This serves as the foundation for the fsQCA, which we employ to determine which combinations of factors were used to account for success. In each of these stages, we are attuned to the way in which ingredients manifest scientists’ perspectives and experiences, as well as narrative conventions associated with the sharing of such experiences.
Common themes
Contrary to folklore about lone scientists, Citation Classic authors quite frequently mention their relationships with others. While many kinds of professional and personal RELATIONSHIPS were viewed as critical to an article’s eventual success, collaborators were mentioned most often (in 58% of the 367 essays). This was particularly true among the subset of essays describing a coauthored article: 93 percent of these 230 essays mention collaborators. Authors often mentioned the benefits not only of a division of labor but also of camaraderie: [my collaborators] had the essential and rare ability to identify the many very similar tree species, without which I was helpless. They also knew a lot about biology and ecology of the trees and forests in Queensland, as well as being all-around good company. (ID 87.025)
Our analysis not only confirms bibliometric research on the importance of coauthor ties (Moody, 2004; Wuchty et al., 2007), but also the more historical work that emphasizes colleagues and mentors (Collins, 2000). Colleagues were a mainstay of professional networks (mentioned in 43% of essays) and perceived as particularly important by those who published alone (51% of the 137 sole-authored essays). One-third of scientists mentioned their days as a student, learning and receiving guidance from teachers, advisors, and supervisors, as in the following instance Our thinking was honed by hours of discussion and sometimes dispute with Walker. Only much later did I come to realize that Walker’s resistance was intended not only to get us to think more clearly but also to strengthen our resolve to prove him wrong. (82.019)
Scientists occasionally mentioned personal relationships, which are obscured in bibliometric data and downplayed in most historical analyses. For example, one neurobiologist claimed that his work would not have been possible without his wife, about whom he said, ‘She was perhaps less efficient than the modern computers but surely much more reliable’ (82.217).
Novelty is also a dominant theme (in 93% of the essays), but the NOVELTY that is described is more nuanced and incremental than the ‘radical overhauls’ of folklore. Scientists’ descriptions allowed us to differentiate between two forms of novelty: new results and new theory. New results were more prevalent than new theory. In all, 62 percent of Citation Classic authors attributed the positive reception of their work to its new result or finding – empirical documentation of something that had not yet been documented. For example, one author stated that his article provided the ‘first convincing evidence that mycotoxicoses could be economically important in farm animals’ (92.018). Moreover, while new theory can also contribute to success (37% of essays mention this aspect of novelty), the complete paradigmatic overhauls of folklore are, as Kuhn (1962) reminds us, uncommon: only 13 percent of the scientists claimed to have drastically altered the intellectual landscape. Apparently, scholars typically innovate only to the degree that is possible within accepted standards of knowledge (Collins, 1968: 1139).
Is influential science immediately accepted (as folklore has it), or is repeated testing and validation required before acceptance (as the scientific gold standard suggests)? Our analysis supports the latter view. More than one-third of scientists (35%) in our sample thought that their success derived, at least in part, from the fact that their results were shown to be CORRECT. One scientist noted that his article ‘withstood the test of time’ (82.283). Another writes, ‘Our description remains a fair analysis of the spectrum of the neurological complications of AIDS. The frequency of nervous system involvement in AIDS has been confirmed in many studies’ (92.063). The import of confirmatory studies is underscored by this scientist, too: ‘Subsequently … the measure has proved to be more robust than originally anticipated’ (92.024).
Successful science is often thought to be useful as well. Various forms of UTILITY were mentioned in 88 percent of the essays. Lending support to folklore depictions, an engineer wrote, ‘In my opinion it was this discovery and its association with strain and lattice defects that became the cornerstone of today’s reliable laser technology, which is why the paper is often cited’ (82.309). However, as revealed in Table 1, two of the most common ways to demonstrate utility are promoting further research in the area (59%) and making one’s work adaptable (51%) to other contexts. For example, a plant scientist claimed that he was ‘gratified by how this work has set the course for the field’ (92.071). Making one’s work adaptable to other settings is another way to link one’s work to future work: ‘By now, the scale has been translated into many languages, and several investigators from foreign countries have collected norms of their own, often on much larger samples than in original studies’ (87.223). Influential articles likely hold utility for subsequent researchers, helping them further their own research in an area. For example, a research article may clarify extant research and make it more accessible to others, focus on an important problem that the scientific community needs answered, or it may have important commercial applications.
Scientists’ accounts confirm the relevance of themes derived from sociological studies (relationships, novelty, verified results, and the utility of the work), but also allow us to glean new insights. We see, for instance, the ways in which deductively derived themes are experienced and recollected – for example, the many kinds of relationships that are important (not just coauthors and colleagues), the ways in which an article can be useful (as a foundation for future research, and not just commercially), and the different forms novelty takes (with new results emphasized more frequently than new theory). We also are privy to factors that are otherwise difficult to measure and thus are unstudied in sociological work. The remainder of this section will discuss the inductive themes that emerged in these accounts, specifically, the role of challenges, an unsettled disciplinary environment, and timing in the creation of influential science.
In the contingent discourse (Gilbert and Mulkay, 1984) evident in Citation Classic essays, authors were also open about the CHALLENGES they encountered.
12
A full 65 percent of essays mention at least one challenge to the development of their work, including competition (16%), resistance from others in the field (22%), initial failure (14%), and effort (26%). One scientist lamented that ‘publication was delayed because the manuscript was lost in the editorial office’ (87.021). To illustrate the Herculean effort sometimes involved in research, one plant and animal scientist studying the migration patterns of geese told a story in which his mentor, Dr. Hanson, encouraged him to shoot as many geese as possible for his study. The author continues, The only time I regretted following Hanson’s advice was when our freeze-dryer broke down (as I recall it cost Charlie’s research grant $3,000 to fly in a refrigerator repairman) … Anyway, my assistant and I each had to backpack 60 lbs of frozen goose samples 30 miles to Eskimo Pt. where we begged and borrowed freezer space. (92.080)
Another scientist recalled, ‘Many futile efforts were first spent, however, and the program was appropriately called Sisyphus after the Greek giant condemned by the gods to roll a big stone up a mountain only to have it roll down again’ (82.285). From these examples, it is clear that the challenges scientists faced were varied and often unexpected.
A lack of consensus or sense of UNSETTLEDNESS, which appears at some historical time points (Kuhn, 1962) and in some fields, can be gleaned from scientists’ accounts. Unsettledness can be discomforting and also serve to motivate scientists to adjudicate competing perspectives and findings. Almost half (45%) of the essayists mentioned their dissatisfaction with extant research. As one psychologist said, ‘I wanted very much to redress what I considered a misguided application of behavioral psychology to personality – a trend that I considered both wrongheaded and philosophically naïve’ (82.216). This scientist then went on to develop an alternative approach in an article that was cited 280 times between 1973 and 1982. Other scholars drew attention to deficiencies in their fields: ‘in the early 1960s, our knowledge of the chemical nature of insect hormones was very poor’ (82.277) and ‘the problem was handicapped by the lack of adequate kinetic data in man and the lack of a suitable method of quantifying beta blockade in human subjects’ (82.260). Another quarter of the essays in our sample mentioned debates raging in their field or subfield that they aimed to address or even adjudicate.
While most studies of scientific success make it appear methodical and based on some combination of effort, novelty, and merit, it is clear that chance plays an important role for a large minority of scientists (Merton and Barber, 2004). This factor is one that could not be revealed in anything but personal reflections. Our content analysis also reveals that a number of authors (43%) claimed to benefit from good TIMING and widespread interest in their topic. Campbell and Fiske, who developed the multitrait multimethod validation technique, noted that ‘there are several reasons for the immediate acceptance and wide application of the paper. One is that the Zeitgeist was ready for it’. Another scientist stated, ‘It was my good fortune to arrive at the right time at the right place’ (77.035). One clinical medicine practitioner claimed that the significance of his work was overblown because of the popularity of the topic: while it was not the first study of its type, it was published at a time when interest in diet and cancer was increasing rapidly. In consequence, it seems to have contributed to the development of that interest to a degree disproportionate with the substance of its findings. (87.247)
These inductively derived themes document the relevance of additional factors that have been neglected in studies of scientific success: the kinds of challenges scientists faced and how they overcame them; a sense of unsettledness in the field, which may motivate scientists; and the impact of external conditions like timing and luck. We move on to the second stage of our analysis to assess whether and how these voiced themes work in conjunction.
Combinations of themes
The QCA results reveal that when we incorporate scientists’ own perspectives, five main recipes for successful science emerge, each with three or four ingredients (see Table 2). 13 Looking across all recipes, we note some important patterns. First, the set RELATIONSHIPS (which includes not just coauthors but also colleagues, mentors, and friends) appears in every recipe, attesting to its prevalence in scientists’ success stories. This is perhaps unsurprising, given the prevalence of coauthorship ties in the bibliometric literature (Moody, 2004; Wuchty et al., 2007) and the prevalence of collegial and mentor ties in the historical literature (Collins, 2000). Second, UTILITY is also well represented – appearing in four of the five recipes. Third, some form of novelty (NEW RESULTS or NEW THEORY) appears in three of the five recipes, though never together. Fourth, CORRECT and TIMING do not appear in any recipe, suggesting that subsequent validation and good timing are not discussed frequently enough to be part of a general recipe of success stories. Fifth, CHALLENGES – a theme that only can be culled from accounts – appears in the most dominant essay (Recipe 1) that has the highest raw and unique coverage score, and thus is empirically the most important (Ragin, 2008: 54).
fsQCA output: reduced configurations.
fsQCA: fuzzy-set Qualitative Comparative Analysis.
Cutoff = 4 (62% of essays); solution coverage = 0.54.
Our goal is to understand how ingredients work together, and for this we focus on these recipes, particularly the dominant Recipe 1. 14 It contains three ingredients – the ubiquitous RELATIONSHIPS, UTILITY, and one of the few ingredients unavailable from other sources, namely, CHALLENGES. The asterisk uniting the ingredients indicates that these three ingredients work together (the Boolean ‘and’) rather than in a substitutable way (the Boolean ‘or’). The dominant Recipe 1 suggests that scientists most commonly attribute their success to the combination of influential relationships, usefulness of the research to others, and overcoming challenges along the way.
To explicate, we summarize and interpret one particularly illustrative essay that has perfect membership in this dominant recipe. (The full text of this recipe is provided in Appendix 2.) From this essay, we learn that Dr. Aserinsky set out to measure eyelid movements while people slept. This was a difficult task that required effort, perseverance, and meetings with his advisor to revise the scope of analysis. Despite such challenges, however, Dr. Aserinsky discovered what is now known as Rapid Eye Movement (REM). He calls this an ‘odd situation’ because his results contradicted his own sponsor’s sleep theory and were incompatible with Freud’s understanding of sleep functions. Even after Aserinsky expressed these doubts, his sponsor encouraged him – their relationship continued to be fruitful even after the contradictory results became apparent and other members of the department expressed ‘dubiousness’ about the work’s merit. Challenges continued as the article was rejected by two journals; eventually, an abbreviated version was accepted by the distinguished journal Science. Later studies extended his ideas to the study of schizophrenia.
This essay reveals that a mentoring relationship played a pivotal role in the initiation of the project, especially for helping to confront the challenges Dr. Aserinsky faced and broadening the utility of the research. First, his advisor helped him reformulate the problem when the initial idea was found to be infeasible. Second, his advisor was helpful in encouraging Aserinsky’s research even when it faced another potential challenge: it contradicted his own research and other well-known sleep theories in the field. Third, his advisor encouraged Aserinsky to extend his work and make it adaptable to a different population, schizophrenics, thereby extending its usefulness. Clearly, Aserinsky’s relationship with his advisor, sponsor, and eventual coauthor (N. Kleitman) is critical in this example; however, it does not function independently of the other ingredients. Indeed, the mentor helped the author overcome obstacles and challenges and helped him extend his work and broaden its utility. The ingredients RELATIONSHIPS, UTILITY, and CHALLENGES combine, in ways like this, to form the dominant recipe.
Recipes 2–4 are very similar to Recipe 1 in that they feature an intersection of RELATIONSHIPS and UTILITY, but each of these secondary recipes substitutes CHALLENGES with one of the following themes: NEW RESULT, NEW THEORY, or UNSETTLED. For instance, in one essay we read that Dr. Sanders, a chemist, began working on his topic at the encouragement of his advisor, published his new finding, and in retrospect attributes the high number of citations to the usefulness of his work: ‘Anybody could magically persuade impossibly difficult spectra to reveal a wealth of new information for a trivial cost in chemicals’ (82.245). In other words, his work was taken up by many subsequent scholars because it simplified an existing problem in a very inexpensive way. Consistent with Recipe 2, this story shows that the author’s RELATIONSHIP with this advisor prompted the research, but its influence came from a NEW RESULT that was USEFUL to others studying in this area.
Recipe 3 also highlights the role of relationships (in motivating research) and utility (in gaining a wide reception), while also showing that scientists attribute their success to new theoretical breakthroughs. A plant and animal scientist reports that after experimental psychologists in his lab pushed him to perform experiments on fish, he began studying muscle innervations in fish. From these experiments, he developed an alternative explanation for fish myotome: a combination of electromyography and measurement of metabolite levels after exercise quickly led to the view that in fish the myotome is divided into two parts: one operating anaerobically during short bursts of high-speed swimming, the other aerobically during long periods of sustained cruise swimming. (82.228)
This new theory was useful to others as it explained differences between dogfish and carp, but more crucially, it ‘suggested that the fibre types of terrestrial animals should be looked at in terms of their evolution from the fish arrangement’ (82.228).
Recipe 4 is similar; however, in this case, the research was deemed influential because it was published during an UNSETTLED time within the field. As a result, it was able to address long-standing problems or discrepancies. In one essay with full membership in Recipe 4, Dr. Transatti writes that his interest in electrochemical behavior stemmed in part from teaching and forming relationships with students, but that he was dissatisfied with the lack of fit between predictions derived from the literature and reality. He conducted a ‘critical search in the literature’, which he says was successful because it ‘enlarged the horizon of the possible interpretation of the structure of an electrochemical interface’ (87.048).
From these secondary recipes, we see that results and new theory never coappear in one recipe; rather, success is attributed to one or the other, but not both. This finding adds to extant literature, which often does not distinguish new results from new theories. While we recognize that retrospective accounts are susceptible to some reinterpretation (e.g. what appears to be a new result at time of publication could, 20 years later, be perceived as a new theory), scientists’ ability to distinguish them shows that they rarely coexist in a given article. This aligns with a reading of Kuhn (1962) who suggests that a series of new results published in separate articles may, eventually and jointly, bring forth new theoretical paradigms.
Finally, Recipe 5, which includes RELATIONSHIPS, CHALLENGES, UNSETTLED, and NEW RESULTS, reveals a recipe for success that accords well with folklore. Successful science is portrayed as identifying an unsettled problem in the field, conducting research to try to solve the problem (with the help of others, and with some challenges along the way), and presenting a new result that helps adjudicate the problem. For instance, one clinical biologist explains that his collaborative research on Alzheimer’s disease overcame challenges from competitors to arrive at a new finding that helped resolve a dispute about the role of cholinergic deficits in the disease (87.044). However, given its low raw and unique coverage scores, this recipe only describes a small subset of essays.
Discussion
How do scientists come to understand their scholarly success? What attributions do they make? To investigate this question, we drew upon a unique data source (accounts), implemented a unique analytic tool (fsQCA), and integrated scholarship on cultural conventions associated with account-giving. Both theoretically and empirically, this approach provides a more holistic, nuanced, and complete understanding not only of scientific success but also the context surrounding it (Hermanowicz, 2007).
As a data source, essays written by highly cited scientists provide a rare glimpse into the development of scientific excellence. In addition to corroborating some key ingredients that extant sociological literature highlights (e.g. the importance of relationships and novelty), Citation Classic essays reveal the various ways in which such ingredients are thought to matter and also offer additional ingredients, many of which can only be gleaned from personal reflections. For example, we were able to elucidate the diverse kinds of relationships that were deemed critical to scientific success (e.g. colleagues, students, friends, spouses), rather than focusing on the oft-studied and easily observable coauthorship ties. Instead of merely confirming that originality and ‘novelty’ are critical to scientific success, we demarcated forms of novelty (Dirk, 1999; Guetzkow et al., 2004) and were able to show that new results are deemed more important to a scientific work’s success than new theory.
We empirically tapped factors like ‘challenges’ and good ‘timing’ that would otherwise remain obscure to analysts. We were able to glean authors’ motivations, such as their dissatisfaction with extant research or the state of the field. Additionally, these accounts demonstrate that contextual factors, like a sense of unsettledness within the scholarly community, were important for gaining wide reception. These reflective, narrative data demonstrated the importance of factors that can be hidden even from scientific ethnographers and from scientists themselves at the time of publication.
Use of fsQCA allowed us to not only consider the frequency with which themes were mentioned but also to assess whether and how they work in combination. This stands in stark contrast to the literature on scientific influence that tends to showcase just one or two key factors. We found a single, dominant recipe that characterizes a large number of Citation Classic essays: RELATIONSHIPS*UTILITY*CHALLENGE. Our analysis is the first to identify the role of challenges and the first to demonstrate that relationships, utility, and challenges work together. Authors tend to mention all three of these ingredients when recounting the development and reception of their highly cited work. Our brief analysis of one essay (on the discovery of REM) with full membership in this recipe shows how these three ingredients work in conjunction. In this scientist’s common experience, relationships were both supportive (his advisor) and challenging (other members of the department were dubious), challenges mounted as his research contradicted well-known work in the field and was rejected by two journals, and yet the work was later used to study another population – schizophrenics, thereby documenting its subsequent widespread utility.
While much of what we offer is new (e.g. identification of heretofore obscured factors, and delineation of how multiple factors work in conjunction), external reinforcement of the key ingredients enhances the validity of our results. Our dominant recipe contains ingredients that are highlighted in both the scholarship on scientific influence (e.g. Collins, 2000; Guetzkow et al., 2004; Hermanowicz, 2006; Latour, 1987) and the scholarship on conventions associated with award receipt (e.g. Mukerji, 1996; Mulkay, 1986; Yearley, 1988). Not surprisingly, we find significant correspondence between the factors that previous scholarship has identified as important, the factors that individual scientists deem important, and the factors associated with the giving of accounts. This suggests that our focus on highly cited works – essentially a form of selection on the dependent variable – does not create much, if any, bias.
We find very little correspondence, however, with folkloric perceptions about the nature of scientific success. Science is not solitary and impersonal. Rather, scientists repeatedly mention the importance of support from not only coauthors, but also more informal relationships with colleagues, friends, and spouses. New scientific results are not immediately accepted. Rather, there are all sorts of roadblocks to getting new ideas published, and even once published, replication and validation bolster the legitimacy of successful work and further demonstrate that it is, indeed, correct. New theories do not drive science. Rather, new results, perhaps collectively, are more critical to a positive reception and eventual theoretical change. Commercial applications (e.g. patents and licenses) are not the only way science can be useful. Rather, new ideas and results can be adapted to new settings and populations and can make future work more efficient and less costly.
In sum, by analyzing reflections by scientists in multiple time periods and disciplines, by relying on scientists’ own analytic capacity, and by integrating research literatures (on scientific success, folklore, and accounts) and methods (content analysis and QCA), we have discovered new insights about the cultural context of scientific success. First, we identified multiple recipes that characterize paths to scientific success. Relationships appear in every recipe and the utility of the work appears in five of the six recipes. According to scientists themselves, these are key ingredients for scientific success. The dominant pathway comprises relationships, utility, as well as challenges (such as initial failure, expenses, and resistance). Second, we demonstrated the importance of additional factors (like challenges, unsettledness, and good timing) that are difficult to capture with other data sources. Third, we decomposed these factors to reveal their constituent forms (e.g. friends as a form of relationships, new results as a form of novelty). Fourth, we showed that these factors – many of which were highlighted individually in previous research – actually work in conjunction.
We hope our work pushes research forward in a slightly new direction. We have demonstrated the utility of autobiographical accounts (Gross, 2002, 2008), and hope that not just historians but other scholars of science will make greater use of them. While historians (Chung, 2006; Hargittai, 2004; Kurian, 2002) and the National Academies of Science have amassed large volumes of scientific biographies, few scholars have analyzed patterns among them as we have here. We also hope that future research will continue our focus on how themes work in conjunction rather than isolation and assess the relevance of the dominant recipe across contexts and time periods. Perhaps recipes for scientific influence depend on a discipline’s age, degree of interdisciplinarity (Rhoten and Pfirman, 2007), degree of consensus (Collins, 1994), or the extent to which it is pure versus applied. Perhaps, scholars will find more recipes, or more diverse recipes, in recent years, which have seen an influx of women and some minorities into many scientific fields. These conjectures will drive our future research, which will explore variation across time periods and disciplines. Answers to these nuanced questions will not only refine theories of scientific discovery but also help produce tailored and possibly more effective research and science policy.
Footnotes
Appendix 1
Appendix 2
Acknowledgements
This research was improved considerably by feedback from Jane Zavisca, Neil Gross, Joe Hermanowicz, Joshua Guetzkow, Charles Ragin, as well as anonymous reviewers and the editor of Social Studies of Science.
Declaration of conflicting interests
The authors declare that there are no conflicts of interest.
Funding
This research was funded by the American Sociological Association’s Funds for the Advancement of the Discipline (FAD) and the STS and SciSIP programs at the National Science Foundation (award #1057559 to Erin Leahey (PI) and Charles Ragin (Co-PI)).