Abstract
Journalists apparently maneuver between their inability to validate every single bit of information and the ramifications of publishing unverified reports. This study is the first attempt to uncover and characterize the reasoning which underlies the journalistic journey from skepticism to knowledge. We draw on the philosophical field of the ‘epistemology of testimony’ and analyze a robust data set. Data consist of detailed cross-verification measures – a reification of journalistic skepticism – underlying a large sample of individual news items in Israeli print, radio, online, and television news (N = 847), following a reconstruction of work processes. Far from being passive recipients of second-hand information, we theorize that reporters make systematic use of ‘evidence of (sources’) evidence’ – a common but previously unarticulated evidence type.
Introduction
Journalists are sometimes described as communicators of facts, who provide, or should provide, the general population with the knowledge necessary for informed democratic participation (Patterson, 2013; Schudson, 2009; Waisbord, 2013). But if the term ‘knowledge’ presupposes any form of certainty, and if the term ‘facts’ signifies direct confirmation, then it is unclear how news – a doubly mediated form of information for both source-reliant reporters and news-reliant audiences – can give rise to firm and well-supported beliefs about reality. Thus, the capacity of communication processes to yield knowledge poses a serious problem: how can evidence be mediated? Essentially, the same problem was discussed by 18th-century Scottish Enlightenment luminaries such as David Hume, for whom second-hand information in itself carried no evidentiary weight, and Thomas Reid, for whom others’ say-so could serve as an acceptable anchor for knowledge (Hume, 1902 [1748]; Reid, 2000).
Unfortunately, the existing literature within the field of journalism studies has not clarified the concepts of knowledge and evidence (more on which presently). Even when the practice of journalism and its products have been closely observed and analyzed, the process by which journalists moved from relative uncertainty and skepticism to higher levels of confidence vis-a-vis the information they covered remained mysterious and appeared arbitrary (Ekström, 2002; Ettema and Glasser, 1985, 1998; Muñoz-Torres, 2012). Thus, in its theoretical contribution, this study draws on relevant philosophical literature offering a journalism-centered notion of evidence. In its empirical contribution, the study provides detailed data about journalistic cross-verification, which is skepticism embodied in practice (as opposed to mere afterthoughts in the reporters’ minds). In what follows, we refer to journalistic skepticism embodied in practice as ‘practical skepticism’ (Reich, 2011b: 31).
We shall employ quantitative methods to measure the extent of practical skepticism in journalism, but not for its own sake. Rather, we are interested in explaining why journalists engage in or avoid practical skepticism. Either practical skepticism and its paucity can be fully explained in terms of sources’ characteristics and reporters’ relationships with them, as much of the literature had suggested, or alternatively, specific features of journalistic knowledge-seeking efforts are the likelier explanation. Data will be obtained through face-to-face reconstruction interviews with a sample of reporters from Israeli print, online, radio, and television news (as well as from various news beats, including political, financial, and domestic affairs news). These reporters will describe in detail the circumstances of cross-verification in every news item in which it occurred (N = 847).
Theory
According to a significant portion of the literature, journalists are always in a state of unbridgeable uncertainty with respect to the subject matter they cover (Allan, 2004; Ericson et al., 1987; Fishman, 1980; Sigal, 1986; Zelizer, 2004). In part, scholars have arrived at this judgment because of journalists’ many documented failures in providing impartial accounts of issues and events (Bennett, 2003; Entman, 2005; Hallin, 1989). In part, scholars have adopted this view because of their own philosophical skepticism about the existence of reality and about the ability to adjudicate claims pertaining to a hypothetical reality (Gauthier, 1993, 2005; Lau, 2004; Muñoz-Torres, 2012). Their reasoning seems clear enough. If no amount of information about a state of affairs can mitigate the inevitable uncertainty with which journalists are faced, it is assumed that journalists gain confidence in their own reporting through non-empirical means. These non-empirical means include primarily the social, organizational, and political status and power of their sources and the nature of journalists’ relationships with them (Ericson, 1998; Gans, 2004; Molotch and Lester, 1974; Romano, 1987; Tuchman, 1972). According to most scholars of journalism, these factors – but not some hypothetical reality – will determine whether reporters believe something to be a fact.
Moreover, insofar as most of the information obtained by reporters constitutes second-hand information (Reich, 2009; Sigal, 1986), no direct evidence is generally available to the journalist. Hence, the above scholarly position entails a specific model of journalistic news-gathering, one in which journalists whimsically allocate trust to specific sources. We may dub this position, which does not attribute to journalists an earnest evidentiary effort, as the ‘trust-based view’ of news gathering (with ‘trust’ denoting the adoption of evidentially unsupported beliefs received through interactions with sources).
The trust-based view stresses the decisive role of exchanges and social interactions with sources, in reporters’ decisions to believe information. These dynamics involve the negotiations of mutual advantages and meanings between journalists and sources (Manning, 2001), 1 but not assessments of evidence giving rise to factual knowledge (Allan, 2004; Ekström, 2002; Ericson, 1998; Ericson et al., 1987; Ettema and Glasser, 1985, 1998; Fishman, 1980; Sigal, 1986). According to Gaye Tuchman (1972), ‘the newsman must question facts by going to the source, but some alleged facts must simply be accepted as “true”’ (p. 664). Richard Ericson (1998) has argued that ‘[T]he journalist must establish the fact value of a story on the basis of a source’s face value […]’ (p. 85). Mark Fishman (1980) had observed that ‘[…] a newsworker will recognize an official’s claim to knowledge not merely as a claim, but as a credible, competent, piece of knowledge’ (pp. 144–145).
Other journalism researchers diverge from this view, despite some tacit philosophical affinity. These scholars argue that while the information journalists gather may never result in the attainment of certainty, it nonetheless carries evidentiary weight. In other words, some classes of information which journalists are professionally obligated to obtain – particularly things journalists can see with their own eyes or hear with their own ears – render some states of affairs more probable than others (Harcup, 2009; Kovach and Rosenstiel, 2007; Lau, 2004; Lippman, 1922; McKane, 2006; Meyer, 2002; Patterson, 2013; Schudson, 2005; Ward, 2004). In popular culture, the image of tirelessly cross-verifying journalists has been immortalized in the film All the president’s Men, portraying the Watergate Scandal (Woodward and Bernstein, 2007). We may dub this minority position among journalism scholars as ‘the evidentiary view’ of news-gathering.
Whether knowledge can or cannot arise from mediated information beyond direct experience has been the subject of intense philosophical debates going back to the 18th century (Audi, 2006; Coady, 1973, 1989; Faulkner, 2007, 2011; Hume, 1902 [1748]; Keren, 2012; Moran, 2005; Reid, 2000). But for such a debate to take place, the mere possibility of knowledge – defined as evidence- and reasoning-based beliefs which happen to be true (Lemos, 2007) – had to be recognized. This is far from obvious as ancient Greek philosophers such as Socrates and Pyrrho have questioned the very possibility of knowledge over 2 millennia ago (Gorgias, 509, 527 cited in Eagan, 1997; Long, 1986), much like present day journalism scholars do today. In the 18th century, however, Scottish Enlightenment philosophers, David Hume and Thomas Reid, have recognized the possibility of knowledge, but have disagreed about its preconditions. For Hume, the precondition of knowledge about the world had been direct experience of some sort (Hume, 1902 [1748]). On his view, without sense data received by a cognizing subject, knowledge about the world could not arise. In contrast, Reid (2000) discerned that much of what human beings come to know arises not only from sense data but also from the testimony of others.
Moreover, according to Reid, the widespread phenomenon of testimony-based knowledge could not be explained by, or traced back to, the sense data received by the knowers. To explain how testimony could give rise to knowledge, Reid (2000) pointed to two previously unnoticed ‘forces of nature’ (p. 119): the instinct of truth-telling and the instinct of trust in fellow human beings. The former was deduced from the introspectively spontaneous character with which truth emerged (without any conscious effort, as opposed to lying which required such effort) as well as from the experience with children, who generally ‘keep to the truth’ (p. 119) even before they internalize any moral commitments to honesty. Likewise, the instinct of trust was inferred from the tendency of children to trust their elders ‘until they meet with instances of deceit and falsehood’ (p. 120).
Thus, despite observing a widespread tendency to rely on testimony, Reid did not immediately leap to the conclusion that something whimsical or willful was taking place (i.e. people merely deciding to believe). Unlike journalism scholars who assumed journalistic arbitrariness born of interactions with sources of second-hand information, Reid sought to explain the very possibility of testimony-based factual knowledge. Thus, he located reliance on testimony in a naturalistic context, on a par with the instinctive way in which human beings connect sense data to ‘external things’ (Reid, 2000: 117).
However, subsequent philosophers sought to ground testimony-based knowledge in a more comprehensively rational calculus involving a formulable principle (rather than merely an instinct). Alvin Goldman (2002, 2010), a prominent epistemologist, suggested that knowledge may be transmitted from one person to another on rational grounds. According to this conception, knowledge can arise from a testifier’s say-so, if and only if the testifier has the evidence and argument to support his or her claim. In the case of journalism, this transmissional conception would suggest that in order to know what their sources know, journalists must possess evidence about the evidence (and argument) possessed by these sources. Thus, to continue Goldman’s line of reasoning, evidence of someone else’s evidence or lack thereof, may also be legitimately regarded as evidence, even if it does not directly entail the existence of some extra-mental phenomenon or the occurrence of some event.
Compare this broadened conception of evidence to the conception of evidence offered in prominent journalism studies. Gaye Tuchman (1972) explicitly mentions ‘presentation of supporting evidence’ as a journalistic procedure, but immediately qualifies that ‘[S]upporting evidence consists of locating and citing additional “facts,” which are commonly accepted as “truth”’ (Tuchman, 1972: 667, scare quotes and emphasis in original). Tuchman’s idea of common acceptance itself, however, is not one necessarily involving factual knowledge, as common acceptance of false beliefs is equally possible. Moreover, as Tuchman makes clear earlier, a reporter covering a series of charges hurled by politicians at one another ‘cannot himself confirm the truth’ (p. 665) of those charges, and he ‘cannot’, on Tuchman’s account, ‘prove that’ a politician’s ‘assessment is “factual”’ (p. 665). Whether the word ‘cannot’ in these remarks denotes an impossibility in principle or merely difficulties of access, it is clear that Tuchman is not referring here to evidence which entails facts (with or without scare quotes). Gans (2004) similarly acknowledges that journalism works with ‘empirically gathered information’ but promptly warns that such empirically gathered information may not adjudicate factual claims or may do so ‘only when there is agreement on the concepts and methods’ (p. 311) – an ‘agreement’ (a la common acceptance above) which is not guaranteed according to Gans (‘… many journalistic facts are not amenable to empirical testing …’, p. 306) and which Gans believes to arise, inter alia, from ‘value judgments’ (p. 306).
In the absence of evidence, it is unsurprising that trust in sources is regarded by these scholars as the main determinant of journalists’ beliefs. As Ericson (1998) points out, ‘factuality for journalists is in any case grounded in having a source person in the know say it is so’ (p. 87). Worse still, Gans’ (2004) description of journalistic trust pools together empirical assessments of sources’ say-so with instances of journalists arbitrarily favoring the say-so of this or that source. On one hand, Gans acknowledges that journalistic judgments of sources’ trustworthiness depend on empirical facts: whether they offer ‘self-serving information’, ‘try to be accurate’, ‘honest’ or whether they have ‘lied’ in the past – issues which ‘can be evaluated over time’ (p. 130). On the other hand, Gans notes that journalists tend to trust [S]ources who cooperate and treat them cordially … who take reporters into their confidence … [S]ources in positions of formal authority … people who are similar to them … conservatives and liberals are trusted more than ultraconservatives and socialists, pragmatists more than ideologists; upper-middle-class people more than others. (p. 130)
But if the notion of trust includes both empirically verifiable forms of reliance and stereotypical, credulous, and faith-based reliance on testimony, then scholars risk an all-inclusive approach to the concept of trust, which indiscriminately devours all forms of belief formation, irrespective of the beliefs’ rational or irrational origin.
We offer a way out of this conceptual morass, by proposing that the empirical portion of Gans’ description of trust (in all its permutations) should be regarded as yet another form of evidence complementing direct experience. If direct experience has been viewed since Hume as evidence par-excellence, then the indirect signs of veracity accompanying mediated information may be regarded as ‘evidence of evidence’, to borrow Goldman’s view. Far from being simply a terminological novelty, this form of second-order evidence is theorized to be an actual factor in the formation of journalistic knowledge, capable of accounting for the findings of the largely descriptive scholarly work on journalistic verification.
Studies of journalistic verification seem to suggest that the extent of verification in journalism is quite limited. One British study had found that less than half of the news stories studied contained any verification (Lewis et al., 2008). A German study had discovered verification to take up only 5.5 percent of journalistic working time (Machill and Beiler, 2009). Qualitative studies have similarly documented instances of apparently incomplete verification. Shapiro et al.’s (2013) study of Canadian journalists has characterized their reasoning as follows: ‘[A] small, easily checkable, fact needs to be checked; a larger but greyer assertion, not so much’ (p. 668), concluding that ‘verification for a journalist is a rather different animal from verification in scientific method which would hold every piece of data subject to a consistent standard of observation and replication’ (pp. 668–669). Similarly, Diekerhof and Bakker’s (2012) study of Dutch journalists concludes that ‘reporters often withdraw from checking sources’ (p. 7) and they ‘… “outsource” the judgment of credibility to other parties, even to those who may have a stake in the issue’.
These conclusions apparently lend support to the idea that journalists are generally not in the business of obtaining knowledge, as least if it is understood as true beliefs supported by evidence. However, a careful reading reveals back-handed acknowledgements that such far-reaching conclusions are unwarranted. For example, Lewis et al. (2008: 28) inexplicably fault for lack of verification a Daily Mail article citing a single expert in diseases of the liver, commenting on the health consequences of the availability of alcohol in the United Kingdom and referring to evidence from international studies (The scholars’ stringent standard of verification is all the more surprising as they do not argue that the expert’s statement or rendering of the evidence lacked merit). Machill and Beiler (2009) note that the low share of cross-checking may be explained by ‘… the lack of any necessity to do so since journalists know the majority of the sources and consider the material provided by news agencies to be trustworthy’ (p. 182). Shapiro et al. (2013) question the very usefulness of the term ‘verification’ – the main focus of their study – as a denotation for a separate stage of fact-checking, pointing out that it is ‘… demonstrably integrated into reporters’ everyday routines and so reliant on the knowledge built out of a reporter’s previous experience …’ (p. 669). Likewise, Diekerhof and Bakker (2012) approvingly cite the view that in journalism ‘sources serve as a form of evidence’ (p. 3), and although they do not clarify exactly what it means, they point out that ‘[J]ournalists do not think it is necessary to verify a source if a colleague has already used that source’ (p. 6) and that ‘… journalists send their stories, before publishing, to the experts and ask them to check the facts’ (p.7).
We set aside for now that contrary to Shapiro et al. (2013), scientific verification resembles journalistic verification in that it does not hold ‘every piece of data subject to a consistent standard of observation and replication’ (p. 669), as careful analyses of scientific work and practicing scientists have noted (Collins, 1975; Hardwig, 1991; Sokal, 2008). Instead, we argue that if documented instances of non-verification in journalism involve citation of and consultation with apparently unimpeachable scientific experts, sources who have been previously relied upon by colleagues, sources whom reporters know, as well as reporters’ own knowledge built out of their experience, then the significance of such non-verification is questionable. Even if two documents (or a document and an oral statement, two oral statements, etc) have not been technically juxtaposed, other evidence could have been evaluated by journalists and knowledge could have arisen.
What other kinds of evidence obtainable by journalists may give rise to knowledge? – as noted, in addition to Hume’s favored sense data, or what we might call ‘direct evidence’ (which includes independent juxtaposition of sources), there exists also a form of second-order evidence, which we have dubbed earlier ‘evidence of evidence’ following Goldman. This concept can be broken down further. In order for a journalist to gauge whether an individual believes in and possesses evidence to support his or her claim, any information about that individual’s mental state and about his or her process of arriving at a conclusion is necessary. If an individual is insincere in making his or her claim, it means they do not believe their own claim. Similarly, if an individual has engaged in sloppy reasoning, his or her claim is not supported by argument and evidence. Naturally, the information about mental states and the quality of reasoning of sources is often ambiguous and dependent on the journalist’s familiarity with the person or the subject matter. But such information, however impoverished, can be obtained, and it can sustain journalists’ source-related hypotheses (Flemming, 2014; Stocking and LaMarca, 1991), thus making them falsifiable (as opposed to dogmatic beliefs, see Popper, 1963). We shall refer to information about sources’ mental states as ‘psychological evidence’ and to information about sources’ quality of reasoning as ‘procedural evidence’, denoting the mental tasks involved in arriving at a conclusion. Here are non-exhaustive examples of what could comprise such evidence:
Psychological evidence
Clues regarding the sincerity of a source can be inferred from the latter’s demeanor. Although no general rules or guarantees of accuracy can ever be offered, it is often possible to sense that another person is experiencing discomfort while conveying information (especially if there is prior acquaintance and familiarity), that a source’s speech is evasive, contradictory, or otherwise incoherent, that a witness to an event seems overly eager to convey negative information about so-and-so or that they inappropriately raise their voices or lack confidence. Albeit inherently speculative and potentially misleading (Leslie, 2011), these signs are not useless in evaluating the mental states of others.
Procedural evidence
Signs of the quality of an official’s or expert’s reasoning will optimally precede or follow their respective claims, or at least would be made available to the journalist on request, in the form of evidence and argument (a potential sign of reliability in itself). If made available, such signs would be initially parsed by the journalist to see whether independent sense can be made of them, and if so, the conclusion would either follow from the evidence and argument or it would not (Collins and Evans, 2008). If due to complexity the journalist is unable to evaluate her source’s claim, the source’s record of past accuracy on similar matters can be drawn upon. For example, a reporter interviewing a missile-defense expert about a controversial technical claim has scarce opportunity of independently verifying the source’s say-so, yet she can check whether this source has made similar claims in the past and whether they have been recognized by authorities in the field to be accurate. If such a record is non-existent, the journalist may still gauge the level of self-criticism and reflectivity that her source exhibits.
However, these two classes of indirect evidence neglect one of Reid’s main insights. Specifically, the above classes of evidence ignore the natural propensity of truth-telling which Reid identified, as well as the social conditions which could put a break on this arguably natural propensity. As Reid (2000) points out, It may well be that some temptations to falsehood would be too strong for the natural force of truthfulness unless forces of honour or virtue were bought to its aid. (p. 119)
Such ‘temptations’ are likely products of the social environment. Thus, beyond the evaluations of sources’ sincerity and reasoning discussed above, journalists may also take into account potential political, organizational, or social rewards that may accrue to their sources and the costs that their sources may incur by deciding to deceive or to tell the truth. We call this calculus of social rewards and costs by the name of ‘social evidence’. Here is an illustration of what ‘social evidence’ may include.
Social evidence
Circumstances and conditions within society, such as a source’s social position, political role, or professional bona fides, affecting the probability that in conveying information to a journalist, this source may take the risk of deceiving or feel compelled to tell the truth. Contrary to previous claims (Diekerhof and Bakker, 2012: 6; Gans, 2004: 130), evidence of social position or political role does not guarantee, for example, that high officials will never lie for fear of public discredit (as had happened over and over again, see Schudson, 2009), but it suggests that since ‘even … the greatest liars … speak truth a hundred times for every lie they tell’ (Reid, 2000: 119), they will tend not to lie about readily refutable points. Others may not be motivated to lie at all, and still others may be dedicated liars – all depending on specific circumstances. The point is not that information about sources’ social or political position automatically entails either deception or truthfulness – it does not – but merely that it can be used in a rational evaluation of the likelihood of a claim’s veracity.
Although previous studies have documented lapses in journalistic fact-finding under particular circumstances (most prominently, in the context of foreign policy coverage; Bennett et al., 2008; Hallin, 1989; Lewis et al., 2006), their main objects of inquiry were the factors behind journalists’ failure to acquire factual knowledge. In other words, they did not seek to account for how such knowledge might be acquired. In contrast, random samples of news items or highly variant news items in the above-mentioned studies of journalistic verification did not select for circumstances in which lapses in journalistic fact-finding were expected (and sometimes did quite the opposite by selecting some items involving quality journalism). They therefore constituted a more normal reflection of day-to-day journalism, which appears to successfully accumulate and convey at least some factual knowledge. Nonetheless, these studies have lacked – or so we have argued – in conceptualization of journalistic knowledge and its building-blocks. We have also proposed our own theoretical account for the low rates of verification (in the sense of two sources being juxtaposed), by introducing a previously unarticulated form of evidence (‘evidence of evidence’).
Even so, it is possible that our account is flawed. It is further possible that the traditional theories of journalistic news-gathering, particularly those belonging to the above-mentioned ‘trust-based view’ are correct in proposing that journalistic practices of fact-finding are merely rituals brought about by source characteristics and journalist-source exchanges and negotiations. If this is the case, then journalists are not led to believe anything on the basis of evidence, but rather choose or decide to believe due to specific interactions with sources (including sympathies, negotiations, trust, etc.). Gaye Tuchman (1973) had argued that a reporter ‘can decide’ which of today’s assignments require his presence at hearings, which can be covered by telephone, which can be reconstructed through interviews with key informants … (p. 124)
If so, then a sufficiently large random sample of news items, adequately attuned to the news-gathering process and the various sources, would exhibit a perfect relationship between verification and its absence and the sources’ political and organizational power and roles, credentials, and the nature of reporter–source relations.
Contrariwise, it is possible that cross-verification would be wholly independent of sources’ characteristics and journalists’ relationships with and trust toward them, as the ‘evidentiary view’ may entail.
All of the aforementioned possibilities are embedded in the following research questions:
RQ1. How do the sources’ institutional, political, and professional roles relate to the practice of cross-verification?
RQ2. How does the frequency of contact with sources relate to the practice of cross-verification?
RQ3. How does the perceived credibility of sources relate to journalistic cross-verification?
Methodology
Since no straightforward empirical measures of journalistic knowledge or comprehensive measures of evidence-use in journalism exist within the field journalism studies, such entities can only be detected indirectly. Nonetheless, assuming that journalists indeed seek knowledge and obtain evidence, these steps are likely to have observable consequences. But what kind of consequences? – for one, journalists relying on evidence do not automatically believe information, not even when their sources are highly trusted or are long-time acquaintances. Moreover, if journalists indeed seek factual knowledge or evidence, the social status, political or organizational power, and credentials of various sources should not decisively determine journalists’ level of confidence. These can be viewed as practice-centered consequences, since they can be revealed in detailed reconstructions of journalistic work processes which include some of journalists’ evidence-seeking behavior (i.e. their practical skepticism in the form of cross-verification), sources’ status, power and perceived credibility, as well as their credentials and frequency of contact with journalists.
Thus, our research consisted primarily of a series of face-to-face reconstruction interviews, a method that has demonstrated its viability in exploring different facets of news processes (Anderson, 2009; Brueggemann, 2011; Phillips, 2012; Reich, 2009; Shapiro et al., 2013). For this study, a sample of Israeli news reporters representing political, financial, and domestic affairs news beats from all four media types (television, print, radio, and online news) was interviewed face-to-face, recreating not only the extent and circumstances of cross-verification behind each of their sampled items but also reporters’ entire work process, reaching the level of individual news items and thereby achieving maximal granularity. The interviews were preceded by three consecutive stages:
Random selection of beats and reporters: a full list of reporters and news beats at 10 leading Israeli news organizations, representing all four media, 2 was prepared in 2 months of byline monitoring.
Identification of all published items: the sampling period extended over 4 weeks – long enough to supply a rich mix of stories, but not long enough to tax participants’ memories.
Random sampling of news items: to limit interview duration to 60–75 minutes, the sample included 8–11 items per reporter (the exact number varied according to medium and size of news organization).
Special seating arrangements were used to avoid infringement of source confidentiality. For print, online, and radio items, the reporter (with a pile of sampled stories or printouts) and interviewer (with a pile of questionnaires) sat on opposite sides of a table with a screen placed between them to give the reporter privacy each time he or she was asked to choose another item from the sample pile and detail how it was obtained. Television reporters were given a laptop with video clips of their sampled items and a pair of earphones, enabling them to watch their items in privacy.
To examine the rate of cross-verification and the possible circumstances which may have given rise to it, we analyzed the complex set of journalistic contacts with varying sources in different circumstances as a simple two-phase process. During the first phase of this process, news discovery, the reporter first becomes acquainted with the existence of a potential new story. During the second phase, news gathering, she or he obtains further information. Seeing the process as having these two phases was originally proposed by the media economist, John H. McManus (1994), and later utilized and developed by Reich (2009).
Consistent with our research questions, the following variables were defined:
Cross-verification: the juxtaposition of two news sources (see below) against each other with the express intention of ascertaining the information’s reliability. To minimize the chances for social desirability bias, the use of the term ‘verification’ (and related terms, such as ‘triangulation’, ‘cross-checking’, and the like) was eliminated from our questionnaire. Instead, the phrasing of the question referred to whether another source had been used in order to confirm or refute information provided by an earlier source. Each time a journalist would answer in the affirmative, the interviewer would not automatically mark the answer, but instead asked the reporter whether he or she was certain that cross-verification had actually occurred, in light of the above definition. In a notable portion of cases, this led reporters to retract their positive answers, indicating a reasonable level of sincerity.
News source: every human, documentary, technological or organizational factor that contributed a layer of information to the item, including fragments of information, story leads, and responses.
Source type: a category determined according to each source’s institutional, political, and professional roles, differentiating between senior sources, non-seniors, government, and business sources, public relation (PR) practitioners, and experts (professionals and academics). Reporters were also asked to specify for each source his or her political/institutional affiliation, such as government, politics, and business.
Frequency of contact: ranging between sources encountered for the first time in the respective item and regulars relied upon on a monthly/weekly/daily basis.
Perceived credibility: the proportion of trustworthy sources in the item.
The correlations between the extent of cross-verification and proportions of the reliance on various source types and trustworthy sources were tested for Pearson product-moment correlations coefficient (Pearson’s r). For ‘frequency of contact’ with sources, and its relationship to cross-verification, we used the Mann–Whitney U test.
Findings
The following data on journalistic cross-verification and its relationship to news sources’ social, professional, and political characteristics, perceived credibility, as well as reporter–source relations are the first of their kind in their level of detail. They are also the first quantitative attempt to test scholarly claims about the nature of journalistic fact-finding, over an unprecedentedly large and diverse sample. The data indicate that cross-verification was present in 56 percent of our sample, across print, television, radio, and online news outlets. This finding is close to an earlier finding of verification in nearly half of a random sample of content analyzed news items (Lewis et al., 2008). It appears that random samples of news items reproducibly indicate that instances of journalistic verification are about as likely as an outcome of a coin-flip. In other words, cross-verification is not an overwhelming phenomenon, but it is neither a negligible one.
Given the diversity of our sample, we may also indicate in passing that female reporters were slightly more likely to cross-verify than male reporters (1.62 instances as compared to 1.16 on average, respectively), that financial reporters were less likely to cross-verify than national and local reporters (19% of the items as compared to 82%, respectively), that broadcast reporters were more likely to cross-verify than print and online media (53% of news items, as compared to 46%, respectively), that only 19 percent of the news items containing information from other news organizations were cross-verified, as compared to 81 percent of the items that did not contain such information, and that only a slight difference was found between reporters who worked in journalism for 6 years or less and those who have worked for 19 years or longer (22% of the items as compared to 25%, respectively). Surprisingly, no notable differences in the rate of cross-verification were found between popular and quality newspapers, suggesting that the quality of evidentiary news work may have been expressed otherwise.
The decision to engage in practical skepticism undeniably arises in the context of journalists’ interactions with different sources, of different institutional, political, and professional backgrounds, as well as different levels of familiarity with, and trust toward them. But can source characteristics, their perceived credibility, and the frequency of contact with them truly explain why journalists cross-verify (or don’t)? – this question is tackled in the next section, after we detail the findings.
Sources’ characteristics comprise their political, institutional, and professional roles, status, power, and credentials. With regard to RQ1, and to our surprise, both professionals and academics which typically serve as expert sources were not only not treated with automatic deference but were even slightly but significantly met with greater instances of cross-verification (r = 0.08, p < 0.05 for both). Still, reporters decided not to cross-verify many of the leads originating from governmental sources (r = −0.121, p < 0.001), but not those originating from business sources, which were apparently – and surprisingly – unrelated to the decision to cross-verify. Notwithstanding this exception, reporters decided not to cross-verify many of the leads originating from public relations sources (r = −0.125, p < 0.001), which are regarded as notoriously manipulative, and which constitute a service afforded primarily by high-ranking sources. Nonetheless, the picture changes drastically beyond the initial lead – that is, during news-gathering – where neither governmental sources nor public relations sources correlate strongly with the decision to cross-verify.
As far as RQ2 was concerned, frequently contacted sources were less cross-verified (U = 117, 406, p < 0.001; see Table 1). However, fully 12 percent of the items involving sources contacted daily included cross-verification.
Cross-verification and frequency of contact with sources.
Interestingly, when examining RQ3, we discovered that the perceived credibility of sources was unrelated to journalistic cross-verification. This finding seems to contradict the commonsensical understanding that the level of trust in sources affects the decision to cross-verify them.
Discussion
Practical skepticism – or cross-verification – is not the only path to knowledge that is available to journalists, but merely one constituent of knowledge. This much is clear from our data, unless one assumes that in nearly half of their news assignments, journalists mindlessly believed baseless information. 3 While it is undeniable that some source characteristics and the nature of journalist-source interactions are related to journalists’ decision to juxtapose two sources in order to assess the reliability of information, we find no support for the idea that this relationship is inevitable (thus belying the ‘trust-based view’, but cf., Ericson, 1998; Tuchman, 1972). However, the relationships that have been discovered are consistent with our theoretical proposition suggesting that second-order evidence (‘evidence of evidence’) complements and often displaces cross-verification. Is such an inference justified?
Consider the following empirically grounded argument in support of this inference. The most striking, albeit negative, finding in favor of our theory had been the lack of any relationship between perceived credibility of news sources and journalists’ decision to cross-verify. This finding is highly counterintuitive. While it would seem that trustworthy sources require no further cross-verification, a large portion of highly trusted sources were nonetheless cross-verified (see also Reich, 2011a). But why would this happen if journalists operated solely on the basis of trust without seeking evidential support? – the simplest explanation is that this was not the case. The likelier scenario, then, is that in many cases, journalists earnestly sought to gain knowledge and evidence regarding what had actually transpired.
Yet, some readers might rightly object that according to our own findings, cross-verification was far from systematic, as some government sources, some public relations sources, and some frequently contacted sources were often met with lower levels of cross-verification. Indeed, the findings indicate that journalists cannot be characterized as blanket cross-verifiers or as scrupulous adherents of a ‘discipline of verification’ (Kovach and Rosenstiel, 2007: 71; thus belying the ‘evidentiary view’ of news-gathering). However, it would be incorrect to view these cases as clear-cut instances of credulity on the journalists’ part, for several reasons. For one, while it is true that in their capacity as providers of initial leads, the above-mentioned sources were met with significantly less cross-verification, they were much more often met with cross-verification during the news-gathering phase of reporting. Similarly, a notable portion of sources contacted daily did not escape cross-verification.
More importantly, all of these sources could plausibly be parsed for indirect clues of veracity comprising the ‘evidence of evidence’: high-ranking, government, and public relations sources typically face journalists with a baggage of relatively discernable interests which may tempt them to deceive or deter them from it (‘social evidence’), while frequently contacted sources’ sincerity could be assessed on the basis of prior acquaintance with their demeanor (‘psychological evidence’).
Another one of our findings provides clear indication of journalists making use of ‘procedural evidence’. The documented tendency to avoid blind deference to experts and even a modest but statistically significant tendency to cross-verify the information they conveyed suggests that some expert sources’ argumentation or evidence seemed problematic to journalists. Correspondingly, our descriptive data about a sizable yet less frequent cross-verification efforts of information from other media suggests that journalists adopt a wait-and-see approach toward colleagues’ reliability, presumably taking into account past accuracy (or once again, ‘procedural evidence’; but cf. Diekerhof and Bakker, 2012). In light of these points, we may posit that cross-verification generally occurs when all other paths to evidence and knowledge have been exhausted.
Although we sought to account for important features of journalism practice, the ‘evidence of evidence’ is not as uncommon or as unique to journalism as it might seem. In everyday life, news audiences also commonly rely on evidence which they themselves do not posses, but which is justifiably believed to be possessed by various sources of information, be it quality media known for their accuracy (Entman, 2005), doctors, scientists, colleagues (Collins and Evans, 2008; Hardwig, 1991), measuring instruments (Goldberg, 2012; Knorr-Cetina, 1999), or even passers-by consulted for directions (Keren, 2012). Although, as noted, the ‘evidence of evidence’ does not guarantee factual knowledge (what evidence does?), large swathes of human and journalistic knowledge are impossible without it. Thus, it is only fitting that we identify indirect signs of veracity as a respectable form of evidence, complementing but sometimes displacing independent verification.
Future studies should strive for more comprehensive empirical indicators of journalistic evidence-use and knowledge, beyond the select practices which achieved a mythical professional status in journalism (cross-verification being one). These could perhaps allow for a more direct observation of ‘evidence of evidence’ at work, as well as its degree of universality across newsrooms and news cultures. Nonetheless, as far as Israeli journalists’ commitment to cross-verification is concerned, we find little reason to doubt the generalizability of the Israeli case. Beyond similar findings elsewhere, the attitude toward cross-verification among the mainstream media in Israel will no doubt ring familiar to students of journalism in other countries. Indeed, the key ethical documents adhered to by the Israeli media (the Nakdi Guidelines, the regulations of the Second Authority for Television and Radio, and the Rules of Professional Ethics of the Israeli Press Council) converge on the necessity of carefully verifying information.
Footnotes
Acknowledgements
The authors thank Inbal Avraham and Yifat Naim for their assistance in data collection, as well as Tali Avishay-Arbel for her statistical advice.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Israel Science Foundation (Grant No. 1104/11).
