The oracle or the crowd? Experts versus the stock market in forecasting ceasefire success in the Levant

Introduction

Forecasting is a booming business (Sherden, 1997; Bueno de Mesquita, 2009). It remains, however, both practically and academically unclear who should fulfil the everlasting wish of policymakers and the public to know the future before it has happened. This article contrasts predictions of political violence produced by two important economic sectors – the media and the financial industry. While the former provide their audience with forecasts about political developments through commentaries and editorials, the latter generates predictions through the valuation of politically sensitive assets.

Evidently, the judgements made by journalists and stockbrokers on particular political developments feed into each other. However, the appraisals of the two industries differ in important ways. First, mass media generate direct forecasts through their more or less explicit predictions of how a certain process will evolve. The financial industry, by contrast, only generates indirect predictions through the assessments of how political events affect the returns on particular investments. Second, while the accuracy of a forecast directly impinges on the income and possibly the job security of financial traders, failed predictions will only exceptionally bring havoc to the career of a media commentator. Third, whereas the financial industry acts as an anonymous and often fairly large group, editorial and commentaries are typically written by a specific newsperson or by a small team of journalists.

For the past decades, the social scientific literature has debated the question of whether individual experts (‘oracles’) or groups of laypeople (‘crowd’) provide better forecasts. As two non-fiction bestsellers show, both positions enjoy widespread popularity. Taleb (2007) warned at the outbreak of the Great Recession in a polemic that the financial community had collectively downplayed the risk of financial crises and thus had not taken sufficient precautions against what he calls ‘extremistan’. His rant about the industry’s mistaken adherence to the normal distribution for forecasting crises echoes Le Bon’s (1895) infamous thesis that the masses are foolish and only a few enlightened individuals, if anyone at all, are able to correctly augur a looming catastrophe.

The position that ‘oracles’ predict the future more precisely than groups of highly paid forecasters in the finance industry is in contrast to the ‘crowd’ conjecture, popularized by Surowiecki (2004) and advocated at least since Condorcet’s (de Caritat, 1785) establishment of the Jury Theorem and Galton’s (1907) classic work on belief aggregation. ¹ According to this alternative viewpoint, large groups of laypeople do much better in ascertaining the truth than individuals who possess ample personal knowledge in a particular area. Tetlock’s (2005) path-breaking study on the accuracy of expert forecasts further supported the scepticism towards the usefulness of expert knowledge, although his most recent work and related studies qualify this position (Mandel & Barnes, 2014; Mellers et al., 2014; Tetlock & Gardner, 2015).

We examine these contradictory expectations through an evaluation of the accuracy of newspaper commentaries and financial markets in predicting the ex ante success of ceasefires in the Levant. Sadly, the Middle East is a region which has been plagued by conflict over the past 60 years. Israel has rarely, if at all, experienced a period of peace in its short history. Over 20 ceasefires have been declared since the end of the Cold War alone, either unilaterally or jointly, and all have, ultimately, failed to bring permanent peace to the region. The 24 ceasefires examined here, concluded at various times between Israel and Hezbollah, Hamas, the Palestinian Liberation Organisation (PLO) and several other Palestinian groups between 1993 and 2014, have achieved varying degrees of relative calm.

It is against this background that the current study is conducted. Guided by Fleischer & Buccola (2002), we propose that the Israeli tourism industry listed on the Tel Aviv Stock Exchange (TASE) should be especially sensitive to the success or failure of ceasefire agreements. This theory is supported by Neumayer (2004) who finds strong evidence that human rights violations, conflict and political violence have a negative effect on tourist arrivals. Furthermore, we expect that, while export and import sectors suffer from war and conflict, certain industries, such as the defence sector, may in fact profit from such events (e.g. Schneider & Troeger, 2006; Guidolin & La Ferrara, 2007). We extend these expectations regarding the redistributive effects of violent events to hypothesize that financial traders will carefully evaluate whether a ceasefire will hold, as the success or failure of such an agreement will directly affect the returns on specific investments.

The predictive power of the stock market will be compared to that of newspaper commentators in forecasting the success or failure of ceasefire agreements. We acknowledge that journalists, with their need to dramatize their stories and to please an often biased audience, rather resemble what Tetlock (2005), drawing on Berlin (1957), has called ‘hedgehogs’ – forecasters whose predictive capacities are underdeveloped despite their expertise in the domain for which the predictions were produced. Yet, as both policymakers and the media audience will continue to pay attention to pundits, we believe that juxtaposing the forecasts of the media and the financial industry is an endeavour of considerable theoretical and practical importance. This is particularly the case as other competitive evaluations of forecasts compare different types of experts or intelligence reports (Mandel & Barnes, 2014; Mellers et al., 2014; Tetlock, 2005; Tetlock & Gardner, 2015), rival game-theoretic models (Thomson et al., 2006; Schneider, Finke & Bailer, 2010; Bueno de Mesquita 2011) or varying econometric specifications of the same time series or panel of cases (Brandt, Freeman & Schrodt, 2011, 2014; Rost, Schneider & Kleibl, 2009), but hardly ever contrast predictions obtained through different sources. That is despite Meehl’s (1954) pioneering study comparing clinical and statistical predictions.

To evaluate substantively different ex ante predictions of the same processes, we have conducted content analyses of three quality newspapers (Haaretz, Jerusalem Post, New York Times) and event studies of two assets (Dan Hotels, Imco). The analysis suggests that we can use carefully selected stock market indices to predict political developments that are economically important. While a financial asset from the tourism sector, Dan Hotels, is a better predictor of ceasefire success than media experts at the New York Times, Haaretz, and the Jerusalem Post, an asset from the Israeli defence sector, Imco, largely fails in forecasting the fate of ceasefires. In other words, the evidence thus rather supports the ‘crowd’ over the ‘oracle’ thesis. However, neither source provides truly satisfactory predictions.

Theory and literature

There is a growing literature that advocates the usage of advanced predictive techniques to improve the empirical evaluation of theoretical claims and of the early warning schemes used in foreign policy decisionmaking (Schneider, Gleditsch & Carey, 2011). Yet, the predictive accuracy of a particular methodology, as sophisticated as it might be, further depends on the quality of the information that feeds the forecasts. This caveat is particularly important for judgemental data that are based on the expertise of individuals or groups.

Tetlock (2005, 2010) shows that expert political judgement is not necessarily a reliable method of producing accurate predictions. Forecasters fall easily into the trap of overconfidence whenever they are not accountable for their judgements, if strong prior beliefs guide their assessments and if the time horizon for the predictions extends over several years. ² Newer experimental research in social psychology shows that the identification of good forecasters, the sharing of information and systematic training might resuscitate the role of individual experts (Mellers et al., 2014). Tetlock & Gardner (2015: 74) argue that the main advantages of individuals with a high accuracy in predicting events – ‘foxes’ or ‘superforecasters’ to use their terminology – is their ability to aggregate information across different sources, making them comparable to Galton’s crowd: ‘They integrate perspectives and the information contained within them. The only difference is that the process occurs within one skull.’ Mandel & Barnes (2014: 10987) conclude in an evaluation of strategic intelligence reports that ‘tempered optimism’ is warranted. Accordingly, well-calibrated expert predictions ‘explained 76% of the variance in geopolitical outcomes’. As we do not have access to such files, we examine the forecasts of media commentators on the anticipated success of truces in the Levant.

The new sociopsychological evidence suggests that journalists or experts, interviewed by them, will almost by necessity not belong to the super-forecasters. As Tetlock & Gardner (2015: 72) maintain, ‘editors, producers, and the public […] go looking for hedgehogs, who just happen to be bad forecasters’. ³ The pressure to come up with a dramatic assessment or an opinion that lives up to the prejudices of the audience of a particular media outlet risks transforming these commentators into oracles. Note that our evaluation is confined to the subgroup of experts who act in public and, for better or for worse, shape public opinion with regards to the success or failure of ceasefire agreements as well as other important political developments. This explicit focus on journalistic opinions is the main difference between this article and others which analyze news through word counts and other techniques to predict the risk of armed conflict (e.g. Chadefaux, 2014).

Hypothesis 1: The more optimistic the reaction of media experts to the announcement of a ceasefire is, the more successful the ceasefire will be.

The main reason why financial market information should improve forecasts for some political developments relates to the costliness of the decisions that traders make. A broker incorporates relevant economic and political information when deciding upon selling or buying a particular security. If a financial market heavily depends on political decisions, traders build up expectations about the likelihood of certain scenarios in the political realm. Stock markets have systematically responded to escalatory and de-escalatory moves in three conflicts throughout the 1990s (Schneider & Troeger, 2006; see also Brune et al., 2015) and to the deliberations of the European Council on security policymaking (Bechtel & Schneider, 2010). Chadefaux (2017) shows in a sweeping historical examination that markets have indeed the ability to forecast wars, but that they systematically underestimate the carnage. As some economically important events, such as upcoming meetings between heads of states, are easily foreseeable, it would be irrational for traders to refrain from partly anchoring their current buying and selling decisions on them. To put it differently, if traders refrain from building sensible forecasts with regard to important political developments, they not only jeopardize the value of the securities they are trading with, but also possibly harm their personal careers. Obviously, markets can be wrong in their assessments, and, if Paul Samuelson’s famous dictum ‘Wall Street indices predicted nine out of the last five recessions’ (Newsweek, Science and Stocks, 19 September 1966) holds, they might have a tendency to overpredict.

Asset prices reflect the collective wisdom that the market holds on how certain political events affect the returns on a particular investment. Media commentaries, by contrast, are individual assessments, although the commentators are undoubtedly also concerned with the extent to which a certain opinion deviates from the beliefs and prejudices of the media audience. In the following, we assess whether stock market data can be used to forecast the success of ceasefires in the Levant. This conflict, which has been dominated by the confrontation between Israel and the Palestinians throughout the past six decades, certainly belongs to the category of disputes that have general economic consequences (Schneider & Troeger, 2006) and also affect individual sectors (e.g. Fleischer & Buccola, 2002). As developments within this conflict region are well reported, several researchers have already attempted to forecast escalations for the various dyads (e.g. Brandt, Freeman & Schrodt, 2011, 2014). This article moves beyond these event data-based time-series models in order to produce predictions by including information from a quasi-expert collective – the stock market.

To the authors’ knowledge, only a few studies have tried to use financial market information to predict violent conflicts. Bueno de Mesquita (1990) descriptively uses the money market discount rate to evaluate the expectations contemporary agents had over the Seven Weeks’ War between Austria and Prussia and the crucial battle at Königsgrätz. Moreover, Schneider (2014) uses aggregated stock exchange information to predict ex post the ups and downs of the conflict between Israel and the Palestinians.

The theoretical background which enables us to consider the stock markets as collective ex ante forecasters is the efficient market hypothesis (EMH) according to which ‘a market in which prices always “fully reflect” available information is called “efficient”’ (Fama, 1970: 383). In other words, the EMH claims that the price of an asset reflects all relevant information at a given point in time. It is important to note that not all market participants need to be well informed for the EMH to work. Indeed, it is sufficient that only a few traders are well informed, assuming that the errors made by non-informed traders cancel each other out. ⁴

One can also expect the stock market to react to the announcement or the anticipation of a ceasefire, and to develop its own expectations as to the probability of its success. However, the effect of a successful ceasefire agreement is not homogenous across sectors. Firstly, empirical evidence supported by sound theory would suggest that the tourism industry is negatively affected by conflict. Fleischer & Buccola (2002) find that foreign tourist demand for Israeli hotel accommodation is moderately sensitive to terrorist attacks and that local tourism provides little buffer for reductions in foreign tourist arrivals. In a cross-country study of tourist arrivals between 1977 and 2000, Neumayer (2004) demonstrates that human rights violations and conflict had a negative effect on the tourist industry.

Hypothesis 2a: The stronger the rally of tourism stocks in anticipation of the announcement of a ceasefire is, the more successful the ceasefire will be.

Secondly, the literature is also united in its belief that the defence and security sector should benefit from increased tensions, therefore suffering under peace. Berrebi & Klor (2005) establish that terrorist events in Israel have a significant and positive effect on Israeli defence as well as on security stocks traded in US markets. In addition, Camyar & Ulupinar (2015: 641) demonstrate that war ‘exercises a positive, but surprisingly weak, distributive effect on the US arms sector’.

Hypothesis 2b: The stronger the fall in defence stock prices in anticipation of the announcement of a ceasefire is, the more successful the ceasefire will be.

Finally, if one takes the EMH a step further and notes that the stock market, being information efficient, should take into account all relevant information, that is, including the information conveyed by the media, then it is logical to expect that the stock market should provide a better forecast of the likelihood of the success of a truce than experts do. This logic is also supported by the empirical evidence of Nofer & Hinz (2014) in the analysis of European stock prediction communities versus expert assessments.

Hypothesis 3: The stock market is a better predictor of ceasefire success than are media experts.

Methodology

The Levant offers a unique opportunity to study the effect of conflict on the stock market. The Tel Aviv Stock Exchange (TASE), founded in 1953, is a mature and highly capitalized market that is likely to be sensitive to changes in the Arab–Israeli conflict. Therefore, we will consider the body of ceasefires occurring between Israel and any other party in the period 1993–2014. The term ‘ceasefire’ is defined here as the explicit agreement to cease violent attacks against the other party. There are certainly cases where the agreement is implicit or de facto; however, these are excluded from this study as they lack a date of announcement leading to intangible stock-market and press reactions.

The list of ceasefires was built using the CAMEO dataset for the Levant from the Penn State Event Data Project (PSEDP). After removing all events in which Israel was not involved or that occurred before 1993, the events coded as ceasefires were selected. This results in a body of 324 events, which we then went through one by one to verify whether each was an actual ceasefire or incorrectly coded. ⁵ The fact that the actual body of ceasefires was reduced to 24 during this process justifies our high level of caution when using machine-coded data. ⁶

In the case of the Levant, the operationalization of the dependent variable, Ceasefire success, is even more problematic than it might be elsewhere. This is due to the special nature of the Arab–Israeli conflict. Ceasefires that have occurred in the past two decades between Israel and a second party tend to be less durable and far less strictly observed than in other regions. For example, while most conceptualize a ceasefire as a total halt to all violent events between the two parties, most of the 24 ceasefires covered a period in which violence still continued at least sporadically. Due to this phenomenon it becomes increasingly difficult to conclude when the truce ended and when full-scale hostilities were renewed. For this reason, we employed the number of violent events occurring within seven and 14 days, respectively, from the official start of the truce in order to measure the success of a ceasefire. ⁷

The data for the dependent variable were gathered from the same source used to define the body of cases, that is, the CAMEO dataset for the Levant region of the Penn State Event Data Project (PSEDP). ⁸ As we are working with overdispersed count data, a negative binomial model will be employed.

The predictive power of the stock market

In order to measure the predictive power of the stock market, the event study method was utilized. Event studies examine the returns to financial assets, that is, the changes in price of a particular asset between one period and the next. Such studies attempt to measure investor reactions to specific incidences which are of particular interest to the researcher. Chen & Siems (2004), for example, apply the technique to assess the effects of terrorism on global capital markets.

In order to perform an event study, the time frame of analysis is divided into two periods, the estimation and event windows. Financial data from the estimation window are used to assess the ‘normal’ returns of the asset of interest. These normal returns are then compared to the returns occurring in the event window, in which the incident of interest occurs. Deviations from the normal returns, predicted according to the estimation window, are called ‘abnormal returns’ and can be interpreted as the result of the event that is, in the case of this study, the announcement of a ceasefire. The event window will include the day of the announcement, plus or minus a few days. We will apply an event window of [–3, 0]. ¹⁴ The estimation window chosen for the purposes of this study was from 50 days before the event until ten days before the event [–50, –10]. Typical event studies utilize continuous periods for an estimation window often of up to 100 weeks before the occurrence of an event (Guidolin & La Ferrara, 2007). Our approach allows for a satisfactorily large estimation window of 40 days, without leading to any difficulties with significantly overlapping estimation and event windows.

To measure normal performance we use the market model which assumes a linear relationship between the return of a given security and the return of the market portfolio in which the proportion invested in each security corresponds to its relative market value.

Formally expressed, for any asset i, R_it and R_mt are the returns on the asset during period t and the market portfolio respectively:

R i t = α i + β i R m t + ε i t

1

where ε_it is the disturbance term with a mean of zero and α _i and β_i are parameters. By applying the statistical model, one can calculate the market model parameters for the estimation window employing an ordinary least squares regression. Using the market parameters generated in this stage and the returns on the market portfolio for the event window, we are able to predict the expected normal returns for the security of interest during the event window.

The next step in the event study process is to measure the abnormal returns to the security of interest over the event window. Once again, this is done by using the market model parameter estimators. We can measure abnormal returns to security i for event time τ, AR_iτ, within the market model as:

A R i τ = R i τ − α i − β i R m τ

2

where α _i and β_i are the previously estimated market model parameters for security i, and R_mτ are the returns to the market portfolio over the event window τ. Finally, the abnormal returns are aggregated over the event window in order to accommodate the multiday event windows. These cumulative abnormal returns (CARs) then function as the independent variable in the second stage of the analysis and are used to predict the success of the ceasefire agreement.

In order to employ the statistical model, it is necessary to select a stock that should be strongly affected by the announcement of a ceasefire. Furthermore, a market portfolio, which is typically a broad-based stock index, is also necessary for the application of the market model. Tourism securities were chosen predicated on the assumption that as they mainly provide services in Israel, they should be more sensitive to developments in local conflict. Moreover, defence stocks were selected as their products were mostly designed for military use. We report the two results obtained for the one asset in each sector which provided the strongest results. ¹⁵ The market portfolio, used to control for market fluctuations, is the TA-100 which includes a broad portfolio of securities from a wide range of industries.

All financial data have been retrieved through Thomson Datastream. On a number of occasions, a ceasefire is announced on Friday or Saturday, when the TASE is closed. Past studies (e.g. Daly & Paytakhti Oskooe, 2015) have tried to circumvent this problem of events on non-trading days by interpolating returns in order to fill in missing values. However, this procedure is unlikely to adequately substitute for missing return observations (Bechtel & Schneider, 2010). For this reason, the dates of ceasefire announcements on Friday or Saturday were artificially transferred to Sunday, when investors are first given the opportunity to trade on the basis of the latest ceasefire announcement.

Results

It is important to note that there are certain ceasefire agreements on which the newspapers did not report at all. This is the case for Haaretz in six of the 24 ceasefires and in three ceasefire events for the Jerusalem Post. A long history of military censorship and possibly self-censorship, even among leading Israeli media might, at least partly, explain this divergence (e.g. Rinnawi, 2007). Table I lists the 24 ceasefires to be analysed in the present study.

OPEN IN VIEWER

Table I.

Ceasefires and the predictions of the different ‘forecasters’

		Number of violent events	Success/ failure	Predictions
	Date			‘Crowd’		‘Oracles’
				Dan Hotels	IMCO	Haaretz	JP	NYT
1	31 July 1993	9	S	S	F	S	S	S
2	05 March 1996	12	S	F	F	S	S	S
3	26 April 1996	7	S	F	F	S	F	F
4	30 September 2000	69	F	F	F	–	–	F
5	08 October 2000	51	F	F	F	–	F	S
6	17 October 2000	52	F	F	F	F	F	F
7	02 November 2000	53	F	S	S	F	F	F
8	23 May 2001	22	F	F	S	F	F	F
9	13 June 2001	15	F	S	F	F	F	F
10	18 September 2001	13	S	S	F	S	F	S
11	16 December 2001	9	S	F	F	–	F	S
12	21 February 2003	9	S	S	F	–	S	F
13	29 June 2003	5	S	S	S	F	S	S
14	18 January 2005	6	S	S	S	–	–	F
15	07 February 2005	3	S	S	S	S	S	S
16	25 September 2005	11	S	S	S	F	–	F
17	30 July 2006	52	F	F	F	–	F	S
18	12 August 2006	29	F	F	F	S	F	F
19	25 November 2006	7	S	F	S	F	F	F
20	17 June 2008	6	S	F	F	S	S	S
21	16 January 2009	12	S	F	S	S	F	F
22	13 March 2012	6	S	S	F	F	F	F
23	21 November 2012	10	S	F	S	F	F	F
24	26 August 2014	4	S	S	S	S	F	F
	Predicted successes (S), failures (F)			11S, 13F	10S,14F	9S, 9F	6S,15F	9S,15F
	Correctly predicted successes			9/16	8/16	8/16	6/16	7/16
	Correctly predicted failures			6/8	6/8	4/8	7/8	6/8

Success is defined as the occurrence of up to 14 violent events within 14 days following the ceasefire declaration. Predicted success is defined as CAR > 0 for ‘crowd’ sources, and journalists’ evaluative scores > 0 for ‘oracle’ sources.

The list of ceasefires reflects only a partial overlap between the non-reported ceasefires for the two Israeli newspapers. This stresses that military censorship possibly accounts for the partial silence of the media. The Jerusalem Post, the most conservative daily, is the most pessimistic media outlet with a prediction of six successes and 15 failures, while the NYT anticipates 15 failures and nine successes.

Table II reports the tests of the first two hypotheses. If Hypothesis 1 finds support within the data, there should be a negative and significant relationship between the cumulative evaluative score (CES) and the number of violent events occurring in the 14-day period following a ceasefire for the three newspapers. ¹⁷ The event study analysis provides a test of the second hypothesis with its two subhypotheses; namely that (a) cumulative abnormal returns (CARs) of tourism stocks should negatively correlate with violations of a ceasefire; and (b) defence CARs should positively correlate. We report here the results for a threshold level of 14 violent events during two weeks after the truce was announced, and for the two assets under examination, with an event window spanning over three days. Alternative specifications with event windows covering five days for the two assets, a time horizon of seven days and a threshold of ten events are reported in the Online appendix.

OPEN IN VIEWER

Table II.

The effects of ceasefire-induced cumulative abnormal returns and newspaper ceasefire assessments on the number of subsequent violent events (time frame 14 days; 14 violent events)

	Crowd I	Crowd II	Oracle I	Oracle II	Oracle III
Dan Hotels	0.21** (0.01)
Imco Industries		0.01 (0.02)
Haaretz CES			–0.12 (0.08)
Jerusalem Post CES				–0.332* (0.12)
New York Times CES					–0.005 (0.05)
Constant	3.01** (0.18)	2.98** (0.18)	2.70** (0.18)	2.53** (0.19)	2.92** (0.18)
N	24	24	18	21	24
Log (pseudo) likelihood	–94.43	–95.23	–65.44	–78.34	–94.92
Alpha	0.67 (0.19)	0.70 (0.20)	0.51 (0.18)	0.48 (0.16)	0.68 (0.20)

Standard errors are given in parentheses. The larger standard error, be it robust or conventional is reported. In the negative binomial models this is generally the conventional standard error. †p≤0.1; p*≤0.05; **p≤0.01 using a one-tailed test.

What immediately become apparent in Table II are the Alpha values, which are significantly different from zero, indicating that the negative binomial model is superior to the Poisson model. The coefficients and the related incidence rate ratios are rather small. This means that both the stockbrokers and the journalists rather under- than overestimate the possible violence following a truce. This pattern is in line with the historical evidence assembled by Chadefaux (2017) that financial markets have over almost two centuries underestimated the level of violence. The opinions of Jerusalem Post commentators correlate negatively and significantly with the number of violent events which occur in the following two weeks, while the impact of the equivalent assessments made by Haaretz and NYT journalists are not significant.

The results nevertheless support the first theoretical expectation, demonstrating an increase in the chance that the ceasefire will be successful the more optimistic the reaction of media commentators to the announcement of a ceasefire becomes. Furthermore, the CARs of Dan Hotels are statistically significant and in the expected (negative) direction. This provides moderate support for Hypothesis 2(a) which states that the stronger the rally of tourism stocks in anticipation of the announcement of a ceasefire is, the more likely the ceasefire is to be a success. The correct, that is, negative, signs displayed by the tourism coefficients support the concept that tourism securities should react negatively to anticipated ceasefire failures, while the influence of the defence stock is not significant. Hence, only the tourist company, Dan Hotels, but not the defence firm, Imco Industries, reacts strongly and in the expected direction to the ceasefire agreements.

Comparing predictive accuracy

Figure 1, which reports the performance metrics accuracy, recall and precision, offers some interesting findings with regard to the ‘oracle versus crowd’ controversy. Note that Accuracy 1 refers to the number of correct predictions out of the 24 ceasefire events, whereas Accuracy 2 refers to the number of correct predictions out of the number of predictions made by the specific predictor. Firstly, it is clear that Dan Hotels is the best predictor of the fate of ceasefires, in terms of Accuracy 1 and Recall. In fact, Dan Hotels predicts the fate of ceasefires with 62% accuracy using the thresholds of 14 and ten violent events. The Recall score, which measures the rate of true positives, is 56% (58%) for the 14 (10)-event threshold which is stronger than those of the competing forecasters. In terms of precision, or rate of false positives, Dan Hotels perform very well with the 14-event threshold, while being somewhat weaker using the ten-event threshold.

OPEN IN VIEWER

Figure 1.

Predictive accuracy of financial market assets and newspaper reports

It is important to notice the accuracy, recall and precision scores produced by the Jerusalem Post. While the accuracy and recall scores are below those of the securities, the precision scores are very high, reaching 100% for the 14-event threshold. The conservative nature of this journal might account for the tendency among its commentators to underpredict successful ceasefires. Out of a total of 21 predictions made using JP commentaries, only six predicted ceasefires to be successful. However, those six were correctly predicted. Haaretz, with only 18 predictions out of the 24 ceasefire events, also scores well in the precision category and offers the most accurate forecasts with regards to the second accuracy criterion. Imco Industries offers good predictions with regard to the accuracy scores, but does not convince with regard to the recall measure. The predictive ability of the New York Times commentators is limited, although its precision score is almost as good as those of the two assets.

O’Brien (2002: 805) sets a 55% rate of accuracy, recall and precision which he describes as ‘minimally acceptable’. All of the performance metrics of Dan Hotels score above this threshold, while Imco Industries predictions manage to do it three times. The New York Times fails three times to meet this minimum threshold, and the two domestic newspapers twice. In his later paper, O’Brien (2010: 95) sets a higher threshold of 67%, which is met by all ‘forecasters’ only once.

From the above analysis it is possible to draw two cautious conclusions. Firstly, Dan Hotels performs better than Imco Industries, the New York Times, Haaretz and the Jerusalem Post, as a predictor of the fate of ceasefires. Secondly, Imco Industries performs at least as well as the newspapers. These conclusions provide conditional support for Hypothesis 3 that movement in the share price of selected securities is a better predictor of ceasefire success than experts at the media outlets examined in this article. This acceptance of the ‘crowd’-hypothesis is, however, dependent on the standard of assessment.

The Brier scores largely confirm these results. For an estimation window of five days, Dan Hotels have a Brier score of 0.218; the corresponding figure for Imco Industries is 0.222. Interestingly, the New York Times forecasts, with a Brier value of 0.216, are slightly more accurate than the predictions derived from the event study analyses, while the two Israeli newspapers reach values of 0.249 (Haaretz) and 0.267 (JP). ¹⁸

Conclusion

Our evaluation of the competing ‘crowd’ versus ‘oracle’ theses about the usefulness of using the news or the financial industry for the prediction of political events has two main implications. First, the statistical results show that, through analysis of stock market reactions, policymakers are better able to judge the likely success of a ceasefire and can in turn create policy using this information. Whether the actions resulting from this insight are that, in light of a promising ceasefire, greater labours are made to rebuild conflict-damaged areas, or that, facing a weaker agreement, greater efforts are made to stabilize the peace, this knowledge can be beneficial. Furthermore, as the results of this article show, one of the most conventional methods of information gathering in policymaking circles, the reading of newspaper commentaries, is often inferior to the insight that can be gained from an analysis of the stock market.

Second, the results of this article have relevance for the academic world. The study offers evidence for the argument that semi-experts – the ‘crowd’ – can make better predictions through their valuation of politically sensitive stocks than single experts or ‘oracles’ can. A promising avenue of future research is the combination of expert- and crowd-knowledge in the spirit of Colaresi & Mahmood’s (2017) supervised machine learning approach. Third, the study contributes to the slowly growing body of political economy literature that utilizes financial markets as a tool for examining the link between security policymaking and the economy (Schneider & Troeger, 2006; Guidolin & La Ferrara, 2007; Bechtel & Schneider, 2010). Fourth, this article compares different types of forecasts and thus moves beyond the extant literature which mainly relies on the same source of information in the model evaluation.

We are aware that other types of experts might have performed better in predicting the accuracy of ceasefires than the media commentators selected for this study. This might be particularly the case for intelligence analysts (Mandel & Barnes, 2014) and the ‘superforecasters’ identified by Mellers et al. (2014) and Tetlock & Gardner (2015). Our evaluation of whether or not different methods of prediction provide a better glimpse of the future is therefore only an initial step for the competitive study of conflict forecasts which rely on judgemental information.