Assessing the Relationship between Economic News Coverage and Mass Economic Attitudes

Conceptual Issues

Let us consider the reasons we might expect—and might not expect—economic news coverage to influence consumer sentiment. We begin with the observation that public perceptions of economic performance are clearly related through some causal chain to economic performance. ² When times are good, people are more positive in their assessments than when times are not so good. Evidence of this relationship is abundant. Summarizing the empirical research, De Boef and Kellstedt (2004, 647) write that “we have long known that economic conditions influence consumer sentiment.”

It may be the case that the positive correlation between economic performance and economic assessments reflects a direct causal link, with no media influence. Unlike some social and political phenomena that most people experience exclusively through the media (e.g., foreign conflicts), members of the mass public routinely experience the economy, which may shape their perceptions of economic performance. People get and lose jobs. Their earnings increase and decrease. The prices they pay for goods and services change from month to month. Relatedly, through social interaction, people may learn about the experiences of friends, family, colleagues, neighbors, and others. Collectively, all of these experiences could produce aggregate public opinion about the economy that reflects overall national economic performance. When times are good, more people get and keep jobs, earn more money, buy more goods and services, and so on. These experiences in turn produce more positive assessments of the economy. When times are bad, the reverse takes place. In short, collective economic experiences may directly lead to collective economic opinion. No direct, intervening, or any other effect of the media is necessary. ³ Indeed, it may be that “economic issues are precisely the type least likely to be influenced by mass media, presumably because of the immediacy and accessibility of personal experiences” (Mutz 1992, 484).

Yet the tone of news coverage of the economy may also influence collective economic opinion (Kiewiet 1983; Nadeau et al. 1999; Soroka 2006; Soroka, Stecula, and Wlezien 2015). Content analyses of news coverage (television and newspapers) routinely reveal that the economy receives a substantial amount of coverage, both in absolute terms and relative to other issues (Boydstun 2013; Harrington 1989). News coverage of the economy may provide citizens with context for understanding their own economic experiences and, thus, judging economic performance. And people are likely exposed to at least some of the news coverage of the economy. The exposure might be direct or indirect, and it might be intentional or unintentional. In any case, the sizable amount of news coverage of the economy in combination with many citizens’ regular exposure to it suggest the plausibility of the proposition that mass economic attitudes are influenced by news coverage of the economy.

Importantly, some portion of the economic news coverage that may be influencing collective perceptions of economic performance may be biased, in the sense that it is overly optimistic or pessimistic than economic performance warrants. In other words, a part of news coverage about the economy may be “extra-economic,” meaning simply that it strays from economic realities. There are at least four reasons that a portion of news coverage may be extra-economic in nature. First, journalistic incentives might create a negativity bias in reporting (Soroka 2006). Second, journalists may unintentionally bring their subjective (and inaccurate) perceptions of the economy to bear on their reporting. Third, the information available to news outlets at the time of reporting might not be accurate. For example, during the presidential campaign period between January and October of 1992, the government’s initial reports of economic performance indicated an average monthly increase of twenty-five thousand jobs, a 1.5 percent increase in personal income, a 2.3 percent growth in consumer expenditures, and a GDP growth rate of 2.0 percent. Over time, those estimates have been revised upward considerably and suggest that economic performance was indeed much better. The jobs, income, consumer expenditure, and GDP data now indicate increases of eighty-two thousand jobs, 2.7 percent, 4.8 percent, and 4.4 percent, respectively. In short, the revised estimates show that the economy in 1992 was performing two to four times better than initially reported. If the media faithfully and accurately reported the available economic information at the time, the tone would have been more negative than justified by economic reality, which only became apparent in official government reports afterward. From this perspective, Bush’s complaints about being blamed for a poor economy were legitimate (Hetherington 1996). Fourth and finally, the resource and agenda limitations of the media, along with the complexity of the economy, make it possible that even the most able and well-intentioned members of the media might present a picture of the economy that does not perfectly reflect economic reality.

A related consideration is that, just as economic assessments may be caused by economic performance, so, too, might news coverage of the economy. Amid their myriad goals, members of the media are motivated by accuracy in reporting (Cook 1998; Graber and Dunaway 2014). Existing empirical evidence substantiates the proposition that economic performance and the tone of news coverage move together (e.g., Casey and Owen 2013; Fogarty 2005; Goidel and Langley 1995; Hollanders and Vliegenthart 2011; Nadeau et al. 1999; Soroka 2006, 2012; Soroka, Stecula, and Wlezien 2015; Wu et al. 2002). One would expect a positive correlation (perhaps a very strong one) between the tone of news coverage of the economy and mass economic assessments, even if there is no causal relationship between the two because both are caused, at least in part, by economic reality.

Taking all of these considerations into account, we expect that economic performance directly influences media tone and economic attitudes. We also expect that the part of economic news coverage not driven by economic performance may have a significant influence on how citizens view the economy. In other words, we expect that just as actual economic performance directly influences economic attitudes, so too does media tone have an independent and direct effect.

If public perceptions of the economy are shaped by economic news coverage, either in lieu of or in addition to the influence of economic performance, the implications are significant. In a world in which economic news coverage perfectly reflects economic fundamentals, it does not matter politically (or normatively) whether economic attitudes respond to news coverage. In this case, news coverage provides the same information as the economy itself. This would not deny a causal influence on citizen perceptions; it could be that citizen responses to changing economic performance occur only because citizens hear about them through media coverage. Such a situation would not be of normative concern. Our concerns should begin at the point at which there are deviations between economic performance and economic news coverage. It is important to assess whether this extra-economic aspect of economic news coverage influences citizens’ economic assessments because we want to know whether news outlets have sway over citizens’ perceptions of the world.

These conceptual issues also pose an analytical problem. Since we have good reasons to think that economic performance influences both citizens’ economic perceptions and economic news coverage, and since we also have good reasons to think that economic news coverage might shape economic perceptions, how are we to disentangle empirically the relationships?

Empirical Challenges in Modeling Consumer Sentiment

The conceptual issues highlighted above present a central problem for assessing the relationships among economic performance, economic attitudes, and media tone: whether or not media tone causes economic attitudes, we expect the two variables to be positively correlated. One solution to this problem is to include economic performance measures along with a measure of media tone in models of economic attitudes. The logic behind this approach is that if the public responds to both (or only directly to media), then a relationship between media tone and economic attitudes will be evident even with economic performance in the model. For this approach to work, however, we must do an exhaustive job of capturing “economic performance” by including all relevant economic indicators that are correlated with economic media tone. Otherwise the estimated effects of media tone will be biased.

Although previous research makes important contributions, most studies include a relatively modest set of economic indicators and a limited lag structure. The result may be that media coverage measures serve as proxies for economic performance and pick up its effects in addition to any media effects. For example, Soroka, Stecula, and Wlezien (2015) control for current changes and one lag of the leading economic indicator index in their effort to determine the influence of media sentiment on consumer sentiment. ⁴ MacKuen, Erikson, and Stimson (1992) model sentiment as a function of current values of change in unemployment, inflation, and quarterly annualized growth in the leading economic indicator index, along with survey measures of attention to economic news. Goidel and Langley (1995) include the current unemployment rate, percentage change in the CPI over the last month, percentage change in the unemployment rate over the last twelve months, and percentage change in GDP over the last quarter—all at time t—along with counts of positive and negative news stories. De Boef and Kellstedt (2004) model sentiment as a function of a more extensive set of economic variables, including the lagging economic indicator index, the coincident economic indicator index, inflation, unemployment, and the federal funds rate—all at time t—along with a variety of political variables. Doms and Morin (2004) include the most comprehensive set of economic indicators in their effort to isolate the effect of news stories mentioning recession. They include percentage change in year over year change in the Standard & Poor (S&P), lags of monthly changes in the CPI, lags of the monthly unemployment rate, change in payroll employment, and current change in gas prices in their models of consumer sentiment. Their study is perhaps the strongest test of the influence of media coverage to date. They find that counts of the words “recession” and “economic slowdown” have an additional effect on consumer sentiment, beyond economic performance. As we will show below, even these more extensive sets of economic indicators are not enough to capture the full effect of economic performance. As such, much of the previous research reporting an effect of media on economic attitudes is prone to omitted variable bias and is, therefore, potentially misleading. If stock market performance, for example, is important to consumers’ evaluations of the economy, omitting market prices and their changes from models of consumer sentiment means those effects—because they are covered in the media—will be attributed to the media, rather than economic performance, where they properly belong.

We advance efforts to isolate the effects of economic performance and economic media tone on economic attitudes by fitting models of both media tone and economic attitudes as a function of a large array of economic indicators (including many lags and different period growth rates of each) in an effort to purge both measures of the variance that is explained by economic performance. Then, we examine the relationship between the residuals in these models for evidence of a direct relationship between media tone and perceptions of the economy. We also estimate a single model of economic perceptions including all our economic indicators and media tone. Throughout our analyses, we focus on the period from January of 1980 through December of 2014, using the month as our temporal unit of analysis.

Measures of Economic Performance

Our chief goal is to purge both consumer sentiment and media tone of their respective economic causes. This means we need to be comprehensive in our selection of economic indicators. We include seven sets of measures of economic performance designed to capture the many dimensions of economic performance. The first four are identical to those used in De Boef and Kellstedt (2004): monthly and quarterly growth rates in the consumer price index (all urban consumers, all items, known as CPIAUCSL) as calculated by the Bureau of Labor Statistics (BLS) and downloaded from the database of economic data maintained by the Federal Reserve Bank of St. Louis – the Federal Reserve Economic Data (FRED); monthly and quarterly growth rates in the Conference Board’s Index of Lagging Economic Indicators ⁵ ; monthly and quarterly growth rates in the Conference Board’s Index of Coincident Indicators ⁶ ; and the monthly (civilian) unemployment rate (UNRATE) as calculated by the BLS and downloaded from FRED. ⁷ They capture quantities of central concern to families—prices and the job market—and, via the Conference Board indices, the “cyclical turning points” in the economy as measured by statistics that tap manufacturing activity, the labor market, financial conditions, and incomes. ⁸ And they capture short (monthly and quarterly) rates of change in aggregate economic performance. In addition, we include annualized changes in these same measures and unemployment twelve months prior to capture longer-term trends in economic performance. We also include three other sets of measures that tap economic performance: monthly, quarterly, and annualized growth rates in the number of jobs added (BLS); the monthly, quarterly, and annualized growth rate in real disposable per capita income (A229RX0, chained 2009 dollars, seasonally adjusted annual rate), downloaded from FRED; and monthly, quarterly, and annualized growth rates in the average daily closing stock prices in the S&P composite index, in 2016 dollars as reported by Shiller (2015) and downloaded from his website. ⁹ These measure trends in job growth, average incomes, and financial market performance. In total, we include fifty-seven right-hand-side economic variables, after accounting for the full set of lags and complement of growth rates. ¹⁰ We believe this comprehensive set of measures taps into the full range of economic activity as it may be perceived by consumers making judgments about economic performance.

Measuring the Tone of Economic News Coverage

Scholars have assessed media coverage of the economy using a variety of strategies. The usual approach is to identify stories about the economy, typically in the New York Times, and apply some coding rules or a dictionary to the text to create a measure of the tone of news coverage. Some have counted the frequency of use of the term recession as an indicator of negative tone (Blood and Phillips 1995; Doms and Morin 2004), or broader set of terms (Hopkins and King 2010). Others have applied sentiment dictionaries developed to capture the positive versus negative tone of discussion across any policy issue (and thus not specific to the economy) (Soroka, Stecula, and Wlezien 2015). Still others have generated and applied dictionaries in the specific context of the research question (De Boef and Kellstedt 2004). These different measures have significantly expanded our understanding of the causes and effects of media coverage of the economy.

Here, we borrow Barberá et al.’s (2016) dataset of economic media coverage, which builds on recent innovations in treating “text as data.” Barberá et al. measure the tone of economic news (i.e., positive, negative, or neutral) using supervised machine learning (SML) techniques (Gareth et al. 2013; Grimmer and Stewart 2013; Klebanov, Diermeier, and Beigman 2008; Lowe 2008; Monroe, Colaresi, and Quinn 2008; Monroe and Schrodt 2008; Van Atteveldt, Kleinnijenhuis, and Ruigrok 2008). Briefly, SML involves three steps. First, human coders label the tone of a sample set of texts. Second, the features of the labeled text (words and phrases) are used to predict the tone assigned by the humans in the sample. “In this way the classifier learns the relevant features of the dataset and the weight assigned to each” (Barberá et al. 2016, 10). The results are evaluated using cross-validation, in which the accuracy of predicted tone is compared with (out of sample) subsets of the human-coded data. The results from multiple classification methods are compared before the best classifier is applied to the full set of available texts in the final step. Monthly measures of media tone can then be created by computing the average predicted probability that the tone of an article is positive across all articles in a given month.

Barberá et al. (2016) develop and validate their measure of tone of the U.S. economy as presented in the New York Times from 1948 to 2014. ¹¹ In what follows, we rely on a measure of tone generated by Barberá et al. (2016) using an expanded universe of media comprising the four national newspapers with the highest circulation in the United States: New York Times, Washington Post, Wall Street Journal, and USA Today. Although newspapers, and these newspapers in particular, do not capture the full extent of the media environment, they continue to originate the majority of policy-based content that is then circulated and filtered through the rest of the media system (Althaus, Edy, and Phalen 2001; Golan 2006; Haider-Markel and Cagle 2004; McCombs and Funk 2011).

Briefly, the measure was developed as follows: A sample of stories from each of the four newspapers over the period they are available electronically was selected using an extended keyword search. ¹² A subset of four thousand articles from each newspaper was then randomly selected and human-coded using CrowdFlower. In the next step, the human-coded data were used to train a classifier on each individual newspaper. ¹³ The result is a predicted tone for each article in each newspaper. A monthly measure of tone was calculated for each paper by averaging across the predicted tone of the full set of articles in a given month. In the final step, the results for each newspaper were averaged to create a measure of the weighted average sentiment across the newspapers. The weights were based on the number of relevant articles in each paper in each month. For the analysis presented here, we focus on the time period from January of 1980 through December of 2014.

The measure of media tone is presented in Figure 1. The series has a mean of 27.5 percent positive and a standard deviation just under 4.4 percentage points over this time period. It ranges from approximately 13 percent positive to 37 percent positive, confirming the tendency for the media to focus on the negative (Soroka 2006). The series moves as we would expect given our knowledge of economic history. It is relatively low in the early 1980s and early 1990s, grows more positive over much of the Clinton years, and declines again in the early 2000s before rebounding and then bottoming out in the 2008 recession, after which it climbs slowly upward (but still remains consistently well below the long-run mean) to the end of the series. Notably, the series is fairly choppy. Given that the media is primed to cover new information, this is not surprising, but undoubtedly some of the noise is due to measurement error.

OPEN IN VIEWER

Figure 1.

Tone of economic news coverage by month, 1980–2014.