User engagement with online news: Conceptualizing interactivity and exploring the relationship between online news videos and user comments

Abstract

With the emergence and rapid acceptance of online news come new and varied opportunities for user engagement with content, along with alternative metrics for capturing those behaviors. This study focuses on interactive engagement with online news videos. We propose a theoretical framework for conceptualizing user engagement on a continuum from exposure to interactivity. Furthermore, we make a distinction between user–content (e.g. commenting) and user–user (e.g. replying to another user’s comment) modes of interaction. We then explore publicly available measures of these concepts and test a series of hypotheses to explore commenting and conversational behaviors in response to YouTube news videos. We conclude by discussing the theoretical and practical implications for advancing our understanding of user engagement with online news.

Keywords

Digital journalism engagement interactivity news videos online news user comments YouTube

The news media landscape has undergone drastic changes over the past decade, as digital platforms are creating unprecedented competition for all legacy, or traditional, media (i.e. newspapers and magazines, as well as broadcast TV and radio news). All of this is part of a broader trend toward convergence in media industries, defined by features such as cross-platform availability of content, less organizational control over production and distribution, greater user empowerment, lower barriers to entry, and increased participation and interactivity among consumers and producers of content.

In terms of sustainability and audience building, some (Hayes, 2008; Webster, 2011) describe this industry transition as shifting from a “push” to “pull” model of mediated communication, especially for video content. The suggestion is that producers in the past relied on a linear model and would simply push content through a limited number of distribution channels with the expectation that users would pay attention. Now, producers operate in a highly competitive and nonlinear environment, characterized by features such as time-shifting technologies and “on demand” access that enable greater user control over when, where, and how they access media. In this context, news organizations must do more to pull users in, offering something not only more useful, informative, compelling, and gratifying than their competition, but also more engaging.

Interactivity is one of the unique defining features of engagement with online news (Deuze, 2003). MacGregor (2007) argues, “To engage is a new value, but not really a news value. This focus on the quality of interaction heralds change” (p. 293). As such, indicators of interactive engagement are complementing more traditional measures of exposure (circulation, ratings, etc.) in understanding the behavior of news audiences.

The notion of engaging users by encouraging them to interact with both content and other users has had substantial traction in marketing and advertising circles for some time, not least because these activities can now be captured and examined via social media tools (Napoli, 2011; Peterson and Carrabis, 2008). With the challenges facing many news organizations, the ability of journalists to offer engaging content to attract, or “pull,” users would seem to be more important than ever.

This study aims to better understand indicators of engagement with online news, specifically the commenting and conversational practices of users in response to online news videos. In what follows, we clarify the concept of engagement and propose a framework for conceptualizing interactivity and its centrality to the understanding of engagement with online news. Next, we explore various indicators of user behavior to illustrate the usefulness of interactivity as an organizing principle. This involves disentangling popularity and exposure metrics from measures of interactivity, each of which constitutes a differing level of engagement. We draw on publicly available online behavioral data to explore the relationships among these metrics, and we integrate these data with a content analysis of news videos to analyze the relationships between content features and interactivity. We contrast commenting practices between “hard” and “soft” news, “objective” and “biased” news, and professional and citizen journalism. The central goal of this study is to further our understanding of the diverse ways that users experience and interact with the news.

User engagement and interactivity

Engagement is a broad phenomenon that describes all sorts of user attention and involvement with media (see Napoli, 2011: 97–98). At its most basic level, it begins with exposure, but most perceive engagement as constituted of both psychological and behavioral experiences (Brodie et al., 2011; Calder et al., 2009; Malthouse and Calder, 2011; Napoli, 2011; Oh et al., 2010), and it can be a property of the users, the medium, or both (O’Brien and Toms, 2008).

A common thread in the research on engagement is the involvement, real or perceived, of the user in either producing, consuming, or disseminating information (Deuze, 2003; Hargittai and Walejko, 2008; Jenkins, 2006; Napoli, 2011; Oh et al., 2010). This involvement is often expressed as interactivity, which is understood as a fundamental component of the broader phenomenon of engagement. Research on interactivity often highlights the multi-directional flow of information. The interactive nature of online media enables the audience to not only receive information but also disseminate and remake it (Jenkins, 2006; Thurman, 2008). Consistent with this line of thinking, Schultz (1999) defines interactivity as a “chain of interrelated messages” in a two-way or reactive system of communication that allows for input from individuals on all sides of the message.

Recent research has applied a uses and gratifications perspective to understand the drivers of interactivity. Ruggiero (2000) notes the potential for interactive media to fulfill certain psychological needs. Subsequent empirical research finds three primary motivations for interactivity: information seeking, socialization or social interaction, and entertainment. This work further distinguishes motives for user–content and user–user interactivity (discussed below), where the former is driven by the need for information and the latter is motivated by the need for social interaction (Chung and Yoo, 2008; Ko et al., 2005; Yoo, 2011). Additionally, a need for entertainment appears to drive both forms of interactivity (Chung and Yoo, 2008; Yoo, 2011). This theme is also central to the work of Clay Shirky (2010), who argues that the Internet enables new forms of collaboration as it affords people an easy entrée into cultural production, as opposed to mere consumption.

Regarding online journalism specifically, Schmitz Weiss and De Macedo Higgins Joyce (2009) suggest the Internet offers unique opportunities for interactivity because it “allows for a closer relationship with the audience; a shortened social space” between media producer and consumer (p. 593). Related, immediacy is a primary characteristic of interactivity in online news media, one that allows users to publicly register their response to a news article or video (Lange, 2007). Commenting on news stories is a way users can engage with content that they did not have available to them until recently. We argue that these comments, as well as replies from other users to already posted comments, can be seen as representative of high-level interactive engagement between the user and content and between the users themselves, respectively.

We distinguish interactivity with online news videos in two broad ways, building on theoretical work with interactivity in a variety of online contexts (Chung, 2008; Chung and Yoo, 2008; Kiousis, 2002; McMillan, 2002, 2005; Massey and Levy, 1999; Nagar, 2011; Schultz, 2000; Stromer-Galley, 2000; Yoo, 2011). User–content interactivity involves a user interacting with content and producers, such as posting an initial comment to a video thread. This represents a basic form of feedback for the content creator. Alternatively, user–user interactivity consists of interactions among two or more users, such as a user replying to another comment already posted on the video thread by a different user. This back and forth among viewers of the news video is representative of a dialogue, or conversation, between the commenters.

Both forms of interactivity signal a highly engaged user, where the individual is not just exposed to content, but demonstrates evidence of cognitive processing. The practice of commenting highlights an active user that is challenging, supporting, or at the very least reflecting on the news. While it is one thing to simply read or watch a news story, making the decision to publicly contribute your reaction or opinion in response to the story indicates an individual that is more invested, aware, and attentive—in other words, more engaged—with the content.

To sum up, we view the ability to act, interact, and co-create online, as a key characteristic of online media, which distinguishes it from other media platforms. We theorize a continuum of engagement, from exposure to interactivity, where more (quantity) and better (quality) ways to interact with content and with other users indicate deeper engagement. We now turn to a discussion of how these concepts are measured.

Measuring media use: From exposure to engagement

Despite the growing body of work on engagement and interactivity with the news, the vast majority of research on news consumption has traditionally relied on rather simplistic measures of raw exposure, with user behavior captured by gross metrics like ratings, circulation, and, more recently, page views and the like. Extending this line of inquiry, we argue that interactivity is distinct from popularity (i.e. exposure), though it is related. We agree with Beckett (2010) that engagement is a composite of various metrics of exposure and interactive behaviors, but make the case for thinking about engagement as a continuum that ranges from less (popularity or exposure) to more (interactivity) user engagement.

Research on audience behavior has long been interested in examining general exposure to media, both from a practical standpoint for its producers and also for the theoretical implications it brings forth. The delivery of news online contributes to a more refined measurement of usage for two reasons: (a) it allows measurement at the level of an individual story rather than the entire news product (e.g. newspaper, television news broadcast), and (b) it provides more precise ways to measure whether a specific piece of content was consumed. These new measures of usage, which aim to capture popularity in some form, typically consist of raw measures of exposure (e.g. page views, traffic, hits, etc.). For instance, Chatzopoulou et al. (2010) treat the number of views of a YouTube video as a basic index of popularity.

While these measures of exposure are certainly important, they are limited in what they tell us beyond whether or not a user encountered the content. Without some measure of engagement, we cannot assess how users respond to what is in front of them. In the past, researchers relied on surveys, self-reports, or letters to the editor, but the Internet has dramatically broadened the range of behaviors vis-a-vis content: emailing, sharing, “liking,” recommending, social bookmarking, and so on. Indeed, we are increasingly seeing the integration of traditional exposure metrics and new indicators of engagement to provide a more complete picture of user behavior (Napoli, 2011).

We distinguish here between “viewing” and other common popularity indicators, such as “rating,” “ranking,” “favoriting,” and “liking.” These behaviors suggest a different level of user experience, one that involves some action on the part of the user (Deuze, 2003). While viewing is a classic measure of exposure, the other metrics essentially operationalize the concept of recommendation. Moreover, while all forms of online use require some activity, we argue that viewing requires minimal interactivity and does not necessarily signal engagement. Instead, exposure is but one aspect of engagement. Likewise, recommendation metrics capture very basic user activity (i.e. clicking), but it is questionable whether they represent deeper engagement.

Interactive features available online, such as direct feedback, comment sections, and interactions with other users, allow users to take a much more active (and interactive) role in the user experience, and thus they clearly suggest deeper engagement with the content. Like page views, these features are publicly available and can be tracked, making it possible to more accurately measure these types of social behaviors.

Recent scholarly work is beginning to take advantage of publicly available user comments and highlights a growing interest in this phenomenon. Nagar (2011) analyzed user comments on mainstream news sites as an expression of public opinion and forum for the surveillance and exchange of ideas. Larsson (2011) studied commenting as one of several interactive features available on Swedish newspaper websites, finding the option varies across types of users in terms of appreciation and use. For instance, “Prosumers” were active commenters, while “Lurkers” were more likely to read others’ comments without partaking. Hujanen and Pietikäinen (2004) studied commenting as a new mode of participation, although they found that Finnish youth rarely engaged in this practice when visiting news websites. This work exemplifies a new trajectory in research on news consumption that focuses on new ways of understanding user behavior.

As digital platforms afford us greater ability to track behavior in more subtle ways, we can disentangle popularity and interactivity, examine the relationship between them, and better understand these alternative measures of engagement. We proceed to analyze several different indicators of popularity (from viewing to recommendation metrics), along with the two modes of interactivity, user–content and user–user. The analysis will shed light on the relationships among interactivity and popularity metrics, as well as the role that certain content features may play in helping to explain these measures of user behavior. In the next section, we outline three sets of hypotheses. The first deals with the relationship between popularity and interactivity, while the last two make inferences about the role of content in explaining interactivity.

Research hypotheses

The first goal of our study is to examine the two types of interactive engagement (user–content and user–user) in relation to broad patterns of popularity. As both types are increasingly integrated in the study of user behavior, we propose two hypotheses to better understand the relationship between popularity and interactivity. The predicted relationships are based on a common notion that more popular content is also more engaging. Siersdorfer et al. (2010) focused on the comment sections of YouTube videos, but with an emphasis on popularity (i.e. “likes”) of comments, ultimately finding that positive comments are associated with greater popularity. Studying the behavioral characteristics of users on the social networking site Digg, Jamali and Rangwala (2009) also link interactivity with popularity ratings; they found that the time of the comment (i.e. time passed since original post) as well as the actual word length of a post or comment was related to high levels of popularity. Lee et al. (2010) proposed a model to predict the popularity of online content derived from the lifetime of threads as well as the number of comments on such threads, where longer lifetime/more comments leads to greater popularity. Taken together, these studies offer empirical justification consistent with the theoretical link between popularity and interactive engagement. Therefore, we propose the following hypotheses:

H1a: There is a positive relationship between popularity and comments (user–content interaction).

H1b: There is a positive relationship between popularity and conversation (user–user interaction).

A secondary goal of this study is to explore the relationship between interactivity and a variety of content characteristics. In the following sections, we review two primary distinctions made in the research on journalism content, hard/soft news and objective/biased news, and propose additional hypotheses to explore the relationship between interactive engagement and these content features.

News: hard versus soft

In recent studies of news content, the provision and selection of hard versus soft news is an area that is garnering a great deal of research attention (Baum, 2002, 2003, 2004; Boczkowski, 2009; Boczkowski and Peer, 2011; Gans, 1980; Hamilton, 2004; Patterson, 2003; Prior, 2003; Schaudt and Carpenter, 2009; Tewksbury, 2003; Zaller, 2003). For instance, Boczkowski and Peer (2011) found a “choice gap,” where journalists provide more hard news than users seem to desire. Despite this gap, other research suggests that news organizations are still trending toward offering more soft news, overall. Hamilton (2004) argues that profit motivations lead to more soft news across all types of news organizations, where the quest for marginal viewers that prefer soft news provides an economic explanation for the rise in soft news over recent years. As we witness the “softening of news” (Boczkowski and Peer, 2011), many wonder about the social and political implications of less public affairs content. For instance, will this lead to a less politically involved citizenry? What about the impact on informed voters? How does this contribute to or detract from the health of a deliberative democracy?

To date, most research on hard/soft news has explored content (subject and format), production, antecedents, and outcomes. Comparing the frequency of comments and conversation posted to hard and soft news videos would offer an additional contribution to this body of research, specifically as it relates to the generative deliberation prospects of each content type. Based on past evidence of a preference for soft news among audience members, and assuming a positive relationship between popularity and interactivity, we should see a positive relationship between more popular content (i.e. soft news) and both types of interactive engagement. We propose the following hypotheses:

H2a: Soft news videos will exhibit more comments (user–content interaction) than hard news videos.

H2b: Soft news videos will exhibit more conversation (user–user interaction) than hard news videos.

Views: Bias versus objectivity

In addition to exploring hard and soft news content, we also compare interactivity across several indicators of bias and objectivity in news videos. The “objectivity norm” has been a feature of modern journalism since the late 19th and early 20th century (Schudson, 2001). Recently, however, many scholars wonder whether market forces and economic imperatives resulting from an environment of increased competition, niche content, and declining profits will lead to a shift away from objective news. A recent study of online news videos showed that biased content was more popular than objective content (Peer and Ksiazek, 2011). Similar to the previous hypotheses, this suggests that biased news videos, because they are more popular, should also exhibit a positive relationship with interactive engagement. Thus, we offer the following hypotheses:

H3a: Videos that break from the objectivity norm will exhibit more comments (user–content interaction).

H3b: Videos that break from the objectivity norm will exhibit more conversation (user–user interaction).

Method

In order to test these hypotheses, we collected a sample of YouTube news videos over a 3-month period. The resulting dataset includes measures of popularity and commenting behaviors, as well as hand-coded content characteristics for a subsample of the videos. The following sections describe the sampling and data collection, details of the specific measures used in this study, and the analytical procedures used to test the hypotheses.

Sampling and data collection

The dynamic nature of the Web makes it extremely difficult to select a purely random sample of news videos. To cope with this, we first chose a single site to study, as even a sample of sites would not be random. YouTube seemed a natural choice, as it has long offered various measures of user behavior (Burgess and Green, 2009; Cheng et al., 2008; Jarboe, 2011). It is the most popular online video property in the United States, with over 136 million unique viewers—a figure that is about three times that of the next closest competitor—and about 16.5 billion total streams during May 2012 (NielsenWire, 2012, 22 June). It is also an aggregator of video content, thus offering a site that includes videos from a variety of news outlets throughout the world, both professional and amateur (Pew Research Center, 2012).

Next, we sampled a composite week, a form of cluster sampling that is common in content analysis of television programs. We set our parameters for March–June 2008 and selected a random Sunday, Monday, Tuesday, Wednesday, Thursday, Friday, and Saturday during that time frame. We crawled the pages of the top 100 videos in YouTube’s “News” category, ranked by number of views, for each of the 7 days. These videos include both repurposed news clips (e.g. a 60-Minutes segment uploaded to YouTube) along with videos made exclusively for the web, which are often produced by amateur or citizen journalists. We collected publicly available popularity and interactivity metrics for each video. This provided our initial dataset of 700 news videos to test H1a and H1b.

We also collected a smaller subsample of videos that consisted of the top 40 news videos for 5 days in the March–June 2008 time frame.¹ These 191 videos were hand-coded for a variety of content and production characteristics, and crawled for popularity and interactivity metrics.² These data were used to test H2a, H2b, H3a, and H3b.

Measures

In order to test the first two hypotheses about the relationships between popularity and interactivity, we used five variables measured for all 700 videos. To measure the popularity of each video, we used three indicators that are publicly available on YouTube: Number of Views is a measure of raw exposure to a video (i.e. the number of times a video has been viewed), Number of Favorited captures the number of times a video is marked as a favorite, and Number of Ratings indicates the number of times a video received a rating. To gauge the level of interactivity for each video, we used two variables. Number of Comments represents the total number of comments posted directly to a video (i.e. by entering text in the “Respond to this video” text box) and indicates the degree of user–content interactivity for a given video. To measure user–user interactivity, we captured the number of replies within the set of comments for each video. A reply to an existing comment is registered when a user chooses to click the “Reply” button associated with a particular comment rather than post a comment as described above. Instead of using the absolute number of replies, we created a Ratio of Replies variable, which represents the proportion of replies to total comments for a given video, and thus offers a more conservative measure than the raw number of replies.³ Table 1 provides descriptive statistics for all five variables. While the values presented in the table are real values, all variables were log transformed to address skewed distributions before conducting the analyses.

Table 1.

Descriptive statistics.

	Min	Max	Mean	SD
	Popularity ←→ Comments
	N = 700
Number of Comments (User–content interactivity)	.00	3375.00	78.013	213.61
Ratio of Replies (User–user interactivity)	.00	.84	.19	.18
Number of Views	201.00	772,637.00	12,083.47	37,832.76
Number of Favorited	.00	2209.00	31.32	142.70
Number of Ratings	.00	3035.00	72.44	174.94
	Content ←→ Comments
	N = 191
Number of Comments (User–content interactivity)	.00	3375.00	145.8115	337.17332
Ratio of Replies (User–user interactivity)	.00	.64	.1879	.15327

SD: standard deviation.

Note: fairness, sourcing, and agenda are nominal variables and thus are not included.

To test the remaining hypotheses, we used the same interactive engagement metrics above, first comparing them across hard and soft news videos, then between biased and objective videos. Four journalism graduate students coded the content of the subsample of videos. The students received extensive training and were closely supervised. They followed a strict protocol and achieved an overall percent agreement score of 82% and a K_n score of .72 (see Peer and Ksiazek, 2011).⁴

Videos were classified as hard or soft news by recoding an existing “content topic” variable in our dataset. Hard news consisted of the following topics: politics, US elections, economics and business, foreign affairs, War on Terror, crime, and natural disasters. Soft news included the following topics: sports, arts, entertainment, culture, science, technology, health, medicine, fitness, weather, sex, animals, and weird/humorous. Videos were characterized as biased or objective based on three hand-coded variables: Fairness, Sourcing, and Agenda. Fairness indicates whether the video presents more than one side of the relevant issue, with multiple sides indicating objectivity. Sourcing asks whether the video uses an outside/secondary source for information, balance, or an alternative point of view, where the use of sources is an indicator of objectivity. Agenda asks coders to determine whether the video seems overly opinionated or seems to be promoting an agenda, where the lack of an agenda characterizes more objective videos.⁵

Analysis

The unit of analysis in our study is the video, such that each popularity and interactivity metric as well as all content variables are measured for each individual video. To test H1a and H1b, we analyzed the correlations among the popularity and interactivity measures. To test the other hypotheses, we used t-tests to compare the mean scores for both interactivity variables across different types of content. For H2a/H2b, we compared the mean number of comments and ratio of replies between hard and soft news videos. For H3a/H3b, we did the same between objective and biased videos, using fairness, sourcing, and agenda as our indicators of the relative objectivity of a video.

Results

The first two hypotheses predict a positive relationship between popularity and commenting (H1a), as well as conversation (H1b). Hypothesis 1a is supported (see Table 2). There are moderately strong correlations between the number of comments posted to a video and all three popularity variables (Number of Views, Number of Favorited, and Number of Ratings). Commenting is positively associated with more exposure to a video, greater likelihood of marking a video as a favorite, and choosing to rate a video. Overall, we can say there is a positive relationship between popularity—both in terms of exposure and recommendation—and user–content interaction.

Table 2.

Pearson correlation matrix for popularity, user–content interactivity, and user–user interactivity variables (n = 700 videos).

	Number of Comments	Ratio of Replies	Number of Views	Number of Favorited	Number of Ratings
Number of Comments	–	−.500*	.470*	.511*	.594*
Ratio of Replies		–	−.181*	−.215*	−.207*
Number of Views			–	.522*	.527*
Number of Favorited				–	.663*
Number of Ratings					–

p < .001.

Hypothesis 1b predicts a positive relationship between popularity and user–user interaction, or conversation within a comment thread. This hypothesis is not supported. In fact, Table 2 shows a negative relationship between ratio of replies and all three popularity variables. This suggests that less popular videos—in terms of raw exposure (Number of Views) and being marked favorite or rated—are more likely to generate conversation among users.

The remaining hypotheses predict that soft (H2a/H2b) and biased (H3a/H3b) news videos will exhibit more commenting and conversation. These hypotheses were not supported. Regarding both types of interactivity, no significant differences were found between hard and soft news or biased and objective news (see Table 3). For ease of interpretation, Table 3 lists the objective categories in the left column (multiple sides, sources, no agenda) and the biased ones in the right column. While soft news and biased news have been shown to be more popular than hard and objective news, respectively, there are no significant differences in the likelihood of commenting or conversing across these types of news content.

Table 3.

T-test results comparing user–content and user–user interactivity across hard/soft, objective/biased, and repurposed/non-repurposed videos.^a.

	Hard and soft news (n = 162)
	Hard		Soft
	M	SD	M	SD	t	df
Number of Comments	160.737	370.419	56.286	39.885	−.002	24.538^b
Ratio of Replies	.207	.155	.141	.127	1.850	160
	Fairness (n = 94)
	Multiple sides		One side
	M	SD	M	SD	t	df
Number of Comments	169.821	236.577	184.564	318.563	.924	92
Ratio of Replies	.211	.139	.208	.133	−1.399	92
	Sourcing (n = 156)
	Sources		No sources
	M	SD	M	SD	t	df
Number of Comments	140.590	299.347	171.808	423.037	1.408	154
Ratio of Replies	.194	.151	.219	.161	−.037	154
	Agenda (n = 125)
	No agenda		Has agenda
	M	SD	M	SD	t	df
Number of Comments	153.098	230.296	218.156	507.363	1.261	113.026^b
Ratio of Replies	0.208	0.139	0.196	0.149	−0.657	123
	Repurposed vs non-repurposed (n = 166)
	Repurposed		Non-repurposed
	M	SD	M	SD	t	df
Number of Comments	103.487	214.069	239.327	521.528	3.297*	164
Ratio of Replies	0.193	0.156	0.209	0.155	−0.084	164

SD: standard deviation; df: degrees of freedom (both italic).

Means are based on real values for ease of interpretations, while t statistics are calculated with log-transformed values to normalize skewed distributions.

Equal variances not assumed.

p < .01.

A post hoc analysis of interactivity between repurposed and non-repurposed videos suggests that non-repurposed videos, or those made exclusively for the web, garner higher levels of user–content interactivity, t (164) = 3.297, p < .01, but reveals no significant differences in terms of user–user interactivity. In other words, the distinction between professionally produced content and news videos created by citizen journalists does seem to predict interactive engagement, at least in terms of posting a comment to a video. Users appear more likely to post comments to videos produced by non-professionals than to those videos that originally appeared as segments on a TV news program.

Discussion

Producers and consumers of news exist in an increasingly interconnected world, with interactions among journalists, users, and content occurring at many levels. This study set out to better understand user engagement and interactivity with online news videos. We began by theorizing that the primary ways in which users interact with the news are manifestations of high-level engagement. We then offered a broad picture of the relationship between traditional popularity metrics, long used to gauge exposure and usage, and more recent engagement metrics. We integrated those popularity metrics under the umbrella of engagement and conceptualized engagement on a continuum from exposure to interactivity. In doing so, we proposed a distinction between user–content interactions (e.g. commenting on a news video) and user–user interactions (e.g. replying to another user’s comment). While we found a positive relationship between popularity and posting a comment to a video, we found a negative relationship between popularity and conversation among users. In other words, viewers of online news videos are more likely to engage in user–content interaction for popular videos, but are more likely to engage in user–user interaction with less popular ones.

These findings lend empirical support to our conceptual framework for interactive engagement with online news. Building on recent work that theoretically and empirically conceptualized interactivity in a variety of new media contexts (Chung, 2008; Chung and Yoo, 2008; Kiousis, 2002; McMillan, 2002, 2005; Massey and Levy, 1999; Nagar, 2011; Schultz, 2000; Stromer-Galley, 2000; Yoo, 2011), this study establishes that we can distinguish between user–content and user–user interactivity with the news. The study presents evidence that these behaviors are distinct in that they manifest differently with more or less popular content and depending on the source of the video.

We would encourage researchers to continue to explore the complexities of engagement and interactivity as new contexts emerge. For instance, news websites now regularly display a variety of more granular metrics alongside stories (e.g. Facebook likes, Tweets, and the number of shares and recommendations through a variety of specific social media platforms like Facebook, Google+, LinkedIn, Delicious, Pinterest, and StumbleUpon). These offer opportunities to explore engagement beyond general measures like views, favorites, and ratings that stem from manifestations of preference that are internal to a given news site. It is possible that users interact in different ways with the news when using social sharing and recommendation options that demonstrate their preferences to their social network, external to the site where they first encounter content. For instance, there may be differences in how and why users demonstrate anonymous preference on a given site (e.g. by simply viewing or rating content), as compared to identifiable manifestations of preference that are accessible to one’s social networks beyond that site. Similarly, higher level engagement with news may take place outside the original news site. Many of these social platforms allow users to engage in user–user interactivity vis-a-vis content that is shared (e.g. engaging in a discussion on Facebook in response to a news story that was shared by a friend). It would be interesting to compare user–content and user–user interactivity as they play out on news sites and on other sites, and to further explore news sites that do it well.

We also analyzed user–content and user–user interactivity as they relate to different types of news content. Looking specifically at two themes that have garnered a great deal of research attention, we found no differences in the likelihood of engaging with hard versus soft news or objective versus biased news. Significantly, while there has been a strong interest in studying these content distinctions, and past empirical research has demonstrated significant differences in terms of popularity, this study found that these content distinctions have little relationship with interactive engagement at multiple levels. While the news industry comes under increasing scrutiny for offering softer and more partisan news coverage, these trends may hold less weight when it comes to engaging users.

From both applied and normative perspectives, interactivity and engagement are positive user behaviors that should be encouraged. For news organizations seeking to “pull” users in (Hayes, 2008; Webster, 2011), evidence of more engaged users can signal greater potential for paid subscribers, as well as more lucrative advertising rates. In other words, interactive engagement translates into increases in both revenue streams for news organizations (selling content to users and selling eyeballs to advertisers). Engagement is also positive in a normative sense, where higher levels of interactive engagement with the news may signal more active citizens, as well as the application of “cognitive surplus” to the realm of public affairs (Shirky, 2010), and perhaps the existence of a kind of public discussion that is essential to democracy (Barber, 2004; Herbst, 1998; Kim et al., 1999; Manosevitch and Walker, 2009).

Yet if the content of the news does not predict these behaviors (at least in terms of the hard/soft and objective/biased distinctions), news organizations need to look elsewhere. In our post hoc analysis, we found that non-repurposed videos exhibit more user–content engagement. This suggests that news organizations may need to look to citizen journalism for ways of creating more engaging content. For instance, a recent study found substantial differences in production and content elements when comparing professional and citizen-produced news videos (Peer and Ksiazek, 2011).

Importantly, these findings suggest that the influence of “public measures”—indicators of popularity or preference made publicly available (Espeland and Sauder, 2007; Webster, 2010, 2011)—may be greater than that of the actual content. That is, public measures serve as a heuristic, a signal to the user that a video is worth viewing. It is now common to find lists of “Most viewed” or “Most recommended” stories on a news website, and other equivalent lists are now commonplace across the Internet. Similarly, the popularity indicators used in this study are publicly available to YouTube users. In other words, a user is able to see the popularity of a video prior to viewing, commenting, or engaging in any other behavior relevant to that video. The strong positive relationships between popularity and commenting could indicate the influence of these public measures on the likelihood of engaging in user–content interactivity. At the same time, the negative relationship between popularity and user–user interaction may indicate that negative public measures inspire more conversation among users, or vice versa, which could be due to the perception of polarized tastes. Moreover, the lack of evidence that the content of the videos makes a difference for either type of interactivity would seem to lend further plausibility to these interpretations about the influence of public measures. Future research can address this question by tracking changes in interactivity and popularity over time, making use of the dynamic nature of the Web to explore causality.

While public measures of popularity may influence user interactivity, it is also possible that underlying social and psychological drivers of interactivity may explain the divergent relationships between popularity and the two forms of interactivity. Earlier, we noted that user–content interactivity is motivated by the need for information, while user–user interactivity is driven by the need for social interaction (Chung and Yoo, 2008; Ko et al., 2005; Yoo, 2011). It is quite possible that users post comments to stories (user–content) when they are already primed to be interested in the story (i.e. through the information-seeking function of media use). At the same time, users may reply to other users (user–user) on more specialized, niche topics (i.e. less popular videos) where they may perceive a greater likelihood of connecting with other users on a social level.

Conclusion

This study took a quantitative approach to understanding the relationships between popularity and interactive engagement metrics, as well as between these metrics and different types of news content. While it is common to quantify engagement, future research would also benefit from a qualitative understanding of these interactive behaviors. Analyzing what users are saying in their comments and conversations would contribute to a richer understanding of interactive engagement practices, how these practices relate to news content, and begin to address questions about the nature and quality of comments and conversations. For instance, Houston et al. (2011) found that qualitative dimensions of comments (e.g. partisan slant) can lead to perceived bias in the news content to which those comments are posted.

Future research could also benefit from addressing some limitations of this study. First, we only recorded replies when the user clicked the “reply” button associated with an existing comment. While this was the most efficient way to collect data on 700 videos with thousands of comments and replies in many cases, we do not account for comments that may have signaled their reply status by addressing another user or comment in the text of the comment (e.g. “NewsJunkie, …” or “@NewsJunkie: …”) or in a video reply. Second, while we were able to capture a snapshot of user behavior over a 3-month period, longitudinal research would contribute to a more complex understanding of the relationships among interactivity, popularity, and content features. For example, Lee et al. (2010) found a positive relationship between the lifetime of comments and popularity. It would be useful to expand that research to include interactivity and content. Finally, longitudinal data would also allow future research to disentangle causality—something we could not do with our data.

Another fruitful line of inquiry could investigate the antecedents of interactivity. Building on recent work (Chan and Leung, 2005; Chung, 2008; Chung and Yoo, 2008; Ko et al., 2005; Yoo, 2011), it would be useful to know more about the motivations and predictors of user comments and conversation around news videos and other types of content. Related, understanding the nature of the conversation surrounding news videos would be of both theoretical and practical interest. In a normative sense, we might be interested in the degree to which conversations around news videos are civil or hostile. In a political era that many characterize as polarized, it would be interesting to see whether and how this plays out in online deliberation. In a practical sense, news executives have spent considerable time and effort to manage the interactive capabilities of their digital presence. Many organizations have policies to discourage inter-user hostility among commenters (e.g. required registration, protocol for reporting hostility, encouraging digital media personnel and journalists to moderate). The industry would benefit from research that aims to better understand the conversations users are having in response to the news they present. Finally, future research might explore potential outcomes of interactivity. These could include loyalty to a news brand, learning (Tremayne, 2008), and influence on perceptions about content (Houston et al., 2011).

In the context of an increasingly interconnected world, this study focused on interactive engagement with online news videos. The results show that users engage with content in various ways and at differing levels, ranging from exposure to recommendation to interactivity. News and other online organizations may wish to apply lessons from this research, as they continue to pursue greater user engagement. But even as we advance our understanding of how people engage with the news, many questions still remain about why they choose to do so.

Footnotes

Acknowledgements

We thank four coders—Josephine Lee, Jon Sonnheim, Jessica Bobula, and Daniel Ellman—for their careful work and dedication, Susan Anderson and Beth Bennett for their guidance and experience, and Mike Smith and the Media Management Center at Northwestern University for financial and logistic support.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Notes

Author biographies

Thomas B Ksiazek (PhD, Northwestern University) is an assistant professor in the Department of Communication at Villanova University. His research interests include digital media use, audience behavior, journalism studies, and network analysis.

Limor Peer (PhD, Northwestern University) is Associate Director for Research at the Institution for Social and Policy Studies (ISPS) at Yale University. Her prior position was Research Director at Northwestern University’s Media Management Center and Readership Institute, where she supervised research on media audience, content, and management strategy and taught classes on communication theory, public opinion, media and society, and statistics for journalists.

Kevin Lessard completed his Master of Arts in Communication at Villanova University in 2012, with a certificate in Journalism and Public Relations. His research interests primarily revolve around the intersections of political communication, journalism and mass media studies. He is currently a Media Analyst with Visit Philadelphia™, the official tourism agency for the Philadelphia region.

References

Barber

(2004) Strong Democracy: Participatory Politics for a New Age. Berkeley, CA: University of California Press.

Baum

(2002) Sex, lies, and war: how soft news brings foreign policy to the inattentive public. American Political Science Review 96(1): 91–110.

Baum

(2003) Soft news and political knowledge: evidence of absence or absence of evidence? Political Communication 20(2): 173–190.

Baum

(2004) Circling the wagons: soft news and isolationism in American public opinion. International Studies Quarterly 48(2): 313–338.

Beckett

(2010) Getting beyond just pageviews: Philly.com’s seven-part equation for measuring online engagement. Nieman Reports, 26 October. Available at: http://www.niemanlab.org/2010/10/getting-beyond-just-pageviews-philly-coms-seven-part-equation-for-measuring-online-engagement/ (accessed 27 July 2012).

Boczkowski

(2009) Rethinking hard and soft news production: from common ground to divergent paths. Journal of Communication 59(1): 98–116.

Boczkowski

Peer

(2011) The choice gap: the divergent online news preferences of journalists and consumers. Journal of Communication 61(5): 857–876.

Brodie

Hollebeek

Juric

(2011) Customer engagement: conceptual domain, fundamental propositions, and implications for research. Journal of Service Research 14(3): 252–271.

Burgess

Green

(2009) YouTube: Online Video and Participatory Culture. Cambridge: Polity Press.

10.

Calder

Malthouse

Schaedel

(2009) An experimental study of the relationship between online engagement and advertising effectiveness. Journal of Interactive Marketing 23: 321–331.

11.

Chan

Leung

(2005) Lifestyles, reliance on traditional news media and online news adoption. New Media & Society 7(3): 357–382.

12.

Chatzopoulou

Sheng

Faloutsos

(2010) A first step towards understanding popularity in YouTube. In: Proceedings of INFOCOM IEEE conference on computer communications workshops, San Diego, CA, 15–19 March, pp. 1–6. New York: Institute of Electrical and Electronics Engineers (IEEE).

13.

Cheng

Dale

Liu

(2008) Characteristics and potentials of YouTube: a measurement study. In: Noam

Pupillo

(eds) Peer-to-Peer Video: The Economics, Policy, and Culture of Today’s New Mass Medium. New York: Springer, pp. 205–217.

14.

Chung

(2008) Interactive features of online newspapers: identifying patterns and predicting use of engaged readers. Journal of Computer-Mediated Communication 13(3): 658–679.

15.

Chung

Yoo

(2008) Audience motivations for using interactive features: distinguishing use of different types of interactivity on an online newspaper. Mass Communication & Society 11(4): 375–397.

16.

Deuze

(2003) The web and its journalisms: considering the consequences of different types of newsmedia online. New Media & Society 5(2): 203–230.

17.

Espeland

Sauder

(2007) Rankings and reactivity: how public measures recreate social worlds. American Journal of Sociology 113(1): 1–40.

18.

Gans

(1980) Deciding What’s News: A Study of CBS Evening News, NBC Nightly News, Newsweek, and Time. New York: Random House.

19.

Hamilton

(2004) All the News That’s Fit to Sell. Princeton, NJ: Princeton University Press.

20.

Hargittai

Walejko

(2008) The participation divide: content creation and sharing in the digital age. Information, Communication & Society 11(2): 239–256.

21.

Hayes

(2008) Jump Point: How Network Culture Is Revolutionizing Business. New York: McGraw Hill.

22.

Herbst

(1998) Reading Public Opinion: Political Actors View the Democratic Process. Chicago, IL: The University of Chicago Press.

23.

Houston

Hansen

Nisbett

(2011) Influence of user comments on perceptions of media bias and third-person effect in online news. Electronic News 5(2): 79–92.

24.

Hujanen

Pietikäinen

(2004) Interactive uses of journalism: crossing between technological potential and young people’s news-using practices. New Media & Society 6(3): 383–401.

25.

Jamali

Rangwala

(2009) Digging digg: comment mining, popularity prediction, and social network analysis. In: Proceedings of the international conference on web information systems and mining, Shanghai, China, 7–8 November, pp. 32–38. New York: Institute of Electrical and Electronics Engineers (IEEE).

26.

Jarboe

(2011) YouTube and Video Marketing: An Hour a Day. 2nd ed. Hoboken, NJ: Wiley.

27.

Jenkins

(2006) Convergence Culture: Where Old and New Media Collide. New York: New York University Press.

28.

Kim

Wyatt

Katz

(1999) News, talk, opinion, participation: the part played by conversation in deliberative democracy. Political Communication 16(4): 361–385.

29.

Kiousis

(2002) Interactivity: a concept explication. New Media & Society 4(3): 355–383.

30.

Cho

Roberts

(2005) Internet uses and gratifications: a structural equation model of interactive advertising. Journal of Advertising 34(2): 57–70.

31.

Lange

(2007) Searching for the “you” in “Youtube”: an analysis of online response ability. In: Proceedings of the ethnographic praxis in industry conference, Keystone, CO, 3–6 October, pp. 31–35. Hoboken, NJ: Wiley.

32.

Larsson

(2011) Interactive to me—interactive to you? A study of use and appreciation of interactivity on Swedish newspaper websites. New Media & Society 13(7): 1180–1197.

33.

Lee

Moon

Salamatian

(2010) An approach to model and predict the popularity of online contents with explanatory factors. In: Proceedings of the international conference on web intelligence and intelligent agent technology, Toronto, ON, Canada, 31 August–3 September, pp. 623–630. New York: Institute of Electrical and Electronics Engineers (IEEE).

34.

MacGregor

(2007) Tracking the online audience: metric data start a subtle revolution. Journalism Studies 8(2): 280–298.

35.

McMillan

(2002) Exploring models of interactivity from multiple research traditions: users, documents and systems. In: Lievrouw

Livingstone

(eds) The Handbook of New Media. London: SAGE, pp. 163–182.

36.

McMillan

(2005) The researchers and the concept: moving beyond a blind examination of interactivity. Journal of Interactive Advertising 5(2): 1–4.

37.

Malthouse

Calder

(2011) Engagement and experiences: comment on Brodie, Hollenbeek, Juric, and Ilic (2011). Journal of Service Research 14(3): 277–279.

38.

Manosevitch

Walker

(2009) Reader comments to online opinion journalism: a space of public deliberation. Paper prepared for presentation at the 10th international symposium on online journalism, Austin, TX, 17–18 April. Available at: https://online.journalism.utexas.edu/2009/papers/ManosevitchWalker09.pdf (accessed 2 November 2013).

39.

Massey

Levy

(1999) Interactivity, online journalism, and English-language web newspapers in Asia. Journalism & Mass Communication Quarterly 76(1): 138–151.

40.

Nagar

(2011) The loud public: the case of user comments in online news media. PhD Dissertation, University at Albany, State University of New York, Albany, NY.

41.

Napoli

(2011) Audience Evolution: New Technologies and the Transformation of Media Audiences. New York: Columbia University Press.

42.

NielsenWire (2012) May 2012: top US online video sites. Available at: http://blog.nielsen.com/nielsenwire/online_mobile/may-2012-top-u-s-online-video-sites/ (accessed 24 July 2012).

43.

O’Brien

Toms

(2008) What is user engagement? A conceptual framework for defining user engagement with technology. Journal of the American Society for Information Science and Technology 59(6): 938–955.

44.

Bellur

Sundar

(2010) A conceptual model of user engagement with media. In: 60th annual conference of the International Communication Association, Singapore, 22–26 June.

45.

Patterson

(2003) The search for a standard: markets and media. Political Communication 20(2): 139–143.

46.

Peer

Ksiazek

(2011) YouTube and the challenge to journalism: new standards for news videos online. Journalism Studies 12(1): 45–63.

47.

Peterson

Carrabis

(2008) Measuring the immeasurable: visitor engagement. Report, Web Analytics Demystified, 7 September. Available at: http://www.webanalyticsdemystified.com/downloads/Web_Analytics_Demystified_and_NextStage_Global_-_Measuring_the_Immeasurable_-_Visitor_Engagement.pdf (accessed 5 January 2012).

48.

Pew Research Center (2012) YouTube and news: a new kind of visual news. Report, Pew Research Center, Project for Excellence in Journalism, 16 July. Available at: http://www.journalism.org/2012/07/16/youtube-news/ (accessed 18 July 2012).

49.

Prior

(2003) Any good news in soft news? The impact of soft news preference on political knowledge. Political Communication 20(2): 149–171.

50.

Ruggiero

(2000) Uses and gratifications theory in the 21st century. Mass Communication & Society 3(1): 3–37.

51.

Schaudt

Carpenter

(2009) The news that’s fit to click: an analysis of online news values and preferences present in the most-viewed stories on azcentral.com. Southwestern Mass Communication Journal 24(2): 17–26.

52.

Schmitz Weiss

De Macedo Higgins Joyce

(2009) Compressed dimensions in digital media occupations: journalists in transformation. Journalism: Theory, Practice, Criticism 10(5): 587–603.

53.

Schudson

(2001) The objectivity norm in American journalism. Journalism 2(2): 149–170.

54.

Schultz

(1999) Interactive options in online journalism: a content analysis of 100 US newspapers. Journal of Computer-Mediated Communication 5(1): 0. DOI: 10.1111/j.1083-6101.1999.tb00331.x.

55.

Schultz

(2000) Mass media and the concept of interactivity: an exploratory study of online forums and reader e-mail. Media, Culture & Society 22(2): 205–221.

56.

Shirky

(2010) Cognitive Surplus: Creativity and Generosity in a Connected Age. New York: Penguin Press.

57.

Siersdorfer

Chelaru

Nejdl

(2010) How useful are your comments? Analyzing and predicting YouTube comments and comment ratings. In: Proceedings of 19th international World Wide Web conference, Raleigh, NC, 26–30 April. Available at: http://www.l3s.de/~siersdorfer/sources/2010/wfp0542-siersdorfer.pdf (accessed 27 July 2012).

58.

Stromer-Galley

(2000) Online interaction and why candidates avoid it. Journal of Communication 50(4): 111–132.

59.

Tewksbury

(2003) What do Americans really want to know? Tracking the behavior of news readers on the Internet. Journal of Communication 53(4): 694–710.

60.

Thurman

(2008) Forums for citizen journalists? Adoption of user generated content initiatives by online news media. New Media & Society 10(1): 139–157.

61.

Tremayne

(2008) Manipulating interactivity with thematically hyperlinked news texts: a media learning experiment. New Media & Society 10(5): 703–727.

62.

Webster

(2010) User information regimes: how social media shape patterns of consumption. Northwestern University Law Review 104(2): 593–612.

63.

Webster

(2011) The duality of media: a structurational theory of public attention. Communication Theory 21(1): 43–66.

64.

Yoo

(2011) Modeling audience interactivity as the gratification-seeking process in online newspapers. Communication Theory 21(1): 67–89.

65.

Zaller

(2003) A new standard of news quality: burglar alarms for the monitorial citizen. Political Communication 20(2): 109–130.