The Blue Check of Credibility: Does Account Verification Matter When Evaluating News on Twitter?

Introduction

Social media websites are increasingly spaces for encountering, consuming, and sharing news. ¹ One attribute of these sites is context collapse. ² A diverse array of sources—from political to news and commercial—populate users' social media news feeds. No matter their history, reputation, or content shared, these sources are visually similar. An understudied potential difference between these online sources is the account verification (shown as a check mark) that social media sites, like Twitter, provide to accounts that either apply for verification or are internally granted verification. According to Twitter, “The blue verified badge on Twitter lets people know that an account of public interest is authentic.” ³

In principle, the badge provides a mark of authenticity. It signals that Twitter has confirmed the accounts' legitimacy and, as such, its presence (or absence) should impact how users make credibility judgments. It is this rationale that prompted Twitter to pause its public verification process in late 2017, amid concerns that Twitter users “think of it as credibility, like Twitter stands behind this [account].” ⁴

Our study explores how powerful this relatively subtle cue is for users when they encounter and evaluate partisan news on Twitter. We use an experimental design to address the following questions: to what extent does account verification impact perceptions of credibility? And if so, how do partisan cues and account ambiguity affect this process?

Hypotheses and Research Questions

Credibility is an audience-based concept reflecting the extent to which individuals find a source or message believable, accurate, and trustworthy. ⁵ A long line of research finds that people rely on simple cognitive heuristics, or mental rules-of-thumb, to quickly render credibility judgments. ⁶ Twitter account verification appeals to a number of these heuristics, especially endorsement and expectation violation.

Endorsement offers “conferred credibility,” transferring credibility from another source. Twitter, when granting an account-verified status, provides an endorsement that the account is legitimate and authentic. In contrast, expectation violation refers to basic “surface characteristics of websites and sources” ⁶ that, when absent, can signal that a source or message is not credible. It may be that having a verified Twitter account is expected of accounts that brand themselves as news. Twitter users may look for this norm violation, much like they look for misspellings or the design layout of websites, ⁷ as a quick cue about the source or message credibility. Given this, we expect:

H1: The presence of a verification check will result in higher evaluations of (a) tweet credibility and (b) account credibility.

We also consider whether verification intersects with two additional source-related factors. First, we expect verification to have a differential impact based on level of source ambiguity. Under conditions of low ambiguity—when Twitter users have ample knowledge of an account—verification should have limited influence. The credibility of prominent mainstream news sources, for example, will be uninfluenced by verification because these sources have name recognition and reputations useful in assessing credibility. ⁸ It is under conditions of increased ambiguity that verification should matter. High ambiguity exists when Twitter users encounter an unknown source; while medium ambiguity exists when a source is unknown, but the account name provides cues about the source (i.e., the name “Conservative Tribune” gives users information about the source, even if users are unfamiliar with it).

RQ1: Will a verification check influence (a) tweet credibility and (b) account credibility more for high versus medium ambiguity accounts?

Second, people often judge sources and messages that are congruent with their political ideology to be more credible. ⁶ We explore how this process interacts with verification status. On one hand, conditions of congruency and incongruency may reach a sufficing principle—these are primary and powerful heuristics that allow people to make credibility judgments. ⁹ As such, verification status may not matter. On the other hand, subtle “proximal cues” that accompany news stories (e.g., timestamps) can increase judgments of credibility, but only for low-credibility news sources. ⁸ Extending this logic, a verification check may increase credibility, but only for incongruent sources (typically seen as less credible). Given competing expectations, we ask:

Our final research question tests whether verification depends both on ambiguity and congruence.

Methods

To test these hypotheses, data were collected from 616 participants recruited from Amazon Mechanical Turk using the Turk Prime interface on February 5, 2018. ^* Participants were offered $0.60 for participating in a 6-minute survey. Participants were 36 years old (M = 36.13, SD = 11.42), 50% male, and 80% white, with 49% of the sample having at least a 2-year degree.

After answering a short pretest questionnaire, participants were randomly assigned to 1 of 12 conditions across three experimental factors. Across all conditions, participants viewed a single tweet, with the content held consistent and ideologically neutral, and then answered a series of questions about that specific tweet (Fig. 1). First, we manipulated account verification of the source of the tweet, as indicated by the presence (or absence) of the blue check mark used by Twitter. Second, we manipulated the ideological positioning of the account that posted the tweet such that three sources favored Republicans, whereas three sources favored Democrats. ^† We then matched these sources with participants' party affiliation (N = 319 Democrats, N = 167 Republicans), as reported on a 7-point scale in the pretest questionnaire, to create a measure of congruence (e.g., source matched participants' affiliation) versus incongruence. We classified those who “leaned” toward a party as partisans, and excluded true Independents (N = 130) from our analyses. Third, we manipulated the ambiguity of the account that posted the tweet: low ambiguity, defined as mainstream partisan sources (Fox News, MSNBC), medium ambiguity, defined as lesser-known sources with partisan cues (Conservative Times, Forward Progressives), and high ambiguity, defined as lesser-known sources without partisan cues (Daily Caller, Palmer Report).

OPEN IN VIEWER

FIG 1.

Example of Twitter verification manipulation. (a) Unverified tweet from the Conservative Tribune. (b) Verified tweet from the Conservative Tribune. Note: Both these posts are also classified as medium in account ambiguity and ideologically congruent (among Republican participants) and incongruent (among Democrat participants). All contents of the post, including engagement metrics, were kept constant across all conditions.

Results

We test H1 using an analysis of covariance, which entered verification as the experimental factor while controlling for the six sources examined in this study. Our first hypothesis is unsupported. Verification did not exert a main effect on tweet credibility, F(1,489) = 0.07, p = 0.79, partial η² = 0.000, or account credibility, F(1,489) = 0.05, p = 0.82, partial η² = 0.000.

The next hypothesis and research question test whether the effects of verification are conditioned by characteristics of the source itself—namely, the ambiguity of the account and its congruence. A series of three-way analysis of variance with verification, ambiguity, and congruence as factors confirm that verification still does not have a main effect on tweet or account credibility (Table 1). Next, we test H2, which predicts that verification matters more for high and medium ambiguity accounts, compared with low ambiguity accounts; and RQ1, which compares high and medium ambiguity accounts. We find verification is not conditioned by source ambiguity for either type of credibility. ^§

The final research questions explore whether verification depends on congruence, as well as account ambiguity. We find no support for these expectations; congruence does not moderate the effects of verification (RQ2), nor is there a three-way interaction between these factors (RQ3). ^** Instead, there is a main effect of congruence and an interaction with ambiguity. ^††

Discussion

This study tests the prevailing assumption that Twitter account verification badges influence how people make credibility judgments. Our results consistently show that verification, on its own or combined with other source-related cues, did not influence evaluations of a news tweet or its source. Even when people were unfamiliar with the source (and were provided no partisan cues), the presence of a verification mark did not alter credibility. Instead, account ambiguity and congruency were more powerful cues in assessing credibility.

These findings may underscore the lost value of Twitter verification badges. The number of Twitter accounts granted verification status doubled, from 150,000 to 300,000 accounts, in <3 years. ¹⁰ Such growth parallels Twitter opening (and then closing) the application process to the general public. What was once a useful heuristic for assessing credibility may now be unhelpful due to its prevalence. Of course, this assumes people notice the presence (or absence) of a verification mark. Our study found that only 61% of our participants correctly noticed this cue when it was present, whereas 39% said a source was verified when it was not—although for some participants, source knowledge (especially for low ambiguity sources) may have led to misreporting. ^‡‡

Results from this study suggest innovations to verification appearance is needed. One option is to attempt to restore the value of a verification badge. This could occur by being more selective—something Twitter is currently doing by eliminating bot-verified accounts and accounts that have broken terms of service. But equally important is helping users notice this distinction—for example, by making the verification badge bigger or through public campaigns. However, there is some indication that Twitter plans to provide verification status to every user. ⁴ In this situation, it might be useful to have verification badges that have different colors—perhaps signifying individuals verses companies, or different types of organizations (e.g., nonprofits, government, media, news). By making these account cues more prominent and informative, Twitter can draw boundaries between sources and better appeal to users who process online information heuristically. ⁶

There are several study limitations that should be considered. The use of an MTurk sample limits our ability to generalize the findings to the population at large, although emerging research suggests experiments on MTurk often produce similar results to general population samples. ¹¹ Additionally, more research is needed to determine if our results apply to other social media sites like Facebook. Lastly, we focused on source-related factors and did not manipulate tweet content. An overtly opinionated tweet or one that appeals to certain news values may produce an influence of verification on credibility. Ultimately, there is still much work to be done in this area. While people undoubtedly rely on heuristics in making credibility judgments online, it appears that the verification mark on Twitter is currently not one of those cues.