Measuring Organizational Legitimacy in Social Media: Assessing Citizens’ Judgments With Sentiment Analysis

Abstract

Conventional quantitative methods for the measurement of organizational legitimacy consider mainly three sources that make judgments about organizations visible: news media, accreditation bodies, and surveys. Over the last decade, however, social media have enabled ordinary citizens to bypass the gatekeeping function of these institutional evaluators and autonomously make individual judgments public. This inclusion of voices beyond functional and formally organized stakeholder groups potentially pluralizes the ongoing discussions about organizations. The individual judgments in blogs, tweets, and Facebook posts give indication about the broader fit between an organization’s perceived behavior and heterogeneous social norms and therefore constitute an indicator of organizational legitimacy that can be accessed and measured. We propose the use of social media data and sentiment analysis to study the affect-based responses to organizational actions by citizens. We critically discuss and compare the method with existing quantitative methods for legitimacy measurement and apply them to a recent case in the banking industry. We discuss the value of the method for studying the process of legitimacy construction as the expression and negotiation of normative judgments about organizations by various evaluators.

Keywords

measurement organizational legitimacy sentiment analysis social media Twitter

Organizational legitimacy is generally defined as the social acceptance of organizations and their actions (Suchman, 1995). Based on Suchman (1995), business and society scholars have conceptualized the process of legitimation, or legitimacy construction, as the expression and negotiation of normative judgments by various evaluators in a public discourse (Palazzo & Scherer, 2006; Schultz, Castelló, & Morsing, 2013). Conventional quantitative measurements have taken into account three sources that make these normative judgments visible. With quantitative content analysis, researchers have assessed the evaluation of organizations by news media (e.g., Pollock & Rindova, 2003), a powerful evaluator that influences the public opinion about organizations (Carroll & McCombs, 2003) and gives direct, selective access to individual judgments (Lee & Carroll, 2011); measurements based on accreditation bodies reveal the balanced expert-evaluation by influential evaluators that legitimize organizations through visible linkages (e.g., Baum & Oliver, 1991); surveys give access to the evaluation of organizations by the general public or specific stakeholder groups whose judgments enter the public domain through condensed public rankings and indices (e.g., Fombrun, 2007).

These measures are valuable to assess evaluations about organizations based on the norms, evaluation criteria, and selection processes of respective evaluators. However, they only partly account for the plurality of norms, values, expectations, and concerns of ordinary citizens, actors that have been regarded as increasingly important from a normative perspective that calls for a stricter democratic accountability of corporate behavior (Matten & Crane, 2005). News media content, for example, is shaped by various selection processes (Shoemaker & Reese, 2013) and its analysis, therefore, does not account for opinions, organizations, or events that are not selected (Vergne, 2010). The observation of linkages to accreditation bodies gives, at best, indirect indication about the evaluation by ordinary citizens. Survey-based measures, while giving direct access to citizens’ judgments, are typically only calculated and published once a year (Fombrun, 2007) and have been criticized to cover limited and only predefined organizational aspects, for which respondents might have also insufficient knowledge (Helm, 2007).

In sum, conventional measures of organizational legitimacy capture a mere fraction of the plurality of citizens’ judgments, because the possibilities for citizens to express their opinions through institutional evaluators are limited.

With the rise of social media, however, ordinary citizens have begun to create autonomous public arenas, where organizational activities are continuously discussed and evaluated (Whelan, Moon, & Grant, 2013). While opinion pages of newspapers provide selected citizens a limited forum to express opinions (Lee & Carroll, 2011), social media have given citizens the possibility to bypass this gatekeeping function of news media (Papacharissi, 2009). Although some scholars have questioned the democratic potential of Internet technologies (Hindman, 2009), others conceive the variety of debates in social media as potentially more democratic (Castells, 2007; Whelan et al., 2013).

Recently, researchers have started to grasp organizational legitimacy in social media. For example, Castelló, Etter, and Nielsen (2016) and Colleoni (2013) have analyzed stakeholder tweets about organizations to measure the outcomes of different communication strategies. More generally, scholars have studied how stakeholders raise their voices and express their opinions about organizations online (Etter & Vestergaard, 2015; Hunter, Le Menestrel, & De Bettignies, 2008).¹ We argue that the systematic and direct access to these voices can contribute to the understanding of the construction of legitimacy in a “normative context that becomes transnationalized, fragmented, pluralized, more complex and less understandable” (Palazzo & Scherer, 2006, p. 79). A legitimacy measure based on social media data, then, has the potential to complement extant measures and contribute to a more encompassing understanding of legitimacy based on the judgments by various evaluators.

To measure citizens’ judgments in social media, we suggest the use of sentiment analysis, a method that has recently gained attention by business and society scholars (Castelló et al., 2016), yet still lacks a thorough and critical discussion. This method bases on the techniques of natural language processing, text analysis, and computational linguistics, and measures the affective orientation of sentences toward an object (Pang & Lee, 2008). It, therewith, gives indication about “affect-based responses” through which ordinary citizens bestow legitimacy to organizations (Haack, Pfarrer, & Scherer, 2014, p. 634). The method allows assessing judgments about organizations (a) in large amounts of social media data, (b) to develop time-sensitive measures, and (c) to access judgments about understudied organizational issues by important actors of civil society.

After a discussion of conventional measurements, we will introduce the method of sentiment analysis, and show with an illustrative case study based on Twitter data how it can complement conventional measurements of organizational legitimacy.

Organizational Legitimacy

The Construction of Organizational Legitimacy

Suchman’s (1995, p. 577) notion of moral legitimacy as the “explicit normative evaluation of the organization and its activities” has gained particular attention by business and society scholars, who argue that legitimacy is constructed through the expression and negotiation of normative judgments in a public discourse (Castelló et al., 2016; Palazzo & Scherer, 2006; Schultz et al., 2013). This conceptualization has met particular resonance in business and society research, because it accounts for the pluralization of norms and expectations toward organizations. For some scholars, then, legitimacy emerges from a deliberative discourse among various actors with the active participation of organizations (Palazzo & Scherer, 2006). For others, legitimacy is communicatively constructed, often involving dissent and beyond corporate knowledge and control (Castelló et al., 2016; Schultz et al., 2013). Inherent to both views is that legitimacy assessments are expressed in a discourse of evaluators (Bitektine & Haack, 2015). The evaluators, or the sources of legitimacy, are “the internal and external audiences who observe organizations and make legitimacy assessments” (Ruef & Scott, 1998, p. 880).

There are many evaluators that asses an organization’s conformity to specific standards and norms (Ruef & Scott, 1998). To reach these judgments, evaluators take into consideration various aspects, such as organizational accomplishments, the way organizations operate, the way organizations are structured, and the evaluation of their leaders (Suchman, 1995, pp. 579-582). For example, news media evaluate organizational outcomes and actions, including various dimensions of (ir-)responsible behavior (Lee & Carroll, 2011); accreditation bodies assess organizational compliance with professional norms, such as a bank’s ability to protect depositor savings (Deephouse & Carter, 2005); and survey-based ranking assess the public’s evaluation for various aspects, such as the protection of the environment (Fombrun, 2007). Because different evaluators tend to have varying interests and use diverse criteria and norms in their evaluation, attention to various evaluators is important (Meyer & Scott, 1983).

How Citizens’ Judgments in Social Media Construct Legitimacy

Social media have given ordinary citizens the possibility to express and negotiate judgments about organizations online (Castelló, Morsing, & Schultz, 2013; Whelan et al., 2013). This evaluation of corporations is often expressed in conversations, which, for example, take place under a certain hashtag, a linguistic marker (e.g., #panamapapers) that is used to categorize conversations by topics (Albu & Etter, 2016). Studies have shown that approximately 20% of all social media content is about organizations (Jansen, Zhang, Sobel, & Chowdury, 2009). Furthermore, judgments in social media are impactful because ordinary citizens increasingly use them as information sources to assess and negotiate the appropriateness of organizational actions (Castelló et al., 2013; Whelan et al., 2013). Twitter, for example, is generally regarded as a reliable source of information (Westerman, Spence, & Van Der Heide, 2014), and the evaluation of corporation, their actions, and services are deemed as credible (Sotiriadis & van Zyl, 2013). Because of their impact, credibility, and increasing volume, judgments in social media can be considered as contributing to the co-construction of organizational legitimacy.

In contrast to institutional evaluators, such as news media and accreditation bodies, that typically endorse or question organizations with more or less rational and balanced evaluations (Deephouse & Carter, 2005; Meyer & Scott, 1983), citizens bestow legitimacy to organizations through affect-based responses (Haack et al., 2014). Affect-based responses, such as joy (positive affect) or disappointment (negative affect), are the affective reactions to a perceptional input, such as organizational actions, and form the basis of individual judgment formation (Haidt, 2001). Affect-based responses are verbalized and expressed as evaluative judgments about organizations and their actions. Similar to endorsing articles in news media (Deephouse & Carter, 2005), positive judgments, then, can be considered as legitimizing organizations, while negative judgments can be considered as de-legitimizing organizations. Consequently, we argue that legitimacy is constructed through individuals’ expression of positive and negative judgments in social media.

Although institutional evaluators cover citizens’ judgments to a limited degree, social media facilitate their expression in various ways (Papacharissi, 2014). Different than institutional evaluators that follow professional rules, norms, and conventions (Baum & Oliver, 1991; Shoemaker & Reese, 2013), ordinary citizens are less restricted, which allows them to express their individual feelings and opinions as semi-private utterings, such as outrage and praise, in a personal style of writing (Papacharissi, 2014).

In sum, additional to judgments by institutional evaluators, social media make the judgments of ordinary citizens public that contribute to the construction of organizational legitimacy (for an overview of sources see Table 1).

Table 1.

Overview of Sources of Organizational Legitimacy.

Sources of organizational legitimacy
Source	News media	Accreditation bodies	Surveys	Citizens in social media
Evaluator type and impact	Institutional evaluator with impact on public opinion about organizations (Carroll & McCombs, 2003) and influence on social media discourses (e.g., Etter & Vestergaard, 2015)	Institutional evaluator that gains news media attention; judgments are used by financial analysts, investors, decision makers, and other stakeholders (e.g., Pollock & Rindova, 2003) Standardized and balanced expert	Institutional evaluator that gains news media attention; judgments are used by decision makers, managers, and other stakeholders (Fombrun, 2007)	Individual evaluator that can gain news media attention (e.g., Pfeffer, Zorbach, & Carley, 2014); used mainly by online peers due to high credibility (e.g., Banning & Sweetser, 2007)
Selection of organizations and evaluation criteria/norms of evaluator	Selection of organizations and their actions based on news value (e.g., Galtung & Ruge, 1965); evaluation based on journalistic standards, such as fact-checking; norms, such as objectivity; and processes, such as editorial meetings (e.g., Shoemaker & Reese, 2013)	Evaluation of a selected range of organizations based on predefined criteria (Haack, Pfarrer, & Scherer, 2014), such as a bank’s ability to protect depositor savings (e.g., Deephouse, 1996) or staff qualifications of health care institutions (e.g., Baum & Oliver, 1991)	Standardized (non-)expert evaluation of a selected range of organizations based on predefined criteria, such as positive social impact, governance, and environmental responsibility (e.g., Fombrun, 2007)	Non-standardized selection of organizations based on personal interest and involvement (e.g., boyd, 2010); non-standardized evaluation, based on individual norms and feelings (e.g., Papacharissi, 2014; Haack et al., 2014)
Frequency of publication of judgments	Continuously through various (online) news media outlets	Typically yearly accreditation, licensing, renewal of membership; published through reports and additionally news media	Typically published in yearly rankings/indices, often through news media and magazines (e.g., Fortune’s Most Admired Companies)	Continuously through various social media platforms
Indication for legitimacy development	Short- to long-term	Long-term	Mid- to long-term	Short- to long-term
Indication for citizens’ judgments	Limited by news media agenda (McCombs, 2013); selected, direct expression of citizens’ judgments through letters to the editor (Lee & Carroll, 2011)	Indirectly, at best; ordinary citizens hardly affect judgments of accreditation bodies, but they are influenced by their judgments (Baum & Oliver, 1991)	Limited by predefined evaluation, criteria, and organizations, for which respondents might have insufficient knowledge (Schultz, Mouritsen, & Gabrielsen, 2001)	Judgment expression in any tone or style; biased by various factors, such as self-censorship (Nekmat & Gonzenbach, 2013), exageration (e.g., Marwick, 2005), digital divides (e.g., Wei & Hindman, 2011)

Conventional Quantitative Measurements of Organizational Legitimacy

Applying quantitative methods, researchers have so far mainly considered news media, accreditation bodies, and surveys for the measurement of organizational legitimacy.

News Media Content Analysis

News media are the most extensively explored sources of organizational legitimacy (Bansal & Clelland, 2004; Deephouse, 1996; Vergne, 2010), also called “media legitimacy” (Bitektine, 2011, p. 154). News media (de-)legitimize organizations by making organizational activities visible and evaluating them (Suchman, 1995). Consequently, legitimacy measurements based on news media content measure how organizations are covered and framed. The method assigns meaning to a selection of words to assess the positive, neutral, or negative tone of news media coverage about an organization.² The aim of the analysis is to build a tonality or favorability index, such as the Janis–Fadner imbalance coefficient (Deephouse & Carter, 2005; Janis & Fadner, 1943).

Because news media are a major influencer of the public opinion about organizations (Carroll & McCombs, 2003), it can be argued that news media analysis is a valid method to assess organizational legitimacy. Furthermore, studies have shown that news media influence discourses about organizations on social media, for example, by setting the frames during a crisis (Etter & Vestergaard, 2015; van der Meer & Verhoeven, 2013), and, therefore, indirectly cover citizens’ voices, too. The method can also give selective direct access to citizens’ opinions through the analysis of letters to the editor (Lee & Carroll, 2011). Nevertheless, news media have their own agendas, and content production is influenced by various factors and selection processes (Shoemaker & Reese, 2013). Therefore, news media analysis provides foremost a valuable measure for the evaluation of a powerful institutional evaluator, but gives only limited indication about the judgments of citizens.

Analysis of Organizational Compliance With Accreditation Bodies

Other researchers have examined the role of accreditation bodies which establish and monitor the rules and norms that determine the way organizations in a given domain should perform their activities (Deephouse, 1996; Pfeffer & Salancik, 1978). It has been argued that organizations gain legitimacy, also called “linkage legitimacy” (Bitektine, 2011, p. 156), through visible institutional linkages to these prominent and legitimate actors (Baum & Oliver, 1991).

Linkage measures capture the organizational compliance with regulations and standards by quantifying compliance with rules, standards, and regulations. Frequently used measures include longitudinal data on the number of registered, licensed, rated, or certified organizations (Baum & Oliver, 1991; Deephouse, 1996; Deephouse & Carter, 2005; Pfeffer & Salancik, 1978). Scholars have developed longitudinal indexes related to accreditations or ranking that are repeated over time to analyze the evolution of legitimacy. Organizational linkages are typically coded through either binary variables (Deephouse & Carter, 2005) illustrating the absence or presence of regulator’s actions, accreditation, collaborative relationship or registration of licenses, or ordinal variables that allow to build a ranking system (Ruef & Scott, 1998).

The method is a valuable measure for organizational legitimacy, because accreditation bodies are powerful institutions that judge organizations with a balanced evaluation based on many dimensions and expert knowledge (Baum & Oliver, 1991; Deephouse, 1996). They provide a more or less balanced evaluation, on which other actors, such as investors, journalists, or citizens build their judgment. However, the method gives, at best, indirect indication about the evaluation by ordinary citizens. Furthermore, because organizations are typically accredited once a year, the analysis does not account for the short- to medium-term dynamism of legitimacy formation.

Surveys

A third quantitative method consists of surveys that assess the evaluation of organizations by the general public or particular stakeholder groups (Ponzi, Fombrun, & Gardberg, 2011). Prominent examples of survey-based rankings are Fortune’s Most Admired Corporations (FMAC), the RepTrak (RT), and other polls for decision makers (Qiu & Welch, 2004).³

The merit of this method is that it gives access to the evaluation of organizations by ordinary citizens (e.g., RT) or particular stakeholder groups, such as executives, directors, and analysts (e.g., FMAC). Furthermore, representative samples guarantee that the public is represented in a valid way. Nevertheless, the evaluation is limited to the predefined aspects and items respondents are asked to rate. For example, the FMAC measure has been criticized for being overly focused on financial aspects (Wartick, 2002). Another widely held critique of items and scaling techniques is that surveys directed at members of the general public do not consider that participants might not have distinct knowledge of the criteria for assessing organizations (Rindova, Williamson, Petkova, & Sever, 2005). Schultz, Mouritsen, and Gabrielsen (2001) observed that respondents often use intuition when answering scales. Similarly, in referring to the halo effect (Thorndike, 1920), Helm (2007) argues that respondents judge unknown organizational aspects with aspects familiar to them. Finally, because surveys are costly, they are mostly conducted on yearly basis, which restricts timely monitoring and insights into the short-term development of organizational legitimacy.

In sum, we conclude that conventional legitimacy measures capture the compliance with relatively homogeneous expectations that are based on established standards and predefined criteria, institutionalized selection and evaluation processes, and on particular interests and agendas of respective sources. Although these evaluations might reflect or influence the judgments of ordinary citizens to certain degrees, they only give access to a fraction of the “growing complexity of globalized social networks [that] is accompanied by an internal pluralization of postindustrial societies” (Palazzo & Scherer, 2006, p. 77). In the next paragraphs, we describe how social media make this pluralization visible.

Pluralization of Discourses in Social Media

Potential Democratization of Online Arenas

Because citizens can express their opinions in social media, scholars have heralded these technologies for pluralizing public discourses (Castells, 2007). Although institutional actors, such as news organizations (Hermida, 2012), non-governmental organizations (NGOs; Blumell, 2016), corporations (Etter, 2014), and organized social movements (Etter & Vestergaard, 2015), use social media to disseminate messages, it is particularly ordinary citizens who express themselves online and outnumber institutional actors. Indeed, only 0.05 % of current 300 million Twitter accounts are official verified accounts (businesses, news organizations, celebrities, etc.).

By referring to Crane, McWilliams, Matten, Moon, and Siegel’s (2008, pp. 9-12) notion of “citizenship arenas,” Whelan and colleagues (2013) propose that social media enable the creation of democratic public online arenas. The multi-modal and transnational nature of social media fosters the autonomy of these arenas and alter, if not diminish, control over communications and media (Castells, 2007). In contrast to news media that are typically profit-oriented organizations (McManus, 1994), tend to follow similar topics (Dearing & Rogers, 1996), and report events only if they are newsworthy (Galtung & Ruge, 1965), ordinary citizens have typically no institutionalized pressure to produce content that sells. The publication of content is rather dependent on the personal interest involvement in a topic (boyd, 2010; Papacharissi, 2009). As a result, online arenas are populated by a wide set of social concerns and interests (Castelló et al., 2013). Similarly, while surveys and accreditation bodies evaluate organizations with predefined criteria for predefined issues and organizations, social media allow citizens the expression of judgments about any organization and issue from the top of their head.

Furthermore, ordinary citizens that contribute with content to online platforms are less subject to direct organizational control and influence than traditional news media (Castells, 2007; Etter & Nielsen, 2015). Indeed, corporations influence news media exposure through public relations tactics, such as press releases or personal interactions with journalists (Westphal & Deephouse, 2011). In contrast to journalists, ordinary citizens are less or only indirectly targeted by these public relations tactics and often base evaluations on personal experiences (boyd, 2010). As a consequence, “legitimacy is not only formed in separate spheres of society, within hierarchical orders of stable institutions or powerful elites . . ., but co-constructed by networked publics” (Schultz et al., 2013, p. 685).

Limitations of Pluralization

Despite the heralded potential for increasing participation among previously unengaged citizens, scholars have also argued that social media fall somewhat short of the expectations held by those most optimistic on behalf of their democratic and disruptive potential (Larsson & Moe, 2012). For example, studies have shown that certain user types contribute with larger volume of content to online discourses, and that these heavy users often belong to elite groups with prominent positions in society (Larsson & Moe, 2012). Furthermore, communities or movements give certain opinions more weight in social media and influence the coverage of news media through their mobilizng efforts. The interplay between online movements and news media can lead an echo chamber between social media and traditional media, excluding other voices (Pfeffer, Zorbach, & Carley, 2014). Similarly, social connections tend to base on homophily and, therefore, act as a filter overemphasizing the importance of single topics or opinions (Pariser, 2011). The resulting information echoes create the impression of everybody talking about the same topics or having the same opinions (Sunstein, 2009). This phenomenon is increased by technological filters and algorithms that expose individuals to information based on previous interest and social connections (Pariser, 2011). Furthermore, it has to be acknowledged that, while most citizens have access to social media, not all make use of them to the same degree (Wei & Hindman, 2011). Usage patterns vary depending on age, gender, education, race, and personality (Hargittai & Litt, 2011; Roblyer, McDaniel, Webb, Herman, & Witty, 2010). Nevertheless, despite these digital divides, research also shows that those groups actually expressing political opinions online are diverse (Bakker & de Vreese, 2011).

Finally, utterances in social media are subject to several biases. For example, social media users aim at creating favorable images of themselves (boyd, 2010) and therefore stretch the truth, sometimes to its outer limits (Marwick, 2005). Similarly, citizens often enact self-censorship to accommodate the expectations of an imagined audience (Marwick & boyd, 2011) or when they feel that their opinion is incongruent with the perceived majority opinion of online peers (Nekmat & Gonzenbach, 2013). Furthermore, studies show that organizations increasingly try to influence what employees express in their capacity as citizens online, which is likely to influence what they dare to express in social media (Stohl, Etter, Banghart, & Woo, 2016).

In sum, we conclude that social media have the potential for pluralizing discourses that construct legitimacy. Nevertheless, it has to be acknowledged that not all judgments are necessarily publicly expressed (Bitektine & Haack, 2015; Clemente & Roulet, 2015). Those that are expressed, however, contribute to the formation of organizational legitimacy.

Measuring Organizational Legitimacy With Sentiment Analysis

Social media are transforming individual judgments from something private (or at least concentrated to face-to-face interactions) into something public. The result is a tangible manifestation of judgments in tweets, blog entries, and other forms of texts that can be collected and analyzed.

Collecting Social Media Data and Sampling Techniques

According to our review, social media data have been collected in three ways: through application programming interfaces (API), data crawlers, or paid services (Lomborg & Bechmann, 2014). APIs are software tools that enable the download of information from social media platforms (de Souza & Redmiles, 2009). In addition to text-data that emerge from online interactions and discussions, APIs can give systematic access to author details, time of publication, and geo-location, which can be used to filter the data for certain regions, actors, or time periods. APIs are regulated by the platform owners, who decide what information is accessible.

Although Twitter is widely used by academics not only for its relevance, but also for its generous policy regarding data access, other platforms are less open. For these platforms, researchers can use data crawlers, which collect systematically publicly available information by simulating user behavior. For instance, “Issue crawler” is a crawler developed by the Digital Method Initiative at the University of Amsterdam and collects information about specific issues. The limitation of this approach consists in the need to adapt the crawler for every platform.

Finally, social media data can be collected by paying the service of third-part companies specialized in the systematic data collection.

Regardless of the method, data collection from social media platforms present some challenges to empirical research in terms of sampling (Gillespie, 2010). For example, the interconnectedness of the content and its embeddedness in a network of interrelated discussions and actors challenges the definition of the boundaries of discourses, groups, and often platforms (Jenkins, 2006). This poses the question of delimitation (i.e., the selection of coherent and representative sample for subsequent analysis; Gerlitz & Rieder, 2013). The two most common sampling techniques are topic-based sampling and snowball sampling.

Topic-based sampling techniques extract information containing specific keywords (Bruns & Stieglitz, 2012). In other research fields, topic-based sampling has been used to study the emergence and evolution of topical concerns (Burgess & Bruns, 2011), to explore brand communication (Stieglitz & Krüger, 2011), and to observe homophily (Adamic & Glance, 2005) or communication practices around corporate social responsibility (CSR; Colleoni, 2013). The definition of keywords aims at delineating a discussion around a specific topic or a specific entity. To this end, a list of the representative keywords is defined by the researcher (e.g., “BP oil spill”), usually with the support of domain experts on the basis of their prior knowledge of the topic (Colleoni, 2013). Snowball sampling builds on predefined lists of discussions defined by domain experts, and extend these through “snowballing” or triangulation, by investigating the networks of the initially identified actors (Rogers, 2009). This sampling strategy has been used to sample topic- or activity-based user groups (Paßmann, Boeschoten, & Schäfer, 2013), word of mouth networks (Pedersen, Razmerita, & Colleoni, 2014), or CSR communities (Fieseler, Fleck, & Meckel, 2010), and helps identifying relevant actors and respective content that were not known before. For example, Fieseler and colleagues (2010) have applied this technique to investigate McDonald’s engagement in the CSR blog’s network.

All the described methods are non-probability selection techniques, particularly suited for case-based analysis, which does not guarantee the selection of a representative sample and therefore limits the generalization of the results (Gibbert, Wicki, & Ruigrok, 2008). This does not invalidate the results of the analysis, but it does raise questions about the generality of derived claims in relation to the entire population (boyd & Crawford, 2012).

After the sample has been extracted, the data are transformed into a form that can be automatically analyzed by the algorithm. This pre-processing phase consists of data cleaning, language detection, and text standardization. The data cleaning process is a particular delicate phase, as a consistent quota of the communication in social media is made by fake accounts. For example, Facebook estimated that around 10% of its accounts are fake (Thier, 2012). Based on prior research on spam and fake accounts (Stringhini, Kruegel, & Vigna, 2010), accounts that grow too fast in terms of friend requests and messages sent are eliminated, as well as accounts that are created and closed within a day. The text is then cleaned to extract only relevant text, deleting punctuation, numbers, and stop-words, such as parentheses and conjunctions. Once this first data cleaning is completed, an algorithm for language detection is applied to the text to select only content in one language. Finally, each word is reduced to its word stem, so that words such as computer and computing both become the same term, comput. At the end of this procedure, the data consist of a set of tokens that can be analyzed.

Sentiment Analysis of Social Media Data

Once the sample has been properly transformed, the data are analyzed through sentiment analysis.

Sentiment analysis is part of a computing paradigm that assesses subjectivity (e.g., affect or opinions) in texts (Pang & Lee, 2008). The method usually detects the polarity of sentences, assessing whether individuals are expressing any form of positive or negative sentiment toward an object. A sentiment measure in the form of a single score (e.g., ranging from −100 to +100) or a label (i.e., positive or negative) attached to a word or a set of words represents the aggregated evaluative judgments in a large number of texts for a predefined entity. If a mixed tone is detected, as for instance in the case of two sentences with different sentiment in a single text, the sentiment algorithm computes the final score comparing the amount of negative and positive sentiment. Detection of mixed sentiment, however, is less relevant when dealing with social media data, because it tends to appear in the form of short sentences, such as tweets, which mostly do not include mixed sentiment.

One approach to classify sentiment in texts is lexicon-based. With pre-coded word lists, occurrences of subjective words within texts are computed to detect polarity of a sentence (Rao & Ravichandran, 2009; Thelwall, Buckley, & Paltoglou, 2011). The overall sentiment score represents the sum of all the evaluative expressions of individuals, usually represented as a nominal number normalized by the number of words in a given sentence. The major limitation of lexicon-based approaches is that they rely on pre-coded lists and are unable to detect novel expressions.

A more complex approach is represented by the linguistic analysis which bases inferences on the semantic and syntactic structure of the text to extract its sentiment valence (Cambria, Mazzocco, & Hussain, 2013). In particular, by defining the structure, morphology, and relationship between words, these approaches can identify superlatives, negations, context, and idioms as part of the prediction process (Thelwall et al., 2011) and, therefore, detect subtly expressed sentiments, overcoming several challenges related to the investigation of social media, such as the use of informal language and uncommon abbreviations. By extracting the structure of the discourse, these algorithms try to detect the holder of the sentiment, the target, and the context (Saif, He, & Alani, 2012). When moving from one context to another, opinions and affect can shift from positive to negative, or neutral. Semantics can capture this evolution and differentiate its results accordingly (Gangemi, Presutti, & Reforgiato, 2014). By relying on large semantic knowledge bases, such approaches advance techniques that blindly use pre-coded words and instead rely on the implicit meaning associated with natural language concepts (Cambria, Schuller, Xia, & Havasi, 2013). The major limitation of these approaches is their limited flexibility and, therefore, their reduced results in terms of generalization to other domains due to the strong dependency of the results to the original semantic structure used to build the analytical model (Pang & Lee, 2008).

The most widely used approach for sentiment analysis is machine learning (Agarwal & Bhattacharyya, 2005; Pang, Lee, & Vaithyanathan, 2002; Zhang, Shang, & Jia, 2015), which bases on the automatic discovering of useful information, or novel patterns in large data sets. By using this approach, researchers can create algorithms that, properly trained, can classify unforeseen data and therefore analyze real-time data that are constantly created in social media. Most importantly, by using this approach, the set of words that are evaluated as positive or negative is not confined to the predefined words list but emerges from the ongoing discourses.

To train the algorithm for sentiment analysis, supervised classification is applied (Pang et al., 2002). This requires two data sets: a training set that consists of a corpus of manually labeled text, and a test set in the form of a corpus of unlabeled text, which is used by the researcher to validate the performance of the classification. A supervised learning algorithm, then, results from the training set as the mathematical model that best maps the data. In a first step, the training set is created by categorizing text based on the affective orientation of the sentence. This process is driven by the expertise of the researcher, who identifies those features to be included in the analysis. From this initial set of features, those that best describe and discriminate the different affective judgments are extrapolated. The algorithm provides a percentage of accuracy and the researcher chooses the combination of features with highest accuracy. These features are used to train a classification model which then is able to automatically detect unforeseen data.

The second step consists in the researcher’s assessment of the accuracy of the prediction of the classification model by testing its ability to correctly classify unlabeled data. To do so, cross-validation is applied that involves partitioning the test set into complementary subsets and performing the analysis with the classification model on one subset. The classification model is assessed on a set of quality measures, namely, accuracy, precision, and recall. Through the combination of these measures, the overall ability of the classification model to predict the sentiment of a new corpus of text is assessed. The indexes are calculated by the algorithm, and the researcher chooses the model with the best quality. On average, the performances of sentiment classification of social media data range from 70% to 95% correctly classified entities in terms of f score (Zhang et al., 2015).⁴ Once the text has been classified, the data are graphically represented as trend over time.

Measuring Citizens’ Judgments as a Source of Legitimacy

Through the detection of affective orientations in large amounts of texts, the advanced techniques of sentiment analysis can be considered as a valid method for the measurement of citizens’ judgment in social media data. The method may be recommended for those scholars interested in (a) exploiting the potential of large scale social media data, (b) developing real-time legitimacy indicators, and (c) assessing judgments by new or understudied actors.

Exploiting the Potential of Large Social Media Data Sets

The access to big databases in social media allows analyzing the broadness of (de-)legitimizing issues and conversations about corporations. Online conversations and actors are highly fragmented (Papacharissi, 2009) and, therefore, potentially draw a highly complex picture of the multiplicity of issues (de-)legitimizing an organization. A large data set is, therefore, essential to reliably and directly analyze plural conversations about an organization. This might not have been an issue for those scholars analyzing legitimacy of corporations through news media content analysis, because media agendas are less fragmented than the social media ones (Castelló et al., 2013; Papacharissi, 2009). Here, small-medium databases of media reports cover the issues addressed in news media. Measuring legitimacy through sentiment analysis instead allows researchers to directly assess judgments of ordinary citizens. The big amount of data allows computing queries of databases which extrapolate an overall picture of how ordinary citizens evaluate organizations from the multiplicity of conversations.

Developing Time-Sensitive Legitimacy Indicators

Sentiment analysis may represent a valuable measurement for those researchers who are interested in analyzing (de-)legitimizing content that affects corporations’ legitimacy. Previous studies using evaluator-based measures with means of news media or accreditation bodies assessed legitimacy on a monthly or yearly basis. Consequently, these measures are not suitable to assess legitimacy in a communication environment, where speed and dynamism of communications have increased significantly (Castelló et al., 2013; Castells, 2007). Indeed, yearly measures of legitimacy are helpful in assessing how the dominant voices of institutional actors (de-)legitimize corporations over a longer time period. However, (de-)legitimizing conversations in social media might reach big audiences within a few hours or minutes (Chumley, 2014). Hence, scholars interested in analyzing the evolution of legitimacy in more dynamic situations, such as crises, might use sentiment analysis of social media data.

Assess Judgments of Previously Understudied Issues and Actors

Sentiment analysis of social media data gives access to the judgments of previously understudied issues and actors. Past studies have followed the logic of first identifying the actors (e.g., New York Times) and then analyzing the content (e.g., news media coverage endorsing vs. challenging corporations). Such an approach bases on the existence of few established and identifiable institutional evaluators that for a long time were the only actors that had the resources to publicly evaluate corporations on a regular basis. Today, however, anybody who is able to attract the attention of other users upon any topic can become an important actor in (de-)legitimation processes. Groups that emerge transnationally around an issue raised by a few users may mutate quickly, when new actors join these groups (Chumley, 2014; Hunter et al., 2008). Hence, the assessment of legitimacy in social media requires a measurement that recognizes how evaluations are expressed not only by clearly well-defined actors but also by a spontaneous aggregation of actors that express judgments around a variety of topics and issues. In this regard, sentiment analysis is a powerful method, because it allows for first analyzing the judgments and opinions of actors for a certain entity. In a second step, researchers can identify interest groups propagating such (de-)legitimizing sentiment. This allows scholars to have a flexible measurement of legitimacy that allows grasping the fast evolving nature of social media conversations.

Table 2 displays an overview of discussed measurements of organizational legitimacy.

Table 2.

Quantitative Measurements of Organizational Legitimacy.

Source of legitimacy	News media	Accreditation bodies	Surveys	Citizens in social media
Method to assess judgments	Content analysis	Archivals, longitudinal analysis of secondary data	Questionnaire	Sentiment analysis
What it does	Selects text and assigns meaning to a selection of words to assess favorability, no favorability, or neutrality of media coverage	Assesses presence/absence or different levels of compliance with regulator rules (e.g., membership/accreditation)	Assesses the evaluation of various organizational aspects by respondents	Assesses affective responses toward organizations that are expressed in texts
Sources of data	News media; articles are usually sourced through databases, such as Lexis Nexis or Factiva	Governments, charities, non-governmental organizations, associations	Representative survey panels; members of the general public or specific stakeholder groups	Social media data, such as tweets, blogs, Facebook posts, and so on; sourced through APIs and data crawlers
(Automated) elaboration of data	Coding software for automated coding of text with assistance of dictionaries/word lists, such as DICTION^a (e.g., Hart & Carroll, 2015); Coding software that supports human coding, such as Nvivo or Atlas (e.g., Etter & Vestergaard, 2015); Statistical software for data management and quantitative analysis, such as SPSS, STATA, SAS, or R (e.g., Martin & Boynton, 2005)	Statistical software for data management and statistical analysis, such as RATE (e.g., Singh, Tucker, & House, 1986)	Statistical software for data management and quantitative analysis, such as SPSS, SPSS, STATA, SAS, or R (e.g., MacMillan, Money, Downing, & Hillenbrand, 2005)	Software based on programming languages, such as Python, that are able to process huge amounts of data (e.g., Castelló et al., 2016)
Unit of analysis	News media article (e.g., Deephouse & Carter, 2005)	Reports, rankings, ratings, institutional linkages (e.g., Baum & Oliver, 1991)	Survey items covering an organizational aspect (e.g., Helm, 2007)	Comments, such as tweets, Facebook posts, blog posts, and so on (e.g., Castelló et al., 2016)
Variable types	Binary variable, such as endorsing or questioning (e.g., Deephouse & Carter, 2005); Ordinal variables, such as positive/negative/neutral (Lee & Carroll, 2011); ordinal variables, such as qualification of adjectives, “good environmental actions”, “bad environmental actions” (e.g., Brown & Deegan, 1998)	Binary variable about absence or presence of regulator’s actions, accreditation, collaborative relations or registration of licenses (e.g., Ruef & Scott, 1998); ordinal variable (e.g., Deephouse & Carter, 2005)	Ordinal variable, typically Likert-type scale for various items representing organizational aspects/dimensions (e.g., Fombrun, 2007); from three (Highhouse et al. 2009) to 51 items (Davies, Chun, Da Silva, & Roper, 2003)	Scale variable with the sentiment score that variate from negative to positive (Colleoni, 2013)
Coefficient/index	Favorability index; Tonality index; media endorsement index (Janis–Fadner imbalance coefficient); raw legitimacy vector (Vergne, 2010); typically calculated on annual basis (e.g., Deephouse & Carter, 2005)	Longitudinal index; rankings over times; longitudinal accreditation index; registration/acquisition index (e.g., Deephouse & Carter, 2005; Oliver & Baum, 1991; Pfeffer & Salancik, 1978)	Typically yearly ranking; overall score/index calculated based on evaluation of various items/organizational aspects (e.g., Fombrun, 2007; Helm, 2007)	Sentiment score/index calculated based on the detected sentiment for units of analysis. (e.g., Castelló et al., 2016)
Examples of data sets	News media articles of regional newspapers (e.g., Deephouse & Carter, 2005), national newspapers (e.g., Brown & Deegan, 1998), or mix of national and regional newspapers (e.g., Lee & Carroll, 2011)	Reports by regulators about sanctions against organizations (e.g., Deephouse, 1996); public information about licenses/registers/certifications (e.g., Baum & Oliver, 1991; Pfeffer & Salancik, 1978)	Representative samples of the general public or particular stakeholder groups (e.g., Fombrun, 2007)	Facebook posts from the “wall” of a corporate Facebook site; blog posts and comments about organizations; Tweets about organizations (Castelló et al., 2016; Colleoni, 2013)
Examples of data volumes (samples)	110 coded news media articles (Brown & Deegan, 1998); 460 coded news media articles (Lee & Carroll, 2011); 1,277 coded news media articles (Deephouse & Carter, 2005)	Accreditation of 143 organizations by seven accreditation bodies (Ruef & Scott, 1998); accreditation of 1,028 organizations by one accreditation body (Baum & Oliver, 1991)	Evaluation of global companies by approximately 4,000 executives (Fortune World’s Most Admired Companies); Evaluation of 2,000 organizations across 15 countries by approximately 100 respondents per country (RepTrak)	43,000 tweets (Castelló et al., 2016); 326, 000 tweets (Colleoni, 2013)
Examples of time periods	5 years (Deephouse & Carter, 2005), 10 years (Brown & Deegan, 1998), 25 years (Lee & Carroll, 2011)	5 years (Deephouse, 1996), 17 years (Baum & Oliver, 1991), 46 years (Ruef & Scott, 1998)	5 years (Brammer & Millington, 2005), 15 years (Roberts & Dowling, 2002)	6 months (Colleoni, 2013), 41 months (Castelló et al., 2016)
Examples of research objectives	Effect of news media legitimacy on IPOs (Pollock & Rindova, 2003) or on strategic communication (Brown & Deegan, 1998); effect of downsizing on organizational legitimacy (Lamertz & Baum, 1998)	Effect of organizational legitimacy on survival of organizations (Baum & Oliver, 1991); Effect of isomorphism on organizational legitimacy (Deephouse & Carter, 2005)	Effect of executives’ judgments (FMAC) on financial performance (Roberts & Dowling, 2002); effect of CSR on public evaluation (Brammer & Millington, 2005)	Outcomes of communication strategies on organizational legitimacy (Colleoni, 2013; Castelló et al., 2016)

Note. API = application programming interfaces.

DICTION uses 31 word lists to compute predefined formulas that analyze the text based on five master variables use—that is, activity, optimism, certainty, realism, and commonality (Hart & Carroll, 2015). For instance, optimism is computed by “standardiz[ing] six variables and then add[ing] or subtract[ing] them (e.g., [praise + satisfaction + inspiration] − [blame + hardship + denial])” (Hart & Carroll, 2015, p. 4).

Illustration: Application of Sentiment Analysis

With a case study, we illustrate how citizens’ judgments in social media are measured with sentiment analysis. We then triangulate (Jick, 1979) these results with those from conventional quantitative measures that are applied to the same case and time period. The chosen entity under study is a major Italian bank, ItalyBank (pseudonym) that has recently evoked public criticism and praise for various issues. We analyze the judgments by 5,991citizens in 14,179 tweets. In addition, we analyze 722 newspaper articles about ItalyBank, monitor its evaluation in a national survey, and observe its institutional linkage to a credit rating agency.

Method

Sentiment analysis of tweets

To assess organizational legitimacy as the normative judgment by citizens in Twitter, we first collected tweets through Twitter’s search API for a time period of 1 year (May 1, 2013 to April 3, 2014). The keyword search resulted in totally 36,092 tweets in Italian language mentioning the word “ItalyBank.” To analyze ongoing conversations that construct organizational legitimacy (Schultz et al., 2013), we selected those tweets that included a hashtag (e.g., #italybank or #statehelp). Furthermore, we excluded all tweets that were produced by ItalyBank’s 12 Twitter accounts (totally 423 tweets). This selection process resulted in a total of 14,179 tweets by 5,991 authors for 389 different hashtags, whereby each hashtag typically refers to a specific topic (see Appendix Figure A1 and Table A1 for examples).

To identify positive and negative judgments that construct organizational legitimacy (Haack et al., 2014), the sentiment of each tweet was detected through a machine-learning approach based on a supervised classification model (Crammer, Dekel, Keshet, Shalev-Shwartz, & Singer, 2006). In a first step, to create the training set for the mathematical model, three individual coders coded the same 1,459 tweets (10% of the data set of re-tweets) for a unique sentiment value: negative sentiment (−1), neutral sentiment (0),⁵ and positive sentiment (+1) toward ItalyBank (inter-coding reliability: K alpha = .81, p = .026).⁶ In the next step, we used a Passive–Aggressive (PA) classifier (Crammer et al., 2006) implemented with a pairwise coupling with majority voting method (Hastie & Tibshirani, 1998). Pairwise coupling is an approach that combines different binary classifiers to obtain a multi-class classification (Platt, Cristianini, & Shawe-Taylor, 2000). The quality of the feature extraction and classification model was confirmed by the experimental results obtained through 10-fold cross-validation on the training data set: f measure 0.75 and accuracy 0.8. The learned model and the feature extraction process were then used on all tweets to compute the sentiment values. The sentiment analysis revealed that 34% of the 14,179 tweets were affectively charged, a rate that was found in previous Twitter studies (Hansen, Arvidsson, Nielsen, Colleoni, & Etter, 2011). Based on the idea that individuals bestow legitimacy to organizations through positive and negative judgments (Haack et al., 2014; Haidt, 2001) and following previous research for news media legitimacy (Deephouse, 1996; Deephouse & Carter, 2005), we calculated the Janis–Fadner coefficient of imbalance, ranging from −1 to +1, for monthly measures (Janis & Fadner, 1943). This coefficient gives indication about the overall evaluation of ItalyBank by citizens for the respective months.

News media content analysis

Because ItalyBank is a major national bank and, therefore, its actions are likely to be covered by national newspapers, we chose the two largest national newspapers of Italy based on the criteria of circulation and source authority (Deephouse, 1996): Corriere della Sera and La Repubblica. We searched Factiva database with the keyword “ItalyBank” for the case period, resulting in totally 1,444 articles, 846 articles from Corriere della Sera and 598 articles from La Repubblica. For the study, every second article was coded, resulting in a total of 722 articles. A recording unit was defined as the evaluation of ItalyBank in a single article. Consistent with past research (Deephouse, 1996; Deephouse & Carter, 2005) each recording unit was rated as either endorsing the bank or questioning its legitimacy and given equal weight in the subsequent analysis. A recording unit was rated as questioning when there was evidence that the bank’s action, structure, mission, or performance was being questioned or challenged. Otherwise, the recording unit was rated as endorsing the bank. A subsample of 75 articles, or 10% of the sample, was coded by two coders resulting in a sufficient intercoder reliability of 0.95. Monthly measures of legitimacy were calculated using the Janis–Fadner coefficient of imbalance (Deephouse & Carter, 2005; Janis & Fadner, 1943).

Accreditation

To observe institutional linkages (Baum & Oliver, 1991), we used secondary data provided by Standard & Poor’s (S&P), one of the most influential credit rating agencies in the global (and Italian) financial sector (Langohr & Langohr, 2010). S&P provides expert-evaluation about the likelihood of (financial) institutions to meet their financial commitments, which are regularly reported through public reports. For this study, we consulted the reports about the long-term credit ratings for ItalyBank that can range from D (lowest) to AAA (highest).

Survey

As a survey measurement, we use secondary data provided by RT, one of the most used corporate reputation rankings in research and practice (Ponzi et al., 2011). RT assesses evaluations by the general public for totally seven aspects (“dimensions”). To measure organizational legitimacy, we selected the aspects “Citizenship” (items: “supports good causes,” “positive societal influence,” and “environmentally responsible”), “Governance” (“open and transparent,” “behaves ethically,” and “fair in way it does business”), and “Workplace” (“rewards employees fairly,” “employee well-being,” and “equal opportunities”). The data were originally collected through computer-assisted web interviewing (CAWI) during the first quarter of the years 2013 and 2014, assessing the reputation of ItalyBank and additional four Italian banks. The sample includes 3,000 respondents among Italian citizens and is representative for age and gender. Respondents rate companies on a 7-point Likert-type scale. The computed overall score ranges from 1 to 100 index points.

Results

The results of the sentiment analysis are displayed in Figure 1 that shows the development of judgments in Twitter. The monthly measures indicate how the overall judgment, or sentiment, is persistently negative, with a monthly average of −0.29. Furthermore, a slight increase from May 2013 to April 2014 can be detected. In contrast, the results of the news media analysis, also displayed in the Figure 1, reveal that the overall judgment calculated as monthly measures is mainly positive (i.e., endorsing), with a monthly average of 0.49 (see Figure 1). Also, here a slight increase can be identified with a significant drop in August 2013 (0.03). The survey-based results also reveal a slight increase of the overall evaluation from 56.5 index points in the first quarter 2013 to 57.5 index points in the first quarter 2014. Finally, the observation of institutional linkages reveals a BBB+ rating of ItalyBank in May 2013. On July 12, 2014, ItalyBank was downgraded to BBB with no further changes until end of April 2014.

Figure 1.

Monthly measures (Janis–Fadner coefficient) of normative judgments about ItalyBank in news media and Twitter from May 1, 2013 to April 30, 2014.

Overall, we find an increase of organizational legitimacy as reflected in the judgments of all evaluators during the studied time period, except for the expert-evaluation by the credit rating agency. News media more or less continuously endorse ItalyBank on a high level. However, we also identified a significant drop for August 2013. In contrast to news media, a negative sentiment prevails in Twitter. This can be interpreted with the fact that, even though ItalyBank was only partially affected by the economic and financial crisis, citizens—in contrast to news media—have still and continuously blamed it in a myriad of conversations that took place on social media, for example, under the hashtags #BankItalia and #badbank (see Figure A1 and Table A1). The evaluation by the general public represented in the survey can be interpreted, if not negative, as rather modest considering the possible maximum of 100 index points. Finally, the credit rating by S&P starts on the intermediate rating BBB+ and then decreases to BBB, indicating that ItalyBank is still found to have “adequate capacity to meet its financial commitments,” yet to a lower degree than before (Standard & Poor’s, 2014).

The case study shows how the analysis of four different sources of organizational legitimacy reveals varying and different sorts of judgments. This is not surprising, because these sources base their evaluation on different criteria, norms, and standards and make them visible through different processes. These differences become apparent, when contrasting the mostly endorsing evaluation in news media with the negative sentiment in Twitter, the development of which we were able to monitor in a detailed manner. In contrast, the presented insights through the survey are based on judgments that are only published once a year, and hence, are more suitable for long-term studies. Nevertheless, the survey-based results somehow confirm the negative sentiment of the public in Twitter and give indication about the increase from one year to the next.

In sum, if we understand organizational legitimacy as constructed through the expression and negotiation of normative judgments by various evaluators, the triangulation of results from different data sources gives a multifaceted picture of organizational legitimacy and its development over time. Each data source on its own gives insights for the construction of legitimacy by respective evaluator, and each evaluator has its own particular influence on the perception of organizations. In concerted use, the measures can complement each other, because they deliver complementary judgments from different angles and frequencies, based on different evaluation criteria, and with different time sensitivity.

Discussion

Because there is rarely a single measure or authority that can pronounce the judgment for the whole society, attention to various evaluators can enhance the scholarship around organizational legitimacy. Conventional methods, although powerful in measuring the evaluation by institutional evaluators, capture only a fraction of the possible voices, norms, and aspects in the evaluation of organizations. In this article, we have discussed sentiment analysis of social media data as a method that gives access to the plurality of judgments by ordinary citizens. Our comparison with existing sources and the empirical analysis have shown how using this method can enrich our understanding of the legitimation of organizations “whose legitimacy is based on civil society discourses” (Palazzo & Scherer, 2006, p. 78).

The case study has illustrated how the number of more than 14,000 tweets by almost 6,000 citizens and categorized below 389 hashtags stands in contrast to a few, yet influential institutional evaluators and their judgments. The consideration of the many voices accounts for an understanding of organizational legitimacy that is based on a plurality of expressed opinions (Palazzo & Scherer, 2006; Schultz et al., 2013). Our study has shown how citizens in social media judge the actions of the studied organization more negatively than news media. Similarly, the decrease in citizens’ judgment over time stands in contrast to the increase of positive judgment by accreditation bodies. These diverging evaluations illustrate how judgments of citizens are not necessarily captured by an analysis of institutional evaluators. As an additional method, then, sentiment analysis of social media data can contribute to a more encompassing understanding of legitimacy that is based on the judgments by various evaluators (Ruef & Scott, 1998).

Implications for Research

The use of sentiment analysis and large scale social media data is appealing and offers tremendous potential for various research avenues and theory development. The method allows directly accessing the voices of ordinary citizens, and, therefore, gives also better indication for individual perceptions (Bitektine, 2011) than measures that are solely based on institutional evaluators’ judgments. Consequently, the method can be used to address a wide array of research questions in the business and society field.

Although a sentiment score can be criticized for being reductionist, it is at the same time appealing for researchers, because it allows testing of hypotheses. As a dependent variable, it might be of interest to investigate the antecedents of legitimacy in social media. For example: How do organizational strategies of legitimation (Scherer, Palazzo, & Seidl, 2013) or certain events affect organizational legitimacy? To answer these kinds of questions, the method can add additional value to existing measurements, because corporate actions and events, such as scandals, are increasingly discussed and evaluated in social media (Etter & Nielsen, 2015; Etter & Vestergaard, 2015). As our study has shown, organizations are discussed differently in social media than in news media, and these evaluations might not be detected by surveys that are conducted long after an event has occurred. Sentiment analysis, then, can give fruitful insights into the development of citizens’ judgments about certain events.

Furthermore, over the last years, corporations have invested heavily in social media channels to influence public opinion through various strategies (Castelló et al., 2016; Etter, 2014; Illia, Romenti, Rodrıguez-Cánovas, Murtarelli, & Carroll, 2017). Whether and how citizens in social media are affected by these strategies or rather develop counter-discourses, where organizational actions are questioned and criticized, can be detected by sentiment analysis of social media data. Similarly, sentiment analysis allows analyzing citizen’s perceptions of business, financial, and CSR initiatives, in addition to conventional legitimacy measures. For example, although CSR initiatives might find not much attention by news media because of lacking newsworthiness, they might trigger enthusiasm (or critique) in social media. Measuring legitimacy in social media might, therewith, help to detect, if CSR initiatives meet heterogeneous expectations of citizens.

When using legitimacy as an independent variable, the relationship between citizens’ judgments in social media and various organizational outcomes, such as financial performance, stock market value, sales, and so on might be of interest. Indeed, judgments in social media are used as information source by various stakeholders, given an indication about the affective orientation toward an organization, and hence, might have predictive power for various forms of organizational performance. The big advantage of sentiment analysis, then, is that it can account for the dynamic development of citizens’ judgments, which is more difficult to grasp with costly surveys. Nevertheless, a concerted use with other legitimacy measures, such as a news media–based measurement, can detect if possible effects are stronger, when other evaluators are involved. For example, in the specific case of a triangulation of social media data with news media data, a researcher can assess whether judgments in social media only affect corporate performance, when these judgments are amplified by news media coverage.

Indeed, it has to be acknowledged that various evaluators interact and influence each other: Judgments published in rankings or reports about accreditations and ratings might be covered by news media, and news media coverage can influence social media conversations, as well as vice versa (Pfeffer et al., 2014). These interrelations might be considered as mediating effects, and not as mere triangulation, when analyzing the relationships described above.

Research questions might be addressed through case studies or with representative samples, for example, from a particular industry. Furthermore, quantitative sentiment analysis can be used as a starting point for qualitative analyses that investigate more in detail the affective dynamics of legitimation processes. Here, sentiment analysis of social media data allows accessing emerging legitimacy issues in a heterogeneous context, which might not be captured with other measures.

Finally, future research might also deepen the understanding of some of the theoretical underpinnings that have been presented in this article. For example, based on the notion of affect-based responses (Haack et al., 2014) that are expressed in social media, we have argued that the expression of affective judgments and opinions by ordinary citizens constitute a source of organizational legitimacy. The relationship between affect and legitimacy needs more conceptual groundwork, which, however, was beyond the scope of this article.

Implications for Practice

Gaining legitimacy is an increasingly complex process for organizations (Castelló et al., 2016). In today’s world, organizations face a multiplicity of expectations and norms due to the expansion of corporate activities and the individualization of society (Palazzo & Scherer, 2006). Therefore, it requires a multifaceted understanding of many concerns, voices, and conceptions of truth, and an ability to engage across independent and conflicting interpretations of issues (Schultz et al., 2013). Particularly in regard to CSR issues, legitimacy of organizations is challenged (Scherer et al., 2013). To address CSR issues, stakeholder engagement has become an important organizational activity (Sharma & Henriques, 2005) that is increasingly enacted through social media platforms (Etter, 2013; Etter, 2014; Illia et al., 2017).

Managing CSR and legitimacy in social media is challenging due to the decentralization of communication in indeterminate networks and the complex dynamics enfolded therein (Castelló et al., 2016; Schultz et al., 2013) for which organizations often are not prepared (Etter, 2013). Our analysis has shown that news media clippings alone might not give indication about the concerns of ordinary citizens. In social media, ordinary citizens might discuss issues in a negative way long before—or after—they have been picked up by the news media or been detected by costly surveys. Monitoring social media through sentiment analysis, therefore, might be a valuable tool to detect (de-)legitimizing discourses around a variety of CSR topics. Additional to extant measures, sentiment analysis and the exploration of social media data can give organizations a multifaceted understanding of the many concerns and judgments that ordinary citizens express in social media. The presented study, for example, identified around 400 hashtags, or issues, to which citizens expressed their opinions. The identification of certain issues can be considered as a necessary first step for the co-creation of CSR agendas through participation in non-hierarchical, open social media platforms (Castelló et al., 2016). Furthermore, with its time-sensitive approach, organizations can use sentiment analysis for detecting (de-)legitimizing conversations and issues in a timely manner. Indeed, continuous monitoring is crucial, as conversations can emerge anytime and anywhere.

In sum, sentiment analysis can help to detect sentiment and important issues in social media. As an additional legitimacy measure to surveys, rankings, and media clippings, it gives immediate indication about the judgments of ordinary citizens represented in social media.

Limitations

The limitations of the presented method are the sampling and the delimitation of inherently connected data (discourses or actors). In our study, we analyzed tweets that mentioned the bank’s name, thereby possibly excluding tweets that make reference to the bank without explicitly mentioning it. Similarly, challenges arise from data cleaning processes, such as excluding texts from certain actors. In our study, we found and excluded 423 tweets—less than 3% of originally collected tweets—stemming from the bank itself. There is, however, a possibility that further undetected tweets stem from other institutional evaluators. Furthermore, while we have no indication of manipulation for our study, there exists the possibility of manipulation through public relations firms that distribute positive messages about companies under fake profiles. Generally, manipulations can often be detected or, considering the large amount of sourced data, may not significantly affect results. Finally, the level of accuracy that algorithms can attain in the classification of sentiment in text is limited, because mining sentiment from natural language is challenging. However, techniques are constantly improving, and the accuracy levels with today’s technologies, such as used in this study, range around 80%.

Conclusion

This article has shown how conventional quantitative methods for the measurement of organizational legitimacy are valuable for assessing judgments that are made visible by institutional evaluators. We have also shown how these sources give only limited indication about judgments by ordinary citizens that have been regarded as increasingly important from a normative perspective that calls for a stricter democratic accountability of corporate behavior (Matten & Crane, 2005). To assess these judgments, we have proposed, critically discussed, and illustratively applied the use of social media data and sentiment analysis.

Our article does not favor one method over the other. We believe that each method has value on its own and provides a valid measure for organizational legitimacy. Sentiment analysis accommodates researchers and practitioners that aim at exploring the power of large scale data sets that include a multitude of conversations and judgments by ordinary citizens and, therefore, give indication about the broader fit between an organization’s perceived behavior and heterogeneous social norms. Furthermore, the method allows the investigation of developments in a very timely manner and, therewith, is suited for a communication environment that is characterized by speed and connectivity. Finally, by focusing on the content first and on the actors in a second step, the method is suited for a context, where established and identifiable evaluators are not the only source of legitimacy, but where any actor with access to social media can evaluate corporations publicly and where conversations can emerge everywhere in a transnational network of actors that contribute to the (de-)legitimation of corporations. Alone or in combination with extant measures, the method is powerful to study organizational legitimacy as constructed through the expression and negotiation of normative judgment by ordinary citizens.

Footnotes

Appendix

Table A1.

Selected Hashtags and Explanation.

Hashtag^a	Issue/conversation
#bankitalia	ItalyBank was involved in the decree of BankItalia, the National Bank of Italy, which has re-financed its capitals through national reserves for the amount of about 4 milliards Euro. As ItalyBank is one of the two major owners of BankItalia, people assumed that part of that money went to ItalyBank too.
#alitalia	The Alitalia issue points to involvement of the bank in the recent sauvetage of Alitalia, the Italian airplane national company, an event which attracted ample attention and generated heated discussions and conflicting opinions around it.
#atf (ATF Bank)	The acquisition of an Ukrainian bank by ItalyBank implied a big speculation. The financial loss through the deal evoked negative sentiment.
#badbank	The intention to create an ad-hoc institution with other major Italian banks to externalize risks evoked mostly negative sentiment.
#culture^b	The bank is repeatedly mentioned for its cultural initiatives, such as the art exposition in Bologna organized with artworks owned by ItalyBank.
#social^b	Under the hashtag #social, social initiatives like the collection of money in favor of flooded populations are discussed.
#startup	Highly appreciated and shared information about bank economically supporting startups and innovative technological ideas.

Hashtags are originally in Italian language and translated into English for the purpose of this study.

For these hashtags, ItalyBank launched social media campaigns that promoted the positive discussions and sentiment.

Acknowledgements

The authors thank Andrew Crane and the other editors of this special issue for their helpful guidance during the development of this article. Furthermore, the authors would like to thank Glen Whelan, Patrick Haack, and Alessandro Rozza for their fruitful comments and inputs for prior versions of this article, as well as the three anonymous reviewers for their constructive critique and comments.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Carlsberg Foundation and the UniCredit & Universities Foundation.

Notes

Author Biographies

Michael Etter (PhD, Universität St. Gallen-Hochschule für Wirtschafts-, Rechts- und Sozialwissenschaften) is a Marie Curie research fellow at Cass Business School. His current research interest lies in the challenges that organizations face in a networked and digitalized communication environment. He is particularly interested in the processes of reputation and legitimacy formation. The aspect of corporate social responsibility is, therewith, prevalent in his academic work that has been published in international communication and management journals, such as Corporate Communication: An International Journal, Journal of Business Ethics, Journal of Management Studies, and Management Communication Quarterly.

Elanor Colleoni (PhD, Università degli Studi di Milano) is a data scientist manager in a reputation-based consultancy firm and research fellow at the University of Milan. Her research and work focuses on developing models to measure corporate key performance indicators (KPIs) in relation to customer data.

Laura Illia (PhD, University of Lugano, Switzerland) is an associate professor at IE University (IE HST and IE Business School). Her current research focuses on how issues of social responsibility, communication, social media, branding, identity, and reputation are involved in organizational management. She has been doing research at the University of Cambridge, London School of Economics and Political Science, and University of Lugano, Switzerland. Her works are published in journals such as British Journal of Management, Journal of Business Ethics, Journal of Business Research, Journal of Management Inquiry, MIT Sloan Management Review, and others.

Katia Meggiorin is a PhD student at IE Business School, at the strategy department. Her research interests focus on institutionalization processes in the context of sharing economy, and how issues of social responsibility, social media, and reputation affect organizational performance.

Antonino D’Eugenio is a specialist in social sciences research practice in industry. He currently holds the position of head of executive development at UniCredit. As a team leader, he pioneered the UniCredit Brand and Reputation assessment. He holds a degree in political science from Pisa University and a master’s degree in lean operations.

References

Adamic

L. A.

Glance

(2005). The political blogosphere and the 2004 U.S. election: Divided they blog. In ACM Proceedings of the 3rd International Workshop on Link Discovery (LinkKDD 2005) (pp. 36-43). New York, NY: ACM Press.

Agarwal

Bhattacharyya

(2005). Sentiment analysis: A new approach for effective use of linguistic knowledge and exploiting similarities in a set of documents to be classified. In Bendre

Mukherje

Sangal

(Eds.) Proceedings of the International Conference on Natural Language Processing (ICON) (pp. 21-28). New Delhi, India.

Albu

O. B.

Etter

(2016). Hypertextuality and social media: A study of the constitutive and paradoxical implications of organizational Twitter use. Management Communication Quarterly, 30(1), 5-31.

Bakker

T. P.

de Vreese

C. H.

(2011). Good news for the future? Young people, Internet use, and political participation. Communication Research, 38, 451-470.

Banning

S. A.

Sweetser

K. D.

(2007). How much do they think it affects them and whom do they believe? Comparing the third-person effect and credibility of blogs and traditional media. Communication Quarterly, 55, 451-466.

Bansal

Clelland

(2004). Talking trash: Legitimacy, impression management, and unsystematic risk in the context of the natural environment. Academy of Management Journal, 47, 93-103.

Baum

J. A.

Oliver

(1991). Institutional linkages and organizational mortality. Administrative Science Quarterly, 36, 187-218.

Bitektine

(2011). Toward a theory of social judgments of organizations: The case of legitimacy, reputation, and status. Academy of Management Review, 36, 151-179.

Bitektine

Haack

(2015). The “macro” and the “micro” of legitimacy: Toward a multilevel theory of the legitimacy process. Academy of Management Review, 40, 49-75.

10.

Blumell

L. E.

(2016). Birds of a feather . . . Gatekeeping human rights online news coverage via traditional and humanitarian news media. Doctotal dissertation, Texas Tech University, Lubbock.

11.

boyd

(2010). Social network sites as networked publics: Affordances, dynamics, and implications. In Papacharissi

(Ed.), Networked self: Identity, community, and culture on social network sites (pp. 39-58). New York, NY: Routledge.

12.

boyd

Crawford

(2012). Critical questions for big data. Information, Communication & Society, 15, 662-679.

13.

Brammer

Millington

(2005). Corporate reputation and philanthropy: An empirical analysis. Journal of Business Ethics, 61, 29-44.

14.

Brown

Deegan

(1998). The public disclosure of environmental performance information—A dual test of media agenda setting theory and legitimacy theory. Accounting and Business Research, 29, 21-41.

15.

Bruns

Stieglitz

(2012). Quantitative approaches to comparing communication patterns on Twitter. Journal of Technology in Human Services, 30, 160-185.

16.

Burgess

Bruns

(2011, August 25-27). The use of Twitter hashtags in the formation of ad hoc publics. Paper presented at the European Consortium for Political Research Conference, Reykjavik, Iceland.

17.

Cambria

Mazzocco

Hussain

(2013). Application of multi-dimensional scaling and artificial neural networks for biologically inspired opinion mining. Biologically Inspired Cognitive Architectures, 4(1), 41-53.

18.

Cambria

Schuller

Xia

Havasi

(2013). New avenues in opinion mining and sentiment analysis. Intelligent Systems, 28(2), 15-21.

19.

Carroll

C. E.

(2009). The relationship between firms’ media favorability and public esteem. Public Relations Journal, 3(4), 1-32.

20.

Carroll

C. E.

McCombs

(2003). Agenda-setting effects of business news on the public’s images and opinions about major corporations. Corporate Reputation Review, 6, 36-46.

21.

Castelló

Etter

Årup Nielsen

(2016). Strategies of legitimacy through social media: The networked strategy. Journal of Management Studies, 53, 402-432.

22.

Castelló

Morsing

Schultz

(2013). Communicative dynamics and the polyphony of corporate social responsibility in the network society. Journal of Business Ethics, 118, 683-694.

23.

Castells

(2007). Communication, power and counter-power in the network society. International Journal of Communication, 1, 238-266.

24.

Chumley

(2014, November 19). Barilla pasta maker—humbled by gay pressures—gives LGBTs rights. The Washington Times. Retrieved from http://www.washingtontimes.com/news/2014/nov/19/barilla-pasta-maker-humbled-by-gay-pressures-gives/

25.

Clemente

Roulet

T. J.

(2015). Public opinion as a source of deinstitutionalization: A “spiral of silence” approach. Academy of Management Review, 40, 96-114.

26.

Colleoni

(2013). CSR communication for organizational legitimacy in social media. Corporate Communications: An International Journal, 18, 228-248.

27.

Crammer

Dekel

Keshet

Shalev-Shwartz

Singer

(2006). Online passive-aggressive algorithms. Journal of Machine Learning Research, 7, 551-585.

28.

Crane

McWilliams

Matten

Moon

Siegel

D. S.

(Eds.). (2008). The Oxford handbook of corporate social responsibility. New York, NY: Oxford University Press.

29.

Davies

Chun

Da Silva

R. V.

Roper

(2003). Corporate reputation and competitiveness. London, England: Routledge.

30.

de Souza

C. R. B.

Redmiles

. (2009). On the roles of APIs in the coordination of collaborative software development. Journal of Computer Supported Cooperative Work, 18, 445-475.

31.

Dearing

J. W.

Rogers

(1996). Agenda-setting (Vol. 6). New York, NY: Sage.

32.

Deephouse

D. L.

(1996). Does isomorphism legitimate? Academy of Management Journal, 39, 1024-1039.

33.

Deephouse

D. L.

Carter

S. M.

(2005). An examination of differences between organizational legitimacy and organizational reputation. Journal of Management Studies, 42, 329-360.

34.

Etter

(2013). Reasons for low levels of interactivity: (Non-) interactive CSR communication in twitter. Public relations review, 39, 606-608.

35.

Etter

(2014). Broadcasting, reacting, engaging—Three strategies for CSR communication in Twitter. Journal of Communication Management, 18, 322-342.

36.

Etter

M. A.

Nielsen

F. Å

. (2015). Collective remembering of organizations: Co-construction of organizational pasts in Wikipedia. Corporate Communications: An International Journal, 20, 431-447.

37.

Etter

M. A.

Vestergaard

(2015). Facebook and the public framing of a corporate crisis. Corporate Communications: An International Journal, 20, 163-177.

38.

Fieseler

Fleck

Meckel

(2010). Corporate social responsibility in the blogosphere. Journal of Business Ethics, 91, 599-614.

39.

Fombrun

C. J.

(2007). List of lists: A compilation of international corporate reputation ratings. Corporate Reputation Review, 10, 144-153.

40.

Galtung

Ruge

M. H.

(1965). The structure of foreign news: The presentation of the Congo, Cuba and Cyprus Crises in four Norwegian newspapers. Journal of Peace Research, 2, 64-90.

41.

Gangemi

Presutti

Reforgiato

(2014, February). Framebased detection of opinion holders and topics: A model and a tool. IEEE Computational Intelligence Magazine, 9, 20-30.

42.

Gerlitz

Rieder

(2013). Mining one percent of Twitter: Collections, baselines, sampling. M/C Journal, 16(2), 1-12.

43.

Gibbert

Wicki

Ruigrok

(2008). What passes as a rigorous case study? Strategic Management Journal, 29, 1465-1474.

44.

Gillespie

(2010). The politics of platforms. New Media & Society, 12, 347-364.

45.

Haack

Pfarrer

M. D.

Scherer

A. G.

(2014). Legitimacy-as-feeling: How affect leads to vertical legitimacy spillovers in transnational governance. Journal of Management Studies, 51, 634-666.

46.

Haidt

(2001). The emotional dog and its rational tail: A social intuitionist approach to moral judgment. Psychological Review, 108, 814-834.

47.

Hansen

L. K.

Arvidsson

Nielsen

F. Å.

Colleoni

Etter

(2011). Good friends, bad news-affect and virality in Twitter. In Park

J. J.

Yang

L. T.

Lee

(Eds.), Future Information Technology: Vol. 185. Communications in computer and information science (CCIS) (pp. 34-43). Berlin, Germany: Springer.

48.

Hargittai

Litt

(2011). The tweet smell of celebrity success: Explaining variation in Twitter adoption among a diverse group of young adults. New Media & Society, 13, 824-842.

49.

Hart

R. P.

Carroll

C. E.

(2015). DICTION 7, The Text Analysis Program: Help manual. Austin, TX: Digitext.

50.

Hastie

Tibshirani

(1998). Classification by pairwise coupling. The Annals of Statistics, 26, 451-471.

51.

Helm

(2007). One reputation or many? Comparing stakeholders’ perceptions of corporate reputation. Corporate Communications: An International Journal, 12, 238-254.

52.

Hermida

(2012). Social journalism: Exploring how social media is shaping journalism. In Siapera

Veglis

(Eds.), The handbook of global online journalism (pp. 309-328). New York, NY: John Wiley.

53.

Highhouse

Brooks

M. E.

Gregarus

(2009). An organizational impression management perspective on the formation of corporate reputations. Journal of Management, 35, 1481-1493.

54.

Hindman

(2009). The myth of digital democracy. Princeton, NJ: Princeton University Press.

55.

Hunter

M. L.

Le Menestrel

De Bettignies

H. C.

(2008). Beyond control: Crisis strategies and stakeholder media in the Danone Boycott of 2001. Corporate Reputation Review, 11, 335-350.

56.

Illia

Romenti

Rodrıguez-Cánovas

Murtarelli

Carroll

C. E.

(2017). Exploring corporations’ dialogue about CSR in the digital era. Journal of Business Ethics 146, 39–58.

57.

Janis

I. L.

Fadner

R. H.

(1943). A coefficient of imbalance for content analysis. Psychometrika, 8, 105-119.

58.

Jansen

B. J.

Zhang

Sobel

Chowdury

(2009). Twitter power: Tweets as electronic word of mouth. Journal of the American Society for Information Science and Technology, 60, 2169-2188.

59.

Jenkins

(2006). Convergence culture: Where old and new media collide. New York: New York University Press.

60.

Jick

T. D.

(1979). Mixing qualitative and quantitative methods: Triangulation in action. Administrative Science Quarterly, 24, 602-611.

61.

Lamertz

Baum

J. A.

(1998). The legitimacy of organizational downsizing in Canada: An analysis of explanatory media accounts. Canadian Journal of Administrative Sciences, 15, 93-107.

62.

Langohr

(2010). The rating agencies and their credit ratings: What they are, how they work, and why they are relevant (Vol. 510). New York, NY: John Wiley.

63.

Larsson

A. O.

Moe

(2012). Studying political microblogging: Twitter users in the 2010 Swedish election campaign. New Media & Society, 14, 729-747.

64.

Lee

S. Y.

Carroll

C. E.

(2011). The emergence, variation, and evolution of corporate social responsibility in the public sphere, 1980–2004: The exposure of firms to public debate. Journal of Business Ethics, 114, 115-131.

65.

Lomborg

Bechmann

(2014). Using APIs for data collection on social media. The Information Society, 30, 256-265.

66.

MacMillan

Money

Downing

Hillenbrand

(2005). Reputation in relationships: Measuring experiences, emotions and behaviors. Corporate Reputation Review, 8, 214-232.

67.

Martin

R. M.

Boynton

L. A.

(2005). From liftoff to landing: NASA’s crisis communications and resulting media coverage following the Challenger and Columbia tragedies. Public Relations Review, 31, 253-261.

68.

Marwick

A. E.

(2005). “I’m a lot more interesting than a Friendster profile”: Identity presentation, authenticity and power in social networking services. In Conference Proceedings Association of Internet Researchers 6.0. Chicago, IL.

69.

Marwick

A. E.

boyd

(2011). I tweet honestly, I tweet passionately: Twitter users, context collapse, and the imagined audience. New Media & Society, 4, 114-133.

70.

Matten

Crane

(2005). Corporate citizenship: Toward an extended theoretical conceptualization. Academy of Management Review, 30, 166-179.

71.

McCombs

(2013). Setting the agenda: The mass media and public opinion. New York, NY: John Wiley.

72.

McManus

J. H.

(1994). Market-driven journalism: Let the citizen beware? Thousand Oaks, CA: Sage.

73.

Meyer

J. W.

Scott

W. R.

(1983). Organizational environments: Ritual and rationality. Beverly Hills, CA: Sage.

74.

Nekmat

Gonzenbach

W. J.

(2013). Multiple opinion climates in online forums: Role of website source reference and within-forum opinion congruency. Journalism & Mass Communication Quarterly, 90, 736-756.

75.

Pak

Paroubek

(2010, May). Twitter as a corpus for sentiment analysis and opinion mining. Proceedings of the seventh conference on International Language Resources and Evaluation (LREC’10), 10, 1320-1326.

76.

Palazzo

Scherer

A. G.

(2006). Corporate legitimacy as deliberation: A communicative framework. Journal of Business Ethics, 66, 71-88.

77.

Pang

Lee

(2008). Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1-2), 1-135.

78.

Pang

Lee

Vaithyanathan

(2002). Thumbs up? Sentiment classification using machine learning techniques. In Proceedings of the ACL-02 conference on empirical methods in natural language processing (Vol. 10, pp. 79-86). Stroudsburg, PA: Association for Computational Linguistics.

79.

Papacharissi

(2009). The virtual sphere 2.0: The Internet, the public sphere, and beyond. In Chadwick

Howard

(Eds.), Routledge handbook of internet politics (pp. 230-245). London, England: Routledge.

80.

Papacharissi

(2014). Affective publics: Sentiment, technology, and politics. Oxford, UK: Oxford University Press.

81.

Pariser

(2011). The filter bubble: What the Internet is hiding from you. Harmondsworth, UK: Penguin.

82.

Paßmann

Boeschoten

Schäfer

(2013). The gift of the gab: Retweet cartels and gift economies on Twitter. In Weller

Bruns

Burgees

Maahrt

Puschmann

(Eds.), Twitter and society (pp. 331-345). New York, NY: Peter Lang.

83.

Pedersen

Razmerita

Colleoni

(2014). Electronic word of mouth communication and consumer behaviour: An exploratory study of Danish social media communication influence. Professional Communication Knowledge Management Cognition, 5, 112-131.

84.

Pfeffer

Salancik

G. R.

(1978). The external control of organizations: A resource dependence perspective. New York, NY: Harper & Row.

85.

Pfeffer

Zorbach

Carley

K. M.

(2014). Understanding online firestorms: Negative word-of-mouth dynamics in social media networks. Journal of Marketing Communications, 20, 117-128.

86.

Platt

Cristianini

Shawe-Taylor

(2000). Large margin DAGs for multiclass classification. In Proceedings of neural information processing systems (pp. 547-553). Cambridge, MA: MIT Press.

87.

Pollock

T. G.

Rindova

V. P.

(2003). Media legitimation effects in the market for initial public offerings. Academy of Management Journal, 46, 631-642.

88.

Ponzi

L. J.

Fombrun

C. J.

Gardberg

N. A.

(2011). RepTrak™ pulse: Conceptualizing and validating a short-form measure of corporate reputation. Corporate Reputation Review, 14, 15-35.

89.

Qiu

Welch

(2004, September). Investor sentiment measures In Working Paper No. w10794. Cambridge, MA: National Bureau of Economic Research.

90.

Rao

Ravichandran

(2009). Semi-supervised polarity lexicon induction. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics (pp. 675-682). Stroudsburg, PA.

91.

Rindova

V. P.

Williamson

I. O.

Petkova

A. P.

Sever

J. M.

(2005). Being good or being known: An empirical examination of the dimensions, antecedents, and consequences of organizational reputation. Academy of Management Journal, 48, 1033-1049.

92.

Roblyer

McDaniel

Webb

Herman

Witty

J. V.

(2010). Findings on Facebook in higher education: A comparison of college faculty and student uses and perceptions of social networking sites. The Internet and Higher Education, 13, 134-140.

93.

Rogers

(2009). The end of the virtual—Digital methods. Amsterdam, The Netherlands: Amsterdam University Press.

94.

Ruef

Scott

W. R.

(1998). A multidimensional model of organizational legitimacy: Hospital survival in changing institutional environments. Administrative Science Quarterly, 42, 877-904.

95.

Saif

Alani

(2012). Semantic sentiment analysis of twitter. In Cudre-Mauroux

Heflin

Sirin

Tudorache

Euzenat

Hauswirth

. . . Bernstein

(Eds.) The Semantic Web - ISWC 2012 (pp. 508-524). Boston, MA.

96.

Scherer

A. G.

Palazzo

Seidl

(2013). Managing legitimacy in complex and heterogeneous environments: Sustainable development in a globalized world. Journal of Management Studies, 50, 259-284.

97.

Schultz

Castelló

Morsing

(2013). The construction of corporate social responsibility in network societies: A communication view. Journal of Business Ethics, 115, 681-692.

98.

Schultz

Mouritsen

Gabrielsen

(2001). Sticky reputation: Analyzing a ranking system. Corporate Reputation Review, 4, 24-41.

99.

Sharma

Henriques

(2005). Stakeholder influences on sustainability practices in the Canadian forest products industry. Strategic Management Journal, 26, 159-180.

100.

Shoemaker

P. J.

Reese

S. D.

(2013). Mediating the message in the 21st century: A media sociology perspective. New York, NY: Routledge.

101.

Singh

J. V.

Tucker

D. J.

House

R. J.

(1986). Organizational legitimacy and the liability of newness. Administrative science quarterly, 31, 171-193.

102.

Sotiriadis

M. D.

van Zyl

(2013). Electronic word-of-mouth and online reviews in tourism services: The use of twitter by tourists. Electronic Commerce Research, 13, 103-124.

103.

Standard & Poor’s. (2014, March). Research update: Italy-based UniCredit “BBB/A-2”—Ratings affirmed on increased provisioning and capital support plan; outlook negative. London, UK: Ratings Direct.

104.

Stieglitz

Krüger

(2011). Analysis of sentiments in corporate Twitter communication: A case study on an issue of Toyota. In ACIS 2011 Proceedings (pp. 1-11). Sydney, Australia.

105.

Stohl

Etter

Banghart

Woo

(2016). Social media policies: Implications for contemporary notions of corporate social responsibility. Journal of Business Ethics, 20, 431-447.

106.

Stringhini

Kruegel

Vigna

(2010, December). Detecting spammers on social networks. In Proceedings of the 26th Annual Computer Security Applications Conference (pp. 1-9). Austin, TX.

107.

Suchman

M. C.

(1995). Managing legitimacy: Strategic and institutional approaches. Academy of Management Review, 20, 571-610.

108.

Sunstein

C. R.

(2009). Republic. com 2.0. Princeton, NJ: Princeton University Press.

109.

Thelwall

Buckley

Paltoglou

(2011). Sentiment in Twitter events. Journal of the American Society for Information Science and Technology, 62, 406-418.

110.

Thier

(2012, October). Facebook has a billion users and a revenue question. Forbes. Retrieved from http://www.forbes.com/forbes/welcome/?toURL=http://www.forbes.com/sites/davidthier/2012/10/04/facebook-has-a-billion-users-and-a-revenue-question/&;refURL=https://www.google.co.uk/&;referrer=https://www.google.co.uk/

111.

Thorndike

E. L.

(1920). A constant error in psychological ratings. Journal of Applied Psychology, 4, 25-29.

112.

van der Meer

T. G.

Verhoeven

. (2013). Public framing organizational crisis situations: Social media versus news media. Public Relations Review, 39, 229-231.

113.

Vergne

J. P.

(2010). Toward a new measure of organizational legitimacy: Method, validation, and illustration. Organizational Research Methods, 14, 484-502.

114.

Wartick

S. L.

(2002). Measuring corporate reputation definition and data. Business & Society, 41, 371-392.

115.

Wei

Hindman

D. B.

(2011). Does the digital divide matter more? Comparing the effects of new media and old media use on the education-based knowledge gap. Mass Communication and Society, 14, 216-235.

116.

Westerman

Spence

P. R.

Van Der Heide

(2014). Social media as information source: Recency of updates and credibility of information. Journal of Computer-Mediated Communication, 19, 171-183.

117.

Westphal

J. D.

Deephouse

D. L.

(2011). Avoiding bad press: Interpersonal influence in relations between CEOs and journalists and the consequences for press reporting about firms and their leadership. Organization Science, 22, 1061-1086.

118.

Whelan

Moon

Grant

(2013). Corporations and citizenship arenas in the age of social media. Journal of Business Ethics, 118, 777-790.

119.

Zhang

Shang

Jia

(2015). Sentiment analysis on microblogging by integrating text and image features. Advances in Knowledge Discovery and Data Mining Lecture Notes in Computer Science, 9078, 52-63.