Analyzing social media messages of public sector organizations utilizing sentiment analysis and topic modeling

Abstract

In this paper, we perform sentiment analysis and topic modeling on Twitter and Facebook posts of nine public sector organizations operating in Northeast US. The study objective is to compare and contrast message sentiment, content and topics of discussion on social media. We discover that sentiment and frequency of messages on social media is indeed affected by nature of organization’s operations. We also discover that organizations either use Twitter for broadcasting or one-to-one communication with public. Finally we found discussion topics of organizations – identified through unsupervised machine learning – that engaged in similar areas of public service having similar topics and keywords in their public messages. Our analysis also indicates missed opportunities by these organizations when communication with public. Findings from this study can be used by public sector entities to understand and improve their social media engagement with citizens.

Keywords

Facebook Twitter social media sentiment analysis topic modeling public organizations unsupervised machine learning

1. Introduction

Increase in use of social media globally, has created new possibilities for public organizations to engage with citizens (Gorodnichenko et al., 2018). Research has shown that an increase in social media interactivity of an organization affects the quality of organization-public relationship (Saffer et al., 2013). Thus governments around the world are striving to increase openness and transparency into the working of their institutions (Bertot et al., 2010b). Social media can play a very constructive role in this regard by allowing public organizations with the opportunity to increase institutional responsiveness and enhance citizen engagement (Lorenzi et al., 2014). Hence efforts are now being made by various governments at the national level to utilize these platforms. One such example is that of the Open Government Directive in the United States, which supports open governance by encouraging government agencies to benefit from social media platforms (Coglianese et al., 2009). Since the start of this initiative, government agencies in the U.S. have come a long way in using social media for communication with citizens. According to a survey by the International Association of Chiefs of Police (IACP), approximately 96.4% of 553 U.S. law enforcement agencies use social media (Huang et al., 2017).

Facebook and Twitter are two of the largest social media networks in terms of the users. Facebook is the most popular social network and has around 1.7 billion active users worldwide in 2020 (Stats, 2018). Twitter on the other hand, averaged around 330 million monthly active users as of year 2020 with as many as 500 million messages a day (Stats, 2017b). This massive number of people utilizing social media for information gathering and expressing their views has provided a unique opportunity specifically with regards to communication between government institutions and citizens. These platforms can also be used by researchers and policy makers to collect data and gauge citizen sentiment towards government policies and initiatives (Sharma et al., 2018).

In this paper, we investigate the nature of communication on Facebook and Twitter by public sector organizations operating in Northeast United States. We collect their social media posts and analyze them for sentiment and content. Along with this, we also perform topic modeling to identify main themes of their message (Rodriguez & Storer, 2020). Organizations analyzed in this study operate in the areas of law enforcement, public transport, utility services and department of motor vehicle (DMV). The goals of this study are following:

•
Observe frequency, sentiment and content of social media messages by organizations operating in disparate areas of public service.
•
Evaluate how and why their messages are similar or different from each other.
•
Perform topic modeling on their social media messages to identify main themes of their messages and how these themes are similar or different for these organizations.

Findings of this study will help public organizations gain better understanding of their online messages, allowing them to improve public engagement through social media communication.

In the next section of the paper we will review the existing literature while in Section 3 we formulate our research questions. In Section 4 we discuss our methodology while in Section 5 we describe our data analysis. In Section 6 we discuss our analysis while finally in Section 7, we conclude our paper.
2. Previous research

In this section of the paper we review some of the existing research in the areas of social media usage for public engagement along with sentiment analysis and topic modeling.

Organizations and individuals are now employing social media for mass communication, ranging from product marketing to political campaigning (De Vries et al., 2012; Yaqub et al., 2017; Kim & Ko, 2012). By using the already available social media platforms, public organizations can save funds that would be required to develop and maintain customized solutions to interact with public (Lee & Kwak, 2012).

One novel example in this regard is the research in use of social media communication during natural disasters (Wang & Zhuang, 2017). A wide range of studies suggest that information sharing networks, such as Twitter, can be very useful in times of crisis by quickly and effectively disseminating relevant information (Sakaki et al., 2013). Thus various studies have used Twitter messages during natural disasters such as hurricanes and earthquakes to analyze communication (Subba & Bui, 2017; Wang & Zhuang, 2017).

Social media also helps in increasing openness and transparency into the working of public organizations (Bertot, 2010). Studies have shown that from civic services to police departments, information sharing and public engagement through Twitter can lead to greater transparency and more confidence of citizens on their state and local institutions (Heverin & Zach, 2010). For example, law enforcement agencies are actively using social media platform as this enables them to handle crimes more effectively while promoting a harmonious relationship between police and community. One such example is the use of Twitter by the London Metropolitan Police (MET) and Greater Manchester Police (GMP) during the British riots of 2011 (Denef et al., 2013). Similarly a study into use of Twitter by police departments in large U.S. cities observed the frequent usage of the micro-blogging platform by these organizations to disseminate crime and incident related information (Heverin & Zach, 2010).

Studies have also indicated that by using social media police forces can increase their perceived legitimacy by enabling transparency and participation (Grimmelikhuijsen & Meijer, 2015). One such example is that of Boston Police Department’s use of social media in the aftermath of bomb explosions during Boston Marathon (Davis et al., 2014). BPD was able to utilize Twitter successfully to keep citizens informed on the status of the investigation along with providing assurance and assistance. Similarly a study into use of social media by five U.S police departments discovered public responding by liking posts of self-promotion by the department more significantly than any other category of messages (Williams et al., 2018). This indicates positive impact of public engagement messages by law enforcement agencies on social media.

This increase in use of social media by organizations and individuals belonging to all walks of life has caused a surge in analyses of data on social media. For example rise in use of Twitter for election campaigning has greatly expanded research in the area of election prediction through data analytic (Tumasjan et al., 2010; Shi et al., 2012). Studies have also looked at the potential of utilizing Twitter location feature to gauge candidate support not only at the national level but also at regional and state level (Yaqub et al., 2020).

Other than law enforcement, research studies have appraised the potential role of social media in working of other public sector organizations. For example, studies have evaluated the potential of performing sentiment analyses on tweets shared by transit riders to gauge their satisfaction with service provided (Collins et al., 2013). This could provide public transport authorities with valuable direct feedback from citizens in terms of their journey satisfaction, enabling them to improve their service quality (Nikolaidou & Papaioannou, 2018). Using social media, public transport authorities can also share reliable, and timely information with passengers and other travelers during disruptions such as large sporting events (Cottrill et al., 2017). Hence it is not surprising to observe a 2016 study discovering 100% and 93% presence of U.S. transit agencies on Twitter and Facebook respectively (Liu et al., 2016).

Researchers have also looked at the role Twitter can play in development of transport policy and planning along with effective operations of transport system (Schweitzer, 2014). The key benefit of using social media is the minimal cost of data collection along with the ability to gather real time data from a large segment of users (Nikolaidou & Papaioannou, 2018).

We can conclude our literature survey by stating that the analysis of social media data can provide public organizations with valuable and timely information allowing them to enhance their services and react in a timely manner to address public needs (Young, 2017). However, some of the most pressing issues involved with utilizing social media data are that of data reliability and the sample bias, as not all citizens utilize social media (Nikolaidou & Papaioannou, 2018; Anwar & Yaqub, 2020).

3. Analysis of social media communication

In this section, we propose the research questions which will be evaluated through data analysis.

3.1 Sentiment analysis of organizational communication

Sentiment analysis of social media messages is being used in vast array of areas related with governance and public trust (Gorodnichenko et al., 2018, Yue et al., 2019). These analysis range from predicting resentment against government policies to forecasting general election results (Calderon, 2015; Tumasjan et al., 2010). Similarly, role of sentiment in online message propagation remains a popular topic of research. For example, some studies have claimed that tweets with positive sentiment reach a wider audience than those with negative sentiment (Ferrara & Yang, 2015b). Hence in-order to reach a wider audience, organizations should create a positive message.

However, while it would be prudent to create a positive message, we also believe that the nature of organization’s operations also plays an important role in determining the sentiment of organizational communication. For example, law enforcement agencies primarily use social media to disseminate information and alerts regarding crimes, traffic alerts, safety notices to citizens on an urgent basis (Huang et al., 2017). This would be different from messages by a department of motor vehicle or a power utility, where most messages would be less negative in sentiment. Hence we propose the following research question:

Research Question 1: How the nature of organization’s operations affects the sentiment of it’s social media messages?

3.2 Frequency analysis of organizational communication

Organizations have various goals when they engage with public on social media. These goals could be disseminating information or creating a call for action. Hou and Lampe (2015) create a framework to evaluate the social media communication of non-profit organizations. They divide the communication into 3 categories, information, community and action. We believe that for some organizations, there is a greater and urgent need to disseminate information frequently than others. For example alerts by a public transport authority such as MTA (New York City) or SEPTA (Philadelphia), where most messages pertain with daily train and bus schedules and delays would be more urgent and numerous than a power utility where messages would be less frequent. Using our Facebook and Twitter messages, we evaluate how frequently organizations are communicating with citizen and how it relates with their daily operations. Hence we propose the following research question:

Research Question 2: How frequently these organizations are using social media for communication? How does it relate with their operations?

3.3 Message framing and direct communication on Twitter

Message framing on Twitter is an important area of research. Adding hashtags (#) to keywords in tweets by message posters makes them searchable by other online users and play an important role in the context of coverage and discussion on social media (Shwartz-Asher et al., 2016). The use of hashtag allows users to become part of trends and also enables them to reach a large audience by making their messages searchable.

Furthermore, Twitter not only allows it’s users to broadcast their message to many people but also lets them interact one-to-one by addressing a person directly. Using “@” before a particular user name, is an effective way for users to interact with each other (Tang et al., 2015). This enables them to respond to other user’s tweets paving way for a dialogue and bi-directional communication. Various research studies analyzing conversations on Twitter have stated that people not only use Twitter to post opinions but also engage in interactive discussions (Tumasjan et al., 2010). Nonetheless, direct messaging creates complexities for users in terms of target audience, especially for users with a large following in having to handle multiplicity and one-to-one conversations at the same time (Marwick & Boyd, 2011). Management of audience becomes challenging as the number of followers grow. Almost all Twitter accounts of the organizations used in this study have followers ranging from tens of thousands to millions of unique users.

Hence, we propose that as organizations use social media to inform public and broadcast critical alerts, they will be more concerned about making their tweets more searchable than ordinary users having fewer followers and lesser number of messages. By framing their messages on Twitter using hashtags, organizations are able to reach a broader audience, making their messages more searchable. We also examining tweets of these organization for direct messages, as this can help us understand agencies’ social media practices in the context of engaging public users for building community relationship along with addressing citizens’ needs. Thus we have the following research question:

Research Question 3: How many organizations use # to make their tweets more searchable to users and how much do these organizations engage in direct messaging with other users, creating a conversation and answering their queries directly?

3.4 Topic modeling of social media messages

Finally, we perform topic modeling of social media messages by these organizations to evaluate the important discussion points. Topic modeling is an unsupervised machine learning technique that uses Latent Dirichlet Allocation (LDA). This is a probability based technique and is used for detection of topics present in a text (Ilyas et al., 2020; Anwar et al., 2021). Various research studies have utilized topic modeling using LDA to evaluate social media communication (Maier et al., 2018; Guo et al., 2016). For example Guo et al. performed topic modeling on 77 million tweets collected during the US elections of 2012 to gauge the usefulness of computer-assisted text analysis to generate insights into data (Guo et al., 2016). The study concluded that LDA based analysis performed better than dictionary based approach, along with the inference that the approach yielded some valuable information from the large tweet data set.

As discussed in research question 1, we look at the importance of the nature of organization‘s work in determining sentiment of organizational communication. We expect similar results in our topic modeling analysis of social media messages. Hence the three law enforcement agencies, which are part of our study, will have similar topics in their messages and will be substantially different from the topics created from the messages by the department of motor vehicle or a power utilities. Hence we propose the following research question:

Research Question 4: How the nature of organization’s operations affects the topics being discussed in their messages on social media? Do organizations operating in similar areas have similar topics in their online messages?

4. Methodology

In this section of the paper, we discuss the process of preparing our data to answer research questions proposed in the previous section. This includes following:

•
Data collection and storage
•
Sentiment Analysis
•
Topic modeling

4.1 Data collection and storage

For this study, we downloaded Facebook and Twitter posts of 9 public sector organizations operating in the areas of law enforcement, transportation, utility services and department of motor vehicles. We selected 3 police departments, while 2 organizations from each of the other 3 areas were selected. All of these organizations have official pages on both Facebook and Twitter. We collected their last 1,561 Facebook posts and 3,200 tweets starting from 9th April 2018.

We utilized Facepager software to download social media messages from Facebook. Facepager was developed for fetching publicly available data from social media platforms such as Facebook, YouTube, Twitter and other websites (Facepager, 2018).

We downloaded Twitter messages of the organizations using Twitter developer API (Twitter, 2021). The API allows users to download last 3,200 public tweets for any Twitter user account.

These messages were stored as cvs files and were cleaned and analyzed for sentiment. They were then stored in MySQL database for data analysis. Figure 1 shows the steps involved in processing and analyzing data.

Figure 1.

Steps involved in data analysis.

4.2 Sentiment analysis

We use open source sentiment analysis tool SentiStrength, to evaluate sentiment of social media posts in our dataset (Stats, 2017a). SentiStrength was developed specifically to capture sentiment of short, informal texts and has been used widely in research studies performing sentiment analysis of social media data (Calderon, 2015; Ferrara & Yang, 2015a).

The software operates by assigning two scores to each text it analyzes. Hence, it assigns both a negative and a positive score to each text, with negative score ranging between [ $-$ 1, $-$ 5] and positive score ranging between [1, 5]. A score of $-$ 1 indicates a weak negative sentiment while a score of $-$ 5 indicates a very strong negative sentiment. Similarly a score of 1 indicates a weak positive sentiment while a score of 5 indicates a very strong positive sentiment. Thus in order to classify a social media post as positive or negative, we assign a total sentiment score to each text by adding both positive and negative sentiment scores assigned by SentiStrength.

$\displaystyle\textit{sentiment}(\textit{total})=\textit{sentiment}(\textit{% positive})+\textit{sentiment}(\textit{negative})$

Thus, a total sentiment score of 4 for a text indicates a strong positive sentiment while a score of $-$ 4 represents a strong negative sentiment. Texts with score of 0 are classified as neutral.

The software has 60% accuracy when calculating positive sentiment while 72% accuracy when calculating negative sentiment of a given text (Thelwall et al., 2011; Thelwall et al., 2010).

4.3 Topic modeling

Topic modeling is a popular unsupervised machine learning technique in the domain of natural language processing. Topic modeling utilizes a statistical approach to detect abstract topics that can occur in a group of documents. We have utilized Latent Dirichlet Allocation (LDA) technique for topic modeling, which is an unsupervised generative probabilistic method of modeling text into topics. It represents topics by word probabilities, where the words with highest probabilities in each topic indicate topics in which corpus documents belong (Jelodar et al., 2019). LDA was first introduced by Blei, Ng and Jordan in 2003 and is one of the most popular methods for performing topic modeling over a corpus (Blei et al., 2003).

Various extensions of LDA have been proposed over time as well. Dynamic Topics Modeling (DTM) obtains evolution of topics over time for corpus of documents which are chronologically arranged (Blei & Lafferty, 2006). Similarly, Labeled-LDA (LLDA) is another extension, where the model can be trained with labeled documents making it an example of supervised machine learning (Ramage et al., 2009).

In this study however, we use the basic LDA technique for topic modeling. We organized social media posts of each organization in 2 documents, one containing Twitter messages while the other containing Facebook posts. Thus for the 9 organizations which are part of this study, a total of 18 documents were created. Python’s gensim library was utilized for topic modeling on these data files (Saxton, 2018).

5. Data analyses results

We now present the results of our data analysis. We present these results in terms of the research questions proposed in Section 3 of the paper.

5.1 Sentiment analysis of organizational communication

The first research question relates with sentiment analysis of organizational communication on social media. Figure 2 displays the sentiment of Facebook and Twitter posts of each department. We notice similar sentiment intensity for all organizations on both platforms.

We observe here that sentiment of messages by the three police departments is negative on both Facebook and Twitter. Majority of their messages are crime alerts and contained words like “wanted”, “robbery”, “alert” etc. Table 2 shows some typical tweets by the police departments analyzed in this study.

Table 1
Facebook and Twitter messages posted by each organization on average in a day. We can observe that some organizations are more active than others in utilizing social media. We can also see that every organization uses Twitter more frequently than Facebook

Organization	Daily FB posts	Daily tweets
Boston Police	1.85	4.80
Philadelphia Police	1.73	4.93
NY Police Dept.	1.12	10.22
NY Dept. of Motor Veh.	0.90	2.5
Penn Dept. of Motor Veh.	1.76	4.48
NY Energy	1.82	4.33
PPL Electric	1.76	4.13
NY Metro	0.90	228.57
SEPTA	1.26	103.23

Table 2

Example of a typical tweet by police departments. Last column displays overall text sentiment

Org.	Tweet text	Sent.
New York Police	“WANTED: Six male teens for assaulting a 14-year-old boy in front of 1251 Stebbins Ave #Bronx at 10:30pm on Sunday”	$-$ 2
Boston Police	“UPDATE: Victim Identified in Death Investigation in the Area of 1 Costello Circle in South Boston”	$-$ 2
Phila. Police	“Wanted: Suspect for Commercial Robbery in the 17th District”	$-$ 1

Figure 2.

Sentiment of Twitter and Facebook messages of public sector organizations.

5.2 Frequency analysis of organizational communication

The second research question looks at how frequently public organizations utilized Facebook and Twitter to communicate with citizens. Figure 3 shows the number of daily posts on both platforms by each organization. We observe that on Facebook, almost all organizations have a similar rate of posts per day. However on Twitter, we observe much variance as some organizations such as SEPTA and MTA, transportation authorities of Philadelphia and New York City respectively, tweet far more frequently than others. These organizations have to communicate regularly with their riders, keeping them up-to-date with any bus or train delays. Table 3 shows typical Twitter updates by these organizations.

Table 3
Example of a typical tweet by transport authorities of Philadelphia and New York

Org.	Tweet text	Sent.
SEPTA	“Norristown: Train #8266 going to Elm St is operating 10 minutes late. Last at Queen Lane.”	1.5 cm
MTA	“Northbound 6 trains are running with delays because of NYPD activity at 116 St”	1.5 cm

Table 4

Correlation matrix between direct tweets and tweets containing hashtag (#)

–	Direct messages	Tweets with #
Direct messages	1	$-$ 0.8137
Tweets with #	$-$ 0.8137	1

Figure 3.

Daily tweets and Facebook posts of public sector organizations.

5.3 Message framing and direct communication on Twitter

The third research question looks at how messages are framed and how much public organizations engaged in direct communication with citizens on Twitter. The objective here is to identify organizations that actively engage in direct conversations with citizens. This creates online dialogue through utilization of bi-directional nature of the micro-blogging website. Various studies have looked at the utilization of Twitter by public and non-profit organizations for two-way symmetrical communication (Lovejoy et al., 2012; Waters & Williams, 2011). To assess for dialogue and message framing, we look at all tweets by an organization that start with the expression “@” or those that contain a hashtag (#). Other studies have also utilized a similar approach to gauge interactive discussions on Twitter (Tumasjan et al., 2010). Figure 4 shows the percentage of tweets that are direct messages or contain a hashtag for each organization. Here we observe an inverse correlation between hashtags and direct messages in tweets by the organizations. Table 4 displays this negative correlation as statistically significant with a $P$ value of $<$ 0.01.

Figure 4.

Percentage of tweets that are direct messages or have hashtags for all organizations under study.

5.4 Topic modeling

Finally we perform topic modeling on the Twitter and Facebook messages of the nine organizations. Here we want to evaluate how these topics are similar or different from each other.

We first create word clouds of Facebook and Twitter messages of the organizations. Figure 5 shows the word clouds of the 3 police departments included in this study. We can observe here that words like Wanted, Suspect and Crime are present in all three word clouds. These words convey a negative sentiment and hence we observe in our analysis that these organizations have a negative message sentiment.

Figure 5.

Word clouds of social media messages of the three police departments.

We observe similar results during topic modeling of me sages by these organizations. Here, we compare topics created on messages by police departments with the topics created from social media messages from power providers. We can observe from Fig. 2 that police departments have negative sentiment messages while the power providers have the most positive sentiment messages.

We can observe from Fig. 6 that majority of the words included in the topics are positive in nature. From Fig. 7 we can observe topics generated from social media messages of the three police departments, NYPD, PPD and BPD respectively. We can observe words like arrest, robbery, suspect and crime in the topic for police organizations.

Figure 6.

Topic created from social media messages of PPL and NYPower compny repectively.

Figure 7.

Word clouds of social media messages of the three police departments.

6. Discussion

In this study, we proposed four research questions, analyses of which provide us with an insight on the nature of communication by public organizations on social media.

For our first research question, we analyze the sentiment of social media organizations and evaluate whether the organizational operations have an impact of the message sentiment. Analyses of messages on the two social media platforms show us that message sentiment is indeed influenced by nature of organization’s operations. We observe that sentiment of all three police departments is negative on both Facebook and Twitter while it is positive for other organizations.

Various researchers have looked at the use of social media by law enforcement agencies. A study into the use of Twitter by police departments in large U.S. cities showed that they primarily use Twitter to disseminate crime and incident related information (Heverin & Zach, 2010). Similarly, a study analyzing Twitter and Facebook posts of five U.S police departments observed that when police use Twitter to engage in self-promotion, the public responded by liking these posts significantly more than other categories such as broadcasting information and announcementsf (Williams et al., 2018). However the study highlights that there is lesser number of such ‘networking’ posts and highlights this as a missed opportunity by the police departments to effectively utilize social media. We think that a similar pattern is observable in our sentiment analysis of police department messages. Majority of social media messages on both Twitter and Facebook by police departments are crime related announcements and broadcasts as shown in Table 2. These posts have a negative sentiment due to the nature of message as they are crime related alerts. We also observe consistency in sentiment on both platforms, Facebook and Twitter, for almost all organizations (Fig. 2). This is caused by posting the same message, usually at the same time, on both Facebook and Twitter, to reach maximum audience. As the study into community policing agenda observed, engagement with public through self-promotion and networking would increase the effectiveness of social media usage by the police departments (Williams et al., 2018).

For the second research question, we looked at the communication frequency of organizations. Here we observe all organizations using Twitter far more often than Facebook. While the number of daily Facebook posts of all organizations is somewhat similar, there is a huge variance in daily Twitter messages, as shown in Fig. 3. MTA and SEPTA (transportation authorities of New York and Philadelphia respectively) are far more active on Twitter than Facebook as they tweet over 228 and 103 times a day on average respectively. This indicates the use of Twitter as a tactical tool by these organizations, where train and bus delays or other transport related disruptions were broadcast regularly to the users. Table 3 shows a typical Twitter alert by these organizations. Research studies evaluating social media usage of public transport organizations have shown similar results. A study of social media usage by transportation agencies of US discovered all 100% keeping a presence on Twitter with half of the agencies frequently use social media to provide transit system information and updates (Liu et al., 2016).

In this case too, nature of organization’s operations determine their level of engagement on social media. Organizations utilizing Twitter most frequently are both public transport providers as they frequently communicate any potential delays in buses or subways urgently to their riders. As time is a critical factor, Twitter enables them to engage with their riders quickly, allowing commuters to arrange for any alternate means of transportation if required. This also means that the utility of their messages are for a shorter duration of time. However like other studies evaluating social media communication of public transport agencies, we discover missed opportunities in terms of public engagement. It has been observed that less than a quarter of transportation agencies frequently use social media to provide transit-related livability or sustainability benefits (Liu et al., 2016). Highlighting benefits such of public transport in terms of reduced congestion and increased safety and positive environmental impact can be highlighted to the public using social media.

For the third research question, we did not observe any clear pattern in terms of utilization of hashtags (#) by organizations on Twitter. All organizations used hashtags some extent except for New York metropolitan area transportation authority (MTA). Similarly all organizations except SEPTA utilize direct messaging function of Twitter and communicate directly with users, responding to their concerns. Other studies evaluating usage of Twitter by public organizations for dialogue with citizens have shown mixed results. Some studies have claimed that most public organizations are utilizing Twitter primarily to broadcast updates and disseminate information rather than two-way symmetrical communication (Lovejoy et al., 2012, Waters & Williams, 2011). We did however observe an inverse correlation between hashtags and direct communication (Table 4). This shows us that either an organization is focused on using Twitter for broadcasting or direct one-to-one communication.

Finally, we perform topic modeling analysis of messages by the public organizations. Figure 7 shows the LDA generated topic for messages by the three police departments. From these topics we are able to evaluate main themes of discussion by the organizations allowing us to better understand their discussion topics.

We observe that police organizations discussed crime alters and hence had far more negative words in the word clouds and topics as shown in Figs 5 and 7 respectively. Hence topics of all three departments contain the word “robbery”, while at least two of the topics contain words like “crime” and “suspect”. We see a similar pattern in the word cloud of the three police departments which shows frequency of words in messages and is dominated by terms like “arrest”, “wanted”, “firearm” etc.

Findings from topic modeling relate with the discussion of our first research question where we observe most law enforcement agencies utilize social media primarily to disseminate crime and incident related information to public (Heverin & Zach, 2010; Williams et al., 2018). Hence this represents a missed opportunity in terms of public engagement and networking where law enforcement agencies can post more messages of self-promotion, enabling them to create a more positive image of themselves in public.

As a result of our topic modeling analysis, we can also state that the LDA based topic modeling did generate valuable information from the social media messages of public organizations. We were able to identify the major topics of discussion by these organizations providing us with insights on the nature of their operations and as a result, their online communication. Hence it is not surprising that the police departments had the highest negative sentiment of social media messages as shown in Fig. 2.

7. Conclusion

In this paper, we analyzed 14,049 Facebook and 28,800 Twitter messages by nine public organizations operating in Northeast United States. The primary purpose of this study was to analyze sentiment, content and topics of these messages to answer the proposed research questions.

We proposed four research questions and answer them through data analysis. For the first question, we observe an overall negative sentiment of messages by all law enforcement agencies included in our analysis. This is due to majority of their social media messages as crime related broadcasts. We recommend strategizing their communication by posting more messages of self-promotion and networking, as such messages will help enhance their public image positively. Hence we observe a missed opportunity for law enforcement agencies when it comes to utilizing social media for public communication.

For our second research question, we observe high frequency of Twitter messages by transit agencies. This enables them to quickly communicate transport related delays and emergencies with public, enabling riders to plan for any alternate means of transportation when necessary. However here too we discover few messages communicating benefits of public transport, in terms of enhanced quality of life, with citizens.

For third research question, we do not discover a clear pattern in utilization of hashtags and direct broadcast messages. However for our fourth research question, through topic modeling, we discover a high number of negative words in topics of law enforcement agencies. This highlights use of social media for primarily broadcasting crime related alerts, pointing to dearth of networking and self-promotional messages by these agencies.

We discover that organizations utilize these two platforms based on their requirements. We also discover that sentiment and frequency of messages is determined by the nature of organization’s operations. Finally through word clouds and topic modeling, we discovered the major themes of their social media messages.

Footnotes

Author’s biography

Ussama Yaqub is assistant professor in Suleman Dawood School of Business at Lahore University of Management Sciences, Pakistan. He received his PhD as a Fulbright scholar in the area of Management Science and Information Systems from Rutgers Business School, Newark, New Jersey. His research interests include analysis of Twitter data during election campaigns, social media sentiment analysis and use of internet for bi-directional communication by government sector organizations.

Soon Ae Chun is a professor of Information Systems and Informatics Program, and a doctoral faculty of Computer Science. She is the director of the Information Security Research and Education Lab. She received 2014 Dolphin Award for Outstanding Scholarly Achievement. She serves as the President for the Digital Government Society. Her research areas include Security and Privacy, Semantic data integration, Social Data Analytics and Workflow, focusing on applied areas in Digital Government and Public Health. Her work includes security ontology engineering, knowledge-based disaster and emergency management process and resource discovery, social media in government, policy-based service workflow generation, public engagement and transparency of policy decision making, Geospatial workflow generation, crowd-based environmental planning, and epidemic and public health analytics. Her research has been sponsored by NSF, NOAA, a New Jersey State Agency, and PSC-CUNY. She is an IEEE senior member, a member of ACM Computing Society, the Digital Government Society, and the Beta Gamma Sigma Honor Society.

Vijayalakshmi Atluri is a Professor of Computer Information Systems in the MSIS Department at Rutgers University. Her research interests include Digital Government, Information Security, Privacy, Databases, and Workflow Management. She has published over 150 papers in peer reviewed journals and conferences. In the past, she served as the Vice-chair for the ACM Special Interest Group on Security Audit and Control (SIGSAC), and the Chair of the IFIP WG11.3 Working Conference on Database Security. She was the recipient of the National Science Foundation CAREER Award, the Rutgers University Research Award for untenured faculty, and the 2014 IFIP WG11.3 Workgroup on Data and Application Security and Privacy outstanding research award.

Jaideep Vaidya is the Dean’s Research Professor in the Management Science and Information Systems department at Rutgers University. He received the B.E. degree in Computer Engineering from the University of Mumbai, the M.S. and Ph.D. degree in Computer Science from Purdue University. His general area of research is in data mining, data management, security, privacy, and digital government. He has published over 140 technical papers in peer-reviewed journals and conference proceedings, and has received several best paper awards. He is an ACM distinguished scientist and an IEEE senior member.

References

Anwar

Ilyas

Yaqub

, & Zaman

(2021). Analyzing qanon on twitter in context of us elections 2020: Analysis of user messages and profiles using vader and bert topic modeling. In DG. O2021: The 22nd Annual International Conference on Digital Government Research, pp. 82-88.

Anwar

, & Yaqub

(2020). Bot detection in twitter landscape using unsupervised learning. In The 21st Annual International Conference on Digital Government Research, pp. 329-330.

Bertot

J.C.

Jaeger

P.T.

, & Grimes

J.M.

(2010a). Using icts to create a culture of transparency: e-government and social media as openness and anti-corruption tools for societies. Government Information Quarterly, 27(3): 264-271.

Bertot

J.C.

Jaeger

P.T.

Munson

, & Glaisyer

(2010b). Social media technology and government transparency. Computer, 43(11): 53-59.

Blei

D.M.

, & Lafferty

J.D.

(2006). Dynamic topic models. In Proceedings of the 23rd International Conference on Machine Learning, pp. 113-120.

Blei

D.M.

A.Y.

, & Jordan

M.I.

(2003). Latent dirichlet allocation. the Journal of Machine Learning Research, 3: 993-1022.

Calderon

N.A.

Fisher

Hemsley

Ceskavich

Jansen

Marciano

, & Lemieux

V.L.

(2015). Mixed-initiative social media analytics at the world bank: Observations of citizen sentiment in twitter data to explore “trust” of political actors and state institutions and its relationship to social protest. In Big Data (Big Data), 2015 IEEE International Conference on, IEEE, pp. 1678-1687.

Coglianese

(2009). The transparency president? The obama administration and open government. Governance, 22(4): 529-544.

Collins

Hasan

, & Ukkusuri

S.V.

(2013). A novel transit rider satisfaction metric: rider sentiments measured from online social media data. Journal of Public Transportation, 16(2): 2.

10.

Cottrill

Gault

Yeboah

Nelson

J.D.

Anable

, & Budd

(2017). Tweeting transit: an examination of social media strategies for transport information management during a large event. Transportation Research Part C: Emerging Technologies, 77: 421-432.

11.

Davis

E.F.

Alves

A.A.

Sklansky

D.A.

, et al. (2014). Social media and police leadership: lessons from boston. Australasian Policing, 6(1): 10.

12.

De Vries

Gensler

, & Leeflang

P.S.

(2012). Popularity of brand posts on brand fan pages: an investigation of the effects of social media marketing. Journal of Interactive Marketing, 26(2): 83-91.

13.

Denef

Bayerl

P.S.

, & Kaptein

N.A.

(2013). Social media and the police: tweeting practices of british police forces during the august 2011 riots. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, pp. 3471-3480.

14.

Facepager (2018). Facepager. Accessed: 2018-04-01.

15.

Ferrara

, & Yang

(2015a). Measuring emotional contagion in social media. PloS One, 10(11): e0142390.

16.

Ferrara

, & Yang

(2015b). Quantifying the effect of sentiment on information diffusion in social media. PeerJ Computer Science, 1: e26.

17.

Gorodnichenko

Pham

, & Talavera

(2018). Social media, sentiment and public opinions: Evidence from# brexit and# uselection. Technical report, National Bureau of Economic Research.

18.

Grimmelikhuijsen

S.G.

, & Meijer

A.J.

(2015). Does twitter increase perceived police legitimacy? Public Administration Review, 75(4): 598-607.

19.

Guo

Vargo

C.J.

Pan

Ding

, & Ishwar

(2016). Big social data analytics in journalism and mass communication: comparing dictionary-based text analysis and unsupervised topic modeling. Journalism & Mass Communication Quarterly, 93(2): 332-359.

20.

Heverin

, & Zach

(2010). Twitter for city police department information sharing. Proceedings of the Association for Information Science and Technology, 47(1): 1-7.

21.

Hou

, & Lampe

(2015). Social media effectiveness for public engagement: Example of small nonprofits. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM, pp. 3107-3116.

22.

Huang

, & Hou

(2017). Examining twitter mentions between police agencies and public users through the lens of stakeholder theory. In Proceedings of the 18th Annual International Conference on Digital Government Research, ACM, pp. 30-38.

23.

Ilyas

S.H.W.

Soomro

Z.T.

Anwar

Shahzad

, & Yaqub

(2020). Analyzing brexit’s impact using sentiment analysis and topic modeling on twitter discussion. In The 21st Annual International Conference on Digital Government Research, pp. 1-6.

24.

Jelodar

Wang

Yuan

Feng

Jiang

, & Zhao

(2019). Latent dirichlet allocation (lda) and topic modeling: models, applications, a survey. Multimedia Tools and Applications, 78(11): 15169-15211.

25.

Kim

A.J.

, & Ko

(2012). Do social media marketing activities enhance customer equity? An empirical study of luxury fashion brand. Journal of Business Research, 65(10): 1480-1486.

26.

Lee

, & Kwak

Y.H.

(2012). An open government maturity model for social media-based public engagement. Government Information Quarterly, 29(4): 492-503.

27.

Liu

J.H.

Shi

Elrahman

O.S.

Ban

X.J.

, & Reilly

J.M.

(2016). Understanding social media program usage in public transit agencies. International Journal of Transportation Science and Technology, 5(2): 83-92.

28.

Lorenzi

Vaidya

Shafiq

Chun

Vegesna

Alzamil

Adam

Wainer

, & Atluri

(2014). Utilizing social media to improve local government responsiveness. In Proceedings of the 15th Annual International Conference on Digital Government Research, ACM, pp. 236-244.

29.

Lovejoy

Waters

R.D.

, & Saxton

G.D.

(2012). Engaging stakeholders through twitter: how nonprofit organizations are getting more out of 140 characters or less. Public Relations Review, 38(2): 313-318.

30.

Maier

Waldherr

Miltner

Wiedemann

Niekler

Keinert

Pfetsch

Heyer

Reber

Häussler

, et al. (2018). Applying lda topic modeling in communication research: toward a valid and reliable methodology. Communication Methods and Measures, 12(2-3): 93-118.

31.

Marwick

A.E.

, & Boyd

(2011). I tweet honestly, i tweet passionately: twitter users, context collapse, and the imagined audience. New Media & Society, 13(1): 114-133.

32.

Nikolaidou

, & Papaioannou

(2018). Utilizing social media in transport planning and public transit quality: survey of literature. Journal of Transportation Engineering, Part A: Systems, 144(4): 04018007.

33.

Ramage

Hall

Nallapati

, & Manning

C.D.

(2009). Labeled lda: A supervised topic model for credit attribution in multi-labeled corpora. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pp. 248-256.

34.

Rodriguez

M.Y.

, & Storer

(2020). A computational social science perspective on qualitative data exploration: using topic models for the descriptive analysis of social media data. Journal of Technology in Human Services, 38(1): 54-86.

35.

Saffer

A.J.

Sommerfeldt

E.J.

, & Taylor

(2013). The effects of organizational twitter interactivity on organization-public relationships. Public Relations Review, 39(3): 213-215.

36.

Sakaki

Okazaki

, & Matsuo

(2013). Tweet analysis for real-time event detection and earthquake reporting system development. IEEE Transactions on Knowledge and Data Engineering, 25(4): 919-931.

37.

Saxton

M.D.

(2018). A gentle introduction to topic modeling using python. Theological Librarianship, 11(1): 18-27.

38.

Schweitzer

(2014). Planning and social media: a case study of public transit and stigma on twitter. Journal of the American Planning Association, 80(3): 218-238.

39.

Sharma

Pabreja

Yaqub

Atluri

Chun

, & Vaidya

(2018). Web-based application for sentiment analysis of live tweets. In Proceedings of the 19th Annual International Conference on Digital Government Research: Governance in the Data Age, ACM, p. 120.

40.

Shi

Agarwal

Agrawal

Garg

, & Spoelstra

(2012). Predicting us primary elections with twitter. URL: http://snap.stanford.edu/social2012/papers/shi.pdf.

41.

Shwartz-Asher

Chun

, & Adam

N.R.

(2016). Social media user behavior analysis in e-government context. In Proceedings of the 17th International Digital Government Research Conference on Digital Government Research, ACM, pp. 39-48.

42.

Stats

(2018). Facebook statistics. Accessed: 2019-01-31.

43.

Stats

I.L.

(2017a). Sentistrength. Accessed: 2017-12-13.

44.

Stats

I.L.

(2017b). Twitter statistics. Accessed: 2019-01-31.

45.

Subba

, & Bui

(2017). Online convergence behavior, social media communications and crisis response: An empirical study of the 2015 nepal earthquake police twitter project. In Proceedings of the 50th Hawaii International Conference on System Sciences.

46.

Tang

Xiong

, & Zhu

(2015). Locating targets through mention in twitter. World Wide Web, 18(4): 1019-1049.

47.

Thelwall

Buckley

, & Paltoglou

(2011). Sentiment in twitter events. Journal of the Association for Information Science and Technology, 62(2): 406-418.

48.

Thelwall

Buckley

Paltoglou

Cai

, & Kappas

(2010). Sentiment strength detection in short informal text. Journal of the Association for Information Science and Technology, 61(12): 2544-2558.

49.

Tumasjan

Sprenger

T.O.

Sandner

P.G.

, & Welpe

I.M.

(2010). Predicting elections with twitter: what 140 characters reveal about political sentiment. ICWSM, 10(1): 178-185.

50.

Twitter (2021). Twitter developer. Accessed: 2021-03-11.

51.

Wang

, & Zhuang

(2017). Crisis information distribution on twitter: a content analysis of tweets during hurricane sandy. Natural Hazards, 89(1): 161-181.

52.

Waters

R.D.

, & Williams

J.M.

(2011). Squawking, tweeting, cooing, and hooting: analyzing the communication patterns of government agencies on twitter. Journal of Public Affairs, 11(4): 353-363.

53.

Williams

C.B.

Fedorowicz

Kavanaugh

Mentzer

Thatcher

J.B.

, & Xu

(2018). Leveraging social media to achieve a community policing agenda. Government Information Quarterly, 35(2): 210-222.

54.

Yaqub

Chun

S.A.

Atluri

, & Vaidya

(2017). Analysis of political discourse on twitter in the context of the 2016 us presidential elections. Government Information Quarterly, 34(4): 613-626.

55.

Yaqub

Sharma

Pabreja

Chun

S.A.

Atluri

, & Vaidya

(2020). Location-based sentiment analyses and visualization of twitter election data. Digital Government: Research and Practice, 1(2): 1-19.

56.

Young

J.A.

(2017). Facebook, twitter, and blogs: the adoption and utilization of social media in nonprofit human service organizations. Human Service Organizations: Management, Leadership & Governance, 41(1): 44-57.

57.

Yue

Chen

Zuo

, & Yin

(2019). A survey of sentiment analysis in social media. Knowledge and Information Systems, 1-47.

Analyzing social media messages of public sector organizations utilizing sentiment analysis and topic modeling

Abstract

Keywords

1. Introduction

3. Analysis of social media communication

3.1 Sentiment analysis of organizational communication

3.2 Frequency analysis of organizational communication

3.3 Message framing and direct communication on Twitter

3.4 Topic modeling of social media messages

4. Methodology

• Data collection and storage • Sentiment Analysis • Topic modeling 4.1 Data collection and storage

4.3 Topic modeling

5. Data analyses results

5.1 Sentiment analysis of organizational communication

Table 1 Facebook and Twitter messages posted by each organization on average in a day. We can observe that some organizations are more active than others in utilizing social media. We can also see that every organization uses Twitter more frequently than Facebook

Table 3 Example of a typical tweet by transport authorities of Philadelphia and New York

7. Conclusion

Footnotes

Author’s biography

References

•
Data collection and storage
•
Sentiment Analysis
•
Topic modeling

4.1 Data collection and storage

Table 1
Facebook and Twitter messages posted by each organization on average in a day. We can observe that some organizations are more active than others in utilizing social media. We can also see that every organization uses Twitter more frequently than Facebook

Table 3
Example of a typical tweet by transport authorities of Philadelphia and New York