Visual analysis of online social media to open up the investigation of stance phenomena

Stance and sentiment model

Stance is a topical area of interest in linguistics because the interactive nature of communication between individuals is considered vital. The function of taking stance in the communicative situation is to convey the speaker’s viewpoint of what is talked about and to regulate the exchange between the dialog partners. Communication here works on more than the pure understanding of words. Words are always understood in the light of the contexts and the situations where they are used.^2,3 In doing so, language is used to recontextualize human experiences into written and spoken forms. Its social role is to affect the state of mind of other people and to negotiate meanings in order to bring about cognitive changes.^4,5 Language users construe their expressions to communicate their particular perspective and viewpoint of what is talked about. As the following scheme ⁶ demonstrates, this process of taking stance is evaluative and fundamentally interactional, a type of ongoing negotiation:

Ours is a broad understanding of the process of taking stance, as it is critical to address the subtle but important differences in how people create discourse—imbuing it with their personal word choices as distinct acts of taking stance. This encompasses expressions of subjectivity, ranging from individual words to larger chunks of text. These items express speaker’s (1) sentiments, (2) attitudes, and (3) beliefs, covering meanings of certainty, volition, evidence, emotion, valence, degree, and so on. Following Du Bois, ⁷ we divide the process of taking stance into three parts: (1) speaker evaluation of what is talked about, (2) speaker positioning (epistemicity), and (3) alignment in communication, that is, establishment of agreement or disagreement. Stance has been studied under different headings and scope, such as evaluation,^8,9 sentiment, ¹⁰ and appraisal, ¹¹ and, of course, under the title stance itself.^5,12–14 Yet, at the present time, there is no conclusive and universally accepted definition of linguistic stance.

As stated above, subcategories of stance include sentiment, certainty or uncertainty, as well as other subcategories that are not well-defined yet. For this article, we have limited the scope of our understanding of stance to sentiment and certainty or uncertainty. These subcategories are generally considered to describe the feelings and assessments of an utterance; as such, they can encapsulate an evaluative statement that is deemed to be a stance act. Our approach is based on the expectation that the occurrences of such expressions lead to occurrences of other stance expressions—we denote the particular analyzed subcategories by stance types throughout this article to simplify the notation. From the computational perspective, this approach could be described as “multidimensional” sentiment analysis.

Sentiment analysis

From an operational point of view, stance includes phenomena such as subjectivity, sentiment, belief, trust, and uncertainty. Some of these phenomena, such as sentiment and subjectivity, have enjoyed considerable attention in the NLP community (for instance, see the works of Pang and Lee, ¹⁵ Liu, ¹⁶ and Lin et al. ¹⁷ ), while others, such as belief, trust, or uncertainty, have remained comparatively peripheral (but there is a number of efforts^18,19 to analyze uncertainty and speculation, respectively). Sentiment analysis in particular has become a staple in NLP, both in research and in commercial applications, with a large number of vendors offering solutions for social media monitoring where sentiment analysis is an important part of the analytics suite.

As with any research area that gains popularity in a research community, there has been a wide variety of approaches suggested in the literature. Examples range from simple keyword matching ²⁰ over standard machine learning techniques^15,21 to the use of topic modeling algorithms and latent variable models^22–24 to deep learning architectures.^25,26 State-of-the-art approaches to sentiment analysis now approach, and in some cases even exceed, 90% accuracy on standardized benchmark test suites.^21,27,28

Sentiment analysis is normally considered as a classification problem over two or three classes, where positive and negative define the basic polarity, and neutral is used to describe a lack of attitudinal content. From the perspective of stance analysis, this is a very simplistic ontology of emotions that is likely to be too restricted to be useful for analyzing and describing complex interpersonal processes of taking stance. Current research on sentiment analysis is beyond the standard positive–negative dichotomy and operates over a wider spectrum of emotions, such as Ekman’s ²⁹ six basic emotions (the so-called Big Six): anger, fear, happiness, surprise, disgust, and sadness, ³⁰ or some other multi-class taxonomy of sentiments.^25,31 Another example of a more complex sentiment palette is the RepTrak model used in the RepLab evaluation campaign that includes eight different categories designed specifically for reputation classification. ³²

As opposed to some of more complex approaches based on ML, we opt for a simplistic approach to sentiment classification for the purposes of the visualization tool in order to preserve transparency and simplicity. As previously noted, we have chosen to address stance through subcategories. More specifically in uVSAT, these are based on Ekman’s Big Six emotions, employing the NLP solution of simple lexical matching over lists of attitude terms (which we call stance markers as already mentioned in the “Introduction” section). The main goal at this stage of the project is to facilitate experiments to further improve our understanding of stance in general and our analysis techniques in particular. While our method of sentiment analysis is simple, such a lexical-based approach is still widely used by visualization and VA solutions,^33,34 especially the ones aiming for high performance when processing large amounts of input data. ³⁵ There are also several examples of combining both lexical-based and machine learning–based approaches for sentiment analysis that reports similar ³⁶ or even surprisingly good ³⁷ results when using the lexical approach.

Time-dependent text visualization

A good number of such works address temporal aspects to visualize events, topic competition or evolution, or other time-dependent data. While some of them introduce novel metaphors for visual encoding, multiple techniques combine well-known representations such as line plots, river metaphors, or animated force-directed graphs. Havre et al. ⁴² introduce ThemeRiver, the original technique for temporal data visualization based on a river metaphor that is designed to depict topic evolution in document collections. Dou et al. ⁴³ combine trees, text tags, and rivers in their HierarchicalTopics system to visualize the temporal evolution of topics in corpora. Xu et al. ⁴⁴ combine line plots, stacked charts, and word clouds to depict topic competition in social media document collections. To support the real-time monitoring of streaming Twitter data backed up with automatic text classification, Bosch et al. ⁴⁵ use timeline, word clouds, glyphs, and maps in the ScatterBlogs2 system. For the work in this article, we decided to choose simple visual representations (line plots, text tags, and bubble charts) for the data currently available to us, although we plan to design more specialized visual encodings for other tasks in the future.

Sentiment visualization

While specific problems (and the corresponding analysis techniques) such as topic modeling and event detection have been very popular in text visualization, the interest for sentiment analysis and visualization is also arising in the VA community. Liu et al. ⁴⁶ and Oelke et al. ⁴⁷ describe visualizations for opinion mining of reviews. Wanner et al., ⁴⁸ Cui et al., ⁴⁹ and Rohrdantz et al. ⁵⁰ present approaches for visual sentiment analysis that supports temporal data. Görg et al. ³⁷ describe the fluid integration of sentiment analysis as well as other computational text analyses with interactive visualizations in their system Jigsaw. Online social media data are used for visual sentiment analysis by Wanner et al., ⁵¹ Zhang et al., ⁵² and Hao et al. ⁵³ SentiView, introduced by Wang et al., ⁵⁴ not only facilitates temporal sentiment analysis but also augments it with relation analysis based on graph representation—this is relevant to our long-term research goals involving intersubjectivity and stance analysis. The recent work of Zhao et al. ³³ describes PEARL, a VA system for multidimensional personal emotion or sentiment visualization of Twitter posts over time, and uses an approach similar to ours (based on lexical matching of emotional words pertaining to eight emotion categories and three additional emotion dimensions)—however, our work focuses on the analysis and visualization of data related to multiple posters and sources, and we are interested in categories beyond emotions or sentiment. In general, most of the discussed works involve sentiment analysis as a means rather than the object of research. Our approach, in contrast to theirs, focuses on the analysis of sentiment to bootstrap the research on visual stance analysis. This leads us to the involvement of experts in linguistics as users and the discussion of existing visualization approaches related to the domain of linguistics.

Visualization for linguistic research

InfoVis and VA techniques have been used to facilitate tasks such as the analysis of corpora (e.g. Compus by Fekete and Dufournaud, ⁵⁵ CorpusSeparator by Correll et al., ⁵⁶ Text Variation Explorer by Siirtola et al., ⁵⁷ and those techniques proposed by Regan and Becker ⁵⁸ ), the analysis of relations or reuse (e.g. ShakerVis by Geng et al. ⁵⁹ and techniques proposed by Jänicke et al. ⁶⁰ ), and lexical analysis (e.g. the study by Rohrdantz et al. ⁶¹ ). An additional category of tasks that is worthy of mention is related to semantics: while numerous text visualization techniques use topic modeling, experts in computational linguistics use visualization to facilitate their research on this subject. For instance, Kabán and Girolami ⁶² visualize their own model of dynamically evolving text collections. Another task related to stance analysis is discourse analysis. Existing work on visualization of discourse includes the graph-based approach by Brandes and Corman, ⁶³ Conceptual Recurrence Plots by Angus et al., ⁶⁴ and several recent works that focus on discourse in online social media: Lingoscope by Diakopoulos et al. ⁶⁵ or ConVis by Hoque and Carenini. ⁶⁶

VA for sentiment research

Finally, the work that is most relevant to our approach in this article is dedicated to sentiment visualization which facilitates the research on sentiment for linguists. Gregory et al. ⁶⁷ conduct visual sentiment analysis of document collection with regard to affect bearing words. Their approach involves eight affect categories (positive, negative, virtue, vice, pleasure, pain, power cooperative, and power conflict) and uses IN-SPIRE for visualization purposes. The recent work of Makki et al. ⁶⁸ focuses on sentiment lexicon refinement from reviews dataset which involves user input via interactive visualization. Their sentiment analysis is based on a standard positive–negative dichotomy. The two major differences between these works and our proposed approach in uVSAT are the involvement of online social media text data (which is dynamic with regard to analysis sessions and available for temporal analysis) and the choice of sentiment categories (which is a base for the further analysis of stance).

To the best of our knowledge, the problem of stance analysis and visualization has not been addressed by work in VA or information visualization. Therefore, we would like to raise the awareness of the InfoVis and VA communities in this article by building on the discussed work in text visualization for sentiment analysis and existing work on visual text analytics for linguists.

System architecture and workflow

Figure 2 displays the overall architecture of uVSAT that is implemented as a web application. The back-end consists of a (visualization) server application implemented in Java that communicates with the Gavagai computing server, fetches the HTML content from URI links, processes the text data, and communicates the results in JSON format to the client(s). The front-end is implemented in JavaScript with D3 ⁶⁹ and Rickshaw ⁷⁰ libraries, and it only requires a modern web browser. While the major and cost-intensive computational analyses are processed by the Gavagai and visualization servers, several minor analyses (which do not require intense computations for large amounts of data) are implemented on the client side.

OPEN IN VIEWER

Figure 2.

The architecture of uVSAT comprises front-end and back-end tiers that communicate with external servers.

Data model

uVSAT has been designed to use time-series data from external providers through a RESTful API, ⁷¹ as well as to fetch and process corresponding HTML data from respective web servers. Currently, we use time-series data only from our collaboration partners at Gavagai (although we plan to support other data sources in the future). Gavagai analyzes text data from multiple sources, but for the purposes of the system presented in this article, they use the data fetched from various blogs and forums.

As mentioned in the “Background” section, we focus on the simplest possible type of stance analysis, that is, counting the occurrences of sentiment terms in documents that mention specific target terms. This simple approach allows our partners to support the analysis of large amounts of text data, up to 15 million documents per day. Here, a target can be anything of interest: a person, a brand, a company, a location, an event, or even something abstract such as a concept or an idea—as long as it can be defined by a set of keywords (also denoted by target terms in the context of our tool). Our present set of targets T includes the following

T = { diet , weapons , Hobbit , Coca - Cola , Pepsi }

To detect documents associated with stance, we consider specific markers relevant to sentiment and (un)certainty from several available sources (WordNet-Affect, ⁷² GeneralInquirer, ⁷³ and Compass DeRose ⁷⁴ ), while refining those marker lists is one of the purposes of uVSAT (since the sources above do not differentiate stance from sentiment, etc.). Our choice of analyzed stance types (also denoted by observers in the context of our tool) includes the Big Six emotions (see the “Background” section) as well as two other categories

O = { anger , joy / happiness , fear , sadness , disgust , surprise , certainty , uncertainty }

As an example, weapons is a monitored target which is defined by a list of 3771 keywords, harvested from the Wikipedia lists of weapons. ⁷⁵ Whenever one of these keywords is mentioned in open online media, the entire utterance containing the keyword is analyzed for occurrences of stance markers. Here, utterance is simply defined as a sequence of text defined by delimiter symbols, for instance, the text fragment

I am so sick of people who sell such rifles and so sick of people who buy this distasteful weapon.

contains two occurrences of the stance marker “sick of” and one occurrence of “distasteful,” generating a polarization value of 3 for the target weapons for observer disgust.

To summarize the description of n targets, m observers, and their possible combinations, we can describe the hierarchical structure of the data as {(T_i , {O _i1, …, O_ij })|1 ≤ i ≤ n, 1 ≤ j ≤ m} for targets T_i  ∈ T and the corresponding observers O_ik  ∈ O, for instance, (Hobbit, {disgust, anger, …}).

The occurrence counts are aggregated for each target–observer combination (T_i , O_ik )—for example, Hobbit/disgust or Hobbit/anger (note that we equivalently use the notations (T_i , O_ik ) and T_i /O_ik )—over a specific time frame which is presently set to 1 h. Thus, all occurrence counts for a specific stance type within this time frame [t ₁; t ₂] are summed, resulting in an hourly value v for each combination. These values are then retrieved and visualized by uVSAT as time-series. Because of this aggregation step (which is necessary to reduce the complexity and computational demands), the time-series data describe the general tendencies with regard to stance but do not directly provide any details about the distribution of specific markers. Therefore, further exploration of the original text documents is required from the users.

The Gavagai API also provides URIs to the documents used to calculate the polarization values (taking (T_i , O_ik , t ₁, t ₂) as arguments and returning sets of URIs), although the corresponding HTML content has to be downloaded and processed on our side. Unfortunately, the total amount of available data makes it infeasible for the VA tool to prefetch everything. Therefore, we limit ourselves to queries for specified sets of target–observer combinations across interactively selected time intervals (although we plan to support streaming data in the future).

Analysis challenges

We have designed uVSAT to facilitate users with answering the following questions:

Analytical tasks

These questions and problems can be mapped to the following categories of high-level (analytical) tasks:

In the following section, we discuss our visualization approach in detail, justify the design decisions, and refer back to the above-listed research questions and tasks.

Timeline view

The timeline view tab (cf. Figure 3) provides the users with the interfaces for exploring time-series data for selected targets or observers and specified time intervals. Note that fetching the input data to be analyzed—that is, the initial selection of specific targets, observers, and time ranges—from the Gavagai server is done via a simple dialog box as explained in our use case (cf. the corresponding section). In this section, we concentrate on overall design aspects including visual representation and interaction possibilities.

ROI highlighting

To facilitate the search for ROIs, our tool also supports automatic ROI highlighting (research question Q2). Currently, we use a basic ad hoc algorithm for marking the ROIs based on outlier or differential analysis. As a first step of the algorithm, time-series points x_i are marked, which differ significantly (with regard to threshold parameters θ ₁ and θ ₂) either from the mean value μ _x (standard deviation σ _x is used for comparison) or from the preceding point (judging by the first derivative x ′ i )

A = { x i : | x i − μ x | > θ 1 σ x ∨ | x ′ i | > θ 2 max j ( | x ′ j | ) }

Since the source time-series data are in general noisy, A will result in multiple regions of small size (comprising only one or several points). Therefore, in the second step, we smooth the results by marking neighboring points as parts of ROI, which will result in contiguous regions

ROI = A ∪ { x i : ( x i − 1 ∈ A ) ∨ ( x i + 1 ∈ A ) }

ROIs are highlighted by thick line segments (cf. Figure 3(b)). The algorithm parameters θ ₁ and θ ₂ can be adjusted by the user, which can be used to partially alleviate the problem of noisy data or to increase or reduce the number of highlighted regions to focus on.

Trend analysis

Users have several options of conducting trend analyses over selected time intervals for specified observers (cf. Figure 3(c)). uVSAT supports linear and quadratic time-series trend analysis based on polynomial regression (calculated with the ordinary least squares (OLS) method). We implemented two variations: one can choose to either render trends as overlay plots (cf. Figure 5(a)) or to substitute selected timeline plot segments with trend lines (cf. Figure 5(b)) to reduce the visual complexity of the displayed data (research questions Q1 and Q2). Trend lines are easily distinguishable by the use of dashed lines. Even information about the predicted value change at the current trend rate and a button for removing trend lines are available on hover.

OPEN IN VIEWER

Figure 5.

Trends can be displayed as either (a) overlay plots or (b) instead of original plot segments.

History diagram

Since the workflow of uVSAT involves multiple document view tabs that also may be closed by a user during the analysis process, the need for overview and control of such user actions arises. Our interactive history diagram (cf. Figure 3(d) and Figure 6) provides an overview of the document URI queries sequence, their results, and relations to each other (research questions Q6 and Q7).

OPEN IN VIEWER

Figure 6.

The history diagram allows users to keep track of document queries and navigate between interface states.

In this diagram that supports the so-called analysis provenance, ⁷⁸ nodes represent URI queries and edges represent the detected relations between corresponding query results (this partially resembles the visualization approach described by Cernea et al. ⁷⁹ ). The size of every node is proportional to the number of URI links retrieved for the corresponding query. Nodes are represented by glyphs similar to pie charts (although only qualitative information about relevant observers is used), following the same color coding of observers as the timeline plots. The currently selected node is highlighted in yellow. Since the diagram is used for history navigation, it also contains a dedicated node (depicted by a triangle) that represents the up-to-date interface state. Edges connect only nodes whose query results contain common subsets of URI links. The size of common subsets (i.e. Jaccard similarity of link sets ⁸⁰ ) is mapped to edge opacity, thickness, or both of these attributes (selected as a user setting). The layout of the history diagram is based on arc diagrams by Wattenberg: ⁸¹ nodes are simply aligned along a horizontal axis in the order of corresponding queries, and edges are rendered as curved arcs. We apply a random-order greedy heuristic described by He et al. ⁸² to decrease the number of edge crossings when allocating edges to the upper or lower part of the drawing.

The interactive history covers the following functionalities: every time a user issues a URI links query that leads to the creation of a new document view tab, the state of this new tab and the timeline view tab are saved and a corresponding node is added to the history diagram. When the user clicks on a history node, the timeline view tab state is restored, a document view tab with corresponding state is either created or brought into focus (if currently present), and the user actions temporarily stop affecting the history state (e.g. issuing a new query will not add the resulting state to history)—we have chosen such behavior to keep the history sequential. When the user clicks on the triangle, the previously saved up-to-date state is restored. Under circumstances, this can lead to some document view tabs getting closed.

Document views

A document view tab (cf. Figure 4) basically consists of two areas. The left (smaller) area provides information about all documents fetched based on the selection described at the end of subsection “Timeline view.” Thus, it shows the aforementioned line plot thumbnail used for the query as well as a link list (cf. Figure 4(a)) to HTML documents (blog posts, forum messages, etc.) that were marked as associated with a specific target–observer combination. Users can filter the list by URI domain and sort it by the timestamp value or by polarization value (as reported by the Gavagai server). Polarization values are also used for the color coding of list entries (research question Q3).

By selecting a link from the list, the corresponding document content is fetched, processed at the (visualization) server side, and rendered at the client side. If the content is not available at this time, the corresponding list entry is marked. The document data at this stage are raw HTML which affects the analysis. This is because the source code comments and metadata (such as keywords) often contain text irrelevant to the document content. To direct the user’s focus on textual document data, uVSAT renders the HTML content as plain text by using the Jericho library. ⁸³ All data and analysis results related to the single focus document are shown in the second area on the right-hand side of the document list. This area integrates four subviews: the current document view, the current document details view (not further discussed here), the document marker view, and the current document overview.

It should be noted that uVSAT also provides an opportunity to copy the query link for a given document view tab and to use it in later analysis sessions by opening a tab with identical contents (research question Q6).

Aggregation charts

While the techniques discussed above allow the users to analyze a selected document in detail and provide an indication of interesting documents (by polarization values), the document sets retrieved for certain queries may contain thousands of documents, and the users will benefit from a method that helps them to select documents that are interesting for further stance marker investigation (research question Q5). uVSAT addresses this problem with a technique that we call aggregation charts: it provides an informative overview and means of navigation for the current document set with regard to detected markers and observers (cf. Figures 7 and 8).

OPEN IN VIEWER

Figure 7.

Aggregation charts organized by observer allow users to explore the distribution of documents with respect to the corresponding observer.

OPEN IN VIEWER

Figure 8.

Aggregation charts organized by marker allow users to reverse the flow of analysis: they can concentrate on document distributions with regard to a specific interesting stance marker.

The visual representation is based on basic bubble charts described by Viégas et al. ⁸⁴ Every item in the chart represents a single document which corresponds to the target; the color coding is based on the nominal target values. A single item is visually represented by a glyph consisting of two nested circles. The size of the outer circle is proportional to the total number of corresponding stance markers detected in the document, and the size of the inner circle (filled with a more saturated color) is proportional to the number of unique marker types detected in the document. For instance, a document with 100 occurrences of a marker “good” and 100 occurrences of a marker “bad” has only two unique marker types: “good” and “bad.”

The aggregated data used for these charts can be organized in two ways: by observer and by stance marker. In the former case, a separate chart is visualized for each observer associated with the document set. In the latter case, one individual chart is visualized for each unique marker type (belonging to present observers) that has been detected in at least one document.

Figures 7 and 8 display examples of aggregation charts visualized for a document set based on 1517 URIs retrieved for the target–observer combinations Coca-Cola/joy, Hobbit/joy, and Hobbit/certainty. In Figure 7, the charts are organized by observer: the left chart contains items pertaining to both Coca-Cola and Hobbit; however, the right one does not contain items for Coca-Cola since no corresponding target–observer combination was available. This figure also shows the details for a chart item displayed on hover. An example of aggregation charts organized by stance markers is displayed in Figure 8. There are multiple charts sorted by the corresponding document numbers in decreasing order, and the user can browse these charts with a specific marker in mind. Details for the first chart (marker: “good”) are provided in a tooltip. Here, the currently selected document is highlighted (yellow) in all charts.

Aggregation charts facilitate the quick perception of the distribution of observers or stance markers in all documents, the identification of documents with a large number of stance markers or unique marker types, the navigation to such documents, and the analysis of document properties concerning other observers or stance markers (by brushing the corresponding chart item).

Marker and document export

One aim of our visualization tool is to identify and collect relevant stance markers from a larger number of analyzed documents (research question Q8). uVSAT supports the export of new stance markers from document view tabs by selecting a portion of text in the current document view (depicted in Figure 4(c)), assigning it with arbitrary tags, and exporting it to a JSON file. This approach allows us to collect a dataset of stance markers not restricted by the categories currently used for observers. Moreover, we are able not only to collect stance markers as short phrases (1-grams, ⁸⁵ 2-grams, or similar) but also to collect larger utterances which provide context for stance analysis.

Our tool also supports the export of currently viewed documents and aggregation charts as static HTML pages. In the former case, the document view with highlighted stance markers and target terms, document details, hierarchical markers view, and document overview (essentially, all the data pertaining to the current document on a document view tab) are exported. In the latter case, all aggregation charts that are currently available are exported together with the corresponding document set query (used observers, selected time interval, etc.). This feature allows users to store static data for further manual investigation or referencing, which can be especially helpful for researchers in linguistics.

Timeline data analysis

First, the researcher uses the Load data dialog box and selects all Coca-Cola observers for the time interval 30 January 2014 12:00—6 February 2014 12:00 CET in order to obtain a very broad return of data (cf. Figure 9). The time-series calculated for corresponding observers are loaded from Gavagai API.

OPEN IN VIEWER

Figure 9.

The dialog box used to select the time intervals and target–observer combinations to load time-series data. Note that there are additional observer types (frequency, positivity, and negativity) provided by Gavagai by default that are not associated with concrete stance markers (therefore, they are beyond the focus of our research).

By viewing the hierarchy and overview tabs (cf. Figure 10), the researcher verifies that all of the chosen observers have been loaded and confirms that there are sufficient data to be analyzed.

OPEN IN VIEWER

Figure 10.

Part of the timeline overview: the plots for observers are ordered by mean value in descending order, certainty being the first. Note the spike around 3 February, when the scandal occurred.

The researcher immediately notices the spike of activity on multiple plots around early hours of 3 February CET, which corresponds to the late evening of 2 February EST—the time when the advertisement was aired in the United States (aim A1).

Then, the researcher creates timeline plots by dragging-and-dropping the observer items onto the timeline view. Using the slider control, the researcher concentrates on the time span 3 February 2014 01:00–3 February 2014 19:00 CET. To confirm a conjecture that some of the observers have extremely low counts in the current time span (aim A1), the researcher filters them out. The remaining observers are certainty, joy, uncertainty, and anger (see Figure 11). To start analyzing the textual data, the researcher issues a request for corresponding URIs.

OPEN IN VIEWER

Figure 11.

Timeline view: four observers for target Coca-Cola that are used for detailed analysis are certainty, joy, uncertainty, and anger.

Identifying the document of interest

The resulting URI set comprises 3424 document links. While the researcher could explore this dataset manually, it would take a significant amount of time to achieve aim A2. At this point, the researcher decides to build the aggregation charts for the current document set and to investigate the charts organized by observer. For this, the text document data are fetched from respective web servers and processed by uVSAT.

The aggregation chart for anger (cf. Figure 12) comprises 1948 documents which in total contain 154 unique markers of anger. The researcher immediately identifies two candidate documents with the largest number of corresponding markers which are represented by glyphs with the largest diameters (also, with large shaded areas which means large number of unique marker types). By hovering on these glyphs, the researcher finds out that one of them contains 142 occurrences of anger markers (39 unique types) and another one contains 193 occurrences (41 unique types). The researcher selects the latter glyph by clicking and loads the corresponding document.

OPEN IN VIEWER

Figure 12.

The aggregation chart for anger provides an opportunity to identify the document with the largest number of corresponding stance marker occurrences. There seem to be two candidate documents which are represented by large glyphs (also with large shaded area). By hovering on these glyphs, the one with larger count of markers (in this case, 193 occurrences) is identified and later used for detailed analysis.

The loaded document of interest (depicted in Figure 13) is a blog post ⁸⁷ with a heated discussion in commentaries. To concentrate on the analysis of anger markers, the researcher filters out all markers of other observers. The current document overview plots at the bottom of the screenshot clearly show that the markers of anger, as well as the target terms of Coca-Cola, are evenly distributed throughout the entire document. To refine the analysis, the researcher needs to concentrate on specific markers.

OPEN IN VIEWER

Figure 13.

Document view for a selected document with majority of stance markers filtered out. Besides the Coca-Cola target terms, only the instances of all markers of anger are displayed.

Identifying the markers of anger

The aggregation charts for the current document set can be organized by stance marker instead of observer. The researcher selects this option and explores the resulting set of 605 aggregation charts (one per each unique stance marker type). Since the charts are ordered by marker occurrences number in descending order, the researcher quickly identifies several most frequent markers of anger, thus achieving aim A3 (see Table 1).

OPEN IN VIEWER

Table 1.

Stance markers of anger in the documents.

Marker	Corresponding documents	Unique markers in documents
Hate	579	123
Angry	347	113
Offended	265	92
Outrage	232	91
Fit	206	107

The most frequently used stance markers of anger in the document set related to the use case. These data have been discovered by investigating the details when hovering over aggregation charts’ labels.

Final document analysis

After identifying the most frequent markers of anger using the aggregation charts (here: “hate,”“angry,”“offended,” etc.), the researcher concentrates on the previously selected document and filters out all the other markers. It turns out that some of the identified markers are also among the most frequent markers of anger in the document as well (cf. Table 2).

OPEN IN VIEWER

Table 2.

Stance markers of anger in the selected document.

Marker	Occurrences in document	Rank in document
Hate	40	1
Offended	25	2
Angry	16	3
Outrage	4	8
Fit	3	9

The number of occurrences and ranks of the previously identified stance markers of anger in the current document.

The researcher reviews the current document overview once more (cf. Figure 14) and concludes that the identified markers are also distributed throughout this document. As the observer anger has the marker “hate” prolifically used, the analyst investigates further, addressing the linguistic characteristics that are employed by users who have posted these. The linguist now proceeds with a close analysis of the document giving critical attention to the markers “hate,”“offended,” and “angry,” thus achieving aim A4. The researcher’s conclusion is that the identified document is interesting for further manual linguistic analysis with regard to the flow of the conversation, and so on, as well as for preparation of an ML training dataset. By exporting the document from uVSAT, the linguist achieves aim A5.

OPEN IN VIEWER

Figure 14.

The overview for the previously selected document with only five marker types of anger displayed. Note that even after filtering the other anger markers (cf. Figure 13), numerous instances of these five marker types remain and they seem to be distributed throughout the whole document.

Summary

By using uVSAT, the researcher has been able to achieve his or her analysis aims, that is, exploring the data related to the case, analyzing the stance-related phenomena of anger and exporting the analyzed text data. By being able to interpret the ROIs on the timeline view, the researcher was able to limit a great amount of documents to an amount for a more detailed review. The tool’s ability to visualize multiple markers simultaneously in the document overview positively guided the investigation. By viewing the aggregation charts, the researcher’s decisions were visually supported, and he or she was able to draw the conclusions about stance phenomena in the dataset. The potential for employing these different refinement possibilities lets the researcher review statistical plots that are dynamic and updated as new postings are incorporated into the document view. The analysis features provided by the document view complements the manual stance analysis based on close reading. Overall, the patterns constructed by uVSAT create an ample opportunity for the researcher to employ user-based data en masse.

On a final note, the linguist began with one specific study area. After using uVSAT, the researcher concluded that the data have also revealed three other possible areas of interest: (1) directionality and frequency of the anger markers, that is, who the poster intends as the recipients and how often they appear and respond; (2) instances of how posters modify their use of anger, that is, intensifiers or attenuators; and (3) if anger is negated so as to create a positive meaning. The tool has provided several new potentials for future lines of research that could have gone unnoticed if traditional linguistic investigations were used.

Domain expert reviews

For the time being, our research partners at Lund University have been the primary users of uVSAT. They are familiar with standard tools for corpus analysis (e.g. AntConc, BYU-BNC, WORDSMITH, or Google Ngram Viewer) and manual text analysis. As a kind of project preparation, we introduced basic visualization concepts and techniques to them at the beginning of our collaboration. Their suggestions and feedback during the design and development stage of uVSAT are summarized in the following with regard to general analysis workflow, visualization and interaction techniques, and possible improvements for the tool.

Performance and scalability

In this subsection, we discuss certain aspects that affect the user experience when trying to apply uVSAT for the analysis of rather large datasets: data transmission delays, data processing delays, and user interface responsiveness.

We currently store neither time-series data nor document text data on our visualization server. Hence, uVSAT issues request for time-series data, URIs, and HTML content from external servers on demand. This leads to delays while retrieving the source data. Additional delays occur while transmitting the data between the front-end and back-end components and, finally, while processing the data at the server side.

We address the networking delay by conducting some types of analyses (such as ROI highlighting or trend computations) on the client side. It currently seems, although, that the performance bottleneck is the step of fetching the HTML content from numerous external servers which may have varying connection speed, performance, access frequency limitations, and even availability. We plan to introduce a local database for caching the external data (as well as some processing results), although it can lead to validity concerns (see subsection “Lessons learned”).

As for the UI responsiveness: D3 and Rickshaw use SVG for rendering which may require significant computational resources (and leads to UI lags). On a 2013 MacBook Pro computer with Intel Core i7 processor (2.3 GHz), sensible UI delays start to occur when re-rendering plots with a total of about 3000 points. This is partially addressed with a style of workflow involving preliminary analysis of time-series overview and focusing on selected time intervals.

Lessons learned

Our current visualization approach involves multiple coordinated views based on standard representations. Its main advantage (as opposed to a more complex integrated view) is the ease of user adoption: the primary users of our tool are researchers in linguistics who do not tolerate abundant details or unintuitive visual representations. The corresponding disadvantage, however, is the necessity of large display area to lay out all the views in sufficient size. We plan to address this issue in the future by developing novel visual representations for stance-related and time-dependent text data, having the domain particularities in mind.

The fact that our source data originate in online social media also has certain consequences: the text documents may be edited or deleted at any time. This presents us with a trade-off between data validity and performance. By fetching online data on user’s demand (as uVSAT currently does), every document is analyzed in its up-to-date state (or it is marked as unavailable), but it requires computational resources (and it is also related to inevitable networking delays). Otherwise, if the data are cached while the original data are modified, it would invalidate the detailed analysis of document contents. To address this issue, we plan to involve uncertainty tackling techniques. Another possibility would involve storing the versioned source documents—while in practice, it would require significant resources, in theory, it could provide an analysis opportunity with regard to additional temporal dimension.