Re-spatialization of time series plots

Abstract

Visualizing time series is a ubiquitous aspect of many applications from science to business. The conventional designs of line graphs and temporal parallel coordinates plots focus on the depiction of the temporal context, but typically fail to convey the spatial information associated with data values at each time step. In this paper, we present a study of the design space for reintroducing spatial context to time series plots. We consider a range of options, from color encoding to glyph encoding, from abstract illustration to realistic representation, and from interactive brushing to animated three-dimensional viewing. We evaluate these design options in conjunction with two case studies: (i) observing events in temporal sensory data of seafloor hydrothermal plumes and (ii) observing phonemic dynamics in poems. We report domain experts’ comparative assessment, which provides an insight into the effectiveness of different re-spatialization methods.

Keywords

Re-spatialization visual mapping time series visualization parallel coordinates spatiotemporal data

Introduction

In many applications, datasets feature attribute values in both spatial and temporal contexts. The visual designs for depicting such data primarily fall into three categories.

Time-centric designs^1–6 focus on the variation of attribute values in the corresponding temporal context. Typically time is mapped to one of the spatial dimensions of the display. Line graphs and temporal parallel coordinates exemplify such a design strategy.

Space-centric designs ^7–11 focus on the variation of attribute values in the corresponding spatial context. Typically time is depicted using animation rather than any spatial bandwidth of the display. For instance, several designs for flow visualization (e.g. heatmap, hedgehogs, line integral convolution) depict attribute values and vector properties in each time step of a simulation.

Spatiotemporal designs ^12–17 allow for the depiction of both spatial and temporal contexts on a display without using animation. Typically, the three or four spatiotemporal dimensions associated with the data are projected onto the two spatial dimensions of the display, while attribute values are depicted using localized visual channels such as multivariate glyphs. Visualizing object tracks, usually based on an orthogonal projection along the temporal dimension, exemplifies this design strategy.

The choice of a design strategy is largely application-and/or data-dependent. In some cases, the critical requirement for visualizing temporal variations may override the occasional desire for perceiving the spatial context. In other cases, space-centric or spatiotemporal designs may not be effective in separating different time series because of limited spatial resolution and movement. Multiple coordinated views¹⁸ can provide a linked representation of space and time through two or more views, but can sometimes suffer from higher cognitive demand on users due to context switching when switching focus between views.¹⁹ We wanted to investigate complementary designs for re-introducing the spatial context in-place, such that the user can reason about both space and time in the same plot. This paper focuses on design strategy (A), that is, time-centric designs, and presents a study of the design space for re-introducing spatial context to time series plots, such as line graphs and temporal parallel coordinates plots. We refer to the collection of methods for such a purpose as re-spatialization, distinguishing it from the term spatialization that also encompasses the spatial mapping of non-spatial attributes.

In order to strike a balance between applicability and generality, we conducted this design study in conjunction with two different case studies. One is a classical scientific application, where oceanographers need to observe patterns captured in time-varying volume datasets. The other is a humanities application, where poetry scholars need to observe phonetic flow patterns in poems. The datasets in these two case studies exhibit very different spatial characteristics, and thereby lead to different requirements for re-spatialization (see Section ‘Definitions and requirements’). In both cases, the above-mentioned design strategies (B) and (C) have been explored, but found ineffective (see Sections ‘Case Study I: Visualizing events in temporal plume data’ and ‘Case Study II: Visualizing temporal phonemic patterns in poems’). While initial attempts to deploy strategy (A) showed promising signs of observable temporal patterns, users indicated a noticeable amount of difficulty in understanding the corresponding spatial context at different time steps.

Figure 1(a) shows 27 spatially associated time series summarizing a hydrothermal plume system with normalized scattering strength at $3 \times 3 \times 3$ grid points and are computed from the corresponding time-varying volume dataset. Although one can easily observe both that different time series vary differently over time and that there are diverse conjoint patterns, gaining a sense of spatial context at each time step requires a mental mapping from the identifier of each time series to its spatial position in the $3 \times 3 \times 3$ grid. Animating isosurface images (i.e. design strategy (B)), as illustrated by snapshots in Figure 1(b), has been found ineffective for analytical tasks requiring comparisons of temporally distant frames or observation of quantitative distinctions.^20,21Figure 2 illustrates five different re-spatialization methods, namely color encoding, glyph encoding, selection-based exploration, abstract spatial context (ASC), and realistic spatial context (RSC), which are to be detailed in Section ‘Re-spatialization methods’. We report domain experts’ evaluation in Section ‘Case Study I: Visualizing events in temporal plume data’.

Figure 1.

(a) Time-centric design: the activities of a hydrothermal plume system are represented by 27 time series of normalized scattering strength on a $3 \times 3 \times 3$ grid. (Note: zero-values appear missing due to the use of a log axis.) (b) Space-centric design: snapshots of the isosurfaces of the same plume system at specific time steps (8 time steps).

Figure 2.

Spatial information can be added into a time series plot using different methods. The visualization shows 27 spatially related time series in the same context as Figure 1. While a temporal parallel coordinates plot allows users to observe scalar and temporal variations of different time series over time, its original form cannot convey the spatial context of each time series effectively. Five different methods for re-introducing such spatial contexts are examined in this paper. They are (a) color encoding, (b) glyph encoding, (c) selection-based exploration, (d) abstract spatial context, and (e) realistic spatial context. (Note: The jigsaw icons indicate missing time steps in the original data, for example in time steps 18, 41, and 47.).

Poetry scholars are interested in observing the “flow of sound” among different poetic lines in relation to the geometric features of each phoneme as illustrated in Figure 3(a). Glyph-based visualization (i.e. design strategy (C)) as shown in Figure 3(b) has enabled scholars to examine a single poetic line or compare two–three lines, but becomes ineffective when there are more than three lines. As shown in Figure 3(c), one may deploy a temporal parallel coordinates plot, where a path traces the vowels across each poetic line in a poem. However, the lack of spatial context in such a plot makes it difficult to perceive “flow of sound” in relation to the mouth geometry. In this study, we applied five re-spatialization methods, analogically equivalent to those in Figure 2. We report domain experts’ evaluation in Section ‘Case Study II: Visualizing temporal phonemic patterns in poems’.

Figure 3.

(a) A version of the international phonetic alphabet²² vowel chart. The vowel chart shows three variables: frontness—(from left to right) back, central, and front; height—(from top to bottom) close, mid, and open; and roundedness—all rounded vowels are shown in circles. (b) Glyph-based visualization of the first three lines of “A Drinking Song” by W. B. Yeats depicts the geometric features of each phoneme, with two images on the top illustrating the glyph representation of a vowel transition from close back rounded to close front unrounded. (c) Two temporal parallel coordinates plots show the frontness and vowel height attributes of the first six lines of “A Drinking Song”.

Spatialization is a generic methodology in visualization for mapping non-spatial attributes (e.g. weight, accuracy, and risk) to spatial variables (e.g. length, size, and location). Re-spatialization is different in two ways. (i) There is usually not enough screen space for displaying the original spatial information in its native form, because of, for instance, the use of a spatial dimension for time. (ii) The original spatial information in its native form can serve as the ground truth reference and expectation, while some re-spatialization methods may incur extra effort of familiarizing. In the literature, there is not yet much previous work on this topic. The main contribution of this study is to provide a systematic comparison of five different methods of re-spatialization through two distinct application case studies. Our design and implementation can provide future research on this topic with a set of baseline options, while the evaluation by the domain experts involved can offer insight as to the effectiveness of each method.

Related work

In this section, we give a brief overview of the related work according to the three design strategies described in Section ‘Introduction’. There are numerous visual designs of spatiotemporal visualization, and understandably this section can only cover a small number of these designs. Readers are encouraged to consult several surveys^23–28 and web sites (e.g. http://survey.timeviz.net/ and http://spacetimecubevis.com/) for a comprehensive coverage. As most visual designs for depicting spatiotemporal data involve “overloading” the display space, they naturally fall into different categories of visual multiplexing.²⁹

Time-centric designs

With this design strategy, the temporal dimension is given the principal attention in the visualization, while the spatial dimensions play a secondary role. It typically employs a conventional time series visualization (e.g. line plots and calendar view) as the base representation augmented with spatial information in abstract forms or available on demand.

For example, when one visualizes time-varying variables (e.g. GDP, population, or volume of traffic) associated with different locations (e.g. countries or cities), one may simply display the variables using polyline representations of time series data, and the locations in a textual or iconic form (e.g. names or flags). E. J. Marey’s 1880 train schedule visualization, featured on the dust jacket of Tufte’s book,³⁰ exemplifies such an approach.

This approach of mapping spatial information to a non-spatial visual channel or a display dimension that has not been utilized for conveying temporal variation usually incurs a loss of spatial information and often introduces difficulties in spatial awareness and reasoning. On the contrary, there are many examples of spatializing non-spatial data in conjunction with a time-centric visualization, such as documents and baby names in ThemeRiver,^1,2 and glyph-based event annotation along a time line.³ This suggests that spatialization is in general beneficial, while the absence of intrinsic spatial mapping of spatial data can be problematic.

Boyandin et al.⁴ annotated a pixel-based time series with spatial information at the beginning and the end. In time-varying volume visualization, it is common to use a time-centric representation as an interface for users to select a specific time step and to visualize the corresponding volumetric spatial data. For example, Lu and Shen⁵ used a temporal storyboard, and Lu et al.⁶ used statistics graphics as such an interface for exploring time-varying volume data.

In this paper, we study the time-centric strategy in a structured manner by exploring five different re-spatialization methods and two applications. Unlike the aforementioned works that tended to focus on one method and one application, we hope to obtain a more objective comparison, while maintaining sufficient relevance to applications.

Space-centric designs

With this design strategy, the spatial dimensions of the data are given the principal attention, while the temporal dimension of data has to utilize the display bandwidth spared by visual objects for depicting spatial information. C. J. Minard’s visualization of the fate of Napoleon’s army in Russia³⁰ is one of the most notable examples of this strategy.

Many visual designs in this family use a spatial visualization (e.g. a map) as the background and overlay temporal information (e.g. tracking information) on top of the background. Joshi and Rheingans⁷ used illustrative metaphors, such as speedlines and strobe silhouettes, to convey temporal information. Duffy et al.⁸ used glyphs to represent motion characteristics of cells along their tracks. Bak et al.⁹ used “growth ring” glyphs to depict temporal changes in sensor logs.

Another approach commonly used is to organize a sequence of spatial visualizations along a time line¹⁰ or combine them into an animation. Walton et al.¹¹ extracted moving vehicles from videos and projected them onto a map as temporal information in a spatial context.

Spatiotemporal designs

This design strategy attempts to give a similar level of attention to both space and time. It typically compromises the intrinsic depiction of both types of information with some abstraction, omission, or distortion. Hägerstrand’s space-time cube¹² exemplifies such an intention while featuring some perceptual distortion because of three-dimensional (3D)-to-two-dimensional (2D) projection. Many contemporary designs^13,14 adopted this approach.

Daniel and Chen¹⁵ distorted the temporal dimension of a video-cube, while displaying only spatial data featuring noticeable changes. Botchen et al.¹⁶ distorted a selected sequence of video frames as a spatial context, while depicting critical spatial data along a time line. Drocourt et al. ¹⁷ introduced an abstraction by mapping 2D spatial dimensions to one dimension while mapping the temporal dimensions to concentric rings.

Definitions and requirements

Consider a time series of multivariate data records, $T_{id} = {R_{1}, R_{2}, \dots, R_{t}, \dots R_{n}}$ , where $T$ is a temporally ordered set, and subscript $id$ defines a reference that connects all data records in the set. Each data record $R_{t}$ is associated with a stamp of time or temporal order t, and consists of data values captured at or computed for t. To simplify the notation, we assume that values in $R_{t}$ are organized using a flat structure, and all data records are defined based on the same set of variables. This simplification does not undermine the generality since data records defined with a hierarchical structure can be flattened and those based on different variable sets can be unified using the union of all variable sets. $R_{t}$ can thus be written as $〈 v_{1}, v_{2}, \dots, v_{i}, \dots, v_{m} 〉$ .

In many applications, some values in $R_{t}$ are spatial positions, which are typically difficult to convey in traditional time series plots if there is more than one spatial dimension. To differentiate such spatial positions from other attribute values, we separate them into two subsets $p_{t}$ and $a_{t}$ , that is

R_{t} = 〈 p_{t}, a_{t} 〉 = 〈 v_{1}, v_{2}, \dots, v_{i}, \dots, v_{m} 〉

This paper focuses on conveying $p_{t}$ for different time series displayed in a shared line graph or parallel coordinates plot. Given the above notation, there are two major classes of multivariate time series, depending on whether or not $p_{t}$ changes within the same $T_{id}$ . For example, in a typical time-varying volume dataset, each voxel defines a time series, for which $p_{t}$ does not change. If one displays values of different voxels in a line graph or a parallel coordinates plot, each voxel identifier implicitly defines its spatial position. Nevertheless, as shown in Figure 1(a), it is still difficult for users to “visualize” where each voxel is located based on the voxel identifier. On the other hand, if one tracks a feature (e.g. the center of a vortex) inside the volume, $p_{t}$ changes over the time, and has to be explicitly defined in each $R_{t}$ . When one visualizes multiple features (e.g. plotting velocity vs. time), it is often more challenging to convey $p_{t}$ in such a plot. The problem of visualizing phonemic features in poems as shown in Figure 3(a) falls into this latter class, because when phonemes change along a poetic line, the corresponding positional features in the mouth vary accordingly.

It is helpful to distinguish these two classes as the requirements for re-spatialization are likely to be different. Given a multivariate time series, $T_{id}$ , if $p_{t}$ is mapped to an equivalent visual representation (e.g. a screen location) for all t, this is said to be a stable spatial mapping. Otherwise, it is an unstable spatial mapping.

For the class of stable spatial mapping, the users are usually aware of the correspondence between the spatial location and the identifier of each $T_{a}$ , but may find remembering such correspondence cognitively demanding. Hence the typical requirements for some types of data analysis may include the following questions:

remind me where this time series $T_{id}$ is;

remind me how $T_{a}$ and $T_{b}$ are spatially related, as they exhibit similarity (or deviation) in attribute values;

show me an overview of the attribute values at t in the corresponding spatial context.

On the other hand, for the class of unstable spatial mapping, the users do not know where each $T_{id}$ is at a specific time t, not to mention remembering the spatial context at t. In addition to the above requirements for stable spatial mappings, the typical requirements may also include answers to the following questions.

Where is time series $T_{id}$ now at t?

How has $T_{id}$ changed spatially between $t_{1}$ and $t_{2}$ ?

Are $T_{a}$ and $T_{b}$ spatially co-located at t, since they exhibit some similarity (or deviation) in their attribute values at t?

Does the spatiotemporal pattern in $T_{a}$ between $t_{1}$ and $t_{2}$ also occur in $T_{b}$ , $T_{c}$ , etc.?

What is the spatial distribution of different time series at t, and are there any interesting patterns (e.g. clusters, symmetry)?

Re-spatialization methods

The goal of re-spatialization is to provide visual support to the requirements discussed in the previous section. The main constraint is that we do not have a dedicated screen dimension (e.g. x or y-axis) for encoding spatial data $p_{t}$ in different time series $T_{id}$ . Naturally, there are many design options, such as mapping spatial data to non-spatial visual channels, utilizing unused space, or superimposing a pop-up window for conventional spatial mapping. In this work, we examine five different re-spatialization techniques for helping users observe or recall the spatial context in a time series plot.

Color encoding: mapping positions $p_{t}$ to colors, and associating them with a $T_{id}$ .

Glyph encoding: mapping positions $p_{t}$ to specially designed glyphs, and associating them with a $T_{id}$ .

Selection-based: organizing different positions $p_{t}$ into an ordered list (e.g. using an axis), and using interaction (e.g. brushing) to explore different positions in the list and to highlight those time series or parts of a time series corresponding to the selected positions.

ASC: depicting a spatial context using an abstract representation of a set of positions (and associated data).

RSC: depicting a spatial context using a more realistic representation of a set of positions (and associated data).

The most common approach to re-spatialization is to display a corresponding space-centric visualization at a selected time t.³¹ This is referred to as RSC.

Time-centric visualizations (e.g. line plots) often feature several or tens of time series. In such a situation, the re-spatialization can focus on an ASC at a lower spatial resolution. The two examples as shown in Figures 1 and 3 exhibit this characteristic.

In Figures 1(a) and (b), the original volumetric resolution is simply too fine and too noisy for observing and discovering signature patterns for different events. Hence a $3 \times 3 \times 3$ low resolution volume was derived from the original volume data set at each time t, yielding 27 time series. The choice of size $3 \times 3 \times 3$ in this application was determined by the domain expert involved. This resolution is expected to show most common types of phenomena being analyzed in the plume data (e.g. the plume bending). The general applicability of this technique to other types of volume data would depend upon the task required by the domain expert. Meanwhile, phonemic positions obtained from text are constrained by the International Phonetic Alphabet (IPA) standard translation. In general, the level of resolution that is to be displayed is application-dependent, and small variations of resolution are trivial to implement. A low resolution spatial context offers the ease to distinguish between different locations and to remember them, while its encoding requires a small amount of bandwidth of a visual channel. Hence we refer to such low resolution spatial context as ASC.

Given a small set of spatial positions, such as the $3 \times 3 \times 3$ grid points of a low resolution volume, we can potentially encode them using a non-spatial visual channel. In this work, we considered using color encoding and glyph encoding to represent a small set of spatial positions.

Last but not least, interaction facilitates spatial exploration. If one can select a specific spatial range to make an observation, and select another range for another observation, gradually one can build a spatial context. We refer to this method as selection-based exploration.

Figures 4(b)–(d) illustrate three re-spatialization methods: color encoding, glyph encoding, and selection-based exploration. Each method can be used either on its own, or in conjunction with others in an integrated manner. Questions that naturally arise may include the following.

Can users interpret those colors in Figure 4(b), or glyphs in Figure 4(c), as spatial positions?

How easy is it for users to build a spatial context from partial views, for example, Figure 4(d), selected interactively?

Figure 4.

Comparative illustration of the basic time series plot (a) and three re-spatialization methods: (b) color encoding, (c) glyph encoding, and (d) selection-based exploration.

Figure 5 and Figures 6(a)–(e) show two sets of examples of applying these five methods in our case studies, which will be detailed in Sections ‘Case Study I: Visualizing events in temporal plume data’ and ‘Case Study II: Visualizing temporal phonemic patterns in poems’, where we will also answer these questions.

Figure 5.

The five visual designs that have been implemented in the Plume Explorer software for Case Study I: (a) color encoding; (b) glyph encoding; (c) brushing; (d) abstract spatial context; (e) realistic spatial context.

Figure 6.

The five different re-spatialization designs considered in the humanities application are (above: zoomed in view and below: standard view) (a) color encoding, (b) glyph encoding, (c) selection-based exploration, (d) abstract spatial context, and (e) realistic spatial context. (f) A visualization of a short poem “April Rain Song” by Langston Hughes using an integrated design space of color encoding, glyph encoding, and RSC. A path (from left to right along the x-axis) represents a line in a poem. The first vertical axis on the left is used to mark the line index. The remainder of the vertical axes represent the roundedness attribute (i.e. the shape of the lips during the production of a vowel), where each of these axes is divided into two parts: rounded at the upper half and unrounded at the lower half.

Designing an abstract spatial context

The methods of ASC and RSC are designed to be activated for a specific time step $t_{i}$ upon request by the user. Our goal is to bring the spatial context into focus and integrate this additional visual information into the time series at $t_{i}$ , in order to convey the spatial context. We considered many designs for such an abstract representation, and Figure 7 illustrates the six different designs that were considered. Four of them are 2D designs with a $3 \times 3$ grid and two are 3D designs with a $3 \times 3 \times 3$ grid.

Figure 7.

Six ASC design options to be integrated into a time series. (a)–(d) 2D designs with a $3 \times 3$ grid. (e)–(f) 3D designs with a $3 \times 3 \times 3$ grid.

When an axis is to be spatially enriched, it expands to a rectangle with the same axis duplicated at both sides. Lines of different time series enter the re-spatialization rectangle from the left, connect with their associated spatial positions in the rectangle, and leave the rectangle on the right to become the normal time series again. The approach is similar to that of Yuan et al.³² who use multidimensional scaling on two or more axes to display connected scatterplots between axes, except that our approach favors augmenting the user’s ability to recall spatial position and relate it to the axis values.

Spatial relation for categorical axes

In time series visualization, when two or more lines share the same attribute value at a time step, tracking them can be a problem (as shown in Figure 8(a)). The problem becomes worse with cluttering and occlusion, for example, when lines have to be annotated by glyphs or have to merge into a point within a re-spatialization rectangle for depicting abstract or RSC. When the axes concerned are categorical axes with a small number of category values, we can relax the path of the time series when they pass through these axes. For each value on a categorical axis, we define a “relaxation” region, and spread the passing lines within the region. To maintain some form of ordering, for each passing line, the original position p is moved to a new position that is a function of p and its line index as shown in Figure 8(b).

Figure 8.

(a) Intersecting points with no relaxation. (b) Intersecting points with controlled relaxation.

Case Study I: Visualizing events in temporal plume data

Our first case study represents a scientific application where time series plots are used to visualize four-dimensional (4D) hydrothermal plume data. A hydrothermal plume is primarily characterized by the rapid vertical discharge of high-temperature fluid through particle-rich sulfide chimneys along the axis of mid-oceanic ridges.^33–35 The process is driven by volcanic activity heating groundwater flow under the surface. Hydrothermal plumes exhibit complex temporal behavior, rising through surrounding water until they become denser than the water, at which point the plume begins to expand laterally. These fluxes and interactions with the ocean are studied in order to further scientific understanding of how the plumes may be distorted by different types of currents and geologic events.

Data collection and analysis

In addition to scientific discovery, analyzing the data informs researchers of the data quality and correctness. In this study, we utilize data from the COVIS (Cabled Observatory Vent Imaging System) project^36,37 in which data is collected from a sonar platform placed near targeted plumes. The multi-beam sonar measures returning backscatter resulting from both the turbulent fluctuations and the concentration of metallic sulfide particles within the plume. Sonar scans are performed at regular intervals of around three hours.

For each dataset representing a time step, post-processing is performed in Matlab to form a volumetric dataset which can be further processed or directly volume rendered. The former methods allow for temporal analysis of the plume’s behavior, such as its bending properties which have been the subject of much scientific interest,³³ whilst the latter are useful for judging the plume’s overall shape.³⁸ Assessing the quality and correctness of the datasets is difficult due to its spatiotemporal nature and the uncertainty of the capture method: the sonar platform often produces incomplete or corrupted scans due to network errors, disk space issues, or instrument failure. In addition, the analysis of longer time periods is becoming more commonplace.

Data visualization and exploration

In this case study, the temporal volume data belongs to the stable mapping class, where each voxel defines a time series in which the voxel position does not change. We wish to provide a tool that will allow researchers to explore the changing properties of the plume’s spatial distribution over time, whilst re-introducing the spatial context back into a more traditional time series visualization. As with Figure 1(a), each volume dataset is resampled into a $3 \times 3 \times 3$ grid by taking the mean of the plume’s normalized backscatter values within each block. We favored this approach over more advanced aggregation methods to focus on the effectiveness of the visualization’s ability to represent bending artefacts caused by changes in density at spatial positions. This aggregation allows not only for the plume’s large-scale spatial distribution over time to be analyzed for interesting properties (such as bending), but also provides an effective summarization of the plume’s data quality.

We begin by plotting each of the plume’s positions in a line plot, as in Figure 1(a). Providing an effective visual mapping for 27 voxel positions in a line plot is difficult as one quickly runs out of visual channels (color, stipple, markers, etc.) in which to distinguish each line. In addition, we require a visual mapping that (i) does not cause occlusion problems; (ii) does not cause problems for those with color perception issues; and (iii) is memorable and/or analogous to a spatial position in $3^{3}$ space such that associating each line with its corresponding voxel is not cognitively demanding.

Our five re-spatialization methods discussed in ‘Re-spatialization methods’ are mapped to five visual design options in an integrated web-based application called Plume Explorer (see Figure 2). This time series plot has unique features which we now discuss in relation to the re-spatialization methods.

Using line color and glyphs

Our first two approaches, as seen in Figure 5(a) and (b), map $3 \times 3$ slices through the y-axis to three distinct visual line properties. Such a mapping effectively separates the bottom, middle, and top sections of the plume for independent analysis of each section’s unique characteristics. We provide two visual mappings: line color and line glyph. For each voxel p belonging to a block of $3 \times 3$ voxels, the line color approach maps p to a color representing that block, with a user-customizable color scheme. In comparison, the glyph approach uses specially designed glyphs to denote the bottom, middle, and top sections (shown in Figure 5(b)), which are placed where each p intersects a time axis. To reduce occlusion, we constrained each type of glyph to every third axis.

Axis brushing for spatial axes

Brushing is a standard and widely-used technique in parallel coordinates plots for visually filtering data matching a particular range or intersection of ranges. By introducing X, Y, and Z spatial axes into our time series plot (always shown as the first three axes), we allow the user to brush location as well as data value (see Figure 5(c)). Selecting data value is achieved by brushing on the t-axes, and is useful for identifying spatial areas of the plume that are within a certain value range or exhibit particular temporal patterns of flow. Being able to brush on spatial location achieves the opposite: allowing the user to inspect particular spatial regions of the data along the time axis by brushing on each of the X, Y, and Z axes. Their intersection represents a region in space.

Abstract spatial context

The ASC approach allows the user to reintroduce the context of each p at any point along the time series. In the Plume Explorer application, this is achieved by clicking the icon above the axis. Upon re-spatializing an axis, the axis is split into two and laterally expands to show the abstract representation of the $3 \times 3 \times 3$ space. This ASC is visually represented by a $3^{3}$ grid of small spheres (see Figure 5(d)). We chose to render lit spheres to provide a valuable visual cue of the three-dimensional nature of the scene. In a re-spatialized axis, the data lines emerge from the left, connect to their associated spatial location by intersecting a sphere, and join back to the remainder of the plot on the right. This technique brings back to the visualization the spatial context of the data surrounding this time step.

The ASC was carefully designed to enhance its effectiveness. We use Catmull-Rom splines to perform the connection between axes and spheres as we found that the gradual nature of the curves improves the users’ ability to follow the lines. Each sphere is Phong shaded to provide visual cue as to its shape. The most apparent feature however is that the ASC performs an animated sinusoidal “wiggle” through its y-axis to take advantage of the motion parallax effect (structure-from-motion) to provide the user with a sense of depth. As well as the additional information that is brought by individual animation frames, the sense of depth provides a “visual decluttering” effect,³⁹ allowing the user to locate visually each line’s trajectory and sphere intersection in 3D space. The speed and rotation degree of animation are user-customizable.

During development, our domain expert expressed difficulty in using the single X, Y, and Z axes to reason about 3D areas of the data. Being able to brush directly on the ASC is likely to be more intuitive for those unused to reasoning about data where the dimensions are separated. Plume Explorer fully supports brushing operations on the ASC, as shown in Figure 9. The user first hovers over the ASC with the mouse to “freeze” the axis. Clicking an individual voxel sphere toggles its selection in the brush state, and the brushed sphere gains a yellow halo to indicate that it has been selected. A selection can also be dragged over the spheres to select groups at a time. The ASC and all other axes dynamically update to reflect the brushing operations as they occur.

Figure 9.

Brushing on a re-spatialized axis.

Realistic spatial context

The RSC approach provides the user with a realistic representation of the data context. By hovering over the icon, the user interface provides a Matlab pre-rendered isosurface of the plume at that point in time. The pre-rendered isosurfaces are generated nightly from the data directory and are uploaded into the online tool. Currently, typical representations of the plume data use multiple isosurfaces or volume renderings.^37,38 The main imagery provided in serving the data utilizes multiple isosurfaces combined with another surface representing the seafloor.^37,40 Although only using a single isosurface, the RSC provides a readily available means of enabling the user to access such a spatial context.

Expert evaluation

The evaluation was carried out in a qualitative manner, structured around a questionnaire, a copy of which is included in the supplementary materials. Qualitative expert feedback⁴¹ was used in our evaluation as observing seafloor hydrothermal plumes requires extensive semantic knowledge of 4D hydrothermal plume data that arbitrary participants are unlikely to possess. The questionnaire consists of 11 main questions. Among them, 10 focus on pairwise comparison of the five designs, and one on the integrated design as shown in Figure 2. The questions are then followed by a freeform discussion. The dataset that was used in the evaluation was from the COVIS project.^34,37 The dataset was chosen as it possessed bending properties and contained gaps between the time steps due to corrupted packet data.

We evaluated the five re-spatialization designs, together with an integrated design, by consulting nine domain experts (one of whom is a co-author of the paper). All of the participants have good knowledge about seafloor hydrothermal plumes, and appreciate the need for and difficulties in observing temporal volume data captured by observatories such as Neptune Canada. As scientists working with hydrothermal plumes have diverse research interests and focuses, we did expect that their evaluation would be heavily influenced by personal knowledge and experience. All of the participants have prior data analysis experience, and are familiar with a Matlab pre-rendered isosurface of the plume since this is their primary method of qualitative analysis at present. Each of the evaluation sessions lasted approximately an hour.

The evaluation began with a brief introduction. The introduction included an explanation on how the data is being represented at each time step, followed by a brief description of the five re-spatialization designs. After the introduction, each of the participants was then asked to fill in a questionnaire consisting of 11 main questions. Questions 1–10 represent a pairwise comparison of the five re-spatialization designs. The pairwise comparison was then followed by an integrated design question where the participants were asked to interact with the integrated web-based application Plume Explorer.

Eight domain experts completed the questionnaire. The results of the ten questions for pairwise comparison are summarized in Figure 10. Their comments on the relative merits of each design are as shown in Figure 11. One domain expert preferred to provide feedback without using the questionnaire and rated brushing and RSC positively.

Figure 10.

Eight scientists compared pairs of designs. Each color square represents one person preferring one design over another. A filled square corresponds to a row, and hollow to a column. Grey squares indicate don’t know answers.

Figure 11.

The comments from the eight scientists on the relative merits of the five re-spatialization techniques for visualizing plume data. Those comparative comments agreed by all are underlined, otherwise in italic.

While considering Questions 1–10, several domain experts suggested using different designs in an integrated manner. In answering Question 11, all experts agreed that integrated design was better than individual designs. One expert explained:

The realistic one connects with my prior understanding of what is happening, the abstract and the colored lines help me tease out the temporal pattern and see what parts are changing and what parts are not.

ASC was considered good for conveying local spatial context, while color and glyph encoding could convey spatial changes in time. As one of the experts described:

Having the spatial context (cube) and then selecting individual lines by clicking on the spheres was the easiest way for me to explore individual voxels and understand the spatial context. Being able to select the “Red, Green, Blue” colour scheme was also helpful - this made the lines stand out a lot better than the “underwater” colour scheme.

One expert found that opening two ASCs at the same time was helpful. Experts also found that wiggling helped spatial perception. They suggested implementing similar capabilities for RSC. They also suggested conducting further research into the glyph design to explore its potential, introducing a legend window for lines, and showing how lines related to the isosurfaces.

Case Study II: Visualizing temporal phonemic patterns in poems

This case study was built on early work of poem visualization in support of close reading.⁴² Poetry scholars who worked with Poem Viewer—a web-based poem visualization tool—have found that visualizing locations and degrees of sonic turbulence helps them begin to capture poetic qualities and characteristics (such as relative intensity and dynamism), which contribute to literary experience and inform critical interpretation. This glyph-based visualization has helped scholars observe where similar mouth actions generate distinct but related phonemic sounds; still, these scholars wish to have more effective means to observe the presence and patterns of a poem’s unique ‘sound flow’.

For example, scholars wish to look not only at where and how specific sounds appear in a poem but also at whether phonemic positions in a given poem remain in a specific region of the mouth, evolve slowly from one region to another, or shift more often and abruptly. Since such patterns of sound flow are spatiotemporal, and the phonemic positions may change over time, the problem belongs to the class of unstable spatial mapping (see Section ‘Definitions and requirements’). It is hence particularly challenging for scholars to build mental models for such patterns by observing a space-centric visualization (e.g. Figure 3(b)) or a time-centric visualization (e.g. Figure 3(c)).

Phoneticians have utilized a variety of spatial representations to study the differences between vocalic systems, ranging from tongue position illustrations to x-ray videos depicting muscular motion.⁴³ In this work, we used the IPA,²² to cluster 30 vowel phonemes into $3 \times 3$ groups. As depicted by the boxes in Figure 3(a), the nine vowel position groups are: (left to right) back, central, and front, and (from top to bottom) close, mid, and open. The frontness or backness dimension describes which part of the tongue is raised, while the openness and closedness dimension (commonly referred to as vowel height) describes how far the tongue is raised. The third attribute, vowel roundedness or lip rounding, describes the shape of the lips when a vowel is produced. In Figure 3(a) rounded vowels are indicated by a circle surrounding the corresponding phonetic characters. For studying sound flow in poems, meaningful insights can be derived from such a 2D representation. In this paper, we focus on the visualization of vowel phonemes, though our re-spatialization methods are easily adaptable to consonants. With the increasing use of medical imaging modalities for recording the movements of speech organs, the re-spatialization methods in Case Study I in Section ‘Case Study I: Visualizing events in temporal plume data’ can potentially be deployed for such data.

Visual designs

Figure 6 shows the application of five re-spatialization methods to a temporal parallel coordinates plot of a poem. The poem in the example is “April Rain Song” by Langston Hughes. Each line of the poem is shown as a path, from left to right, intersecting with vertical axes. The first vertical axis on the left indicates the different lines in a poem. The remainder of the vertical axes represent the roundedness attribute. Each of these axes is divided into two parts, rounded vowels at the upper half and unrounded at the lower half. Spatial relaxation is used in order to reduce cluttering and allow for spatial variations at junction points. To make the two roundedness categories more distinguishable, each axis is split into two sections. Although we also have spatiotemporal visual designs that map one spatial dimension (i.e. either frontness or vowel height) to the vertical axes, and use re-spatialization for the other, the five designs discussed below allow us to focus on evaluating re-spatialization methods for time-centric visual designs.

Color encoding

Figure 6(a) shows a partial view of this re-spatialization method. Each of the nine vowel position groups is encoded with a different color (selected using ColorBrewer⁴⁴). As a quick aid to learning and memorization, a small legend is displayed at the top of all vertical axes except the first axis for line number. A possible advantage of this design may be its simplicity. One possible disadvantage may be the difficulty in learning and remembering the mapping between spatial positions and colors.

Glyph encoding

In this design, each of the nine vowel position groups is represented by a glyph, as shown in Figure 6(b). The glyphs utilize two visual channels to encode the different positions. The vowel height is mapped to a color channel, that is, red for close, gray for mid, and blue for open. The frontness is mapped to a combined shape-orientation channel, that is, back is shown as a semicircle on the left, central a circle on its own, and front a semicircle on the right. Similar to the color encoding, a small legend is shown on top of each axis. To make the frontness dimension more observable, we shifted the glyph for a back position slightly leftward and that for a front position slightly rightward. The possible advantages of this design may include the ease to learn and remember, to separate the two dimensions, and to distinguish the three values in each dimension.

Selection-based exploration

In this design, users can select the different vowel positions that they wish to view using a set of nine tick-boxes. The positions that have been selected are encoded using dark gray and those unselected with light gray. Figure 6(c) shows an example of using this design where only the closed back and open front vowel positions have been selected. By interacting with the different vowel positions, users may be able to build a mental model of the spatial context and explore the different spatial relationships between different lines.

Abstract spatial context

In this design, each vertical axis (except the first one) can be expanded to display the abstract representation of the nine position groups. As illustrated in Figure 6(d) each path is connected to the corresponding vowel position in the expanded illustration, where lines are also relaxed to maintain traceability. The possible advantage of this design may be due to its on-demand nature as a user could easily examine where concentrations of phonemic positions occur in a poem.

Realistic spatial context

In this design, each vertical axis (except the first one) can be expanded to display a more detailed representation of the tongue geometries for all the vowel sounds on the axis. The illustrations depict cardinal tongue positions, and are drawn based on x-ray images in the literature.⁴³ The different curves, representing the top of the tongue, also implicitly suggest how different sounds travel above these curves through the mouth. The acoustic structures of the vowels are determined by the position of the tongue in the mouth.⁴³ We used line thickness to indicate the frequencies of the vowel positions, that is, the thicker the line is the more times the vowel positions are repeated at the corresponding temporal step. A possible advantage with this design is that it provides extra information that is not available in the ASC.

Expert evaluation

Similar to the study in Section ‘Expert evaluation’ above, this evaluation was also carried out in a qualitative manner, structured around a questionnaire featuring 11 main questions and freeform discussions (see the supplementary materials). Among the questions, 10 of the questions were focused on pairwise comparisons and one on an implementation where four designs are integrated together (excluding color encoding as it is mutually exclusive with glyph encoding in this case).

Observing sound flow along different lines in poems is a highly specialized visualization task. We thus conducted our evaluation of the five re-spatialization designs, together with an integrated design, by consulting seven humanities scholars (two of whom are co-authors). All have studied poetry, and have experience in close reading. The poem that was used in the evaluation was “A Drinking Song” by W. B. Yeats. Each of the evaluation sessions lasted approximately an hour.

The evaluation session began with a brief introduction. The introduction included an explanation of the IPA vowel chart that was used in the study and its representation, followed by a brief description of the five re-spatialization designs. After the introduction, the participants were then asked to fill in the questionnaire consisting of 11 main questions. Similar to the questionnaire in Section ‘Expert evaluation’ above, Questions 1–10 are designed as a pairwise comparison of the five re-spatialization designs and Question 11 is an integrated design question where the participants were asked to explore the integrated design using an implemented web-based prototype.

Five domain experts completed the questionnaire. The results of the 10 questions for pairwise comparison are summarized in Figure 12. Their comments on the relative merits of each design are as shown in Figure 13. Two domain experts (the 6th and 7th) gave feedback without using the questionnaire. One indicated a preference towards color and glyph encoding, and another preferred glyph encoding.

Figure 12.

Five humanities scholars compared pairs of designs. Each color square represents one person preferring one design over another. A filled square corresponds to a row, and hollow to a column. Grey squares indicate don’t know answers.

Figure 13.

The comments from the five humanities scholars on the relative merits of the five re-spatialization techniques in poem visualization. Those comparative comments agreed by all are underlined, otherwise in italic.

While considering Questions 1–10, several domain experts suggested different ways of combining visual designs. In answering Question 11, all experts agreed that the integrated design was better than individual designs, and liked the flexibility in choosing different designs for different tasks. One suggested that brushing should always be available. Another expert thought the integrated design would provide both an orienting overall view of the poem (through glyph encoding) and the flexibility to focus explorations (via brushing). A number of suggestions for improving the visual designs were made, ranging from using a color wheel metaphor, to changing the contrast of lines during brushing.

Discussions and conclusions

In this work, we studied a challenging problem in spatiotemporal data visualization of how to reintroduce spatial context back into time-centric designs. We examined five design options in conjunction with two application case studies. Based on the evaluation by domain experts who understand the requirements of visualization tasks, we can make the following observations.

The integrated design is much better than individual designs. It provides different levels of support to different tasks, for example, discovering the local spatial context at t vs. observing spatial changes over a period $[t_{1}, t_{2}]$ . We did not observe any user difficulties in using more than one re-spatialization mechanism.

Domain experts suggested that providing further information to accompany the RSC may be required as it would allow them to relate to the data using their existing knowledge. However, doing so may not necessarily provide support for quantitative analysis.

Domain experts are able to relate more to the ASC in 2D than in 3D. The possible reasons may include that 3D spatial perception is more difficult, and glyph encodings are less effective in 3D. The wiggle motion, which received positive feedback, can help to improve 3D perception.

Brushing (selection-based exploration) is considered important by both sets of domain experts, though the humanities scholars appreciated it more. This may be due in part to poetry scholars’ greater desire for flexibility in exploration—a desire facilitated by the comparably smaller data size of poems, which allows for more time to interact with the visualizations. In comparison with RSC, the humanities scholars felt that the spatial context is more clearly related to the poetic context.

Glyph encoding is much appreciated in 2D but not at all in 3D. When combined with brushing, it could potentially offer a good practical solution to many 2D applications. For example, in Case Study II, glyph encoding was able to convey a sense of gradation and relationship to nearby spatial positions, that is, horizontally, vertically, and diagonally.

Color encoding received fairly positive reviews in 2D in terms of spatial context, but was scored lower than glyph encoding due to the difficulties in learning and memorization.

In addition to the above observations the domain experts also found that for the plume study, the ASC provided the best quantitative analysis support since it helped them to recall the spatial context with more ease. They however found that RSC provided the best qualitative analysis support since it provided a level of familiarity with their existing knowledge of isosurface renderings of the data. For the poem data, the humanities scholars found that both glyphs and color encoding helped them to make additional observations as it enabled them to relate to the poem’s phonetic positions spatially. Using RSC on the other hand, they found it difficult to relate to the individual words in the poems. For both the plume and the poem data, brushing was appreciated as it allowed for filtering operations that are essential for analysis.

Like most expert-based evaluations in application case studies, these observations may be specific to the applications, tasks and users concerned. Nevertheless, they do offer a useful template for visualization designers to map out various design options and to anticipate users’ requirements and feedback.

This work demonstrates that the re-spatialization of time series plots is feasible under the condition that the spatial context is of a relatively low resolution. The scalability with regards to the number of time steps is additionally a factor in its usability, which needs to weigh the analytical and interactive functions afforded by our plot against the simplicity of a basic time series plot. As we can see from the two application case studies, low resolution is often desirable either because high-resolution spatial context is too complex and noisy to comprehend (e.g. Section ‘Case Study I: Visualizing events in temporal plume data’), or because data for high-resolution spatial contexts are not available (e.g. Section ‘Case Study II: Visualizing temporal phonemic patterns in poems’). Of course, one should not generalize the findings of this work to that in arbitrarily high resolutions or large datasets, which is a much more difficult challenge. In future work, we wish to explore the design spaces for the other two design strategies mentioned in Section ‘Introduction’ as well as their suitability at various data sizes.

Footnotes

Acknowledgements

The authors wish to thank all those who participated in the evaluation. They are humanities scholars from University of Oxford: Iain Emsley, Alexander Huber, Laura Ludtke, David Robey, and Natasha Ryan; and COVIS data experts: Ben Biffard, Maia Hoeberechts, Darrell Jackson, Reyna Jenkins, Mike Kenney, Russ Light, Li Liu, Sedat Ozer, Jay Takle, and Guangyu Xu.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Havre

Whitney

Nowell

ThemeRiver: Visualizing thematic changes in large document collections. IEEE Trans Vis Comput Graph 2002; 8: 9–20.

Wattenberg

. Baby names, visualization, and social data analysis. In: Proc. IEEE symp. information vis. 23–25 Oct 2005, Minneapolis, MN. 2005, pp.1–7.

Legg

Chung

Parry

et al . MatchPad: Interactive glyph-based visualization for real-time sports performance analysis. Comput Graphics Forum 2012; 31: 1255–1264.

Boyandin

Bertini

Bak

et al . Flowstrates: An approach for visual exploration of temporal origin-destination data. Comput Graphics Forum 2011; 30: 971–980.

Shen

. Interactive storyboard for overall time-varying data visualization. In: Proceedings of the IEEE pacific visualization, 5–7 March 2008, pp.143–150. Kyoto, Japan: IEEE.

Chen

Ribarsky

et al . Year-long time-varying 3D air quality data visualization. In: Advances in Information and Intelligent Systems (eds Zbigniew

William

), Berlin, Heidelberg, 2009, pp.289–306. Berlin, Heidelberg: Springer.

Joshi

Rheingans

. Illustration-inspired techniques for visualizing time-varying data. In: Proceedings of the IEEE Visualization 23–28 October 2005, pp.679–686. Minneapolis, MN: IEEE.

Duffy

Thiyagalingam

Walton

et al . Glyph-based video visualization for semen analysis. IEEE Trans Vis Comput Graph 2015; 21: 980–993.

Bak

Mansmann

Janetzko

et al . Spatiotemporal analysis of sensor logs using growth ring maps. IEEE Trans Vis Comput Graph 2009; 15: 913–920.

10.

Andrienko

. Interactive visual tools to explore spatio-temporal variation. In: Proceedings of the Working Conference on Advanced Visual Interfaces, Lecce, Italy, 25–28 May, 2004, pp.417–420.

11.

Walton

Berger

Ebert

et al . Vehicle object retargeting from dynamic traffic videos for real-time visualisation. Visual Computer 2014; 30: 493–505.

12.

Hägerstrand

What about people in regional science?

Pap Reg Sci 1970; 24: 6–12.

13.

Kapler

Wright

. GeoTime information visualization. In: Proceedings of the IEEE Symp. Information Visualization, 10–12 October, pp.25–32.

14.

Forlines

Wittenburg

. Wakame: Sense making of multi-dimensional spatial-temporal data. In: Proceedings of the Int. conf. advanced visual interfaces, Rome, Italy, 25–29 May 2010, pp.33–40.

15.

Daniel

Chen

. Video visualization. In: Proceedings of the IEEE visualization 2003, pp.409–416.

16.

Botchen

Bachthaler

Schick

et al . Action-based multifield video visualization. IEEE Trans Vis Comput Graph 2008; 14: 885–899.

17.

Drocourt

Borgo

Scharrer

et al . Temporal visualization of boundary-based geo-information using radial projection. Comput Graphics Forum 2011; 30: 981–990.

18.

Roberts

. State of the art: Coordinated & multiple views in exploratory visualization. In: Proceedings of the fifth int. conf. coordinated and multiple views in exploratory visualization Zurich, Switzerland, 2–2 July 2007, pp.61–71.

19.

Wang Baldonado

Woodruff

Kuchinsky

. Guidelines for using multiple views in information visualization. In: Proceedings of the working conference on advanced visual interfaces 2000, pp.110–119.

20.

Fisher

. Animation for visualization: Opportunities and drawbacks. Beautiful visualization. Farnham, Beijing: O’Reilly Media, 2010, pp.329–352.

21.

Robertson

Fernandez

Fisher

et al . Effectiveness of animation in trend visualization. IEEE Trans Vis Comput Graph 2008; 14: 1325–1332.

22.

International Phonetic Association. Handbook of the international phonetic association. Cambridge: Cambridge University Press, 1999.

23.

Andrienko

Gatalsky

Exploratory spatio-temporal visualization: an analytical review. J Visual Lang Comput 2003; 14: 503–541.

24.

Aigner

Miksch

Muller

et al . Visual methods for analyzing time-oriented data. IEEE Trans Vis Comput Graph 2008; 14: 47–60.

25.

Andrienko

Demsar

et al . Space, time and visual analytics. Int J Geogr Inf Sci 2010; 24: 1577–1600.

26.

Zhong

Wang

Zeng

et al . Spatiotemporal visualisation: A survey and outlook. In: Arisona

Aschwanden

Halatsch

et al . (eds) Digital urban modeling and simulation 2012. Berlin, Heidelberg: Springer, pp.299–317.

27.

Andrienko

Visual analytics of movement: An overview of methods, tools and procedures. Information Visualization 2013; 12(1): 3–24.

28.

Bach

Dragicevic

Archambault

et al . A review of temporal data visualizations based on space-time cube operations. In: EuroVis State of the Art Report, Swansea, UK, 9–13 June 2014, pp.23–41.

29.

Chen

Walton

Berger

et al . Visual multiplexing. Comput Graphics Forum 2014; 33: 241–250.

30.

Tufte

ER.

The visual display of quantitative information. 2nd ed.Cheshire, Conn.: Graphics Press, 2001.

31.

Höllt

Magdy

Zhan

et al . Ovis: A framework for visual analysis of ocean forecast ensembles. IEEE Trans Vis Comput Graph 2014; 20: 1114–1126.

32.

Yuan

Guo

Xiao

et al . Scattering points in parallel coordinates. IEEE Trans Vis Comput Graph 2009; 15: 1001–1008.

33.

Rona

Bemis

Jones

et al . Entrainment and bending in a major hydrothermal plume, Main Endeavour Field, Juan de Fuca Ridge. Geophys Res Lett 2006; 33: 971–980.

34.

Di Iorio

Lavelle

Rona

et al . Measurements and models of heat flux and plumes from hydrothermal discharges near the deep seafloor. Oceanography 2012; 25: 168–179.

35.

Jackson

Bemis

et al . Observations of the volume flux of a seafloor hydrothermal plume using an acoustic imaging sonar. Geochem Geophys Geosyst 2013; 14: 2369–2382.

36.

Ocean Networks Canada. Data Archive. University of Victoria, Canada. Downloaded in March 2014.

37.

Rona

Light

Sonar images hydrothermal vents in seafloor observatory. Eos, Trans Amer Geophys Union 2011; 92: 169–170.

38.

Santilli

Bemis

Silver

et al . Generating realistic images from hydrothermal plume data. In: IEEE Visualization, Austin, TX, 10–15 October 2004, pp.91–98.

39.

Holten

van Wijk

JJ.

Evaluation of cluster identification performance for different PCP variants. Comput Graphics Forum 2010; 29: 793–802.

40.

Ocean Networks Canada. Data & Tools, www.oceannetworks.ca/data-tools. (accessed 1 September 2014).

41.

Isenberg

Chen

et al . A systematic review on the practice of evaluating visualization. IEEE Trans Vis Comput Graph 2013; 19: 2818–2827.

42.

Abdul-Rahman

Lein

Coles

et al . Rule-based visual mappings – with a case study on poetry visualization. Comput Graphics Forum 2013; 32: 381–390.

43.

Ladefoged

Disner

SF.

Vowels and consonants. 3rd ed. Chichester: Wiley-Blackwell, 2012.

44.

Brewer

Hatchard

Harrower

MA.

ColorBrewer in print: A catalog of color schemes for maps. Cartogr Geogr Inf Sci 2003; 30: 5–32.