Task Cube: A three-dimensional conceptual space of user tasks in visualization design and evaluation

Abstract

User tasks play a pivotal role in visualization design and evaluation. However, the term ‘task’ is used ambiguously within the visualization community. In this article, we critically analyze the relevant literature and systematically compare definitions of ‘task’ and the usage of related terminology. In doing so, we identify a three-dimensional conceptual space of user tasks in visualization, referred to as the task cube, and the more precise concepts ‘objective’ and ‘action’ for tasks. We illustrate the usage of the task cube’s dimensions in an objective-driven visualization process, in different scenarios of visualization design and evaluation, and for comparing categorizations of abstract tasks. Thus, visualization researchers can better formulate their contributions which helps advance visualization as a whole.

Keywords

Action design guidelines interaction objective task frameworks terminology visualization theory

Introduction

Tasks are an issue discussed frequently in the visualization community as they are pivotal to how we design and evaluate our work. In many, if not all, scenarios of empirical visualization research^1,2 tasks are either central to the study setup (e.g. in controlled experiments or case studies) or an emergent study result (e.g. transcription of observed visual data analysis and reasoning processes). This is further illustrated by the role of tasks in widely used design frameworks such as the Data–Users–Tasks Design Triangle³ or the Nested Model.⁴

However, there is continuing confusion about what the term ‘task’ means in a visualization context. Even if we consider tasks only for visualization users and neither for developers of visualization systems nor for the audience of a presentation, there are many nuances of what such a visualization task can be. It may be as open-ended as ‘detect anomalies in recent public health data’ or ‘identify the main drivers of climate change’ but also as crisp as ‘find yesterday’s most profitable product’ or “buy a train ticket”⁵ p.2433. Already in 1994 it was widely acknowledged that “the notion of ‘task’ is increasingly difficult to pin down”⁶ p.410, and nowadays the word ‘task’ is still regarded as “deeply overloaded in the visualization literature”⁴ p.921. Based on our own experience throughout multiple visualization design and evaluation projects and inputs from fellow researchers, we regard this confusion as unsatisfactory. A commonly agreed understanding and terminology of tasks is needed.

Therefore, this article investigates user tasks in visualization design and evaluation using the three conceptual dimensions, referred to as the task cube, as a theoretical compass.

The primary contribution of this article is the conceptual space and its dimensions perspective, abstraction, and composition, which we define and describe in the section of the same title. We also compare our dimensions systematically to definitions from relevant literature on visualization and human-computer interaction (HCI).

Furthermore, we emphasize the central role of tasks in visualization in the next section. On the one hand, we present visualization as a task-driven endeavor by putting objectives and actions to the foreground of visualization processes. On the other hand, we illustrate the role of tasks in different scenarios of visualization design and evaluation.

Finally we survey 37 categorizations of abstract tasks found in theoretical frameworks and state-of-the-art reports and systematically compare these categorizations along the dimensions of the task cube.

Thus, instead of proposing a categorization of individual tasks, the task cube and its dimensions help visualization researchers navigate the conceptual space of task categorizations—in particular when working on theoretical task frameworks and collecting state-of-the-art reports. In addition, the reduced ambiguity allows visualization researchers to better formulate their contributions. This article builds on contributions presented in a previous workshop article⁷ with revised definitions, an objective-driven visualization process, an illustrative example, and a survey of abstract task frameworks as major extensions.

Conceptual space

In an effort to clear the aforementioned confusion around the term ‘task’, we analyzed the scientific literature on HCI and visualization for definitions of what tasks are and what roles they play in visualization (see section on Comparison to literature). In addition, we surveyed and compared task categorizations published in theoretical frameworks and state-of-the-art reports (see section on Categorizations of abstract tasks). Our materials have a focus on visualization and visual analytics of time-oriented data and graph data. Visual analytics⁸ has a particular emphasis on developing a science of interaction⁹ and to support higher-level reasoning. Nevertheless, we consider our results applicable to the visualization field in general.

Our literature review confirmed that the term ‘task’ is not consistently defined but refers to different concepts. Consequently, our analysis concentrated on characterizing these concepts and we identified three dimensions—abstraction, composition, and perspective—spanning the task cube (Figure 1). For this purpose, we treat composition and abstraction as continuous dimensions along with the why/how dichotomy for perspective. Thus, a concept of task can be placed either on the top or on the bottom plane of the figurative cube. As we demonstrate below these dimensions are non-redundant and sufficiently expressive to explain the conceptual variety of user tasks in different visualization design and evaluation scenarios.

Figure 1.

Overview of the conceptual space of user tasks in visualization with composition, perspective, and abstraction as orthogonal dimensions.

Next we present these dimensions in a consolidated terminology distilled from our literature analysis.

Abstraction. On the one hand, concrete tasks describe what needs to be done in a specific application context, such as ‘find the quarter of Google’s largest revenue’. On the other hand, abstract tasks describe an aim on a more generic level, such as ‘find maximum’. To generalize the concept behind a concrete task, they can be abstracted and expressed using generic categories from task frameworks. For example, Andrienko and Andrienko¹⁰ would classify this as the abstract task ‘indirect lookup’. Thus, abstraction and task frameworks allow systematic study and facilitate the reuse of visualization methods for tasks of the same abstract category.

Composition. When tackling a task, it is common practice to break it down into smaller subtasks.^6,11 Thus, the level of composition can range from long-term challenges, such as ‘end poverty’, to small steps, such as ‘find outliers in economic data’. By the same token, task frameworks often distinguish high-level, low-level, and sometimes levels in between.^12–14 The compositions of tasks form hierarchies for which we propose a continuous scale from high-level to low-level .

Abstraction. can be clearly distinguished from composition: a low-level task is a part of a high-level task, whereas an abstract task is a more generic category of a concrete task. This is illustrated by a low-level task demanding fewer steps to perform than the high-level task encompassing it. In contrast, if an abstract task describes a given concrete task more generically, both tasks still require the same steps. These dimensions conform to the subclass of and part of relationships found, for example, in object-oriented programming and ontology languages.

Perspective. On the one hand, high-level visualization tasks are typically non-routine objectives such as ‘find suspicious patterns’ or ‘what explains this behavior’. For these, the technical term ‘objective’ is adopted from Roth¹⁴ because other terms such as ‘problem’ or ‘goal’ are understood differently in our community. We define it as:

an objective is a question on data that the user raises for inquiry, consideration, or solution while aiming to solve a problem or satisfy a curiosity.

On the other hand, tasks using interactive features of visualization artifacts and categories of some low-level task frameworks are formulated as executable actions, such as ‘map time to y-axis’ or ‘zoom to the orange cluster’. The technical term ‘action’ is widely used and we derive its definition from Gotz and Zhou:¹⁵

an action is a discrete step towards addressing an objective.

Since there is usually no direct relation—no decomposition—between objectives and actions, it makes sense to distinguish between a why perspective and a how perspective and to have two distinct terms to explicitly address these perspectives.

Both objectives and actions can be composed or decomposed at different levels. On the why perspective, users break down large objectives to increasingly smaller subobjectives, intentionally or unintentionally, in order to make ill-defined objectives manageable using visualization tools.^11,16 On the how perspective, blocks of consecutively observed actions can be combined to action sequences, at multiple levels of composition. With experience, users will develop action strategies to solve common subobjectives with a tool and it is possible to analyze such strategies by observing users.¹⁷ In addition, actions can be decomposed further to a level below intentional problem solving such as individual user interface events.¹⁵ Even though the composition hierarchies of objectives and actions are connected, it makes sense to draw a clear line between them along the why/how dichotomy because objectives and actions are different concepts with different properties. Furthermore, actions can be found at the same composition level as intermediate- to low-level objectives with the mapping varying largely depending on visualization tools. For example a specialized tool for electronic health records might have the affordance to select all critical patients by a single intermediate-level action whereas a general purpose time series analysis tool requires a sequence of low-level actions to achieve the same objective.

Neither could we subsume perspective under abstraction because both objectives and actions can be described at different levels of abstraction. Above, we mentioned the concrete objective ‘find the quarter of Google’s maximum revenue’ and classified it as the abstract objective ‘indirect lookup’. Likewise, the action of changing the y-axis of a scatter plot to a different variable can be abstracted as ‘reconfigure’,¹⁸ ‘arrange’,¹³ or ‘visualize’.¹⁹

Together these three dimensions span the task cube (Figure 1), a simple yet expressive model of the conceptual space for user tasks in visualization.

Example

To illustrate these concepts more deeply, we will follow an extensive example based on the Kronos Incident, the fictional application context created for VAST Challenge 2014.²⁰

The concrete, high-level objective of the Kronos Incident²⁰ is to support law enforcement agencies of the fictional country of Kronos in investigating the disappearance of some employees of GAStech, a natural gas production company. The VAST Challenge breaks the objective down to subobjectives of (1) reconstructing the events of the disappearances, (2) reconstructing networks of people that influenced each other, (3) finding missing information, and (4) providing the best course of action for the police force (Figure 2). Of course, these concrete subobjectives need to be broken down further. For example, to specifically analyze the network of the main suspect, an organization named the Protectors of Kronos (POK), we can identify the lower-level objectives as (2a) identifying the leaders, (2b) identifying all members of the extended network, (2c) analyzing the change of the network structure over time, and (2d) finding potential connections between the POK and GAStech.

Figure 2.

Visual task decomposition of the Kronos Incident as described in the Example section.

To break down the concrete subobjective (2a) even deeper, we follow the solution of Saraf et al.,²¹ winners of the Grand Challenge Award. They first (2a1) located co-occurrences of the terms “POK” and “leader” in news articles over time. This resulted in three time ranges when both terms peaked simultaneously. Then they (2a2) browsed through word clouds of persons retrieved from named entity recognition of articles in these time frames.

It is possible to abstract the high-level objective by domain as a law enforcement objective. The subobjective (2a) can be abstracted as ‘discover/explore/compare’ using the three-level why-typology of Brehmer and Munzner.¹³ We can abstract the low-level objective (2a1) of finding co-occurrences of the terms over time as ‘discover/locate/summarize’¹³ or as a ‘synoptic inverse comparison task’.¹⁰ The following concrete objective of finding the most mentioned name in each time frame can abstractly be considered an ‘elementary comparison’ task according to Andrienko and Andrienko.¹⁰

Saraf et al.²¹ solved the concrete low-level objective (2a1) by entering the terms in a search box of their visualization artifact and then used the date range widget for subobjective (2a2). Both concrete actions can be abstracted as ‘filter’ in the frameworks of Yi et al.¹⁸ and Gotz and Zhou.¹⁵

Comparison to literature

In colloquial English a ‘task’ is understood as “a piece of work that has been given to someone: a job for someone to do”.²² The Merriam-Webster Dictionary emphasizes that tasks are characterized as being externally assigned, having a deadline, and being hard or unpleasant. Surveying the HCI and visualization literature we can however discover a shifted focus and a multitude of nuances of this general definition. Next, we analyze these along the three dimensions presented above.

Abstraction. The need for abstraction and categorization of tasks is widely acknowledged. There are many different terms used for abstract tasks in the literature including ‘generic’, ‘general’, but also ‘high-level’. Munzner⁴ p.921 distinguished between a task formulated in terms of an application domain, which she called “problem”, and an abstract task, which she denoted as “operation”. Apart from that we are not aware of any term for a concrete task.

Composition. It appears as a general notion that users break down tasks into smaller, more manageable subtasks.¹¹ Preece et al.⁶ p.412 spoke of a “decomposition of goals into subgoals and tasks into subtasks as the user moves downwards through a hierarchy of systems”. One possible notation to specify such hierarchical tasks models are ConcurTaskTrees.²³ In the visualization literature, the terms ‘high-level’ and ‘low-level’ are often used to distinguish such tasks¹³ but there is no consistent understanding.

In addition, some task frameworks introduce explicitly named composition levels (Figure 3), which contrasts with abstraction, which to our knowledge has no such hierarchies. Norman²⁴ p.15 introduced a composition level above tasks: “an activity is a coordinated, integrated set of tasks”, and emphasized that design should be centered on these activities. Gotz and Zhou¹⁵ p.46 described how “analysts typically follow a divide-and-conquer approach, performing several sub-tasks to achieve the requirements of a single top-level task”.

Figure 3.

Comparison of composition levels in different hierarchies. The grey area denotes whether the level is regarded specific to the used tool.

Tasks at the leaf-level of the hierarchy are often treated distinctively. Preece et al.⁶ p.411 defined an action “as a task that involves no problem solving or control structure component” and is performed by “the human physically interact[ing] with a device”. Tasks at a similar level of composition are also referred to as ‘simple tasks’ or ‘unit tasks’.⁶ Fuchs¹¹ p.10 used ‘basic tasks’ for the leaf-level and differentiates them from ‘actions’, which “describ[e] the functional properties beyond the conceptual task decomposition”. He defines an ‘action’ as “an atomic operation that is executed upon an artifact, by an entity that is involved in the completion of the task (user, computer, . . .)”. Similarly, the action tier of Gotz and Zhou¹⁵ p.46 represents “an atomic analytic step performed by a user with a visual analytic system”. They consider yet another level below actions: “events correspond to the lowest-level of user interaction events (for example, a mouse click or a menu item selection) which carry very little semantic meaning” p.43. Norman²⁴ p.15 described tasks as “composed of actions, and actions [as] made up of operations”. ConcurTaskTrees²³ distinguished at the leaf-level between ‘user tasks’, ‘application tasks’, and ‘interaction tasks’. They referred to tasks composed of such subtasks as ‘abstract tasks’.

The selection of tools also plays an important role in task decomposition (Figure 3). Both Preece et al.⁶ and Cooper et al.²⁵ used it to distinguish between ‘goals’ and ‘tasks’. For Fuchs¹¹ it is the ‘action’ that involves an artifact such as a task-specific visual representation.

Perspective. The distinction between a why and a how perspective of tasks stretches across the literature and has only recently been addressed by Brehmer and Munzner¹³ as ends–means ambiguity.

On the one hand, the why perspective describes a task by the ends or the intended outcomes. Preece et al.⁶ p.411 defined an external task or goal “as a state of a system that the human wishes to achieve”. Roth¹⁴ p.2357 distinguished further between “an ill-defined task, or goal, motivating use of the visualization” and “a well-defined task, or objective, that can support the goal”. In visualization, the why perspective is often formulated as a question or a query to be answered based on the data at hand. Andrienko and Andrienko¹⁰ p.49 “use[d] the word ‘tasks’ to denote typical questions that need to be answered by means of data analysis”, Amar et al.¹² categorized tasks by questions or queries asked, and Munzner⁴ p.921 denoted a task described in domain terms as ‘problem’. In her book,²⁶ p.45 she referred to abstract why tasks as ‘action’. For Gotz and Zhou¹⁵ p.46 the “task tier captures a user’s high-level analytic goals”.

On the other hand, the how perspective addresses the means or actions by which users perform the task. Preece et al.⁶ p.411 defined a task, in particular an internal task, “as the activities required, used or believed to be necessary to achieve a goal using a particular device”. Fuchs¹¹ p.10 understood a task “as a single, conceptually distinguishable but not necessarily atomic step within a composite activity or work flow”. Cooper et al.²⁵ p.15 wrote “both activities and tasks are intermediate steps (at different levels of organization) that help someone to reach a goal or set of goals”. For Schulz et al.²⁷ p.2366 visualization tasks are “activities to be carried out interactively on a visual data representation for a particular reason”. For Brehmer and Munzner¹³ p.2376 “abstract tasks are domain- and interface-agnostic operations performed by users”. Roth¹⁴ p.2357 distinguished a third category: “a system function, or operator, that may support the objective”.

Discussion. What makes the notion of ‘task’ so hard to understand is not only the differences in these three dimensions (Figure 1) but also their interconnectedness. For example, when we follow a concrete user session, we could observe how the user transforms a high-level why task formulated in terms of the application domain into low-level how tasks matching operations of the visualization system.¹⁵ Thus, it is difficult to tackle tasks in the more abstract way needed to draw generalizable guidelines and to characterize tasks on the levels in between.

Bridging between the extremes and taking an abstract view on an intermediate level appears as a promising direction. Yi et al.¹⁸ proposed to categorize actions by user intents, i.e. “what users achieve by using the interaction techniques rather than how the techniques provided by Infovis systems work”. Schulz et al.²⁷ established a five-dimensional design space of visualization tasks that encompasses both why and how as well as dimensions pertaining to the data (characteristics, target, cardinality). Likewise, Brehmer and Munzner¹³ represented a task as a triple composed of why, how and what (data).

In contrast, our work does not aim for such a single intermediate level that captures all diverse forms of tasks. Instead, it emphasizes the diversity of task concepts. The task cube can accommodate for different task categorizations that suit different visualization scenarios and application domains. Thus, it proceeds from previous work considering its dimensions. On the one hand, Munzner⁴ proposed a 2x2 matrix of task concepts by abstractions and composition. On the other hand, Roth¹⁴ distinguished between different concepts for why (goal, objective) and how (operator). Pike et al.²⁸ combined the why/how dichotomy and the level of composition. They described “analytic discourse as the relationship between [interaction] techniques and the user’s goals and tasks, which involve low-level choices about manipulating interactive controls and higher level goals surrounding the problem being investigated”²⁸ p.265. Brehmer and Munzner¹³ proposed three dichotomies to compare task categorizations: level (high vs low), temporality (sequences vs constraints), and specificity (general vs specific). What the task cube adds is a more fine-grained understanding of abstraction and a terminology that can distinguish between why and how tasks. In particular, we need to emphasize that visualization often addresses open-ended objectives. When solving such objectives, the users can follow different strategies and there are often no definite mappings between the why and the how. Furthermore, HCI scholars have long warned against a design approach that focuses on hierarchical analysis of operational tasks and not on the goals and characteristics of the users.^6,24,25,29

Next, we will explore the role of objectives and actions both in a concrete process and in different scenarios of design and evaluation.

Role of tasks in visualization

We conceptualize visualization-supported data analysis as a process that is largely driven by objectives. This process combines and extends existing process models by expanding the gulf of goal formation^16,31 beyond actions towards higher-level objectives and by adding emphasis on objectives and their breakdown to the knowledge generation model³⁰ and the hypothesis-driven model of Lammarsch et al.³² For this purpose we rely on a few technical terms (model, finding, insight, hypothesis, and knowledge) from the knowledge generation model.³⁰

This visualization-supported data analysis process (Figure 4) starts with a user, data, and an objective.³ While the data resides in a more or less structured form within a computer, the user brings in the objective and his or her background knowledge.^5,30 The objective can originate from the user’s personal goals, from the goals of the user’s organization,⁶ or out of pure curiosity.³³

Figure 4.

Objectives and actions in visualization-supported data analysis. The knowledge generation model³⁰ is extended with the notion of an objective residing at the human side, which is tackled through a plan and performed through a sequence of actions. New insights might lead to replanning.

Under consideration of his or her background knowledge about the objective, about the data, and about the available tools, the user breaks the objective down to manageable subobjectives. Then he or she develops a plan, which is a sequence of actions that he or she believes will solve the subobjective and thus will advance towards addressing the overall objective.^6,11 This planning is guided but also limited by the affordances of the tool(s) used. Such a tool is typically a visualization artifact providing interaction with visual representations and models of data. Nevertheless, we should keep in mind that in many realistic data analysis application contexts users have a range of possible tools at their disposal, e.g. other computing systems, pen and paper, or mental models.

The user performs the planned actions by activating a number of events in the visualization user interface that are interpreted by interaction techniques.¹⁵ For example, panning using a scrollbar involves a mouse-down event, several mouse-drag events, and a mouse-up event. Still, we focus on actions as the discrete steps that represent the lowest level of activity of which the user is consciously aware. Above actions we consider action sequences and action patterns as frequently occurring sequences, e.g. sort, then select the top 10. With experience, users can learn strategies involving such action patterns as generic and low-level plans to solve common subobjectives with a tool or family of tools.¹⁷

While performing these actions, the user collects findings—potentially interesting observations from the data. Insights arise from accumulating these findings and setting them in context with the objective and knowledge.³⁰ These insights often lead to an update of the plan as the user develops hypotheses that make new subobjectives relevant. Furthermore, the plan will be iteratively updated as the background knowledge, the availability of tools, or even their goals can change.^{6,16,25,28,34}

Implications for design and evaluation

Above, we observed, from the users’ point of view, concrete objectives and actions in the visualization process. While users can abstract their objective breakdown and action planning from one situation to another and from one visualization artifact to another, abstraction plays an even more important role for visualization design and evaluation. Here, theoretical frameworks serve as bonds between individual research contributions and general guidelines for the visualization discipline. Next, we will walk through different scenarios of visualization design and evaluation, describe the role of objectives and actions, and give examples.

Domain characterization and abstraction. An essential aspect for the understanding of environments and work practices¹ is to determine the objectives of domain experts. As high-level objectives might be too vague, it is often necessary to gather objectives at multiple levels of composition. Subsequently, it is necessary to transform concrete objectives formulated in domain language to abstract objectives that can be matched to categories of task/objective frameworks. The two outer layers of Munzner’s Nested Model⁴ describe these steps.

RelEx, an example study conducted in automotive engineering,³⁵ described objectives at three levels of composition: the high-level objective of “traffic optimization on the signalpath network” in domain terms, mid-level objectives being abstracted to general features of social networks, and low-level objectives as queries on network relations. Another study in IT security³⁶ classified the objective of malware pattern analysis along the three-level why-typology of Brehmer and Munzner.¹³ Both studies analyze objectives in context of the data and users as proposed by Miksch and Aigner.³

Designing visualization artifacts. The next layer of visualization design⁴ is mapping the abstracted objectives to visual representation and interaction techniques, and thus the action affordances provided by the visualization artifact. This step can profit from guidelines translating between abstract objectives and abstract actions.³⁷ An example of such design guidance for visualization of spatial and temporal data is the book of Andrienko and Andrienko,¹⁰ which lays on the solid foundation of their theoretical framework of low-level objectives.

The RelEx design study³⁵ illustrates how design rationale is based on clearly formulated user objectives. Based on the objectives cited above, they analyzed existing tools and postulated design requirements—in particular the requirement for rich exploration of complex, multi-way relations. RelEx meets this requirement with brushing and linking in multiple coordinated views, especially a signalpath view for 4-way relations. Likewise, the LiveGantt design study³⁸ tackles the objective of what-if questions in scheduling by drag and drop actions in its Gantt chart view. The ChronoLenses technique³⁹ addresses two objectives: (T1) single data stream transformation such as removing trends and (T2) cross-data stream analysis such as comparing two streams. These two tasks result in two types of interactive lenses.

Stimuli for experiments. User experience or performance can be evaluated by letting test users interact with visualization artifacts in a controlled environment. The stimuli presented to the test users are typically objectives and can be formulated as question-like ‘which <items> fulfill <. . .>’, as imperative-like ‘identify <items> that fulfill <. . .>’, or even prescribe the answering method e,g. ‘click on <items> that fulfill <. . .>’. As argued above, the evaluation of predefined action sequences like ‘filter by <. . .>, then zoom to <. . .>’ is less relevant for user studies in visualization.

Depending on the study’s hypotheses, a suitable composition level must be found because low-level objectives can be too trivial and high-level objectives too open-ended for quantitative analysis of time and errors. For such experiments other evaluation methods, e.g. qualitative analysis of insights,^40,41 are more suitable.

Research hypotheses are often specific to an abstract objective such as ‘less errors for comparison’, whereas stimuli need to present concrete objectives that act as representative examples for the abstract objective. In some cases a concrete objective is also translated to a concrete objective in a different domain so that a sufficiently large population of test persons can be recruited. Each occasion a test person works on such a concrete objective is called trial ⁴² p.298. Repeated trials with multiple test persons or multiple data (sub)sets are needed to reach statistical power. For example, Javed et al.⁴³ experimentally compared four visualization techniques for time series by the performance of solving comparison, slope, and discrimination objectives. The SemanticTimeZoom technique⁴⁴ was evaluated using 12 objectives, which were categorized based on the Andrienko and Andrienko¹⁰ objective framework.

Setting context in case studies. When case studies are performed to evaluate visualization artifacts under more realistic conditions it is necessary to describe the objective tackled and it is useful to set it in context using abstract objectives. This also applies to evaluation by algorithmic performance measures or qualitative result inspection.²

Studying interactive analysis and reasoning. Interaction remains a topic of great interest in visualization currently. Such studies include case studies with domain experts where observation of user interaction allows to see behind the curtain of visual data analysis and reasoning processes. Interaction logs can be a valuable addition to thinking aloud protocols or diaries or even replace them.¹ From sequences of actions in these logs, interaction patterns can be identified and generalized based on abstract actions. Thus, it is possible to develop shortcuts for frequently used patterns,²⁸ explore how visualization artifacts induce or inhibit certain analytical behaviors, and judge the cost of interaction.³¹

For example, the experiment of Dou et al.⁴⁵ demonstrated to what extent analytical reasoning can be inferred from interaction logs alone. They noted that such inference worked best for highly interactive artifacts but worse when users got along with looking at visual representations. Another example study by Pohl et al.¹⁷ compared the interaction logs from user studies of two visualization artifacts of the same target audience, medical doctors, to identify interaction patterns up to a length of three actions and transition probabilities between actions. In order to make actions comparable, they used the Yi et al.’s user intents¹⁸ for abstraction.

Integration in visualization artifacts. Objectives and actions are not only relevant for visualization designers and researchers, but they can also be made explicit within visualization artifacts for users during visualization runtime thus addressing the visual analytics challenge of “capturing user intentionality”.²⁸ The record of actions performed makes sense as a history mechanism⁴⁶ and for analytical provenance.⁴⁷ Notable examples are the p-set framework,⁴⁸ which focuses on parameter sets, and HARVEST,¹⁵ which builds on an action taxonomy for interoperability. Likewise, the visualization artifacts can support users by allowing them to track open subobjectives as demonstrated in Sandbox⁴⁹ or Aruvi.⁵⁰ Given the user’s objectives the visualization artifact can also recommend suitable settings at runtime. For example, Tominski et al.⁵¹ proposed choosing one of eight color schemes based on a three-dimensional objective space and Schulz et al.²⁷ used their five-dimensional task framework to recommend visual representation techniques for climate impact data.

Categorizations of abstract tasks

Beyond individual visualization design and evaluation projects, tasks are useful for systematic research in visualization. Thus, a number of theoretical frameworks that categorize abstractions of objectives and actions have been conceived. These frameworks are valuable in making the results of empirical research comparable and extracting design guidelines from visualization work.

State-of-the-art reports, in particular, can apply such categorizations to describe and systematically compare visualization artifacts. The objectives addressed and the actions provided allow these reports to structure the surveyed area, to identify similarities not evident from the original description of each individual artifact, to provide design guidance, and even to generalize beyond the scope of the report. Although, it is not uncommon for state-of-the-art reports to develop a customized task categorization for its scope. For example, Alsallakh et al.⁵² categorized set visualization techniques using a list of 26 low-level objectives specific to set-typed data. Dealing with visualization systems for electronic health records, Rind et al.⁵³ extended the user intent categorization¹⁸ with 20 more concrete subintents and used these for categorization.

Yet, the number of task categorizations can be confusing and in addition the abstract tasks found therein are quite heterogeneous. This is also illustrated in the typology of Brehmer and Munzner¹³ that reclassifies abstract tasks from a wide range of existing categorizations. For example, it maps the abstract tasks by Shneiderman⁴⁶ to produce, summarize, navigate, filter, and record. Thus, they occur for both why and how as well as at different levels of their typology. Therefore, we propose to use the dimensions of the task cube to classify and compare these categorizations of abstract tasks.

Method. We conducted a non-exhaustive survey of task frameworks in the visualization literature and identified 37 categorizations (Table 1). Then we classified each categorization (1) as objectives versus actions, (2) by three levels of composition, and (3) by four types of abstraction. As we could not identify general classification rules for composition, we made groups of high-level, intermediate-level, and low-level tasks that are as homogeneous as possible. For abstraction, we distinguished between generic abstractions and abstractions that are tailored for a data type, for a domain, or towards the architecture of tools. In addition, we inform about the expressiveness of each categorization by reporting the number of categories, the number of categories in each dimension, or similar information for more complex categorizations. The classification was first proposed by the first author and then discussed and revised by all authors until consensus was reached.

Table 1.

Survey of abstract objective and action categorizations found in theoretical task frameworks, state-of-the-art reports, and domain characterization studies (not exhaustive).

Task categorizations	Perspective		Composition			Abstraction				Number of categories (in each dimension)
Task categorizations	why	how	HI	IN	LO	GE	DA	DO	TO	Number of categories (in each dimension)
Explore/confirm/present⁵⁴	•		•	○		•				3
Amar and Stasko⁵⁵: prototypical analysis tasks	•		•			•				4
Roth¹⁴: goals (procure/predict/prescribe)	•		•	○		•				3
Thomas and Cook⁸, p.35	•		•			•				3
Amar et al.¹²: low-level components	•			•	•	•				10
Andrienko and Andrienko¹⁰	•			○	•	•				2 x 3 x 2
Brehemer and Munzner¹³: why	•			•	•	•				4 x 4 x 3
Munzner²⁶: why/actions & targets	•			•	•	•	○			6 x 4 x 3 x 11
Roth¹⁴: objectives	•			•	○	•				5
Wehrend and Lewis⁵⁶	•			•	•	•				11
Ahn et al.⁵⁷: network evolution	•			•	○		•			3 x 2 x 10
Alsallakh et al.⁵²; set-typed data	•				•		•			26
Brehmer et al.⁵⁸: dimensionally-reduced data	•			○	•		•			6 (maps to^13,26)
Kerracher et al.⁵⁹: temporal graphs	•			○	•		•			extends¹⁰
Lammarsch et al.⁶⁰: time-oriented data	•			○	•		•			extends¹⁰
Lee et al.⁶¹: graphs	•				•		•			11
MacEachren⁶² p.316: aspects of time	•			•	○		•			7
Pretorius et al.⁶³: multivariate networks	•			•	•		•			25 (maps to⁶⁴)
Roth¹⁴: objectives & operands in cartography	•			•	•		•			5 x 3
Suo⁶⁵: high level tasks in network security	•		•					•		5
Meyer et al.⁶⁶: comparative genomics	•			○	•			•		14
Schulz et al.²⁷: 5-dimensional design space	•	○		•	•	•	○			3 x ≧8 x 4 x 3 x ≧11
Yi et al.¹⁸: intents	•	○			•	•				7
Valiati at al.⁶⁴: multidimensional data	•	○		•	•		•			7
Brehmer and Munzner¹³: how		•			•	•				11
Munzner²⁶: how/idioms		•			•	•				11 (different from¹³)
Chuah and Roth⁶⁷		•			•	•				18
Gotz and Zhou¹⁵: actions		•			•	•				21
Liu and Stasko⁶⁸: mental models		•		○	•	•				6
Sedig and Parsons⁶⁹: action patterns		•			•	•				32
Suo⁶⁵: low-level tasks		•			•	•				32
von Landesberger et al.⁷⁰		•			•	•				3 x 4
Roth¹⁴: operators on spatial data		•			•		•			17
Rind et al.⁵³: electronic health records		•			•			•		20 (extends¹⁸)
Shneiderman et al.⁴⁶		•		○	•				•	7
Heer and Shneiderman¹⁹: dynamics		•			•				•	12
Sacha et al.³⁰		•			•				•	6

The why and how columns denote whether the categorization describes objectives or actions. Three composition levels are distinguished: HI. . . high, IN. . . intermediate, and LO. . . low, and four types of abstraction: GE. . . generic, DA. . . data type, DO. . . domain, and TO. . . tool architecture. The symbol • denotes the primary class of the entry and the symbol ○ represents a partial match, for example if only a few categories fall into this class. The final column describes the expressiveness of the entry by the number of categories or similar details.

Results. Table 1 provides an overview of the surveyed task categorizations and their classification. Along the why/how dichotomy a majority of 24 categorizations is primarily for objectives, while 13 categorizations address actions. For this, we split Brehmer and Munzner’s multi-level typology¹³ into its why and how parts. In Roth’s framework¹⁴ we described three categorizations separately: goals and objectives for why and operations for how. There were a few ambiguities.

Schulz et al.’s five-dimensional space²⁷ has four dimensions relating to why (goal, characteristics, target, cardinality) and one for how (means).

Yi et al.¹⁸ categorized interaction techniques along user intent. Nevertheless their objectives are characterized very similar to actions, e.g. “show me something conditionally” is almost equivalent to a filter action.

The taxonomy of Valiati et al.⁶⁴ mixes objectives like “identify patterns” with actions like “zoom”.

For composition, we identify only few categorizations for high-level objectives. For example, explore/confirm/present are regularly used categories. Alternatively, Amar and Stasko⁵⁵ presented the prototypical tasks of “complex decision making, especially under uncertainty”, “learning a domain”, “identifying the nature of trends”, and “predicting the future”. This is not unexpected as real-world high-level objectives can be very specific to a domain problem. Many objective categorizations can be positioned at intermediate-level, low-level, or both. These ambiguities results from some categorizations having a broad scope (e.g. elementary versus synoptic tasks¹⁰) and that their abstract objectives can describe for various real-world objectives at these composition levels. Action categorizations are at a mostly low composition level, which is obvious because characteristic sequences of actions are specific to the concrete situations they emerged from. For this reason, there is also no abstract task frameworks for high-level actions. A few categorizations encompass intermediate-level actions such as ‘overview’ and ‘relate’⁴⁶ or ‘cognitive offloading’.⁶⁸

Among the four types of abstraction, generic abstractions are most common for objectives (12) and actions (8). This can be explained by the fact that our survey is not exhaustive of the task categorizations collected for state-of-the-art reports and design studies. Still there is a large proportion of objective categorizations tailored by data type (9)—most frequently networks and/or time.

Discussion

We looked at the task cube from various angles of visualization design, evaluation, and research in the previous sections. Reflecting on the dimensions of the task cube, we now observe some general implications.

Usefulness of abstract task frameworks. The long yet non-exhaustive list of task categorizations in Table 1 highlights a large body of research towards the theoretical understanding of tasks. Above, we showed various visualization scenarios that require visualization designers, experimenters, and researchers to explicitly consider concrete tasks. But they need abstract task frameworks to generalize such tasks beyond the particular application context; to systematically compare visualization artifacts in a survey, to generate guidelines from evaluation results, and to apply such guidelines and surveys for designing new artifacts. In addition, abstract task lists can be used to check if any relevant tasks still need to be addressed. Our survey shown in Table 1 supports visualization researchers and designers in finding a categorizations suitable for their scenario and application context.

Suitable levels of composition. In capturing the how perspective, actions take a pivotal level. Defined as discrete step, they should be at the lowest composition level of which the user is consciously aware. However, this is a matter of context and, in particular, the user’s experience, e.g. which granularity the user regards as discrete steps.^6,34 One user’s action pattern might be another user’s action and this might also change within a single user over time as he or she progresses from novice to expert. For example, a user might learn to click through menus and dialog boxes unconsciously because he or she frequently needs to change a setting that is hidden there. Then again, visualization designers have also the possibility to detect the need for such action patterns and provide dedicated interaction techniques as shortcuts.

Likewise, objectives can be broken down to increasingly lower levels. However, decomposition into trivially small subobjectives, such as ‘read value 1, read value 2 . . .’, is often not practical and differs from how users realistically solve objectives. Combining human perception and visualization techniques, they can detect patterns at a larger scope. For example, they can spot clusters in a scatter plot or judge the trend in a line plot without consecutively reading the values encoded for individual data records. Therefore, it is important to consider objectives at an adequate level of composition, in particular when evaluating visualization techniques.

Open-endedness of objectives. Visualization is often characterized as undirected exploration or even as a casual endeavor leading to serendipitous results. In contrast, the term ‘task’ often has connotations as being externally assigned and scheduled within a workflow. We take a pragmatic point of view and understand visualization-supported data analysis as the user steering his or her actions towards solving a reasonably open-ended objective. Such an objective might originate from casual curiosity and/or result in unexpected findings. Still, we assume there is an objective motivating the user’s actions. We also assume that the user has some intellectual interest in the findings. We would not consider an assignment such as ‘pan to the year 1983’ or ‘click on the blue rectangle’ as an objective. Such assignments are purely perceptual and mechanical and do not carry the notion of the user raising an inquiry. Furthermore, objectives in the visualization field involve questions based on data.

This intellectual interest in objectives and their many-to-many mapping to action sequences are our motivations to distinguish between the perspectives why and how. They are also a major difference to the traditional view of tasks in HCI that follows the assumption of a one-to-one mapping between these perspectives.²⁹ Yet, such an assumption holds only for operational tasks and not for such creative tasks as they are needed for the open-ended objectives addressed by visualization.^71,72

Summary of discussion. We assume that much of the confusion and criticism on tasks stems from this traditional view of operational tasks. For example, some visualization experts reportedly¹⁰ p. 148 even “believe that having a defined task is not (or not always) necessary”. Other HCI experts, e.g. Preece et al.⁶ p. 410, warn that “the idea of a task is useful in system development – as long as it is used with care” because otherwise current action sequences would be re-instantiated in future systems, which are too rigid to serve the real objectives of users.

However, we agree with Miksch and Aigner,³ Munzner,⁴ Andrienko and Andrienko,¹⁰ and many other visualization experts that tasks are a useful concept for evaluation throughout visualization design and development. Yet, we propose to use the more precise terminology of ‘objectives’ and ‘actions’ instead of the ambiguous term ‘task’. Furthermore, we recommend understanding perspective, level of composition, and level of abstraction as non-redundant dimensions. While these dimensions are clearly distinct in a theoretical view of the conceptual space, it can be difficult to separate them in practice. This is also reflected in a number of ambiguities in our survey of abstract task categorizations. However these ambiguities might also be inherent as a 3D-covariance that deserves deeper investigation.

Conclusions and future work

In this paper we critically analyzed the usage of the term ‘task’ in visualization and human-computer interaction literature. We propose using ‘objective’ and ‘action’ as a more suitable terminology that reduces ambiguity and allow visualization researchers to better formulate their contributions. In addition, we identified a three-dimensional conceptual space of user tasks in visualization with abstraction, composition, and perspective as orthogonal dimensions. We looked at objectives and action in a concrete visualization process in various visualization design and evaluation scenarios, and in state-of-the-art surveys and theoretical frameworks. Finally, we emphasized the usefulness of tasks, the importance of choosing adequate composition levels, and focusing on open-ended objectives.

Concluding this article, we can identify two important areas for future research. First, to supply the visualization community with design guidelines,³⁷ we need not only further empirical work but also need to abstract, aggregate and compare these results. Second, tasks and in particular objectives are important to visualization as a objective-driven process. Therefore, visualization artifacts should take users’ intents into account either from explicit input or by auto-detection.

Footnotes

Acknowledgements

We thank the anonymous reviewers, the participants of BELIV 2014, and Bilal Alsallakh for their feedback and suggestions.

Funding

This work was supported by the Austrian Science Fund FWF [grant numbers P25489, P22883], the Austrian Federal Ministry for Transport, Innovation and Technology [grant number 845598], and the Austrian Federal Ministry of Science, Research and Economy [grant number 822746].

References

Lam

Bertini

Isenberg

. Empirical studies in information visualization: Seven scenarios. IEEE TVCG 2012; 18(9): 1520–1536.

Isenberg

Chen

. A systematic review on the practice of evaluating visualization. IEEE TVCG 2013; 19(12): 2818–2827.

Miksch

Aigner

A matter of time: applying a data-users-tasks design triangle to visual analytics of time-oriented data. Comput Graph 2014; 38: 286–290.

Munzner

A nested model for visualization design and validation. IEEE TVCG 2009; 15(6): 921–928.

Sedlmair

Meyer

Munzner

Design study methodology: reflections from the trenches and the stacks. IEEE TVCG 2012; 18(12): 2431–2440.

Preece

Rogers

Sharp

. Human-computer interaction. Harlow, UK: Addison-Wesley, 1994.

Rind

Aigner

Wagner

. User tasks for evaluation: untangling the terminology throughout visualization design and development. In: Proceedings of the 5th workshop beyond time and errors: novel evaluation methods for visualization, BELIV (eds Lam

Isenberg

.), 2014, pp. 9–15. ACM.

Thomas

Cook

(eds) Illuminating the path: the research and development agenda for visual analytics. IEEE, 2005.

Aigner

. Understanding the role and value of interaction: first steps. In: Proceedings of the international workshop visual analytics (EuroVA) (eds Miksch

Santucci

), 2011, pp. 17–20. Goslar, Germany: Eurographics.

10.

Andrienko

Exploratory analysis of spatial and temporal data: a systematic approach. Berlin: Springer, 2006.

11.

Fuchs

Task-based adaptation of graphical content in smart visual interfaces. PhD thesis, Universität Rostock, 2011.

12.

Amar

Eagan

Stasko

. Low-level components of analytic activity in information visualization. In: Proceedings IEEE symposiumn information visualization, INFOVIS, 2005. pp. 111–117.

13.

Brehmer

Munzner

A multi-level typology of abstract visualization tasks. IEEE TVCG 2013; 19(12): 2376–2385.

14.

Roth

RE.

An empirically-derived taxonomy of interaction primitives for interactive cartography and geovisualization. IEEE TVCG 2013; 19(12): 2356–2365.

15.

Gotz

Zhou

MX.

Characterizing users’ visual analytic activity for insight provenance. Inf Vis 2009; 8(1): 42–55.

16.

Norman

DA.

Cognitive engineering. In: Draper

Norman

(eds) User centered system design. Hillsdale, NJ: Lawrence Erlbaum, 1986, pp. 31–61.

17.

Pohl

Wiltner

Miksch

. Analysing interactivity in information visualisation. KI – Künstliche Intelligenz 2012; 26: 151–159.

18.

Kang

Stasko

. Toward a deeper understanding of the role of interaction in information visualization. IEEE TVCG 2007; 13(6): 1224–1231.

19.

Heer

Shneiderman

Interactive dynamics for visual analysis. Queue 2012; 10(2): 30:30–30:55.

20.

Cook

Grinstein

Whiting

VAST challenge 2014: the Kronos incident, 2014. http://www.vacommunity.org/VAST+Challenge+2014 (2014, accessed 9 February 2015).

21.

Saraf

Butler

Ramakrishnan

. Safeguarding Abila: discovering evolving activist networks. In: VAST challenge, proceedings of the IEEE conference on visual analytics science and technology, 2014.

22.

Merriam-Webster Dictionary. Task. http://www.merriam-webster.com/dictionary/task (accessed August 27 2014).

23.

Paterno

Mancini

Meniconi

. ConcurTaskTrees: a diagrammatic notation for specifying task models. In: Proceedings of the IFIP TC13 international conference on human-computer interaction, INTERACT (eds Howard

Hammond

Lindgaard

), 1997, pp. 362–369. Springer.

24.

Norman

DA.

Human-centered design considered harmful. Interactions 2005; 12(4): 14–19.

25.

Cooper

Reimann

Cronin

. About face 3: the essentials of interaction design. New York, NY: John Wiley & Sons, 2012.

26.

Munzner

Visualization analysis and design. Boca Raton, FL: AK Peters, 2014.

27.

Schulz

Nocke

Heitzler

. A design space of visualization tasks. IEEE TVCG 2013; 19(12): 2366–2375.

28.

Pike

Stasko

Chang

. The science of interaction. Inf Vis 2009; 8(4): 263–274.

29.

Beaudouin-Lafon

. Designing interaction, not interfaces. In: Proceedings of the working conference on advanced visual interfaces, AVI 2004, pp. 15–22. ACM.

30.

Sacha

Stoffel

. Knowledge generation model for visual analytics. IEEE TVCG 2014; 20(12): 1604–1613.

31.

Lam

A framework of interaction costs in information visualization. IEEE TVCG 2008; 14(6): 1149–1156.

32.

Lammarsch

Aigner

Bertone

. Towards a concept how the structure of time can support the visual analytics process. In: Proceedings of the international workshop on visual analytics, EuroVA, in conjunction with EuroVis (eds Miksch

Santucci

), 2011, pp. 9–12. Goslar, Germany: Eurographics.

33.

Sprague

Tory

Exploring how and why people use visualizations in casual contexts: modeling user goals and regulated motivations. Inf Vis 2012; 11(2): 106–123.

34.

Nardi

BA.

Studying context: a comparison of activity theory, situated action models, and distributed cognition. In: Nardi

(ed) Context and consciousness: activity theory and human-computer interaction. MIT Press, 1996, pp. 69–102. Preprint retrieved from CiteSeer.

35.

Sedlmair

Frank

Munzner

. RelEx: visualization for actively changing overlay network specifications. IEEE TVCG 2012; 18(12): 2729–2738.

36.

Wagner

Aigner

Rind

. Problem characterization and abstraction for visual analytics in behavior-based malware pattern analysis. In: Proc. 11th Workshop Visualization for Cyber Security, VizSec (eds Whitley

Engle

Harrison

.), pp. 9–16. ACM.

37.

Meyer

Sedlmair

Quinan

. The nested blocks and guidelines model. Inf Vis 2015; 14(3): 234–249.

38.

Huh

Park

. LiveGantt: interactively visualizing a large manufacturing schedule. IEEE TVCG 2014; 20(12): 2329–2338.

39.

Zhao

Chevalier

Pietriga

. Exploratory analysis of time-series with ChronoLenses. IEEE TVCG 2011; 17(12): 2422–2431.

40.

Saraiya

North

Duca

An insight-based methodology for evaluating bioinformatics visualizations. IEEE TVCG 2005; 11(4): 443–456.

41.

Smuc

Mayr

Lammarsch

. To score or not to score? Tripling insights for participatory design. IEEE Comput Graph Appl 2009; 29(3): 29–38.

42.

Forsell

Cooper

An introduction and guide to evaluation of visualization techniques through user studies. In: Huang

(ed) Handbook of human centric visualization. New York, NY: Springer, 2014, pp. 285–313.

43.

Javed

McDonnel

Elmqvist

Graphical perception of multiple time series. IEEE TVCG 2010; 16(6): 927–934.

44.

Aigner

Rind

Hoffmann

Comparative evaluation of an interactive time-series visualization that combines quantitative data with qualitative abstractions. Comput Graph Forum 2012; 31(3): 995–1004.

45.

Dou

Jeong

Stukes

. Recovering reasoning processes from user interactions. IEEE Comput Graph Appl 2009; 29(3): 52–61.

46.

Shneiderman

. The eyes have it: a task by data type taxonomy for information visualizations. In: Proceedings of the IEEE symposium on visual languages, VL, 1996, pp. 336–343.

47.

North

Chang

Endert

. Analytic provenance: process+interaction+insight. In: Proceedings of the 2011 annual conference extended abstracts on human factors in computing systems, CHI EA, 2011, pp. 33–36. ACM.

48.

Jankun-Kelly

Gertz

A model and framework for visualization exploration. IEEE TVCG 2007; 13(2): 357–369.

49.

Wright

Schroh

Proulx

. The sandbox for analysis: concepts and methods. In: Proceedings of the ACM SIGCHI conference on human factors in computing systems, CHI, 2006, pp. 801–810. ACM.

50.

Shrinivasan

van Wijk

. Supporting the analytical reasoning process in information visualization. In: Proceedings of the ACM SIGCHI conference on human factors in computing systems, CHI, 2008, pp. 1237–1246.

51.

Tominski

Fuchs

Schumann

. Task-driven color coding. In: Proc. 12th Int. Conf. Information Visualisation, IV, 2008, pp. 373–380. IEEE.

52.

Alsallakh

Micallef

Aigner

. Visualizing sets and set-typed data: state-of-the-art and future challenges. In: Eurographics conference on visualization (EuroVis) – state of the art reports (eds Borgo

Maciejewski

Viola

), 2014.

53.

Rind

Wang

Aigner

. Interactive information visualization to explore and query electronic health records. Found Trends Human-Comput Interact 2013; 5(3): 207–298.

54.

Schumann

Müller

Visualisierung: grundlagen und allgemeine methoden. Berlin: Springer, 2000.

55.

Amar

Stasko

JT.

Knowledge precepts for design and evaluation of information visualizations. IEEE TVCG 2005; 11(4): 432–442.

56.

Wehrend

Lewis

. A problem-oriented classification of visualization techniques. In: Proceedings of the 1st IEEE conference visualization, 1990, pp. 139–143.

57.

Ahn

Plaisant

Shneiderman

A task taxonomy for network evolution analysis. IEEE TVCG 2014; 20(3): 365–376.

58.

Brehmer

Sedlmair

Ingram

. Visualizing dimensionally-reduced data: interviews with analysts and a characterization of task sequences. In: Proceedings of the 5th workshop beyond time and errors: novel evaluation methods for visualization, BELIV (eds Lam

Isenberg

.), 2014, pp. 1–8. ACM.

59.

Kerracher

Kennedy

Chalmers

A task taxonomy for temporal graph visualisation. IEEE TVCG 2015; 21(10): 1160–1172.

60.

Lammarsch

Rind

Aigner

. Developing an extended task framework for exploratory data analysis along the structure of time. In: Proceedings of the international EuroVis workshop on visual analytics, EuroVA (eds Matkovic

Santucci

), 2012, pp. 31–35. Goslar: Eurographics.

61.

Lee

Plaisant

Parr

. Task taxonomy for graph visualization. In: Procedings of the 2006 AVI workshop beyond time and errors: novel evaLuation methods for information visualization, BELIV (eds Bertini

Plaisant

Santucci

), 2006, pp. 81–85. ACM.

62.

MacEachren

AM.

How maps work. New York: Guilford Press, 1995.

63.

Pretorius

Purchase

Stasko

JT.

Tasks for multivariate network analysis. In: Kerren

Purchase

Ward

(eds) Multivariate network visualization. Cham, Switzerland: Springer, 2014, pp. 77–95.

64.

Valiati

ERA

Pimenta

Freitas

CMDS

. A taxonomy of tasks for guiding the evaluation of multidimensional visualizations. In: Proceedings of the 2006 AVI workshop beyond time and errors: novel evaluation methods for information visualization, BELIV (eds Bertini

Plaisant

Santucci

), 2006. ACM.

65.

Suo

A task-centered visualization design environment and a method for measuring the complexity of visualization designs. Computer science dissertations, Georgia State University, Atlanta, GA, 2009.

66.

Meyer

Munzner

Pfister

MizBee: a multiscale synteny browser. IEEE TVCG 2009; 15(6): 897–904.

67.

Chuah

Roth

. On the semantics of interactive visualizations. In: Proceedings of the IEEE symposium information visualization. 1996, pp. 29–36.

68.

Liu

Stasko

JT.

Mental models, visual reasoning and interaction in information visualization: A top-down perspective. IEEE TVCG 2010; 16(6): 999–1008.

69.

Sedig

Parsons

Interaction design for complex cognitive activities with visual representations: a pattern-based approach. AIS Trans Human-Comput Interact 2013; 5(2): 84–133.

70.

von Landesberger

Fiebig

Bremm

. Interaction taxonomy for tracking of user actions in visual analytics applications. In: Huang

(ed) Handbook of human centric visualization. New York: Springer, 2013, pp. 653–670.

71.

Mayr

Smuc

Risku

Many roads lead to Rome: mapping users’ problem-solving strategies. Inf Vis 2011; 10(3): 232–247.

72.

Shneiderman

Creativity support tools: accelerating discovery and innovation. Communications of the ACM 2007; 50(12): 20–32.