An empirical evaluation of three-dimensional pie charts with individually extruded sectors in a geovisualization context

Abstract

This study compares participants’ performance in extracting information from three-dimensional pie charts with individually extruded sectors in a single map frame against two-dimensional pie and bar charts in adjacent map frames. Specifically, we examine the response accuracy and response times of 181 adults who were asked to (1) identify the highest magnitude, (2) estimate a proportion, (3) perform both at the same time, and (4) solve a map-related task using the two chart setups. For each task, charts were shown on backgrounds with increasing visual complexity: a blank, a borders-only, and a choropleth map. Furthermore, we tested whether participants’ performance improved through additional practice with the two chart types. We did not observe any differences in participants’ aggregated response accuracy or response times between the tested three-dimensional and two-dimensional chart types for the (1) highest magnitude task and (2) proportion task. However, participants solved the (3) combination task with two-dimensional pie and bar charts on a blank background more accurately and were faster in fulfilling the (4) spatial task with three-dimensional pie charts. The first difference, however, leveled for participants who gained more practice and who accomplished the combination task on maps with higher visual complexity, whereas the second difference persisted even for more trained subjects.

Keywords

Two-dimensional three-dimensional pie charts bar charts maps performance measurement visual complexity learning

Introduction

Charts are a widespread means to represent multivariate data. Over the years, scientists have developed and formalized various chart types,^1–4 among which pie charts and bar charts are probably the most common. Users and applications of charts are manifold. For example, decision-makers frequently encounter charts in reports about financial data⁵ or when analyzing usage metrics in web dashboards.⁶ If the underlying data were spatially referenced, charts can be placed on a map to depict the proportions of a thematic variable at different locations. This enables, for instance, marine biologists to explore oceanographic data⁷ or educators to display imports and exports of a country in school atlases.⁸

While chart use is rather commonplace, the question of which chart types should be used in which contexts is subject to debate.⁹ To determine this, user performance with pie charts and bar charts has been previously investigated for certain setups and scenarios. In a seminal study on the topic, Cleveland and McGill¹⁰ reported that nearly double as many subjects judged the change in percentage of the second largest value correctly with bar charts than with pie charts when the maximum value has been indicated beforehand. However, Simkin and Hastie¹¹ demonstrated that while participants compared values more accurately with bar charts, they judged proportions of the whole more accurately with pie charts. Later, Spence and Lewandowsky¹² confirmed that bar charts are favorable for comparing magnitudes, while pie charts are advantageous over bar charts when comparing combinations of proportions. In another study, further confirming the previous findings, Hollands and Spence¹³ stated that participants needed more time and made more errors when detecting small differences in changing values with pie charts than with bar charts, whereas the performance was better with pie charts in proportion judgment tasks.

Similar to the pie versus bar chart debate, another research topic is whether and when to add a depth cue to charts (i.e. the three-dimensional or 3D). Arguably, adding a depth cue to two-dimensional (2D) charts increases their attractiveness.¹⁴ It has been demonstrated in a study, wherein a selection of 70 different diagram types were compared, that participants favored 3D pie charts against all others in terms of aesthetic appeal.¹⁵ What people prefer is an important factor to consider in designing graphics, to ensure that the graphics are embraced by users; however, how depth cues on charts affect user performance while solving different tasks is critically important as well. Several experiments demonstrate that people perform better or at least similarly well with 2D charts than with their 3D equivalents. For instance, Siegrist¹⁶ reports that participants were slower and less accurate in estimating proportions in pie charts and magnitudes of bar charts when decorative depth was added. Participants in Hughes’¹⁷ experiment were also less accurate in comparing relative sizes with 3D bar charts than with 2D bar charts. Stewart et al.¹⁸ showed that participants performed a read-off task “equally well” with 2D pie charts and 2D bar charts compared with their 3D counterparts. Participants performed better with 2D charts than with 3D charts when the task type was changed to a spatial transformation task or a trend prediction task. In another study examining different task types, Schonlau and Peters¹⁹ reported that depth cues reduce subjects’ comprehension of pie charts, but not of bar charts. It is important to note that in all of these studies, the third dimension was extraneously used, that is, it was task-irrelevant and did not convey any information and thus essentially would be regarded as “chartjunk” by Tufte.²⁰

Influence of visual variables—such as size, orientation, and color²¹—on user performance with charts has been examined in addition to depth cues. For instance, Zacks et al.²² stated that participants’ judgments of absolute and relative bar heights were less accurate when depth cue was added. Zacks et al.²² altered the length of depth cues and changed their type to rectangles; these modifications led to similar results. Likewise, Fischer²³ examined the effect of 2D and 3D frames as visual aids for 2D and 3D bars. In Fischer’s²³ study, participants needed the most time with 3D bars in 3D frames to compare bar sizes. Fischer²³ interpreted the result as an indication that the working memory load was the highest in the 3D setup. Studying yet another visual variable, Rangecroft²⁴ reported that the orientation of pie chart sectors may influence performance. Participants in Rangecroft’s²⁴ experiment identified the smallest and largest segments more accurately in 2D than in 3D pie charts, whereas the accuracy was low (under 75%) or very low (50% or lower) for half of the 3D pie chart configurations differing in orientation. In a recent study, Kosara²⁵ varied single pie chart sectors in terms of viewing angle, body height, value range, and rotation around the center. Study participants were asked to replicate the altered pie segments interactively in a 2D reference pie chart. Kosara²⁵ observed a higher number of errors for viewing angles of 15° and 30° than those of 60° and 90°, as well as for sector orientations at the side than at the front or back of the chart; however, body heights did not affect the error rates. Note that newer studies, such as Schonlau and Peters’,¹⁹ used colored chart depictions, whereas charts in older experiments, like Siegrist,¹⁶ were mainly drawn in black and white. Even though color is a strong visual variable (e.g. Brychtová and Çöltekin²⁶), Stewart et al.¹⁸ stated that “graph comprehension accuracy was similar for color and black-and-white graphs” (p. 198) as a result of an experiment manipulating color, in which they compared charts both with and without depth cues. Depending on the colors used in the charts, this result could be challenged, but a comprehensive review of the effects of color is beyond the scope of this article.

In this study, inspired by the use of 2D and 3D charts in the Atlas of Switzerland,²⁷ an important objective is to understand how charts can be successfully combined with maps. One can question if it would be better to avoid adding charts altogether, and depict all attributes on side-by-side “small multiple” maps;²⁰ however, visualizing high-dimensional data has been a persistent challenge²⁸ and empirical evidence on the effectiveness of particular representations or their combinations is often missing. Thus, we briefly review some of the previous studies analyzing user performance with charts on maps or similar geographic visualizations.

Cleveland et al.²⁹ compared the performance of high-school students and scientists with proportional circles on statistical maps with “map-like” stimuli (i.e. ticks, border, labels, scale bar) and “non-map-like” stimuli (i.e. a blank background). Circles were randomly positioned in an invisible 4 × 3 grid in both setups. The authors noted that the difference between map-like and non-map-like stimuli is “sufficiently small” (p. 544), thus they did not include it in their analysis.²⁹ As one of the earliest works in cartography examining the role of 3D symbols on maps, Kraak’s³⁰ dissertation included border maps for mono and stereo vision with 3D point symbols (i.e. cubes, spheres) varying in visual variables (i.e. size, color). Three different types of questions involving the subjects to read single, neighbored, or all symbols were formulated. In Kraak’s³⁰ study, geodesy and cartography students were significantly faster and provided more correct answers in stereo than in mono maps with differently sized 3D symbols. In another study, Lewandowsky et al.³¹ conducted two experiments, in which they compared color and grayscale choropleth maps, dot density maps, and maps with pie charts, where all maps included borders of Health Service Areas or US states. Participants were asked to identify and recall clusters of particularly high or low mortality for certain diseases. Lewandowsky et al.³¹ reported that people forgot more clusters on Health Service Areas than on US states boundaries, although the immediate perception of clusters was about the same. Participants identified fewer clusters on maps with pie charts than on monochrome choropleth maps on average. However, Seipel and Carvalho³² did not find any differences concerning accuracy and time when participants judged the relative heights of single bars visualized on a 2D and a tilted 3D base map. The latter results coincide with those of Fischer,²³ who left the background blank in his study.

Whether the background of a chart is blank or not, it is important to remember that most modern maps, atlases, and other geovisualization outputs are presented interactively. Thus, results from studies examining static charts and maps alone might not necessarily apply to more recent visualization work. Examining the role of interaction in participant performance with charts and maps, Bleisch et al.³³ stated that participants needed significantly more time in estimating differences of bars in an interactive 3D geovisualization environment than in a static 2D alternative. However, the authors suspected it was mainly due to the navigation in the 3D space. Participants also recognized bar heights faster with a reference grid than without in the 3D maps.³³

When people face new visualizations, they need to learn how to read and interpret them. This is broadly termed “graphical literacy” and it is a part of the curriculum in schools in many countries. In connection to graphical literacy and learning, Ben-Chaim et al.³⁴ demonstrated that high-school students could improve their spatial visualization abilities through instruction. Ben-Chaim et al.’s³⁴ finding was based on a task to match 2D squares from different views of 3D cubes, which represented buildings on a map, and vice versa. Two years later, in a comprehensive essay, Pinker³⁵ noted that the ability to read graphs can be improved by explicit or implicit instructions with practice and with experience in drawing quantitative relationships. Shah and Hoeffner³⁶ added that it is helpful if readers understand the scientific context and if they are able to switch between chart representations, because they can connect visual features and meaning.

As learning is a lifelong process, professionals may also encounter and need to adapt to unfamiliar visualizations. Krishnamoorthy and North,³⁷ for instance, tested the learnability of a mapping application, including visualizations like a histogram and a scatter-plot in side-by-side frames. In particular, they examined the effect of exploratory learning by guiding users only by the application without providing any additional explanations—apart from the possibility of consulting the help menu. Krishnamoorthy and North³⁷ observed that “[n]ovices tend to stick to initially learnt strategies” (p. 310), which may not be the most efficient ones. The authors suggested “self-disclosure” as a method to overcome this problem, that is, giving context-sensitive help to users while performing a task for the first time. To measure the visualization literacy of laymen, Lee et al.³⁸ developed a test with eight tasks for 12 visualization types. Tasks for pie and bar charts, for example, involved retrieving a value, finding the extremum, and making a comparison. Domain experts judged the content of all tasks for pie charts as equally valid, while they assigned higher scores to the first two tasks for bar charts than to the comparison task. This comparison task was also the most difficult to answer for tested participants in the study of Lee et al.³⁸

We can conclude from the studies reviewed above that people can compare magnitudes well with bar charts, while pie charts might fit better for identifying a portion of a whole. Adding a decorative depth cue to either chart type has an overall negative impact on performance in a variety of tasks. To our knowledge, however, no studies have examined pie charts in depicting meaningful information in the third dimension on the pie sectors. Current off-the-shelf software like spreadsheet programs (e.g. Microsoft Excel) or geographic information systems (GIS) software (e.g. ESRI ArcGIS) cannot generate 3D pie charts with individually extruded sectors, which may explain the lack of studies. Likewise, it seems that measuring the influence of what is shown in the background has not been subject of research yet. Our literature review revealed that charts are often placed on blank or neutral backgrounds in information visualization experiments, whereas in a geovisualization context, a map in the background is often used. Finally, researchers have offered observations on how people learn to work with new visualizations, and charts in particular, suggesting that design of a chart might have a different effect on first-time and returning users.

With this article, we contribute toward a better understanding of chart use by answering the following questions through an empirical study: (1) How well do people perform in chart- and map-reading tasks with 3D pie charts in comparison with 2D charts when the third dimension represents meaningful information? (2) Do the performance differences among people using 2D versus 3D charts persist when the background is no longer blank? (3) How does practice affect performance with “novel” or less usual chart types? Below, we present more precise questions and the related hypotheses.

Hypotheses

This study primarily examines how well people can read 3D pie charts with their sectors extruded proportionally to individual data values: One attribute is mapped on the pie sector angle and another is depicted on the height of the extruded sector. A real-world example, where 3D pie charts represent the number and capacity of wood-fired heating systems, has been implemented in a national atlas.³⁹ The topic of this study pertains to traffic volumes and commuting times for different means of transportation. We compare 3D pie charts with 2D pie and bar charts in adjacent frames, where both the 2D and 3D setups encode identical information. The extrusion in the 3D pie chart is essentially a way of integrating the 2D bar chart onto a 2D pie chart.

Task type

We test the participants’ performance (response accuracy and response times) with four different tasks varying in type and difficulty. We hypothesize that while participants should be more accurate in estimating the magnitude of the sectors with the 2D chart combination due to perceptual skew introduced by the perspective view in 3D pie charts,⁴⁰ they should be faster in using 3D pie charts than using 2D pie and bar charts in adjacent frames because the 2D setup would require more eye movements over a larger space.⁴¹

Background complexity

Differences in performance were often observed in previous studies when the chart was presented on a blank background. As charts have been used on maps for nearly two centuries (e.g. Minard⁴²), we want to examine whether the presence of visual “noise” (i.e. map elements and colors) influences participants’ performance in reading chart values when the background map is not relevant for solving the task in that moment. In geovisualization contexts, the map in the background is typically relevant; however, it might also act as a distractor if one is trying to read information only from the charts. When the map is relevant, attention is usually switched between the background map and the charts. In this study, we examine one task with a meaningful interaction with the background map, but also focus on the effect of “noise” as one of the possible scenarios. We vary the backgrounds from a blank to two “noisy” backgrounds in the first tasks and finally present a task where the map is relevant. For the tasks with visual “noise” in them, we hypothesize that the perceptual advantages offered by each chart type should be more pronounced in the presence of a non-blank background, because the visual complexity created by this “noise” makes the tasks cognitively harder for the participants by forcing them to process more visual input than necessary.^43–45

Practice levels

An additional consideration in a study like ours is whether the proposed visualization type, that is, the 3D pie chart with individually extruded sectors—is “learnable.” A strong motivation for this aspect is the frequent use of charts in school atlases or in other educational materials. We hypothesize that if participants had a chance to get some practice (e.g. by solving similar tasks more than once), their performance would overall improve, especially for a visualization type that is unknown or less familiar to them. More specifically, we expect more practiced participants to show level performance with the new visualization (i.e. the 3D pie chart) and the baseline visualization type (i.e. the 2D pie and bar chart combination).

Methods

Participants

We recruited participants from the second author’s personal and university network by email and social media to participate in the experiment. A total of 181 adults (111 males and 70 females) aged 19–77 years (M = 32.28, SD = 11.10) completed our study. We excluded four participants from the statistical analysis as they exhibited color deficiency in our screening with the Ishihara⁴⁶ test. Most participants (63%) indicated that they hold a university degree, whereas the remaining participants have completed an apprenticeship (15%) or secondary school education (22%). Participants reported that they were familiar with current information technologies: 6% have very poor/poor, 30% average, and 64% good/very good general computer and visualization knowledge and skills. Based on self-reports, participants’ experience with maps and other cartographic products is balanced: 37% have very poor/poor, 36% average, and 27% good/very good cartographic knowledge and skills.

Participants were randomly assigned into one of two groups: 91 adults (53 males and 38 females) aged 22–77 years (M = 33.51, SD = 11.68) to Group A and 86 adults (54 males and 32 females) aged 19–71 years (M = 31.14, SD = 10.51) to Group B. The differences between the groups are explained in detail in the “Procedure” section.

Materials

We created 36 maps varying in chart type, number of frames and charts, and background type (Figure 1). Six maps containing one 2D pie chart and six other maps containing one 2D bar chart were presented to Group A in Tasks 1 and 2. These 12 maps were meant as “fillers” to prevent Group A participants from practicing answering questions with 2D charts in adjacent (=juxtaposed or side-by-side) frames and 3D pie charts. The latter stimuli, which included six maps with one 2D pie chart and one 2D bar chart in adjacent frames as well as six other maps with one 3D pie chart in a single frame, were shown to Group B participants in Tasks 1 and 2 to comparatively assess their performance with these setups. Map sets with the same chart type in each task differed in background (i.e. blank, border, choropleth) to additionally examine the influence of visual “noise” on participants’ performance. The same setup for Group B, comprising three maps each with adjacent 2D charts and single 3D pie charts, was provided to both groups in Task 3, but the background was still not meaningful for solving the task. In Task 4, however, all maps had the choropleth background consisting of seven colors, where each color represented a greater region. Three of the maps comprised seven 2D pie charts and seven 2D bar charts in adjacent frames, and three other maps had seven 3D pie charts in a single frame. The charts were centered in the greater regions in Task 4 and in the map frame in the other tasks. Charts and backgrounds are explained in detail in the following paragraphs, while tasks and groups are described in depth in the “Procedure” section.

Figure 1.

Chart types and background maps used in our study for different groups and tasks: Group A used 2D pie charts or 2D bar charts in a single map frame for Tasks 1 (highest magnitude) and 2 (proportion). For Tasks 3 (combination) and 4 (map-related), Group A used 2D pie and bar charts in adjacent map frames and 3D pie charts with individually extruded sectors in a single map frame. Group B used the latter chart setup for all tasks. A task consisted of three questions. Questions of Tasks 1–3 were performed on a blank map, a borders-only map, or a choropleth map. Questions of Task 4 were performed only on a choropleth map. Chart proportions varied between the questions but were counterbalanced.

We represented fictional values for the topic “Commuting in Switzerland” in all of the charts used in the experiment. The means of transportation was arbitrarily color-coded in the chart segments (i.e. blue = by foot/no commute, green = by bike/moped, yellow = by public transportation, violet = by car/motor bike). The share of people using a certain means of transportation was represented by proportions of 2D and 3D pie chart sectors (e.g. 10%, 20%, 30%, 40%). We chose differences of 10%, as according to Slocum,⁴⁷ differences in two-sectored pie charts smaller than 2.5% cannot be distinguished by participants. Since a recent study shows that people are more effective with basic pie charts,⁴⁸ we did not alter their shapes (e.g. to ellipsis, square) or sectors (e.g. individual radii, translated sectors). The average duration of travel to work (i.e. 0.5, 1, 1.5, 2 h) was proportional to heights of 2D bar charts and extruded sectors of 3D pie charts. We aligned the bars vertically, as it has been shown that people are more efficient in solving tasks with vertical than with horizontal bars.⁴⁹ The assignment of proportions to pie chart sectors as well as to heights of bar charts and 3D pie charts varied for each question in a counterbalanced manner to prevent an additional learning effect. The assignments are not meant to be representative for the topic, since we primarily aimed to control the influence of chart proportions on participants’ performance using the same proportions (i.e. 1:2:3:4) for all 2D and 3D chart segments. Chart size and order of colors of chart segments were also the same in all maps. 3D pie charts were arranged in a way that no occlusions occur, which we could not guarantee when we used real data. The viewing angle of 3D pie charts was 45° and extruded sectors were shaded in black. As shades are not common for 2D chart sectors, we chose a black outline in the corresponding depiction. We created 2D pie and bar charts with Microsoft Excel. As in the study by Schnürer et al.,⁴ 3D pie charts were produced with OpenJSCAD, a JavaScript library for constructing solid geometries, and Blender, a 3D modeling application.

We systematically manipulated the visual complexity of the background on which the charts were displayed, except for the maps with seven charts, since here the background was essential for solving the task. In total, ten maps had a blank background, ten had black lines in the background representing borders of Swiss cantons, and 16 (i.e. choropleth maps) had borders of Swiss cantons as well as seven color-coded NUTS-2 regions⁵⁰ in the background (Figure 1). Borders of Swiss cantons originate from the “Vector 200” data set provided by the Federal Office of Topography swisstopo. The colors of the NUTS-2 regions were chosen close to a map of the Swiss Federal Statistical Office (SFSO)—dark green for Lake Geneva (LG) region, light green for Espace Mittelland (EM), light violet for Northwestern Switzerland (NW), yellow for Eastern Switzerland (ES), orange for Central Switzerland (CS), pink for Zurich (ZH), and red for Ticino (TI).

We provided static maps to the participants so that they are exposed to the same information and cannot change the viewed information by panning, rotating, and zooming. The title was placed in all maps at the top center, the data source at the left bottom, the north arrow at right top, and the scale bar at the right bottom. Legends for angles of pie chart sectors were omitted as their relation to percentages was implied by the questions. Legends for chart segment colors and chart heights were put below data source and scale bar, while the legend for colored NUTS-2 Swiss regions was put at the left top of the map. A short textual explanation was given below the legend (Figure 2). Backgrounds and map elements were created with ESRI ArcMap, apart from the legends, which we generated using Adobe Illustrator.

Figure 2.

Two questions, used in our study for Task 3, containing 2D pie and bar charts in adjacent map frames (left) and 3D pie charts in one map frame (right), both on choropleth background. Participants were asked to select the share of the total traffic volume (=Anteil am Gesamtverkehrsaufkommen) for the transportation means (=Verkehrsmittel) with the longest travel time (=Fahrtzeit). The correct answer was 40%.

Procedure

The experiment (Figure 3) was conducted as an online study and the data collection period was three weeks. All textual descriptions were given in German. Participants were asked to use desktop and mobile devices with a sufficiently large screen so that they would not need to scroll on the page. Participants were ensured of anonymity and confidentiality through a consent form.

Figure 3.

Study design.

In the introductory part of the survey, participants provided demographic information, such as their age, gender, and education level. We also asked about their experience, interest, and skills in using computers, maps, and visualizations using five-point Likert-type scales.

In the main experiment, participants were first familiarized with tasks and maps in a training session. Tasks in the training session were similar to those in the actual experiment; however, proportions of charts (and thus the answers) were different. One example question was given for each task and chart type, except for the last task, for which we gave the participants two example questions because it was the most difficult task. The correct answer was presented after each question to the participants to help them self-assess their accuracy. With the training session, we aimed to acquaint participants with the tasks and visualizations as we assumed that the majority of them would be unfamiliar with 3D pie charts with individually extruded sectors.

The actual experiment in the survey consisted of 24 mandatory questions, that is, one question for each combination of two chart types, three backgrounds, and four tasks. Before the experiment started, participants were asked to answer questions as quickly and accurately as possible. Questions were randomized to control for order effects due to learning, which would favor a certain chart type, background, or task. After every four questions, participants had the opportunity to relax in open-ended breaks. The page layout for each question was as follows: There was a map in the middle of the page, a question on top of the map, and a radio-button box with possible answers below the map (Figure 2). One of four possible answers was correct for the first three tasks and one of seven possible answers (i.e. the NUTS-2 regions) for the last task. The selected answer of participants and the time needed to respond were recorded. There was no time limit for any question.

Tasks and example questions are listed in Table 1. The readability of maximum values in Task 1 is similar to that in Meyer et al.’s⁵¹ experiments, whereas proportion estimations—also known as part or portion of the whole judgments—in Task 2 are based on the study by Simkin and Hastie.¹¹ Tasks 3 and 4 are combinations of Tasks 1 and 2, with the difference that Task 4 additionally involves a geospatial component (i.e. identification of a region). A map-related task, such as Task 4, is representative of chart use in a geovisualization context (e.g. atlases), whereas Tasks 1–3 are also common outside geovisualization (e.g. reports). We designed the tasks to progress from “simple” to “complex” based on our own judgment. Task difficulty was controlled to examine if effects, if any, would persist over more complex tasks as well.

Table 1.

Task types and exemplary questions in our study.

Task type	Example question
Task 1 (highest magnitude): Tick the color of the highest bar or the most extruded pie chart sector.	For 2D pie-and-bar charts: Which bar is the highest one? For 3D pie charts: Which extruded pie sector is the highest one?
Task 2 (proportion): Tick the percentage of a pie chart sector or an extruded pie chart sector in a certain color.	Which share of the whole circle has the orange sector?
Task 3 (combination): Tick the percentage of a pie chart sector from the highest bar or the most extruded pie chart sector in the same color.	Which share of the whole traffic volume has the transportation means with the longest travel time?
Task 4 (map-related): Tick the region from a color-coded means of transportation, a certain pie chart percentage, and the highest bar or most extruded pie chart sector in the same color.	Which NUTS-2 region has the following properties: (a) the highest average travel time for “car/motor bike” and (b) a share of 40% on the traffic volume for “car/motor bike”?

Each task consisted of six questions. For Group A, Tasks 1 and 2 had six questions each for regular pie versus bar charts. For Group B, Tasks 1 and 2 again had six questions each for 2D versus 3D chart setups as described earlier. Tasks 3 and 4 had identical questions for both groups. This design shows whether there is a practice effect for Tasks 3 and 4 because this way, participants in Group B would solve the questions with 2D pie and bar charts in two adjacent frames and 3D pie charts twice as often as participants in Group A during the study. Tasks 1–3 were executed twice on a blank background, twice on a “borders-only” background, and twice on a choropleth map in the background. Task 4 could not be solved without the map; thus, the charts were always shown on a choropleth map. A detailed overview of our study setup is given in Table 2.

After the main experiment, participants took a color perception test⁴⁶ for red–green and blue–yellow, which we used as an inclusion criterion. Participants also responded to a water-level task,⁵² rated their confidence during the experiment, and stated their preferred chart type. In this article, we focus on performance metrics, as described above. Water-level task, confidence, and preference results have been reported earlier by Ritzi (2016).⁵³ The experiment ended with an optional comment box, in case participants had any feedback about the study.

We created the experiment using the online survey tool LamaPoll. Participants did not receive any compensation for their effort and participation was entirely voluntary. In our study, we followed Standard 8 of the Ethical Principles and Code of Conduct for Psychologists.⁵⁴

Statistical analysis

As mentioned in the hypothesis about the task type, we regard performance as a combination of effectiveness (response accuracy) and efficiency (response time) in this study. To quantify response accuracy, we gave one point for correct answers and zero point for incorrect answers, making accuracy a binomial variable. Response time was measured as the time spent by participants to answer a question. We log-transformed response times in all tests to meet the assumption of normal distribution.⁵⁵ We carried out the statistical tests with R (version 3.5.3) and assumed a significance level of α = .05.

Results

In this section, we summarize the results for comparing participants’ performance with 2D pie-and-bar charts (=2D pie and bar charts in adjacent frames) and 3D pie charts (in a single frame). Results for 2D pie versus 2D bar charts in single frames are not reported as they simply replicate the previous findings (i.e. pie charts are better for judging proportions of a whole, while bar charts are better for estimating magnitudes).

Overall performance

Figure 4 shows the main effects for participants’ accuracy (left) and response times (right) with 2D pie and bar and 3D pie charts. The descriptive statistics suggests a slight advantage with 2D pie-and-bar charts in terms of accuracy (M_2D = 91.32%, SD_2D = 28.17% vs M_3D = 89.48%, SD_3D = 30.69%) and with 3D pie charts in terms of response time (M_2D = 18.11 s, SD_2D = 13.91 s vs M_3D = 16.94 s, SD_3D = 12.69 s). However, a weighted Welch’s t-test for the main effects shows that the differences in neither accuracy (t(3120.78) = 1.84, p = .07) nor response time (t(3148.21) = 1.67, p = .09) are statistically significant at the aggregate level.

Figure 4.

Mean response accuracies (left) and response times (right) of questions involving 2D pie and bar charts in adjacent frames and 3D pie charts in a single frame over all tasks. Results of Tasks 1 and 2 (N = 6 × 86) got more weight in Welch’s t-test than results of Task 3 and 4 (N = 6 × 177) to balance the different response counts as only participants of Group B solved Tasks 1 and 2 with 2D pie-and-bar charts as well as 3D pie charts. Error bars: ±SEM.

Effects of task type on performance

Figure 5 depicts the results at the task level for participants who solved all questions with 2D pie and bar and 3D pie charts (Group B). At the left of Figure 5, we see that, irrespective of chart type, the accuracy levels drop as we go from simpler tasks to more complex ones (see Table 1 for their descriptions). Similarly, at the right of Figure 5, we observe that the response time increases in the same direction, confirming that task complexity is highest for the map-related task (Task 4). When we examine the plots for participants’ performance differences based on chart type, a Welch’s t-test shows that subjects are faster in the map-related task (t(257) = 4.6, p < .001, d = 0.29) with 3D pie charts (M = 22.64 s, SD = 11.38 s) than with the 2D pie-and-bar charts (M = 25.62 s, SD = 12.64 s). No other results show significant effects (Table 3).

Figure 5.

Mean response accuracies (left) and response times (left) for the four task types in the experiment. Results of participants with less practice are not included in this chart as they solved only the last two tasks with 2D pie and bar charts in adjacent frames and 3D pie charts in a single frame. Error bars: ±SEM.

For Group A, we report the results in section “Interactions between task type and practice level” because participants in this group solved only the last two tasks with 2D pie-and-bar and 3D pie charts to measure the effect of practice for this task type.

Effects of background complexity on performance

Figure 6 illustrates the participants’ performance with 2D and 3D chart setups as the backgrounds vary. We considered only the results of the first three tasks in this analysis, because the background map is relevant for solving Task 4, therefore making it inherently different from the other tasks. Results did not appear to differ very much (Table 4) between background types or based on the chart types, except in one case: When the background is blank, McNemar’s test confirmed that participants have a higher accuracy (χ²(1, N = 349) = 12.41, p < .001, φ = 0.19) with 2D pie and bar charts (M = 95.70%, SD = 20.31%) than 3D pie charts (M = 89.11%, SD = 31.19%).

Figure 6.

Mean response accuracies (left) and response times (right) for different backgrounds shown for each visualization type. Results of Task 4 (i.e. the map-related task) are not included in this chart, as the task could be performed only on a choropleth map and not on other backgrounds. Error bars: ±SEM.

Effects of practice level on performance

To understand if the performance improves with practice or if the differences based on chart type might disappear once the participants have had a chance to practice, we analyzed the differences between the two groups of participants—one group with less practice (Group A) and another with more practice (Group B). Even though descriptive statistics suggest that participants’ response accuracy is slightly better with the 2D chart combination and that this difference disappears with practice (Figure 7, left), the differences are statistically significant for neither the less practiced nor the more practiced participants (Table 5).

Figure 7.

Mean response accuracies (left) and response times (right) of participants with less practice and more practice. The first two tasks are not included in this chart because less practiced participants solved these tasks with 2D charts in single-map frames only. Error bars: ±SEM.

Response time analysis shows a different pattern: both the less practiced and more practiced participants were faster with the 3D than the 2D chart setup (Figure 7, right). These differences are statistically significant both for more practiced participants (t(515) = 2.78, p < .01, d = 0.12): 3D pie charts (M = 19.11 s, SD = 11.25 s) versus 2D pie-and-bar charts (M = 20.3 s, SD = 12.22 s); and for less practiced participants (t(545) = 3.55, p < .001, d = 0.15): 3D pie charts (M = 22.48 s, SD = 14.88 s) versus 2D pie-and-bar charts (M = 24.62 s, SD = 16.31 s).

Interactions between task type and background complexity

As mentioned earlier, we varied backgrounds for the first three tasks and compared the participants’ performance with 2D and 3D charts against these backgrounds. In other words, the backgrounds themselves were introduced as “noise” in the first three tasks. For Tasks 1 (highest magnitude) and 2 (proportion), we observed no statistically significant difference in participants’ performance in terms of accuracy or response time with the 2D and 3D charts on individual background types (Table 6). For Task 3 (the combination task), participants were more accurate in identifying the proportion of the 2D pie chart sector and the highest 2D bar (M = 94.35%, SD = 23.15%) than in specifying the proportion of the most extruded 3D pie sector (M = 84.18%, SD = 36.60%) on a blank map (χ²(1, N = 177) = 10.32, p < .001, φ = 0.24). No interactions were observed between chart types on border and choropleth maps for this task (Table 6).

Also, as mentioned earlier, participants performed Task 4 (the map-related task) only with a choropleth background. We found no statistically significant differences in participants’ response accuracy between the 2D and 3D chart setup with Task 4 (Table 6). However, participants solved Task 4 faster (t(530) = 7.7, p < .001, d = 0.33) with 3D pie charts (M = 24.43 s, SD = 13.70 s) than with 2D pie-and-bar charts (M = 28.45 s, SD = 15.61 s).

Interactions between task type and practice level

For the simpler tasks (Task 1: highest magnitude and Task 2: proportion), understandably, there were no interactions, because those tasks were meant to allow one group to practice more with 2D pie-and-bar and 3D pie charts than the other group.

For Task 3 (the combination task), participants with less practice solved questions with 2D pie-and-charts (M = 92.31%, SD = 26.70%) more accurately (χ²(1, N = 273) = 10.3, p < .001, φ = 0.19) than with 3D pie charts (M = 83.88%, SD = 36.84%). Among the participants with more practice, we did not find any statistically significant differences concerning accuracy rates for the combination task (Table 7). Response times appear to be similar among participants of both groups with 2D and 3D charts for Task 3.

With Task 4 (the map-related task), participants’ response accuracy did not differ statistically significantly between the charts (Table 7). However, participants solved the questions faster with 3D pie charts than with the 2D chart combination irrespective of their practice levels: less practiced participants were faster (t(272) = 5.13, p < .001, d = 0.31) with 3D pie charts (M = 26.12 s, SD = 15.41 s) than with 2D pie-and-bar charts (M = 31.13 s, SD = 17.58 s); the same results were seen for the more practiced participants (t(257) = 3.51, p < .001, d = 0.22; M_3D = 22.64 s, SD_3D = 11.38 s; M_2D = 25.62 s, SD_2D = 12.64 s).

Interactions between task type, background complexity, and practice level

As backgrounds were varied in the first three tasks and participants with different practice levels had the same questions in the last two tasks, interactions between background visualizations and practice levels could have only occurred in Task 3, that is, the combination task. Here, only participants with less practice (χ²(1, N = 91) = 13.47, p < .001, φ = 0.38) were more accurate with 2D pie-and-bar charts (M = 95.60%, SD = 20.61%) than with 3D pie charts (M = 76.92%, SD = 42.37%) on the blank map, and not the participants with more practice (Table 8). We did not find any interactions between the chart types on other background types (border and choropleth) among participants with different practice levels.

Participants’ comments

Two participants commented in an optional text area for feedback that 3D pie charts are practical because everything is included in one chart and not split into two. Four participants remarked that it is difficult to read 3D pie sector proportions. One participant gave a negative comment about using 3D visualizations in general.

Discussion

Before we interpret the observations presented in this article, it is worth noting that, overall, the success rates are quite high (80%–100%) in our study, in which 3D pie charts with individually extruded sectors were compared against 2D pie and bar charts in adjacent frames. On one hand, these high success rates indicate that the participants took the survey seriously (i.e. they did not guess but solved the tasks). On the other hand, we see a ceiling effect, especially for Task 1 (i.e. estimating the highest magnitude), where 100% of the participants were able to give the right answer. This may partly be explained by the way the task was designed: We gave our participants four options to choose from, which may have made the task too easy to detect any performance differences between 2D pie-and-bar and 3D pie charts, and which is in contrast to Cleveland and McGill’s¹⁰ experiment where participants were asked to respond with exact values. We opted for this task design because, as Spence and Lewandowsky¹¹ posited, the purpose of charts is not in telling exact values (tables should be used for such tasks) but to estimate approximate relative proportions. Our results thus provide evidence that the chart types in this study worked well for such a task.

The response times varied considerably over the entire experiment: Participants took 5.4 s for the easiest question and 29.7 s for the hardest question on average. We believe that these differences in response times are linked to task complexity—the easiest question required only one cognitive action (i.e. identification of the highest magnitude), whereas the hardest question required six mental steps (i.e. identification of the highest magnitude and proportion, color-coding and color-encoding, spatial comparison, and referencing). We speculate that the number of operations involved in harder tasks must have led to considerably higher cognitive load, which, in turn, would explain this gap in response times. Even though we examined outliers and found only one extreme value for the proportion task with 2D pie charts in a single frame (Table 2), which was not relevant for the results we presented, it is still possible that our participants could have been distracted during the online survey. If they did, the distractions most likely occurred randomly, and thus would be distributed on different conditions. Hence, there is no reason to believe that such interference would have favored a particular variable that we examined. Similar to Fischer et al.,⁴⁸ we had breaks after every four questions, which we believe should have helped to counter such disturbance and also fatigue; however, overall response times may be lower in a controlled setting. Below, we provide a more fine-grain interpretation of our findings.

Task type and its interactions with other variables

Among the four task types examined, we formulated a two-level hypothesis regarding participants’ performance: For the first two tasks types, we assumed that the 2D pie-and-bar charts should facilitate overall better (i.e. more accurate and faster) information extraction, because only one size variable (pie sector proportion or bar/extruded pie sector height) is tested in these tasks. For the last two tasks, however, participants interpreted two size variables (pie sector proportion and bar/extruded pie sector height). For these tasks (Tasks 3 and 4), 3D pie charts could offer advantages over 2D pie-and-bar charts, because the 3D setup should mitigate the potential split-attention effects by bringing the multivariate information closer together than the 2D setup, thus restricting the search space. Our results demonstrate that the differences in participants’ performance are not as “clear-cut” as we expected between the two chart setups for many tasks.

Reported as an interaction between the task types and background complexity, we see that in only one of the four tasks (Task 3, the combination task), participants solved one question (i.e. blank background) using the 2D pie-and-bar charts with a higher accuracy than with the 3D pie charts. We believe that this result is more relevant for the background complexity (elaborated more in the next section) than the task type. Nonetheless, if we look at this result from the perspective of tasks types, we see that this task required the reading of two size variables, which contradicts our hypothesis that 3D pie charts should be better suited for tasks because of the compact representation of the information. For the other three tasks, participants’ success rates did not differ between 2D and 3D chart setups. While interpreting “no difference” is statistically not justified, the fact that we do not observe any differences in these cases seems to be in contradiction with some earlier findings (e.g. Siegrist¹⁶), where participants were less successful in solving various chart reading tasks with 3D pie charts with equally extruded sectors. A more detailed examination of the results (Table 2) reveals that participants’ accuracy in estimating proportions with single 2D pie charts (Group A) and 3D pie charts (Group B) on a blank background for Task 2 matches with results of previous studies regarding depth cues in charts, which makes sense because the earlier work also used blank backgrounds.

General change in task difficulty remains only a descriptive observation for response times in Tasks 1–3, that is, there were no significant differences in participants’ response times between 2D and 3D chart setups in these tasks. In Task 4, in accordance with our hypothesis, participants solved the more complex map-related task faster with 3D pie charts than with 2D pie-and-bar charts. A reason for this may be the distance between the pie chart and the bar chart in the side-by-side representation, which would require more eye movement to interrelate corresponding values of the two charts. The larger difference in response times for more difficult tasks is consistent with the results of Stewart et al.¹⁸ with one important difference: participants are faster with 3D charts in our study—aside from the observations of “no difference” which we cautiously interpret as “at least as fast.” This might be because the information depicted by the extruded sectors of 3D pie charts is necessary to solve the task. Note that we do not observe a speed-accuracy tradeoff (i.e. participants did not rush to finish the experiment and took the time they needed to work out the answer); thus, we believe that the interpretation of the response time as a performance measure is meaningful.

Background complexity and its interactions with other variables

Previous studies comparing chart types have been typically conducted on blank backgrounds (e.g. Rangecroft²⁴), whereas this is rarely the case when the charts are used with maps (e.g. in atlases). Thus, we tested the effect of having some visual noise in the background. Note that we use the word “noise” because the background maps in Tasks 1, 2, and 3 were irrelevant for solving the tasks, and these are the tasks in which we have a comparative measure of how background complexity affects the performance outcome. We expected any perceptual advantages offered by either chart setup—2D or 3D—to be amplified when there was noise in the background, because the participants would need all the assistance they could get in the presence of extraneous information. We did not expect differences in time taken by participants in locating the charts with different backgrounds, because the charts were quite salient and easy to detect in all conditions: 3D charts were shaded and 2D charts were outlined, and both chart types were placed in the center of the maps. Here, we will discuss the results only at an aggregate level and not between individual backgrounds per task type, since we also changed chart proportions in the questions to counter-balance for learning effects.

On the whole, varying the background complexity did not create a statistically significant effect in participants’ response times with the 2D and 3D charts in this experiment. In an earlier study, Neider and Zelinsky⁵⁶ reported similar results for a visual search task where neutral and “camouflage” backgrounds caused no difference in efficiency (they did not report effectiveness). However, importantly, and as mentioned in the previous section, we observed that participants’ accuracy was higher with the 2D pie-and-bar than with the 3D pie charts only on the blank background. This difference was no longer statistically significant when we introduced visual noise in the background. As opposed to our hypothesis, this finding suggests that increasing the background complexity reduces the differences between the 2D and 3D chart setups, seemingly impairing participants’ performance in the 2D condition. Why should visual noise in the periphery weaken performance with 2D pie and bar charts? We suspect that it might be related to the size of the frame in which the 2D and 3D charts were displayed in this study: 2D pie and bar charts were presented in two adjacent frames, covering a considerably larger space than the 3D pie charts on the display. This possibly forced the participants to use more information from the peripheral vision as they needed to compare the pie chart with the bar chart, which was placed away. Thus, the presence of visual noise in this operation might have increased cognitive load, as one must actively ignore the incoming signal. The 3D pie chart, on the other hand, was compact in presentation and thus possibly did not require processing peripheral (or perhaps even parafoveal) visual information at all.²⁶ A future study including eye movement analyses would confirm this speculation.

Furthermore, we used a specific color configuration in this study. Following up on this work, it would be interesting to test other color configurations or distances. For example, as demonstrated by Brychtová and Çöltekin²⁶ for choropleth maps, varying the color distance between chart sectors and background might lead to different results; further studies examining this aspect of the visual design would make our findings more robust.

Practice levels and its interactions with other variables

To understand if additional practice would remove some of the issues with the “seemingly more difficult” chart types—the 2D pie and bar chart combination and 3D pie charts—we allowed Group B participants to work with these charts twice as often as Group A participants. Apart from improving overall performance, we speculated that any performance differences with 2D and 3D charts might disappear or at least become weaker, because as they practice, participants might develop strategies to counter the issues with the particular chart types. If participants were more experienced with one rather than the other chart type, this gap might become smaller through practice for either of the two chart types. Note that all participants had a training session at the start of the experiment; thus, Group B’s exposure to twice as many tasks with 2D pie-and-bar charts and 3D pie charts was an additional chance to practice and improve. However, as opposed to the training session, Group B did not receive additional instructions or feedback for these questions.

Confirming our expectations, we observed that, overall, participants with more practice (Group B) responded faster than those with less practice (Group A), although the overall accuracy rates did not differ between the two groups (Figure 7). This finding confirms the common sense understanding that repeating a task is likely to increase efficiency. In some other setups, it may also lead to higher accuracy, but in our experiment, the accuracy rates were already very high, as discussed earlier.

Examining the interactions between the experimental factors revealed an interesting result related to our hypothesis on practical levels: Differences in accuracy between 2D and 3D chart setups vanished for more practiced participants compared with those with less practice for the combination task (Task 3, i.e. identifying the highest magnitude and estimating a proportion at the same time). We believe this is an important result in the 2D versus 3D debate as it suggests that in some contexts, if people are able to practice, 3D charts might not hurt performance. Future studies examining the effectiveness of visualizations might benefit from bearing this in mind (i.e. by controlling for practice levels in the experiments). Following from this observation, it would be interesting to assess the steepness of the learning curve, the number of attempts after which the maximum performance converges, and how well participants can remember their acquired chart literacy skills.

Further remarks and future work

Below, we list some constraints to the study of which the readers should be aware when interpreting or attempting to reproduce the results.

First of all, it is important to remember that participants in this study were highly educated (i.e. 63% with university degrees); therefore, our sample is likely not representative of the general public. Furthermore, our participants were possibly mostly Swiss German, meaning that our results may or may not apply to people with different cultural backgrounds. For example, results may be different if the study is performed in countries where science education or preferences for chart colors differ.⁵⁷

As there is always an illumination model involved for 3D visualizations to produce shaded surfaces, colors in 3D pie charts were not exact matches of those in 2D charts. These differences were very subtle and arguably not perceptually relevant, and hence we believe they do not pose a limitation on our findings. Nevertheless, it would be interesting to examine whether the presence of shadow and outlines influences the performance of reading chart values.

Moreover, we arranged 3D pie charts in a way that no occlusions occur between sectors. When mapping non-fictional values onto the 3D charts, it is likely that they will be less readable or even non-readable—for example, when sectors in the foreground are larger than those in the background. Furthermore, occlusions would occur between extruded sectors in interactive 3D environments when moving and rotating the camera, whereas this is not the case for 2D charts. However, one can adjust the view in an interactive system to find the desired vantage point that enables maximum access to relevant information. Such additional action(s) would probably lower participants’ performance with the 3D pie charts, which remains to be tested in a future study. Furthermore, occlusion occurs not only between the 3D pie charts sectors, but also between the charts and the underlying map. While all of the different map areas are still partly visible in our experiment, point generalization techniques⁵⁸ could be applied to the charts in the cases where the charts fully cover areas. An example solution would be to displace the charts to an “empty” space on the map and add a leader line to link the chart with the associated map area.

In many previous studies involving charts, such as summing up values in a pie chart,⁵⁹ tasks were only performed with one chart, but not for a chart combination (i.e. pie-and-bar charts), as in our experiment. We decided to position the 2D charts in adjacent map frames, since interactive geovisualization tools often offer a split-screen function by which different data sets can be compared. We always placed pie charts within the left frame and bar charts within the right frame in our experiment. Variations in position (e.g. top–bottom) may have led to different results. Moreover, the two static frames in this study were kept adjacent to each other, causing split attention. This effect would possibly be amplified if the information was spread to even more maps, which is the case for other high-dimensional representations such as small multiples.²⁰ Small multiples usually show only one variable per map and would probably facilitate comparison of values of the same category between map areas. However, the interrelation of different category values for a particular map area would be impeded. Other 2D chart types, for example, wing charts,² or interactive techniques such as data lenses or swipe tools,⁶⁰ are possible alternatives to integrate all information in one map, thus to isolate the effect only to the change from 2D to 3D. Finding the best techniques to visualize multivariate data from a cognitive performance point of view may be a worthwhile subject to study in the future for similar configurations.

Further considering experimental control versus ecological validity, Tasks 1–3 should not be regarded as “realistic” use cases because they examine a single chart. Most real-world situations are similar to Task 4 where maps contain multiple charts. However, based on selection, filtering, or highlighting techniques that help to focus the map reader’s attention on a single chart, one can argue that the first three tasks have fairly high ecological validity for digital maps. A different use case to our experiment, where participants were asked to identify the maximum value of bar charts or extruded pie chart sectors, would be to judge the values represented by height. As previous research only tested equally extruded pie chart sectors with meaningless heights, it would be interesting to verify whether the additional mental rotation to compensate the tilted view or summation of sector heights in the foreground and background of the chart decreases peoples’ performance of 3D pie charts when reading height values. This question is motivated by the fact that mental rotation, especially in 3D, is known to be a difficult cognitive task for many people.⁶¹

While questions for Tasks 1 and 2 have been constructed for theoretical purposes, questions for Tasks 3 and 4 tested participants’ understanding of the context as well. In other words, in Tasks 3 and 4, participants needed to read values from the legend, which is a common task in geographic products such as atlases. The design of the questions in Tasks 3 and 4 is similar to that in Schonlau and Peters’¹⁹ experiment and is recommended by Shah and Hoeffner.³⁶

Conclusion

We depicted meaningful information on individually extruded pie chart sectors and compared this 3D chart type against a 2D alternative, in which pie and bar charts were placed in adjacent frames. In eight out of ten comparisons between the two chart types, and participants had similar response accuracies and response times. Participants’ performance did not differ between 2D and 3D chart setups for the easier tasks (i.e. Task 1, identifying the highest magnitude and Task 2, estimating proportions). For the more difficult tasks, however, we observed nuanced differences in participants’ performance with 2D and 3D chart types, which were as follows:

(a) Participants were more accurate with 2D pie and bar charts than with 3D pie charts on a blank background when identifying the highest magnitude and estimating a proportion at the same time (i.e. Task 3, the combination task).

(b) Participants were faster in responding with 3D pie charts than with 2D pie-and-bar charts when performing a spatial comparison task and when using the map legend in addition to the combination task (thus, when executing Task 4, the map-related task).

The difference in accuracy (a) vanished for participants who received more practice during the experiment, whereas the difference in efficiency (b) persisted even after gaining more practice. Increasing the visual complexity of the background—either to border-only or to choropleth maps—also leveled out the accuracy differences, as described in (a).

Based on our results, we believe that information visualizers and map editors can use either chart setup successfully, bearing in mind the discussed experimental constraints (e.g. occlusion). The arguments and previous findings against the 3D pie charts appear to be rather about the extraneous (i.e. decorative) use of the depth cues, and not necessarily detrimental to participants’ performance when the third dimension is used for representing a task-relevant variable. Overall, 3D pie charts with individually extruded sectors, as well as 2D pie and bar charts in adjacent frames, work well to represent multivariate data on static media.

Footnotes

Appendix

Table 8.

Interactions between task type, background complexity, and practice level for 2D pie-and-bar charts and 3D pie charts.

Response accuracy (%)	M _2D	SD_2D	M _3D	SD_3D	χ ²	p	φ	N
Task 3: combination + blank map + Group A: less practice	95.6	20.61	76.92	42.37	13.47	<.001	0.38	91
Task 3: combination + border map + Group A: less practice	91.21	28.47	89.01	31.45	0.08	.77	0.03	91
Task 3: combination + choropleth map + Group A: less practice	90.11	30.02	85.71	35.19	0.56	.45	0.08	91
Task 3: combination + blank map + Group B: more practice	93.02	25.62	91.86	27.5	0	1	0	86
Task 3: combination + border map + Group B: more practice	89.53	30.79	90.7	29.22	0	1	0	86
Task 3: combination + choropleth map + Group B: more practice	89.53	30.79	88.37	32.24	0	1	0	86
Response time (s)	M _2D	SD_2D	M _3D	SD_3D	t	p	d	N
Task 3: combination + blank map + Group A: less practice	18.02	9.95	17.82	12.83	0.5	.62	0.05	86
Task 3: combination + border map + Group A: less practice	18.35	12.1	19.03	11.41	−1.04	.3	0.11	86
Task 3: combination + choropleth map + Group A: less practice	17.96	13.22	19.7	15.67	−1.38	.17	0.14	86
Task 3: combination + blank map + Group B: more practice	14.75	8.83	13.73	8.72	1.21	.23	0.13	91
Task 3: combination + border map + Group B: more practice	15.52	11.26	16.18	8.52	−0.85	.4	0.09	91
Task 3: combination + choropleth map + Group B: more practice	14.66	6.75	16.82	12.07	−0.89	.38	0.1	91

Tasks 1 and 2 were solved with 2D pie-and-bar charts and 3D pie charts by Group B and Task 4 was performed by both groups on a choropleth map. Differences in response accuracy were calculated by McNemar’s test and differences in response time were computed by Welch’s t-test.

Acknowledgements

The authors thank Lorenz Hurni for critically revising the paper.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Raimund Schnürer

References

Wilkinson

. The grammar of graphics. New York: Springer Science + Business Media, 1999.

Schnabel

. Benutzerdefinierte Diagrammsignaturen in Karten, Konzepte, Formalisierung und Implementationen. PhD Thesis, ETH Zurich, Zurich, 2007.

Heer

Bostock

Ogievetsky

. A tour through the visualization zoo. Commun ACM 2010; 53(6): 59–67.

Schnürer

Eichenberger

Sieber

, et al. 3D charts—taxonomy and implementation in a virtual globe. Rev Bras Cartogr 2015; 67(5): 1055–1065.

Ervin

. Pie charts in financial communications. Inf Design J 2011; 19(3): 205–215.

Elias

Bezerianos

. Exploration views: understanding dashboard creation and customization for visualization novices. In: Campos

Graham

Jorge

, et al. (eds) Human-computer interaction—INTERACT 2011. Berlin: Springer, 2011, pp. 274–291.

Kreuseler

. Visualization of geographically related multidimensional data in virtual 3D scenes. Comput Geosci 2000; 26(1): 101–108.

Hurni

. Schweizer Weltatlas. Zürich: Schweizerische Konferenz der kantonalen Erziehungsdirektoren (EDK), 2017.

Spence I. No humble pie: the origins and usage of a statistical chart. J Educ Behav Stat 2005; 30(4): 353–368.

10.

Cleveland

McGill

. Graphical perception: theory, experimentation, and application to the development of graphical methods. J Am Stat Assoc 1984; 79(387): 531–554.

11.

Simkin

Hastie

. An information-processing analysis of graph perception. J Am Stat Assoc 1987; 82(398): 454–465.

12.

Spence

Lewandowsky

. Displaying proportions and percentages. Appl Cognitive Psych 1991; 5(1): 61–77.

13.

Hollands

Spence

. Judgments of change and proportion in graphical perception. Hum Factors 1992; 34(3): 313–334.

14.

Fausset

Rogers

Fisk

. Visual graph display guidelines. Technical report no. HFA-TR-0803, 2008, https://smartech.gatech.edu/bitstream/handle/1853/40573/HFA-TR-0803%20Graph_Guidelines_Final.pdf

15.

Burch

. The aesthetics of diagrams. In: Proceedings of the 6th international conference on information visualization theory and applications, Berlin, 2015, pp. 262–267, https://doi.org/10.5220/0005357502620267

16.

Siegrist

. The use or misuse of three-dimensional graphs to represent lower-dimensional data. Behav Inform Technol 1996; 15(2): 96–100.

17.

Hughes

. Just noticeable differences in 2D and 3D bar charts: a psychophysical analysis of chart readability. Percept Motor Skill 2001; 92(2): 495–503.

18.

Stewart

Cipolla

Best

. Extraneous information and graph comprehension: implications for effective design choices. Campus-Wide Inf Syst 2009; 26(3), 191–200.

19.

Schonlau

Peters

. Comprehension of graphs and tables depend on the task: empirical evidence from two web-based studies. Stat Politics Policy 2012; 3(2), https://doi.org/10.1515/2151-7509.1054

20.

Tufte

. The visual display of quantitative information. 1st ed. Cheshire, CT: Graphics Press, 1983.

21.

Bertin

. Semiology of graphics: diagrams, networks, maps. Madison, WI: University of Wisconsin Press, 1983.

22.

Zacks

Levy

Tversky

, et al. Reading bar graphs: effects of extraneous depth cues and graphical context. J Exp Psychol-Appl 1998; 4(2): 119–138.

23.

Fischer

. Do irrelevant depth cues affect the comprehension of bar graphs? Appl Cognitive Psych 2000; 14(2): 151–162.

24.

Rangecroft

. As easy as pie. Behav Inform Technol 2003; 22(6): 421–426.

25.

Kosara

. Evidence for area as the primary visual cue in pie charts. In: Proceedings of the IEEE VIS short paper, Vancouver, BC, Canada, 2019, https://doi.org/10.31219/osf.io/fcna4

26.

Brychtová

Çöltekin

. The effect of spatial distance on the discriminability of colors in maps. Cartogr Geogr Inf Sc 2017; 44(3): 229–245.

27.

Sieber

Serebryakova

Schnürer

, et al. Atlas of Switzerland goes online and 3D—concept, architecture and visualization methods. In: Gartner

Jobst

Huang

(eds) Progress in cartography. Cham: Springer International Publishing, 2016, pp. 171–184.

28.

Çöltekin

Bleisch

Andrienko

, et al. Persistent challenges in geovisualization —a community perspective. Int J Cartogr 2017; 3(Suppl. 1): 115–139.

29.

Cleveland

Harris

McGill

. Judgments of circle sizes on statistical maps. J Am Stat Assoc 1982; 77(379): 541–547.

30.

Kraak

M-J

. Computer-assisted cartographical three-dimensional imaging techniques. PhD Thesis, Delft University of Technology, Delft, 1988.

31.

Lewandowsky

Herrmann

Behrens

, et al. Perception of clusters in statistical maps. Appl Cognitive Psych 1993; 7(6): 533–551.

32.

Seipel

Carvalho

. Solving combined geospatial tasks using 2D and 3D bar charts. In: 2012 16th international conference on information visualisation, Montpellier, 11–13 July 2012, pp. 157–163. New York: IEEE.

33.

Bleisch

Dykes

Nebiker

. Evaluating the effectiveness of representing numeric information through abstract graphics in 3D desktop virtual environments. Cartogr J 2008; 45(3): 216–226.

34.

Ben-Chaim

Lappan

Houang

. The effect of instruction on spatial visualization skills of middle school boys and girls. Am Educ Res J 1988; 25(1): 51–71.

35.

Pinker

. A theory of graph comprehension. In: Freedle

(ed.) Artificial intelligence and the future of testing. Hillsdale, NJ: Lawrence Erlbaum, 1990, pp. 73–126.

36.

Shah

Hoeffner

. Review of graph comprehension research: implications for instruction. Educ Psychol Rev 2002; 14(1): 47–69.

37.

Krishnamoorthy

North

. Learnability of interactive coordinated-view visualizations. In: Proceedings of the ninth international conference on information visualisation, London, 6–8 July 2005, pp. 306–311. New York: IEEE.

38.

Lee

Kim

Kwon

. VLAT: development of a Visualization Literacy Assessment Test. IEEE T Vis Comput Gr 2017; 23(1): 551–560.

39.

Sieber

Schnürer

Eichenberger

, et al. The power of 3D real—time visualization in atlases—concepts, techniques and implementation. In: Proceedings of the 26th international cartographic conference, Dresden, 2013, http://icaci.org/files/documents/ICC_proceedings/ICC2013/_extendedAbstract/178_proceeding.pdf

40.

Wilkinson

. Graphical methods: presentation. In: Baltes

(ed.) International encyclopedia of the social & behavioral sciences. Oxford: Pergamon, 2001, pp. 6368–6379.

41.

Brügger

Fabrikant

Çöltekin

. An empirical evaluation of three elevation change symbolization methods along routes in bicycle maps. Cartogr Geogr Inf Sc 2016; 44(5): 436–451.

42.

Minard

., 1858, https://commons.wikimedia.org/wiki/File:Minard-carte-viande-1858.png

43.

MacEachren

. The role of complexity and symbolization method in thematic map effectiveness. Ann Assoc Am Geogr 1982; 72(4): 495–513.

44.

Neider

Zelinsky

. Cutting through the clutter: searching for targets in evolving complex scenes. J Vision 2011; 11(14): 1–16.

45.

Schnur

Bektaş

Çöltekin

. Measured and perceived visual complexity: a comparative study among three online map providers. Cartogr Geogr Inf Sc 2017; 45(3): 238–254.

46.

Ishihara

. Tests for color-blindness. Tokyo, Japan: Kanehara Shuppan, 1917.

47.

Slocum

. Analyzing the communicative efficiency of two-sectored pie graphs. Cartographica 1981; 18(3): 53–65.

48.

Kosara

Skau

. Judgment error in pie chart variations. In: Proceedings of the eurographics / IEEE VGTC conference on visualization: short papers, Groningen, 6–10 June 2016, pp. 91–95. New York: ACM.

49.

Fischer

Dewulf

Hill

. Designing bar graphs: orientation matters. Appl Cognitive Psych 2005; 19(7): 953–962.

50.

European Parliament and the Council. Regulation (EC) on the establishment of a common classification of territorial units for statistics (NUTS) (No. 1059/2003), 2003, http://data.europa.eu/eli/reg/2003/1059/2018-01-18

51.

Meyer

Shinar

Leiser

. Multiple factors that determine performance with tables and graphs. Hum Factors 1997; 39(2): 268–286.

52.

Piaget

Inhelder

. The child’s conception of space. New York: Humanities Press, 1956.

53.

Ritzi

. Multivariate Diagramme in kartografischen Produkten. Eine Nutzerstudie zur Evaluation von zwei- und dreidimensionalen Diagrammen in Karten. Master thesis, University of Zurich, Zurich, 2016.

54.

American Psychological Association. Ethical principles of psychologists and code of conduct, 2017, http://www.apa.org/ethics/code/

55.

Lawrence

. The lognormal as event-time distribution. In: Crow

Shimizu

(eds) Lognormal distributions: theory and applications, Vol. 88. 1st ed. New York: Routledge, 1988, p. 387.

56.

Neider

Zelinsky

. Searching for camouflaged targets: effects of target-background similarity on visual search. Vision Res 2006; 46(14): 2217–2235.

57.

Mackiewicz

. Perceptions of clarity and attractiveness in PowerPoint graph slides. Tech Commun 2007; 54(2): 145–156.

58.

McMaster

Shea

. Generalization in digital cartography. Washington, DC: Association of American Geographers, 1992.

59.

Hollands

Spence

. Judging proportion with graphs: the summation model. Appl Cognitive Psych 1998; 12(2): 173–190.

60.

Lobo

M-J

Pietriga

Appert

. An evaluation of interactive map comparison techniques. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, Seoul, Republic of Korea, 18–23 April 2015, pp. 3573–3582. New York: ACM.

61.

Shepard

Metzler

. Mental rotation: effects of dimensionality of objects and type of task. J Exp Psychol Human 1988; 14(1): 3–11.