There is more than one path to a solution, especially when it comes to ill-defined problems like complex, real-world tasks. Until now, the evaluation of information visualizations has often been restricted simply to a measuring of outcomes (time and error) or insights into the data set. A more detailed look into the processes that facilitate or hinder task completion is provided by analysing user problem-solving strategies. The study presented in this paper illustrates how such processes can be assessed and how the resulting knowledge can be used in participatory design to improve a visual analytics tool. For users to be equipped with a problem-solving scaffold, the tools used should allow them to choose their own path to the solution – their own route to Rome. We also discuss how the evaluation of problem-solving strategies can shed more light on the ‘exploratory minds’ of users.
ThomasJJCookKA. Illuminating the Path: The Research and Development Agenda for Visual Analytics2005Los Alamitos, CA: IEEE Computer Society Press.
2.
BertiniEPererAPlaisantCSantucciG. BELIV’08: Beyond time and errors – novel evaluation methods for information visualization. CHI ’08 Extended Abstracts on Human Factors in Computing Systems2008Florence, Italy, New York: ACM Press, 3913–3916.
3.
SarayaiaPBNorthCDucaK. An insight-based methodology for evaluating bioinformatics visualizations. IEEE Trans Vis Comput Graph2005; 11: 443–456.
4.
SmucMMayrELammarschTAignerWMikschSGärtnerJ. To score or not to score? Tripling insights for participatory design. IEEE Comput Graph App2009; 29: 29–38.
5.
ProctorRWVuKP. The cognitive revolution at age 50: Has the promise of the human information-processing approach been fulfilled?Int J Hum-Comput Intn2006; 21: 253–284.
6.
MillerGA. The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psych Rev1956; 63: 81–97.
7.
NewellAShawJCSimonHA. Elements of a theory of human problem solving. Psych Rev1958; 65: 151–166.
8.
Smuc M. Unveiling the exploratory mind. A cognitive approach to Human-Computer Interaction in visual data analysis. (Forthcoming) PhD thesis, University of Vienna, Austria.
9.
CardSKMoranTPNewellA. The Psychology of Human–Computer Interaction1983Hilldale, NJ: Lawrence Erlbaum Associates.
10.
PikeWAStaskoJChangRO’ConnellTA. The science of interaction. Inform Vis2009; 8: 263–274.
11.
JonesG. Testing two cognitive theories of insights. J Exp Psychol Learn2003; 29: 1017–1027.
12.
SimonHANewellA. Human problem solving: The state of the theory in 1970. Am Psychol1971; 26: 145–159.
13.
JonassenDH. Instructional design models for well-structured and ill-structured problem solving learning. Edu Technol Res Dev1997; 45: 65–94.
14.
Valiati ERA, Pimenta MS and Freitas CMDS. A taxonomy of tasks for guiding the evaluation of multidimensional visualizations. In: Bertini E, Plaisant C and Santuppi G (eds.) Proceedings of the 2006 AVI workshop on BEyond time and errors: novel evaLuation methods for Information Visualization (Venice, Italy), ACM Press: New York, 2006, pp.1–6.
15.
FrielSNCurcioFRBrightGW. Making sense of graphs: Critical factors influencing comprehension and instructional implications. J Res Math Edu2001; 32: 124–158.
16.
Amar R, Eagan J and Stasko J. Low-level components of analytic activity in information visualization. In: Stasko J and Ward M (eds.) Proceedings of the 2005 IEEE Symposium on Information Visualization (Minneapolis, MN), IEEE Computer Society 2005, pp.111–117.
17.
Amar R and Stasko J. A knowledge task-based framework for design and evaluation of information visualizations. In: Ward M and Munzner T (eds.) Proceedings of the IEEE Symposium on Information Visualization (Austin, TX), IEEE Computer Society, 2004, pp.143–150.
18.
GickML. Problem-solving strategies. Edu Psychol1986; 21: 99–120.
19.
SchunnCDMcGregorMUSanerLD. Expertise in ill-defined problem solving domains as effective strategy use. Memory Cognition2005; 33: 1377–1387.
20.
RiskuH. Situatedness and artefacts in expert knowledge and activity. In: GadnerJBuberRRichardsL (eds) Organizing Everyday Knowledge: Methods and Case Studies2003Houndmills, UK: Palgrave, 28–42.
21.
SchrawGDunkleMEBendixenLD. Cognitive processes in well-defined and ill-defined problem solving. Appl Cognitive Psychol1995; 9: 523–538.
22.
ClarkA. Being There. Putting Brain, Body, and World Together Again1997Cambridge, MA: The MIT Press.
23.
Lammarsch T, Aigner W, Bertone A, Gärtner J, Mayr E, Miksch S and Smuc M. Hierarchical temporal patterns and interactive aggregated views for pixel-based visualizations. In: Banissi E et al. (eds.) Proceedings of 13th International Conference on Information Visualisation (Barcelona, Spain). Washington, DC: IEEE Computer Society Press, 2009, pp. 44–49.
24.
Shimabukuro M, Flores EF, de Oliveira MCF and Levkowitz H. Coordinated views to assist exploration of spatio-temporal data: A case study. In: Roberts J (eds.) Proceedings of the 2nd International Conference on Coordinated and Multiple Views in Exploratory Visualization (London, England), Los Alamitos, CA: IEEE Computer Society Press, 2004, pp.107–117.
25.
Smuc M, Mayr E and Risku H. Is Your User Hunting or Gathering Insights? Identifying Insight Drivers Across Domains. Paper presented at BELIV 10 (Atlanta, GA), 2010.
26.
Whiting MA, Haack J and Varley C. Creating realistic, scenario-based synthetic data for test and evaluation of information analytics software. In: Bertini E, Perer A, Plaisant C and Santucci G (eds.) Proceedings of the 2008 conference on Beyond time and errors: novel evaluation methods for Information Visualization (Florence, Italy), New York: ACM Press, 2008, pp.8–15.
27.
Robertson G. Beyond Time and Errors - Position Statement. Paper presented at BELIV 08 (Florence, Italy), 2008.
28.
GrantERSpiveyMJ. Eye movements and problem solving: Guiding attention guides thought. Psychol Sci2003; 14: 462–466.
Sieber R, Schmid C, and Wiesmann S. Smart legend-smart atlas. Proceedings of the 22th International Cartographic Conference (La Coruña, Spain), International Cartographic Association, 2005.
31.
Isenberg P, Zuk T, Collins C and Carpendale S. Grounded evaluation of information visualizations. In: E. BertiniA PererC Plaisant and G. Santucci, (eds.) Proceedings of the 2008 conference on BEyond time and errors: novel evaLuation methods for Information Visualization (Florence, Italy), New York: ACM Press, 2008, pp.56–63.
32.
RiskuHMayrESmucM. Situated interaction and cognition in the wild, wild world: Unleashing the power of users as innovators. J Mob Multimedia2009; 5: 287–300.
