A narrative approach to interactive information visualization in the digital humanities classroom

Introduction

The recent rise of “big data” and sophisticated data visualization technologies has spurred interest in developing interactive information visualizations for a wide array of academic projects across the humanities. Following Ferster (2012), I define an interactive visualization as a computerized visual representation of abstract data developed to encourage audience exploration and inquiry. The data incorporated into interactive visualizations may be quantitative or qualitative. While many information visualization techniques were originally developed to explore or analyze numeric or geolocated datasets in the natural and social sciences, interactive visualizations are increasingly being used in journalism (Hullman and Diakopoulos, 2011), history (Ferster, 2012), and in the digital arts and humanities more generally (Burdick et al., 2012). Partly as a result of this interest, information visualization techniques are increasingly being taught in digital humanities classrooms in order to help students explore quantitative and qualitative datasets as well as to enable them to make visual and interactive arguments.

While interactive visualization tools and techniques are increasingly enabling scholars to develop engaging and attractive digital texts, there is concern within the digital humanities community that critical humanistic considerations are being overlooked as scholars adopt visualization methods that originated in the sciences (Burdick et al., 2012; Dalton and Thatcher, 2014). Drucker (2011) calls for a humanist critique of information visualization methods, with particular attention to the rhetorical consequences of selecting visualization genres and categorizing data as well as clearly articulating the designer’s assumptions and interpretations. Similarly, Jessop (2008) describes a need to develop critical methodologies for humanities visualization, including measures to make the reasoning behind data selection and arrangement transparent. Others have discussed the need to interrogate assumptions embedded in platforms and tools that may constrain scholarly practice, for example, by affording visualization creators only specific ways to structure content-management systems (i.e. databases), query data, and produce narratives (Drucker and Svensson, 2016; Monea, 2016).

There are few published case studies describing how information visualization tools have been incorporated into digital humanities classrooms, or humanities classrooms more broadly, while maintaining a critical orientation toward tools and choices made during production. Classroom practices incorporating other visual digital humanities genres like multimedia narrative are more commonly described (Benmayor, 2008). Other scholars have described student activities that do not explicitly include an element of student critical evaluation of how a visualization is generated and structured, for example, using word clouds to rhetorically analyze poems using cluster criticism (Butler, 2011). In a social science example that is relevant to projects using quantitative humanities data, Rom (2015) suggests how technical and design choices might influence the visualizations that students produce, though he does not explicitly take a critical or rhetorical approach. Critical elements that might be added to such activities include analysis of the technical limitations imposed on student authors by software platforms or examination of the consequences of using particular search terms or categorizing data in certain ways.

One humanities field that foregrounds the importance of a critical orientation when developing information visualizations in the classroom is technical and professional communication. As Ballentine (2015) discusses, technical and professional communication shares several points of similarity with digital humanities, including concern for perpetuating humanistic values while employing technological tools and an emphasis on project-based scholarship. Specific points of concern include maintaining a humanistic ethic when developing graphics about injury or death (Dragga and Voss, 2001), adopting a rhetoric of user participation when developing interactive data displays (Kostelnick, 2008), and maintaining a critical perspective on the power relations embedded in computing technologies (Selber, 2004) and code (Maher, 2011). Wolfe (2015) describes two specific classroom activities focused on critically developing data categorization schemes and articulating storylines in data visualizations. In the related field of digital composition, Sorapure (2010) outlines several types of information visualization projects that introduce students to interactive visual composition with textual, personal, or social data. Technical communicators also point out that critical reflection on the use of computing tools should not be confined to the humanities. Selber (2004), for example, has more broadly discussed the importance of combined critical, technical, and rhetorical “multiliteracies” for computing education. Nevertheless, concerns about criticality and rhetorical design traditionally fall within the humanities sphere and are particularly pertinent for the digital humanities classroom.

Narrative information visualization is one approach to visualization development that allows students to use data visualization tools while addressing humanistic concerns. This approach, described as “telling stories with data,” has been used extensively in journalism, for example, in the interactive informational graphics developed by newspapers like The New York Times (Segel and Heer, 2010). Narrative information visualizations are often developed by an author in order to help audiences make sense of a dataset through structured exploration that leads to discovery (Dove and Jones, 2012). The relative proportions of structured narrative to free exploration vary in these visualizations (Segel and Heer, 2010), and the choices that authors make with regard to the inclusion, exclusion, and representation of data are central to the audience’s experience (Hullman and Diakopoulos, 2011). The analysis of large datasets or “big data” is central to many digital humanities projects (Burdick et al., 2012; Dalton and Thatcher, 2014), and taking a narrative research approach can help students interrogate the rhetorical and critical assumptions that they make when analyzing and representing those data.

Ferster (2012) describes a prescriptive model for the development of interactive narrative visualizations for digital history. In this paper, I describe a narrative-oriented approach to information visualization in the digital humanities classroom that extends Ferster’s model through critical and rhetorical reflection. I illustrate how four students in a graduate humanities course titled Narrative Information Visualization learned how to understand the history and theory of narrative information visualization from an interdisciplinary perspective, develop technical skills for creating interactive visualizations, and critically assess and evaluate interactive visualization projects.

A course on narrative information visualization

The Narrative Information Visualization course asked students to learn about three aspects of visualization theory and development: history, theory, and the interdisciplinary nature of the field; tools and techniques for developing visualizations; and visualization conception, construction, and reflection. Ferster’s (2012) “ASSERT” model (ask a question, search for evidence, structure information, envision an answer, represent data, tell a story) provided an overall scaffold for the developmental aspects of the course and guided students in thinking through design choices in visualization development. His text was supplemented with readings taken from the interdisciplinary literature on data visualization, visual rhetoric, and graphic design. Reading responses, class discussions, and data analysis activities were used to help students reflect upon their developing understanding of the field and situate their own interests within it. The course concluded with a major information visualization project, which students accompanied with a final critical reflection on their process and results. This sequence of content and associated activities provided a framework that was flexible enough to enable students to explore their own interests while maintaining a unified approach to project development. The three major parts of the course are described below.

Project entry points

The students in the course were in the first year of an interdisciplinary doctoral humanities program and had different academic backgrounds and interests. Students also had different levels of experience working with visualization tools, though all had at least some classroom experience with coding webpages or databases. Each student included a variety of types of information in their visualization, used different techniques for visualizing the information and creating a narrative structure, and articulated different methodological frameworks to support their rhetorical purposes. The flexible project structure was intended to allow students to tie their course projects to their own disciplinary backgrounds and develop visual narratives in different ways, as appropriate to their areas of interest.

Each student had a different point of entry into the visualization project. As is common with digital humanities projects (Burdick et al., 2012), some students began with argument-oriented goals while others began with the exploration of datasets to draw out narratives. Students One and Four began with initial arguments and then marshaled data in support of those arguments. Student One focused on demonstrating how overlap between two sets of public school curricula, English language and computer animation, would allow standards from both to be taught simultaneously. She conceptualized her project as an argument in storyboard format in aid of justifying the continued offering of computer animation courses. Student Two’s intent was to comment critically on the overall narrative themes in a popular video game, Super Mario Brothers III. He planned to explore the numbers, locations, and programmed actions of the non-player characters (i.e. characters that cannot be controlled by the action of the player) to find patterns that he could use to support a tentative preexisting reading of the game wherein the player is working to uphold an inequitable power dynamic in the game world.

Students Three and Four initially engaged with existing datasets and then developed research questions for those data from which they constructed narratives. Student Three’s focus was the mental models of writing employed by undergraduate students in a metacognitive “writing about writing” composition program. His entry point was a dataset of student before-and-after definitions of writing, which he planned to analyze and use to develop a narrative about how student understandings of the writing process changed after taking the course. Student Four began with a dataset about motor vehicle accidents in the University’s metropolitan area that had resulted in bicyclist injuries or deaths. He was interested in creating an interactive map that would allow him to tell a story about the details of these accidents, such as where and when they occurred. Like Student Three, he planned to explore the information and draw out story elements where he found them, though he had a more specific concept of what he wanted the final visualization to look like.

Asking questions and developing narratives

The processes of asking questions of the datasets and using those questions (or their answers) to develop a primary narrative for the visualization were different for each project. How questions are framed, data are categorized, and narrative interpretations are represented are important considerations for humanities visualization, in part because data visualizations can convey a sense of objectivity to the audience that overshadows the ambiguity inherent in humanistic interpretation (Drucker, 2011). Each student encountered different technical, critical, and rhetorical challenges during this process. Both in-class discussions and the final reflective essays helped the students examine their assumptions about how information should be categorized, what types of trends or correlations they might find in it, and what narratives it might support. These activities together formed the second milestone of the course project.

Students encountered a variety of challenges in learning how to technically operate their software for data analysis and display as well as in navigating the ways in which data structure and software shaped their querying options. They were able to resolve these challenges with varying degrees of success. For example, Students Three and Four both began with existing datasets and used exploratory methods to draw out patterns in those data and develop narratives. Student Three used textual analysis and asked questions about word frequency in student definitions of writing before and after taking a composition course, as compared to word frequency in professional definitions of writing. He found it necessary to use several text analysis tools and comparative datasets as well as carefully structure searches to obtain results that helped him develop narrative themes. Similarly, Student Four began by looking for patterns in a database of traffic accidents involving bicycles. He was initially interested in identifying relationships among the number and severity, location, and circumstances surrounding accidents. As he explored the dataset, he realized that there were no strong correlations that would help him develop narrative themes, given the dataset size, ambiguous data categorization, and limited analysis platform that he had access to.

Another common set of challenges that students encountered related to rhetorical choices about argument and the use of data in constructing their narratives. For example, Student One initially found the large number of English language and computer animation curricular standards for possible comparison to be daunting. She focused on narrowing the scope of her inquiry to standards with similar terminology or instructional goals. Ultimately, she structured her narrative around the curricular standards that could be taught by close reading of a single chapter of Frankenstein. Student Two generated a dataset of the orientations, locations, and types of non-player characters in Super Mario Brothers III. He defined non-player characters in different game locations as friendly or enemy, and used the proportions of these categories plus the game’s scenery to support his preliminary argument. His primary challenges were categorizing non-player characters and organizing this information to achieve the rhetorical effects that he wanted. Student Three’s expectation before beginning his project was that there would be clear differences between student definitions of writing before and after taking a composition course. Initially, however, he found only small word frequency differences. He therefore transitioned to examining phrase-level concepts. Together, the combination of word frequency and phrasing provided a more nuanced central narrative for his visualization. Finally, Student Four selected an open-ended narrative structure that gave the audience freedom to explore the data rather than a strong central message, because of the difficulties he had in identifying strong narrative themes in his dataset.

Representing narratives

As students developed their ideas about the narrative structure of their visualizations, they also began to decide how to represent those narratives visually. Visual representation of a narrative involved multiple choices, such as which digital tools to use to build their projects, how to visually represent data that were in many cases non-visual, how to arrange and divide visualization elements, and how to incorporate interactive features for audience exploration. While the process of deciding how to represent a visualization narrative is described here as the third milestone, occurring after the processes of questioning the dataset and drawing out narrative elements from it, it is important to note that these decisions did not necessarily occur in a linear fashion. For example, every student began their project with at least some ideas about how they might visually represent their data, and there were several instances in which students’ interpretation of their data changed once they had begun to visually represent their narratives. This cycle of questioning, developing a narrative, and representing the narrative allowed thoughtful and nuanced invention and arrangement of material.

Some student projects were more author-centered, in that they had a strong central storyline, linear organization, and limited interactivity, while others were more audience-centered, with a weaker narrative, nonlinear organization, and freer interactivity (Segel and Heer, 2010). Student One’s visualization was the most linear in structure among those of the four students. It took the form of an interactive storyboard depicting a chapter of Frankenstein, and its structure was driven by the novel’s textual sequence. It was also perhaps the most consciously dominated by visual elements, as it was based on image-based storyboarding. Each “frame” combined a passage from the text with an accompanying visual. She used annotations to define key terms and concepts from the reading that were relevant to curricular standards. Interactivity provided a directional narrative and allowed users to explore the visualization to uncover more details.

In contrast, Student Four’s project was the least linear in construction and afforded the most flexibility for audience interaction. It consisted of a background map upon which the locations of accidents were indicated by dots, with a side menu that let the audience filter and display certain types of information (e.g. accidents that occurred at night). After focusing his visualization on the severity of accidents and selecting color to represent this information, his second major narrative decision centered on how indicate the other categories of information that he wanted to provide. He decided to provide a series of menus with filters that would allow the audience to view the data via different types of variables such as lighting conditions or whether the accident occurred on the road or at a sidewalk. As mentioned previously, he ran into unexpected difficulties in creating these filters because of ambiguity in the categorical descriptions of variables.

The software platforms that students used to build and display their narratives affected the shapes of those narratives (Drucker and Svensson, 2016). Student One used professional storyboarding software and Student Four used a geographic analysis platform to display their projects. In both cases, students found their opportunities for including interactive features to be constrained by their software choices. Students Two and Three both used blogging platforms to produce their projects, and later described similar constraints on incorporating interactive features, as well as design of page layout (e.g. menu locations and page widths). Importantly, students’ skill level with the software also influenced how constraining the software platforms were. For example, Student One had long experience with her chosen platform and was able to find solutions for several technical constraints with advanced coding knowledge.

Student Two’s primary narrative subverted the heroic journey of the main player character in a video game. By drawing upon the game scenery and distribution of non-player characters, he argued that the game could be read instead as a system of class-based oppression of the few “friendly” characters against the many “enemy” characters. His narrative consisted of a series of linked pages connected by a central menu system that presented interactive charts of the non-player character distributions and affiliations as well as images of the scenery in various game locations. These visual representations, along with brief textual annotations that posed provocative questions and interpretations, served as the main evidence for his arguments.

As with Student Two’s project, Student Three’s visualization contained both linear story elements and moderate affordances for audience interaction and choice within the overall narrative structure. His focus was on conveying differences in word frequency and phrasing of student definitions of writing before and after taking a composition class. His narrative took the form of a series of linked pages that walked the audience through different ways of exploring the dataset. The audience began their interaction with the visualization on a starting page that provided context for the project, and then navigated around the different pages via a header menu, a narrative structure common to many web pages. On each page, he included either a set of graphs of word distributions that could be rearranged according to choices of the audience (e.g. rearranging terms alphabetically or by frequency), or several illustrative examples of student definitions with key phrases that demonstrated the relative sophistication of student concepts highlighted. Some of his major design decisions concerned how to structure the information that he was presenting into pages, incorporate interactive features that allowed the audience to explore the graphs, and use color to indicate key information.

Student final reflections

The students’ final project milestone was a reflective paper intended to articulate the theoretical underpinnings of their design. Students demonstrated how they considered visualization theory studied during the course, how their projects were framed by theory appropriate to their individual disciplinary interests, and how they overcame specific design challenges during project development. Students’ design processes drew from Ferster’s (2012) “ASSERT” model, with several other sources proving informative as they developed their projects. These included material on narrative visualization structure (Segel and Heer, 2010), rhetorical aspects of visualization design (Hullman and Diakopoulos, 2011), and cognitive science and perception (Liu and Stasko, 2010). In addition to the information visualization theory that they collectively applied to their projects, each student drew upon works specific to their visualization topic areas and theoretical frameworks.

Several critical issues that arose during the development of the visualization projects were raised in the student reflections. Student One, for example, discussed rhetorical tensions between her visual storyboard of the Frankenstein chapter and the interactive textual annotations that communicated the English-language curricular standards. She concluded that while her final project did not achieve the rhetorical ends that she had originally intended (i.e. persuasively arguing that the English and animation curricula should be combined), it did serve as an effective starting point for discussion of the issue. The software platform that she used constrained the overall narrative structure and interactive features of her project. Though she was able to modify some of the interactive features that she had initially planned to include, she accepted other constraints in order to retain the storyboard format as a primary means of expression used in animation classes.

Student Two developed his project as a broad critique of the dominant heroic narrative within a specific video game, which he supported by identifying patterns in his dataset and developing arguments from those patterns. While his primary critical focus was therefore on the game that he was analyzing, he did employ additional critical strategies when deciding how to classify non-player characters. Because his goals were largely persuasive, his focus was on selecting and integrating visuals, interactive features, and textual annotations for rhetorical effect. He reported some technical challenges with integrating various interactive data visualization tools into his final project, but felt that the overall constraints imposed by the platform did not detract from his overall argument.

Student Three used a framework of education theory to structure his visualization project, focusing on student mental models of the writing process and how changes to student definitions of writing might or might not indicate changes in understanding. He used composition studies theory as guidance for exploring how student definitions might change, though he ultimately found that shifts in student definitions were subtler than he had initially anticipated. He reflected critically on assumptions about the initial survey that had elicited student responses, the use and interpretation of terminology in this context as well as on the constraints that qualitative text analysis tools placed on how he could format and analyze data. The primary challenge that he described was the need to use several qualitative text analysis tools and visualization techniques to find ways to develop a narrative and convey it to his audience.

Student Four drew on urban planning theory to situate his visualization of bicyclist injuries within the field of geographic analysis and risk assessment. He began his project with a strong vision of what the final visualization would look like, which was informed by conventions of geospatial accident information display. He experienced major technical challenges including difficulties with displaying the data correctly on a map, and conceptual difficulties with interpreting the data about the characteristics of accidents. In particular, he found that spatial uncertainties in accident locations and ambiguous descriptions of accident context made it difficult to draw conclusions about the locations and circumstances surrounding accidents. His final reflection discussed how identifying these multiple levels of uncertainty helped him understand the subjectivity of data analysis, though he was not able to address that subjectivity in a critical way in his project.

Conclusion

In order to effectively bring information visualization into the humanities classroom, we need to develop approaches that move our students from simply utilizing prevailing data visualization techniques to critically incorporating those techniques into humanities-centered practice (Drucker, 2011). I have sketched the experiences of four students who designed and developed information visualization projects while learning how to recursively connect humanistic theory to their work throughout their decision-making processes. The students in this course learned about how data selection and categorization could affect the types of questions they could ask of those data or rhetorical structures they could build (Monea, 2016). They also reflected upon how their selection of digital tools affected their options for analysis and design (Drucker and Svensson, 2016). These examples demonstrate how a narrative approach to information visualization helps students develop critical, rhetorical, and technical skills that enable them to create their own interactive digital narratives.

One of the strengths of the approach outlined in this paper is that it provides enough structure to help students learn how to examine and develop information visualization projects from a critical humanities perspective, and yet affords enough flexibility for them to pursue their own primary disciplinary interests. Students taking this course were required to continually reflect on and maintain an awareness of the decisions they were making. This helped them be conscious of their critical choices, foregrounded the subjective nature of information visualization development, and made them consider the potential rhetorical effects of their work on audiences. The diversity of their theoretical approaches, initial datasets, and eventual completed narrative visualizations shows that this approach is relevant for students with different digital humanities interests. The individual projects took several forms commonly seen in the digital humanities, including a “filmstrip”-style project with limited interaction (Student One), a georeferenced map with several affordances for audience filtering and exploration (Student Four), and two projects comprised of linked modular interactive graphics and textual arguments that provided moderate affordances for audience choice (Students Two and Three). Thus, individual students were able to develop projects with interactive structures that were tied to the types of narratives that they wanted to tell.

While this paper reports on a graduate digital humanities course, this approach could certainly be modified for undergraduate courses. In an undergraduate course, it would likely be helpful for students to spend a substantial amount of time working with a single dataset and focusing on how to ask critical questions of how those data were derived and defined (Dalton and Thatcher, 2014), and how to use specific data visualization techniques to build a narrative. Focusing on specific techniques and working with a single dataset would help students develop a critical and digital narrative vocabulary and compare their decisions with those of their classmates, thus helping them acquire the technical, rhetorical, and critical literacies advocated by Selber (2004).

Another consideration for those teaching a narrative visualization course is how to prioritize technical, rhetorical, and critical learning objectives, particularly for undergraduates. In this course, rhetorical and critical objectives were prioritized, and students relied to an extent on their existing technical skills. The students with less digital design experience were challenged by the need to identify appropriate design tools and learn to balance narrative elements with aesthetics to the best effect. If development of critical and rhetorical, rather than technical, skills are primary learning goals of the course, it might be helpful for students with limited experience to focus on using a pre-selected suite of tools that are easier for beginners to learn to create interactive narratives (a representative list of such tools is found in Ferster, 2012). It would be important in this situation, however, for students to reflect on the ways in which those tools constrain their research and development process (Drucker and Svensson, 2016; Monea, 2016).

A final consideration for those teaching a similar course is to what extent students might be encouraged to employ generative or imaginative uses of visualization tools. As described by Dalton and Thatcher (2014), digital visualization tools can also be used for the explicit purpose of envisioning and encouraging social change. As the students in this course developed their projects, they reflected critically upon their choices during multiple stages of the design process as well as at several levels of design ranging from data selection and categorization to their overarching narrative. This recursive attention to decision-making at multiple levels reflects contemporary developments in information visualization research (Hullman and Diakopoulos, 2011; Wolfe, 2015). Students were not, however, directed to adopt an explicit ethos of cultural critique when producing their projects. Such an ethos might make a meaningful addition to the critical component of a narrative information visualization course.

The final reflective essay in particular was a key point at which students connected humanistic theory to their visualization projects and situated their projects within humanities disciplines that were appropriate to their interests. Throughout the course, class discussions, reading responses, and other activities that focused on learning about digital techniques required students to consider the critical and rhetorical implications of their choices. These activities reinforced the central role of theory in a humanistic design process and helped students develop their own syntheses of theory in support of their visualization goals. Digital humanities practice is overwhelmingly an interdisciplinary endeavor (Burdick et al., 2012). For students learning to conduct digital humanities research, it is important to learn how to selectively synthesize theory and techniques from different fields in ways that support their research goals. The final essays were an important opportunity for students to articulate how they achieved this and demonstrated their understanding of how information visualization is imbued with humanistic values.