Abstract
Visuality is a concept that crosses boundaries of practice and meaning, making it an ideal subject for interdisciplinary research. In this article, we discuss visuality using a fragment from a video meeting of television producers at Swedish Television’s group for programming in Swedish Sign Language. This example argues for the importance of recognizing the diversity of analytical and practice-derived visualities and their effect on the ways in which we interpret cultures. These different visualities have consequences for the methods and means with which we present scholarly research. The role of methods, methodology, and analysis of visual practices in an organizational and bilingual setting are key. We explore the challenges of incorporating deaf visualities, hearing visualities, and different paradigms of interdisciplinary research as necessary when visibility, invisibility, and their materialities are of concern. We conclude that in certain contexts, breaking with disciplinary traditions makes visible that which is otherwise invisible.
Introduction
Visual practices, underwrit with zeros and ones, divorced from any fixed place, constantly in motion, changing in context and meaning, which traverse today’s porous cultural and geographical boundaries, overwhelm us and our visibilities. These visual practices and visual forms are a ubiquitous, increasingly important interface for communication. The omnipresence and historicity of visual practices are particularly evident in research fields that focus on visual practices in one way or another. Images and visual practices are a central preoccupation of social sciences, such as (visual) anthropology and sociology, in human–computer interaction (HCI) and the humanities.
The current resurgence of the visual is a reminder that visual practices—especially in research but also in other social practices—have long been subordinated to modes of discourse favoring textual, oral, cognitive, and rational discourse (see Banks, 2001; Rose, 2001; Murdock & Pink, 2005; Pink, 2012). Description, transcription and textual records in research practices are often emphasized at the expense of images and nonrepresentational modes of attention, such as emotions, affect, and sensory dimensions. Methods for studying visual practices—visibilities, visualities, and even visualizing technologies—have, for better or worse, been modeled on these logocentric modes of thinking. In an age of ubiquitous visuality, however, we need to explore visual analysis and understanding in ways better adapted and more congenial to visual modes of perception. Our focus in this article is therefore on the role of methods, methodology, and analysis of and through visual practices in an organizational and bilingual setting. We explore resources for reflexivity through a discussion on methodology and analytical frameworks in a broad interdisciplinary setting and question the role of images and visuality in the different phases of our research. We evaluate the methodological and analytical implications of focusing on the visual using an empirical example of a deaf-hearing workplace that challenges received assumptions about visuality and visual analysis. Intended as a contribution to a more general call for new ways of thinking about visualities, this article complements the work of visual anthropologists (Banks, 2001; MacDougall, 1997; Pink, 2007), humanities and media scholars (Mitchell, 2005; Rose, 2001; Thompson, 2005) and HCI researchers (Egido, 1990; Finn, Sellen, & Wilbur, 1997; Olson & Olson, 1997; Tang & Isaacs, 1992) with its focus on “ways of seeing” for those who are deaf.
Approaches to Visuality: Anthropology, HCI, and the Humanities
We begin by exploring various methods and strategies in the study of visibilities and visualizing technologies across disciplinary contexts. Starting with research fields selected on the basis of the authors’ competencies (anthropology, HCI, and the humanities), we provide an overview of these various approaches to the study and analysis of visuality and end with a summary of a more interdisciplinary approach. We then proceed to our empirical example, followed by a contextual and methodological analysis.
Visibility as Visual Culture in Anthropology
In anthropology, the study of visibilities often falls under the banner of visual anthropology. Visual anthropology is characterized by a broad scope of research topics, approaches, and methods addressing visual culture. A number of compendiums focus on the methods available for visual anthropologists and the potential for innovative data-gathering tools suited to the study of visual means of communication (Pink, 2007, 2012; see also Murdock & Pink, 2005). The main trends in this area of research can be divided into roughly four intersecting and sometimes overlapping categories. These are (a) research in the form of ethnographic films (as visual products of research and representations of it), (b) research on ethnographic films, (c) research on visual images and artifacts produced by others, and finally (d) visual research on visual images produced by anthropologists. A more recent dimension requiring a different definition of the visual artifact emerges in visual anthropological studies that center on the creation and analysis of exhibitions, art installations, and written reviews of exhibitions and performances (see Brodine et al., 2011). These different foci divide the field in some ways. For instance, studies researching visual manifestations of a culture and seeing visual forms or artifacts as symbols of culture and meaning differ from other anthropological studies of the visual such as studies of performances, for example, where no visual, tangible record of this type of cultural expression is produced. Some anthropologists, who practice visual anthropology and define it as the study of visual-based practices, argue for the centrality of visual documentation of their studies in addition to conventional text-based publications (see Gürsel, 2009a, 2009b, n.d.).
Visuality From the Perspective of HCI
The focus in HCI is on a technological artifact or system that operates within the realm of vision. Until the mid-1990s, HCI studies on video-mediated communication (VMC) were based on an experimental approach, focusing on the technology, the transmission of data, the audio and video quality, and efficiency (see Finn, 1997). From an HCI perspective, the development of video technology is based on the idea that it should mimic face-to-face communication. With this in mind, Sellen (1992) lists some important differences between video and face-to-face meetings. Unlike our eyes, cameras have a fixed field of view and cannot be controlled by the viewer in the same way. Another distinction is the importance gestures and gaze play in securing another’s attention and how these are tied to the feeling of being “distanced” from others in video meetings. Breaking with the experimental approach, Olson and Olson (1997) performed field studies during which they visited geographically distanced workplaces, observing and interviewing employees. This raised method-related questions about whether it was necessary for the observer to be on site or whether all observations could be conducted from a single location. Can the observer “see” enough of the other locations to understand the situation or is their physical presence essential?
Much later, Hunter, Maes, Tang, Inkpen, and Hessey (2014) addressed the social aspects of VMC. Using expert interviews, a pilot study, and an interactive design process including an evaluation of the final system. Hunter et al. (2014) analyzed virtual environments composed of digital assets layered in 3D space. This study points to an acknowledgment within HCI of the importance of social interaction and relationship-building in instances of VMC. HCI has also recently renewed its focus on the importance of eye contact without relating VMC to face-to-face communication. Eye-gaze direction is in this case portrayed by the eyes of the avatar (Rae, Steptoe, & Roberts, 2011).
These few examples suggest that the visual in HCI hinges on a functionalist perspective of the field and focuses on the technology rather than the cultural contexts of its use, exactly the opposite of previous anthropological perspectives, which tended to prioritize cultural meanings while sidelining the technologies that go into producing them. HCI can therefore provide a complementary addition to studies on the visual, understood as technologies that relay visual data and specifically the importance of gaze (cf. anthropological definitions).
Visuality From the Perspective of the Humanities
Although visual anthropology and HCI deal with the concrete visual, the humanities approach visuality from a philosophical angle based on the text versus image binary. The dominant model of representing the world in the humanities has generally been based on the (logocentric) model of text. Not surprisingly, this model has also dominated visual analysis. Art historian Mitchell (1994) challenged this logocentrism and proposed a “pictorial turn” in the humanities. He suggests that vision be put on equal footing with language and that practices of looking are as complex as the reading of a text. This inspired all sorts of nonrepresentational theories across the humanities and over into other fields such as cultural geography (Thrift, 2008). Indeed, many have argued that the sensual immediacy of visuality, its sensation, feeling, and affect are a form of representation unrivaled by any text-based metaphor. However, it is worth remembering (as Mitchell does through the notion of “image/text”) that reading is also a visual mode and a text is also a form of visual representation, albeit graphic. Banks and Morphy (1997) argue to the contrary, stating that a neglect of the visual is anchored in the Western bias toward textual exploration and its categorization as belonging to a “higher intellectual canon” (Banks & Morphy, 1997, p. 14). This textual bias has direct consequences for any study of cultures that privilege visual modes of communication, such as deaf cultures. Quite apart from this debate, the visual has inarguably become ever more central to communication with the emergence of the new forms of media associated with the so-called “digital age” (Pink, 2012). A preoccupation with the visual can thus no longer be dismissed as Western ocularcentrism (Jay, 1993; see also Mirzoeff, 1999, pp. 1-33; Sturken & Cartwright, 2009).
The rise of “visual culture” (Mirzoeff, 2002) can be traced through developments in several disciplines in the humanities, but it was not until the 1990s that it became explicit in interdisciplinary fields. A central question in this eclectic tradition is, what actually counts as visual? The focus is not on a specific visual object, but on the problematics of living in environments saturated by images and/or hybrid forms of multimodal experiences—text, image, sound, touch, and so on. From a media studies perspective, sociologist John Thompson (1995) argues that the development of communication media has transformed the nature of social interaction (Jenkins, 2008 cf. Kress & van Leeuwen, 2001; Pink, 2007). An interactional theory of the media not only stresses the transmission and reception of content but also new forms of action and interaction created through diverse uses of the news and popular media. Thus, contemporary mediated interaction takes many different forms. This shift toward interaction is reflected in Thompson’s (2005) more recent research on “the new world of mediated visibility” (p. 31). New technologies such as television and the Internet have created new, wider repertoires of interactional situations. This is especially so when communication occurs through email, user groups, chat rooms, and VMC where dialogical features are prominent.
Thompson (2005) explains that visibility is seldom “pure vision” but “seeing is always shaped by a broader set of cultural assumptions and frameworks, and by the spoken or written cues that commonly accompany the visual image and shape the way in which the images are seen and understood” (p.36). In stating this, Thompson acknowledges the important work of Foucault concerning visibility and power, sketching a trajectory from the society of the spectacle to disciplinary society “in which the visibility of the few by the many has been replaced by the visibility of the many by the few” (2005, p. 39). This identifies a shift from the spectacular display of sovereign power to the normalizing power of the gaze (panopticon) into a permanent state of visibility. For the deaf, this permanent state of visibility is arguably the norm, and was so long before the advent of new media technologies.
Interdisciplinary Work at the Crossroads of Social Science, the Humanities, and Engineering
So far we have sketched ways of theorizing visuality and visual analysis in three different research traditions, some of which have proved to be quite multifaceted. All traditions outlined above certainly have their individual benefits and drawbacks in terms of their objects of study, guiding rationales for research and means of achieving their particular goals. Yet, we argue that there is still a clear need to explore these methods of analysis, methodologies, and disciplinary understandings to arrive at research methods that are better adapted and more congenial to diverse visual modes of perception, such as deaf ways of seeing. In arguing this, we acknowledge that the visual is a resolutely interdisciplinary subject. We hold with Nicholas Mirzoeff (1999), who quotes Roland Barthes in stating, “To do interdisciplinary work, it is not enough to take a ‘subject’ (a theme) and to arrange two or three sciences around it. Interdisciplinary studies consist in creating a new object, which belongs to no one” (p. 6; see also Pink, 2012). The visual and visibility should thus be regarded as distributed border-crossing phenomena, where conventional disciplinary gatekeeping, if operative, is circumvented by a postdisciplinary approach. We also argue against visualization as a detached analytical gaze, echoing both Thomas Nagel’s famous description of philosophy as “the view from nowhere” and Donna Haraway’s (1988) “gaze from nowhere.”
Visibility, Swedish Sign Language, and VMC
The following empirical example of a video meeting and VMC technologies lends itself to different approaches of analyzing visual practices and offers insights into new ways of thinking about visibility.
Deaf Ways of Seeing
In some sociocultural communities, discourse-as-visuality has long been a privileged focus of attention. This particular example is taken from one such community—the deaf community in Sweden. Swedish deaf people are part of a social group whose modes of communication could potentially upset dominant mainstream traditions of discourse-as-text. This particular example concerns a group of employees working on developing and broadcasting television programs in Swedish Sign Language (SSL) at one of Swedish television’s regional offices in Falun. The field site places central importance on visual practices of television production and supports VMC (video meetings) in both Swedish and SSL. This cultural group is founded on a shared social ordering that prioritizes the visual and relies on visibility to communicate. Composed of skilled visual communicators, this group works on creating visual products through the medium of video, products that have no form in the conventional sense.
The following example focuses on communication for the purpose of holding a meeting in which work-related decisions are made. A meeting held by the SVT Teckenspråk editorial team for Culture and Society in 2012 is a good example of work-related visually mediated communication and shows the centrality of being visible in sign language–dominated communication. Communication takes place in Swedish Sign Language, facilitated by interpreters. The primary method-related challenges for the researcher centered on how to best capture the events and represent them in a way that conveys the essence of the video meeting in Swedish and SSL, its hurdles, and character. The following excerpt is a short exchange that occurred toward the middle of the roughly 1-hr long video meeting.
Exploring Deaf Visualities Mediated by Technology
We—an anthropologist, a deaf SVT Teckenspråk employee, and a freelancer collaborating with the editorial team on the program—are in Stockholm, Sweden at SVT Teckenspråk’s offices in Kontorshuset on the edge of Stockholm’s city center. The meeting takes place in a tiny, 4 × 2 metre room in the head offices of the Department for Culture and Society Programming. The SVT Teckenspråk employee, Louisa, dials up SVT Teckenspråk offices in regional Falun to meet with a hearing colleague. After some initial difficulties, a video link is established and Louisa proceeds to position the camera in Stockholm using the remote and zoom function so that only she and the freelancer, Erik, are visible (see Figures 1 and 2). She and Erik are seated at a small round table in the middle of the room with space for only two. I am seated behind them and to their left, so I am not visible to the team in Falun. The purpose is not to exclude me but to make sure that all signs, facial expressions, and movements Louisa and Erik make are visible to the interpreters and any deaf colleagues in Falun. This meeting is unusual in the sense that Louisa and Erik are both deaf and in Stockholm without an interpreter. Louisa’s hearing colleague, Eve, is alone in Falun, accompanied by two Swedish Sign Language interpreters sitting with her, Hanna, and Karin.

The video meeting as seen from the anthropologist’s point of view in Stockholm.

The Stockholm meeting room.
All now present, Louisa plugs her laptop into the power point behind a cupboard while Karin shuts the door to the meeting room in Falun, blocking out some, but not all of the background noise. In Stockholm, we too shut the glass sliding door to our small room, which is already hot and stuffy. Louisa begins the meeting, and Eve and the two interpreters settle down. Although the camera in Stockholm has been adjusted by Louisa to give Falun an optimal picture of her and Erik, the camera in Falun has not been moved. Instead, the people have positioned themselves around the table in places they assume are visible to the camera and to Louisa and Erik in Stockholm. The interpreters have placed themselves center stage. Eve however, is outside of the image most of the time, with only her left arm and part of her green T-shirt visible (see Figures 3 and 4). It is not essential to see Eve from a sign language perspective. The interpreters are the only ones Louisa and Erik will be able to understand, as they do not read lips or hear. However, not being able to see Eve’s face and her expressions means that no one in Stockholm, including me, will be able to gauge her response in the same way. Louisa and Erik might even (and do) fail to notice that she has even said anything as there are no visual cues. The interpreters in Falun are experts used to portraying words and their meaning by adding facial expressions that mirror those of the person they are translating for. However, as the camera is zoomed out, the interpreters appear rather small on the monitor, so their facial expressions and movements can easily be (and are) missed. The small monitor in Stockholm makes it even more difficult to capture subtle expressions. Bad lighting in Falun creates a high-contrast image that again increases the likelihood that small signals will be missed.

The Stockholm view of Falun.

Erik is watching the interpreters to follow the discussion, and Louisa glances at the monitor while taking notes. Eve is looking at the monitor in Falun but is barely visible to us in Stockholm.
With the meeting about to get underway, Louisa mentions that she is going to be taking notes and asks if that is okay with everyone. No objections are made and she begins by typing something on her computer. Moving through the agenda, Louisa starts signing and engaging Eve (via Hanna and Karin) and Erik in a discussion. The interpreters translate what Louisa and Erik sign to Eve, who is making her own notes and rarely looks at the monitor. Both interpreters have their gaze firmly fixed on the monitor to capture and relay everything Louisa signs. They also convey all of Eve’s comments—whether they are muttered while looking down at her notes or said clearly while looking at Louisa and Erik. At one point, I hear Eve chuckle at something Louisa signs (and which the interpreters translate). Eve chuckled while she was looking down at her papers and is barely visible because of where she is seated, just out of the camera’s view. Subsequently, no expression of mirth is visible, and this pleasantry goes unnoticed by Louisa and Erik. As the interpreters prioritize certain aspects of the conversation, and in this instance, rather than adding a long sequence explaining that Eve has chuckled at an incidental, slightly comic comment from Louisa, they choose to let the gap in communication go. The moment has passed, and the business of making television is more important than pleasantries in this meeting. Nevertheless, the tone of the meeting is cheerful, positive, and dominated by the excitement about the new project and collaboration.
We have mentioned this small, seemingly insignificant exchange because it provides the context for what happened a few minutes later. Going back to the meeting, Louisa is taking notes on her computer at the same time as chairing the meeting and discussing specific issues with Eve (“opposite her” on the monitor) and Erik (sitting beside her in the same room). Karin is translating for Eve, who is only barely visible on the monitor and not looking at the camera or her colleagues in Stockholm. Erik, who is outside of my filmed version of events, is presumably watching what Karin is signing. He places his pen and notepad down on the table in front of him. Louisa, whose face is also outside my film frame, is typing notes on her computer. She does not seem to be watching everything that is being signed, only glancing up occasionally to catch small phrases and parts. Eve is moving between being partially visible and not visible on the monitor in Stockholm (see Figure 4). This configuration lasts for about a minute. Both Erik and Louisa begin to take notes, neither looking at the monitor or the group in Falun (see Figure 5). Eve notices that Louisa is not looking but typing, and her tone of voice and hesitation in speech hints that she might be anxious that Louisa is unaware that she is talking. Perhaps, she just wants to have Louisa’s full attention. Eve moves to the left and cranes her neck to the side so that her head is now visible on the monitor in Stockholm while continuing to talk as before. Karin continues signing what Eve is saying without pause.

Louisa and Erik taking notes in Stockholm while the interpreters are signing.
Louisa looks up, perhaps having noticed that Eve is now visible and has moved. Louisa, after a short pause for thought, looks at Erik sitting beside her, and they both begin signing simultaneously (see Figure 6). They are signing to each other rapidly and in turning to face each other, they are no longer facing the interpreters or Eve in Falun. The Falun people only see their profiles. The interpreter, Hanna, is focusing on what Louisa is signing but she has not yet started translating this, even though Erik has already started responding to Louisa in SSL. She could be having trouble deciphering the signs now, they are no longer directed at her, Karin, Eve, and the video camera. There is a slight pause in the spoken conversation while Hanna waits for Louisa to have signed enough for her to interpret clearly. Eve has leaned back in her chair again (as in Figure 4), and her face is no longer visible and her arms are crossed. Finally, Hanna begins speaking what Louisa has signed, but she has missed part of the conversation. Louisa is signing, and in Stockholm, I see her sign, “The seventh of June?” Erik immediately answers that that is fine with a wave-like gesture and a nod rather than a formal sign. He then reaches down to pick up his pen and make a note. This exchange between Louisa and Erik about scheduling happens quickly and is over before Hanna has managed to translate. Still translating for Eve’s benefit, Hanna, who is unsure of the date, eventually interprets what Louisa signed with a questioning tone of voice: “The seventh of June?” Eve does not respond and is looking at the stack of notes on the table in front of in Falun. Still uncertain, Hanna then signs, “seventh June” back to Louisa with a questioning expression on her face and on her own behalf. Eve has not detected the hesitation in Hanna’s tone of voice or seen her puzzled expression. Eve leans forward in her seat, reaching for her pen and paper. In doing so, Eve is now almost 100% visible on the screen in Stockholm. Erik has already begun signing to Louisa, who answers him immediately (see Figure 6). None of this exchange is translated by either interpreter. Hanna is still asking Louisa whether the date really is the seventh of June and is signing to Louisa, who is unfortunately still in the middle of an exchange with Erik and does not see her. Louisa has turned to her left, and Erik has turned to his right to better communicate, so neither of them see the monitor in more than a peripheral way. Both are oblivious of the confusion that is erupting in Falun. Erik and Louisa finish their exchange, and Louisa immediately starts taking notes on her laptop. Hanna lets out an expression of frustration. It is a verbal expression that is only audible so Louisa misses it. Once more Louisa and Erik have an exchange in sign language that yet again goes untranslated. Hanna, still trying to get Louisa’s attention, taps her hand up and down on the table—a conventional way of getting someone’s attention in SSL. Finally, Erik notices, gently taps Louisa on the shoulder and points at the monitor (see Figure 7). Louisa looks up and Hanna signs the question (without speaking) “Seventh June?” twice. Louisa signs back “Seventh June. Yes” and Hanna signs back with “Ok. Ok” (also without speaking). There is a small pause and then Eve begins to talk—just as Louisa returns to taking notes on her laptop and so her comments go unnoticed. The meeting rounds up shortly after this, and I stop filming, right in the middle of Louisa’s final comments. It is an abrupt end to the segment, which also began abruptly and in the middle of a discussion.

Eve in Falun taking notes, finally visible to Louisa and Erik in Falun. Hanna and Karin are laughing about the misunderstandings. Louisa and Erik are signing simultaneously in Stockholm.

Erik taps Louisa on the shoulder so she can see what Hanna is asking her. Eve is again only partially visible and not looking at her colleagues in Stockholm either.
Interpreting Visibility
In this section, we offer some interpretations of this short sequence, drawing on foundations and suggested applications of methods departing from the three research fields of visual anthropology, HCI, and the humanities.
Visual Culture
An anthropological take on this video meeting might begin by anchoring the course of events—the misunderstandings and understandings—in a shared (or lack of a shared) visual culture. The visual culture in these examples is encapsulated in the medium of communication. In this case, there are multiple media—sign language, Swedish, video meeting technology, and interpreters. The meeting format itself could even be considered a medium for communication, as it too sets limitations, dictates requirements, and has its own set of practices. From this perspective, a visual anthropological analysis would focus on the cultural phenomenon of communication in SSL, at SVT Teckenspråk and in meeting situations. When considering visibility, the medium of SSL provides an interesting confirmation but also a counter-example to Thompson’s (2005) discussion on visibility and new technologies (p. 41ff.). Indeed, a lot of what went on during this snippet is tied to the nature of communication in SSL when mediated by new technologies. However, it is also contingent on SSL as a language and the influence of the interpreter. Both are arguably affected by video mediation and take on the kind of fragility Thompson (2005) alludes to (p. 42).
From an anthropological perspective, the cultural aspects of visual communication in SSL are difficult to pinpoint in one short example. The contexts, history, and general practices of visual communication are mere shadows in this example. There is no artifact or image from which to derive cultural meaning. Instead, meaning is evident in the course of events which unfolds—the note-taking which is constantly interrupted by the need for Louisa to follow the conversation; the quick exchange between Erik and Louisa, which hints at the accommodations they make when signing to nonnative SSL users; and the misunderstandings that emerge when Erik and Louisa turn away from the camera. These hiccups are in fact indicative of the way in which sign language communication is carried out. Generally, deaf people turn and face each other as much as possible when signing. Eye contact is desirable and sometimes crucial, depending on the situation. Being able to see facial expressions, and not just the hand formations, is also critical. This means that signing ceases the minute that one person turns away. Communication is interrupted. This is difficult for hearing employees at SVT Teckenspråk to fathom. Indeed, Eve offers the perfect example of how a hearing person takes part in video meetings—looking down at their notes; leaning back and looking away from the monitor to rest their eyes perhaps; fiddling with their phone, staring into thin air yet always listening to what is being said, and managing to follow the thread of the discussion. These are the mechanics of visual culture for the deaf. Yet, this is more about practices and conventions than meaning and as such only a part of what visual anthropologists seek. It is however the meat of an HCI-based inquiry.
Visibility Is Key
One observation that emerges from this example is that communication in SSL and Swedish during a video meeting is about making decisions. Clarity and consensus are as central as is the act of listening. As it turns out, precisely these elements were problematic in this particular video meeting. There was less listening and watching than required, and information was lost on both sides—in Falun and Stockholm. There were misunderstandings and unclear translations, which arose from how much or little the people involved were looking at each other. Establishing visual relations and eye contact are central to sign language. This is best illustrated by Hanna’s failure to get an answer to her request for clarification on the date. Her uncertainty and inability to get an answer hinged on the fact that Louisa and Erik were both engaged in a conversation in Stockholm and were not watching the monitor. Eve was not watching either, and Karin, who was watching, decided not to translate Erik and Louisa’s conversation until Hanna had received an answer to her question. Without this answer, Hanna’s understanding and therefore interpretation of what Louisa and Erik were discussing would be incomplete. It was not until Hanna moved her hand up and down against the table in an exaggerated “wave” that Erik and Louisa realized that she had a question (see Figure 8).

Hanna tries to get Erik and Louisa’s attention by waving her hand up and down (the blur on the right hand side of the image shown on the monitor).
Although some would argue that visual meeting technologies mean that meeting participants occupy the same spatiotemporal realms (see Thompson, 2005, p. 35ff.), it could be argued that this is a strategic social imaginary (Appadurai, 1996), which is selectively deployed during meetings. By not looking at the monitor, Erik and Louisa are no longer in the same meeting as Eve, Hanna, and Karin. They are in Stockholm, absorbed in their own notes while still engaged in mediated communication in the sense that their image is being conveyed to their colleagues in Falun.
This points to another key practicality of both VMC and deaf visuality—gaze and awareness. These concepts emerge in HCI research on collaboration and awareness in video meetings (Dourish & Bellotti, 1992; Gergle, Kraut, & Fussell, 2013; Hinds & Bailey, 2003; Isaacs & Tang, 1993). Although underlining the importance of awareness and gaze for establishing group belonging and effective collaboration, these studies rarely focus on communication in more than one language or even sign language. Typically, such research is carried out within a comparative paradigm that draws exclusively on a hearing perspective and understanding of visibility. By this, we mean that these studies often depict high-quality audio as the most important factor for task completion; video is presented as an added bonus for creating a sense of “being in the same room”. Important considerations focus on the practicalities of how communication functions in work practice: How important is eye contact? Is task completion affected if people are not visible or only partially visible during meetings? Are group relations affected? and so on. In the example of a video meeting presented here, it is clear that a hearing interpretation of visibility brings up numerous issues. The SVT Teckenspråk example however shows how visibility takes on an entirely different significance during meetings in SSL. Misunderstandings caused by partially visible signs when Erik and Louisa turn away from the camera gave rise to delays and meant that some of what Erik and Louisa were discussing never got interpreted. Eve was therefore not party to that discussion as she was looking at her notes and there was no Swedish translation for her to hear. In hearing-only contexts the outcome of not looking and not being seen is less serious. Eve would for example be able to hear and follow the conversation if everyone spoke, even if she was not looking at the screen or only partially visible herself.
In keeping with HCI studies in an organizational setting, it seems logical to frame visuality as part of working toward creating a sense of group belonging. Gaze and awareness lead to fewer misunderstandings and create the shared spatiotemporal properties or sense of co-presence which Thompson (2005) alludes to. In turn, the sharing of space and time helps foster a feeling of togetherness. However, Thompson’s association of new media with increased feelings of copresence is not self-evident. If building this sense of group belonging and establishing copresence through gaze, awareness, and other video meeting practices comes at too great a price (as in the example shown here), new media such as video meeting technology accentuates differences in space, time, and culture rather than conflating them. In the SVT Teckenspråk case, the cost of time and effort required to learn alternative ways of communicating—whether hearing or deaf—is less important than getting the job done and getting by. Miscommunication is a small price to pay for a shorter meeting and less arduous in the short term than consciously working on adopting SSL-oriented or hearing-oriented communication practices.
Thinking in these terms is only remotely related to abstract notions of a deaf visual culture in visual anthropology but relies heavily on analytical methods commonly used in the humanities and HCI. The issue at hand is more about working with multiple visualities: deaf, hearing, and technologically mediated visualities. None of these have material properties or form, so to speak, but all are embodied in the ways in which people interact with the video meeting technology: the tables, chairs, and furniture; the lighting; their computers, papers, and other personal items, and so on. For example, the video meeting technology requires that the interpreters sit at a particular angle to the camera. It also dictates that deaf employees sit within its range and that hearing employees are often outside the camera’s scope—that is, not entirely visible. It dictates a prioritization and new hierarchy within video meetings that puts the deaf first and the hearing last, regardless of their role in the editorial team. Although this is not entirely undesirable because it upsets the conventional order of things, where the deaf come second as a minority among a privileged hearing majority, it does in effect underline the minority status of deaf employees by taking away their right to choose where to sit. The lack of freedom to choose whether they are front or center, visible or not, can negatively affect people’s experiences of video meetings.
Embodied communication and the dimension of spatiality in deaf worlds are topics that have been well explored in deaf studies, although usually from a linguistic perspective, as a brief survey of the Journal of Deaf Studies and Deaf Education confirms. Even so, there are numerous studies that diverge from this trend and point out the centrality of visual communication in deaf culture and communication practices (see Blatto-Vallee, Kelly, Gaustad, Porter, & Fonzi, 2007; Emmorey, Korpics, & Petronio, 2009; Hauser, Cohen, Dye, & Bavelier, 2007; Marschark et al., 2015; Muir & Richardson, 2005 p. 154f.). In HCI too, finding technological means of capturing spatiality has often been of concern (see Rae et al., 2011; Schmitt, Gunkel, Cesar, & Hughes, 2013). Both these fields of inquiry, however, consider visual communication rather than visualities. Nevertheless, visuality is undeniably tied to being deaf, hearing, or the hearing interpreter, and these three categories require different modes and degrees of visibility. The functionalist view also reveals this but does not tie such observations to people, focusing instead on how technology and language dictate ways of seeing, and what is visible or invisible.
The view of visuality drawn upon in this analysis so far is more in line with HCI approaches and difficult to reconcile with more cultural, meaning-based notions of visuality. Although lending itself to theorization in a humanities vein, it does not draw on visual anthropological perspectives explicitly. These perspectives are, however, present. A cultural visuality akin to anthropological studies recognizes that elements such as gaze, awareness, and embodied interaction are founded in alternative paradigms and ways of seeing. For the deaf, these ways of seeing are anchored in movement, tempo, placement of signs, and parts of the body in space and relation to other bodies, as well as the sequencing of signs. For the hearing, ways of seeing are inextricably tied to hearing, sounds, and have been well examined by Berger (1977), who writes that ways of seeing are embodied in images (p. 10). Even the simple observation that the deaf and the hearing have different ways of seeing, fundamentally in accordance with anthropological and humanities perspectives on visuality, goes against HCI’s tendency to homogenize observable visual cues, gaze, and awareness. Although to the HCI researcher, notions such as gaze and awareness can be treated as universal phenomena, the visual anthropologist would categorize them as cultural. It can further be argued that gaze, awareness, and visual cues are emergent and situated (especially from a feminist approach to interaction). Taking a visual anthropological view is not entirely unproblematic, however. Unlike Berger’s (1977) examination of art, the first television series to address the role of sound and its influence on ways of seeing (Berger, 1972) or conventional visual anthropological studies, there are no fixed products produced by deaf and hearing employees to analyze when it comes to video meetings in SSL and Swedish at SVT Teckenspråk.
Invisibility
Although visualities hinge upon being seen, the way people see and who is seen, the example at SVT Teckenspråk illustrates just how central not being seen is to the ways in which people communicate via technology. In the video meeting example, the description of the encounter as well as the transcription already offers some insight into this key concept of visibility or invisibility, as the case may be. The interpreters take center stage in Falun, while Eve sits on the periphery, almost invisible as the only nonfluent sign language participant (apart from the researcher herself who is also on the periphery and invisible to Falun and even Stockholm for the most part). A symbolic interpretivist and perhaps classically anthropological reading of this positioning of people within the room might draw conclusions about power relations and group dynamics. How much do a person’s words weigh if they are not seen during meetings in SSL? SSL is a language founded on visual perception, and Eve does get somewhat overlooked, but this could be partly due to the social relations and power dynamics within the group. We have already mentioned how an interpreter stands as a visual representative of someone’s spoken words in the previous section. This is even more so when that person is not visible. At one point in the meeting, when Eve asks a question that goes unanswered, she decides to drop the issue and returns to taking notes. The interpreter, Karin, is then responsible for re-asking her question and when she finally gets their attention, she needs to make a gesture to Erik and Louisa to explain who asked the question in the first place. Louisa does not need this added reference as she knows that Eve is the only person in the meeting that is not an interpreter and who would ask a question. Erik, however, is new to these types of meetings and has not classified the interpreters as nonparticipants or “invisible technologies” (like the video meeting technology). This creates a paradox for us as researchers who choose to come from a perspective where the interpreter is considered in the same terms as the video meeting technology when it comes to participating in the meeting. It is here that an analysis that focuses on visuality comes in handy, enabling us to tackle these issues in a different manner and focus instead on the consequences of being barely visible rather than the purpose of the meeting and the content of the discussion.
The question of whether people are visible on the monitor in the video meeting room as well as whether they are visible in the video recording of the meeting is also relevant. Erik is only partially visible in the film as is Louisa. Their facial expressions and sometimes only one hand or a bit of a hand are visible, which makes it difficult to follow every exchange in SSL. Although this can create some challenges for transcription, it was a conscious choice made to minimize exposure of the employees at SVT Teckenspråk. There had to be a compromise—their hands at least needed to be visible—if we were going to get even a partially coherent recording of the meeting. It also has less of an effect on the data collected than one might imagine. Given that there is also an audio track and the interpreters speak all that is signed, the conversation itself is—in theory—easy to follow, provided the sound quality and volume are high enough. In this instance that was not the case on either count. This puts an interesting spin on HCI research that highlights the effects of poor audio and video quality on how meeting participants experiences video meetings and brings this question firmly into the research sphere (cf. Rae et al., 2011; Schmitt et al., 2013). Although there are many ethical and methodological reasons that justify omitting a focus on faces, this aspect of video-mediated communication in SSL is significant for use. Not being able to distinguish facial features and expressions due to video quality and size does affect the quality of communication. The group in Falun is tiny on the monitor, and it is often hard to see whose mouth is actually moving and therefore not always possible to determine who is speaking for whom. This has practical consequences, not only for analysis but also for communication in sign language, as studies have shown that deaf people when communicating via video focus mostly on faces “to pick up small detailed movements associated with facial expression and mouth shapes” (Muir & Richardson, 2005).
Not being visible and its impact on VMC raise concerns about how not being visible might affect the analysis and interpretation of these kinds of visualities. If invisible or without form, how can we grasp the significance and meaning of cultural phenomena using traditional methods and paradigms?
Methods and Paradigms
We have already hinted at the centrality of visibility in deaf contexts and how they become enacted in actual events and happenings that a researcher takes part in. Based on an ethnography of visual practices of video meetings in a bilingual (deaf–hearing) workplace, we address questions such as the role of images and visuality in different phases of participant observation such as data collection, analysis, and the rendering of results. We have explored some of the methodological and analytical implications using this example to challenge assumptions about visuality and visual analysis. In this section, we aim to delve deeper into these issues and reflect on our own practice and ask the more specific question: How does the visual affect ethnographic research methods today?
In this example, eye contact and the act of seeing are central. It is therefore feasible to employ typical HCI methods such as eye-tracking to gauge how the eye moves and whether this movement coincides with misunderstandings or fluid conversation. This illustrates the mechanics of VMC but does little to get at the sociocultural aspects unless interpreted in combination with video footage and transcripts, which highlight moments when eye contact and gaze are critical or instrumental to successful VMC. Such a comparison would reveal key instances where hearing modalities of communication collide with deaf modalities and vice versa. Ignoring the burden of conventional visual anthropology, this method does not require a final visual product but views the visual as a mode of perception anchored in the act of seeing. It is not this clear-cut, however, once visual anthropology is brought into the picture. From this perspective, the video transcript is part of a visual culture—that of the researcher. Examination of the act of seeing and a comparison with video documentation brings two potentially different visual cultures and practices into the same methodological approach. There are problems inherent in this, unless a comparative stance is taken. The visual images documented by the anthropologist using a combination of photographs, short segments of film, field notes, and sketches are perhaps not representative of the deaf way of seeing and yet are required to act as such. This has methodological consequences that hark back to early existential crises in anthropology where the white anthropologist’s ability and right to represent “reality” and more specifically the indigenous voice in text and image was called into question (Marcus & Fischer, 1986, p. 9). Interestingly enough, Thompson and his new visibility offers a potential solution.
Although we have not addressed this aspect in this article, Thompson’s (2005) Foucault-inspired theories on visibility can be applied to VMC in sign language (p. 39). This is especially true when we consider that the video meeting technology was designed by hearing engineers and designers specifically for hearing needs. This bias re-affirms the hearing privilege of being invisible or partially visible when and where it suits them. Although the deaf are used to being visible even when communicating in deaf-only contexts, this lack of choice and constant visibility underlines inequalities when it comes to VMC, but also extends into the workplace and society as a whole. Furthermore, it allows the researcher to acknowledge potential ambiguities and the power they take when communicating their representations of VMC in sign language and deaf ways of seeing.
Asynchronous Visuality Without Form
The complexities of communication in two languages via video and with interpreters definitely present challenges to a dominant hearing and logocentric research paradigm and also to methodological practice. Not only have we got deaf visualities and hearing visualities at work in this video meeting, but the recording of the data and their analysis puts a second order of visuality into play—something similar to ethnographic filmmaking and yet not quite. In this example, there is no final presentation of a meeting in the form of a film to be analyzed as a complete visual artifact, but only snippets and screen shots. The video meeting is never shown or re-presented as a film and is instead rendered into a set of still images that poorly capture the dynamics of sign language, speech, and video meetings in general (cf. Anthes, 2009; Broth, Laurier, & Mondada, 2014; Dorsey & Diaz-Barriga, 2010; Favero, 2005; Glass, 2009; Hammond et al., 2009; Kisin, 2011; Sinha, 2009). Instead the researcher falls back on their own cultural bias toward a primarily textual presentation of results (Banks, 2001, p. 3ff.). The conversion of video meetings as unfolding interactions into a permanent, final text messes with the fleeting, momentary, and fluid events of video meetings. Academic representation is plagued by asynchronicity and definitely does not share spatiotemporal properties but rather distorts these, imposing a second order of visuality—that of the researcher.
Temporality is an issue that has previously been raised in relation to the production of visual imagery such as film and live television broadcasting from the perspective of those being studied (see Perry, Juhlin, & Engström, 2014). In this article, we underline the importance of acknowledging that we as researchers introduce a temporality of our own and one that often contradicts that of the actual phenomena we aim to study. Phenomenological perspectives on the visual in anthropology, for example, identify a danger of closure in structuring ethnographic film, so that it imposes a “false fixity and narrative coherence on the fluidity of quotidian social interaction” (Banks, 2001; see also Banks & Morphy, 1997, p. 18). A researcher’s account and documentation of a video meeting through video recordings and the final presentation of these happenings in text and published works misleads the reader or academic audience. It imposes a fixity and sense of closure that does not exist and in this sense, the researcher is, rather than interpreting cultures, co-creating cultures (see Haraway, 2007, p. 4 ff.; more specifically on visual anthropology, see Behar & Brink-Danan, 2012; Westin, 2014).
Although we advocate presenting research results in different visual formats, one small observation we must make is that empirically-centered research often presupposes a study of culture that takes visual form. In typical postmodern style, anthropology and HCI are careful to also focus on the process of creating this form, yet there is still the expectation that this process will result in a final visual artifact—even if it is accepted that this final artifact continues to be configured and reconfigured through human interactions (Suchman, 2007).
Conclusion
In this research project on video meetings between deaf and hearing editorial staff using sign language, there is no absolute visual form that accurately represents the phenomenon under investigation in either SVT Teckenspråk or the researcher’s worlds. The video meeting is not recorded in its entirety nor is there a film made about it. The people taking part in the meetings have no expectation that the meeting will make up a visual artifact to which they can refer back to and discuss by “looking at it.” So, what does this mean for studies of visuality? Is there something we can draw from HCI studies of video and visuality that do not have this exclusive focus on a visual form? Perhaps, a focus on the video meeting as interaction? Here too, HCI’s traditional focus on interactional aspects and relationalities has been criticized for failing to consider the material and its properties (Dourish, 2013). We have also stressed that there is a need within studies of visualities and visual culture from all disciplines to consider a visuality without form. It is not as simple as “turning away” from the “material turn” because, as Banks and Morphy (1997) rightly point out, any recording of visible aspects of behavior essentially becomes material and—like other aspects of material culture—can be re-analyzed at any time (p. 17). In fact, Banks and Morphy argue for including material products of culture and recordings of visible culture within the same discourse. In view of the visual documentation methods used in fieldwork, we too would argue that to deny all consideration for the material (as is usual in discourse on the digital and digital technologies) actually takes away from an understanding of deaf visualities. The visuality of sign language communication and VMC is indeed inherently embodied and thus anchored in the material realm. Therefore, there is merit in strategically employing such an HCI interactional approach as long as it is tempered with a notion of materiality via the concept of embodied interaction inspired by Merleau-Ponty (Svanaes, 2013).
An approach that extends the notion of visual material culture beyond image, artifact, or product fits well with sign language and the fact that this kind of visuality is often rooted in the way the body moves. This deaf visuality is not about visuality alone but also involves a level of embodiment—a materiality somewhat different from that presented in studies of the visual which focus on the visual form, for example, images, films, or artifacts in anthropology, HCI, and the humanities alike. It is a materiality that incorporates spatiality and movement, which are key components of SSL. In this respect, both video meetings as video-mediated communication and sign language represent a kind of temporary, momentary, and embodied visuality where perception and mutual co-creation for communication and creation of a cultural language and community is the “form” that results. Our visuality includes a deaf visuality, the visuality of video-mediated communication and video-recorded research data. This visuality is about the body and its movements in space and time—an embodied materiality that is impermanent in a physical sense (unlike images and film as artifacts). However, like images and film, the way this visual materiality is perceived and interpreted varies and is multiple depending on the viewer, the setting, the situation, and the meanings inscribed (Rose & Tolia-Kelly, 2012).
This embodied materiality extends out of field and into research contexts. On a practical level and with implications for doing research, “film can also work to shift emphasis away from the concrete nature of material forms towards properties that emerge in context, or to processual aspects of their production that may be more salient than the final form that is produced” (Banks & Morphy, 1997, p. 18). To draw on history, a focus on process rather than form might just be the ultimate anticolonialist move. The infamous British classification of Australia as terra nullius comes to mind—a verdict that was founded on European ethnocentrism and its obsession with the need for material forms of culture such as “man-made” buildings and permanent visible infrastructures. Such a notion was alien to the indigenous population, who saw the environment as just as indicative of their culture as anything constructed could be. By claiming that the absence of visual or permanent objects and structures was synonymous with a lack of culture and social structure, the British not only failed to recognize other cultures and their materialities but also ignored their visualities. There is a danger of something similar happening in research if we continue to prioritize research on cultures and cultural phenomena where a visual form, artifact, or product exists. In deaf contexts, to conclude that the absence of visual forms means visuality is not important is actually to ignore a central part of deaf culture. To classify sign language as simply linguistic and without visuality is just as damaging. The example of VMC in sign language in this article shows that visuality is indeed central to understanding the workings of SVT Teckenspråk and Swedish deaf culture in general. However, although our arguments for a concern with the visual and efforts to distinguish between visuality based on visual processes and form could potentially provide solutions for new approaches to visualities, such distinctions may lead to more problems than they solve (Banks & Morphy, 1997; Battaglia, 1997; Born, 1997, p.18f.). Our analysis of in/visibility which draws on perspectives found in visual anthropology, the humanities, and HCI underlines the significance of being seen, seeing, and how different ways of seeing come to play. As such, is it “seeing” as an act, subjectivity, and culture that is our new interdisciplinary object à la Barthes rather than visual culture, visual perception, or visuality. This interdisciplinary lens proves critical to understanding a visuality without form within the context of SSL video meetings at SVT Teckenspråk and offers a means of critiquing and debating methods of data collection and analysis in all fields of research.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by FORTE, the Swedish Research Council for Health, Working Life and Welfare. Project number 2013-1236.
