Seeing like a Rover: Visualization,embodiment,and interaction on the Mars Exploration Rover Mission

Abstract

Based on more than 2 years of ethnographic immersion with the Mars Exploration Rover mission, this paper examines the representational work and associated embodied practices through which the science and engineering team makes decisions about how and where to move their robots. Building on prior work in Science and Technology Studies on the importance of embodiment to visualization, the paper posits that such practices also contribute to the production and maintenance of social order within the organizational context of the laboratory. It thus places visualization technologies and techniques in the context of the social organization of scientific work, contributing to our understanding of representation in scientific practice.

Keywords

embodiment human–robot interaction planetary science representation social organization

Introduction: ‘My body is always the Rover’

On a cold February day in 2008, I sit in a windowless room in a university astronomy building beside Liz,¹ who is staring intently at her screen. Liz is one of the Mars Exploration Rover camera operators. Working with the Panoramic Camera (Pancam)² instrument, Liz codes mission scientists’ image requests for daily upload to the team’s twin robots, nicknamed Spirit and Opportunity. The two Rovers, each about the size of a golf cart, were built by NASA to explore the surface of Mars, looking specifically for evidence of past water. Arriving on Mars in January 2004, the robots do not operate autonomously. But because of the distance between Earth and Mars, it takes approximately 7 to 20 minutes for a signal to travel between the two planets. As a result, there is no ‘joystick’or ‘real time’ operations for driving the robots. Instead, a NASA-selected and funded team of scientists and engineers meets daily to decide upon and code the Rovers’ subsequent day’s activities, and to upload the instructions to Mars, based on what the Rovers have accomplished the day before. The work of the mission, then, is largely engaged in using images and visual work to determine where the Rovers are, what their instrumental findings suggest for Martian geological history, and what they should do next.

Liz is engaged in precisely this visual work when I join her. On her screen is a simulated Mars Rover field of view, assembled from black and white low resolution Navigation Camera (Navcam) images: she must use these images to indicate how and where the Pancams will point, in order to take the high resolution color pictures that her teammates have requested. The image she is setting up now requires her to command the Rover to look downward, to take a close-up picture of a rock located exactly between its front two wheels.

Liz looks at her screen, and tilts her head to one side for a moment. Then, she takes her cell phone out of her purse and places it on her desk in front of the screen. She then raises both hands to either side of her head, forearms perpendicular to the floor, head tilted slightly down, fists open but fingers lightly curled. Slowly, mechanically, she twists sideways from her waist, mindful of the location of her hands relative to her phone (Figure 1). When I ask her what she is doing, she explains:

So that’s [points to her cell phone on the desk] close-up rock, and then I know that there’s a disconnect [raises hands to either side of her face] between left and right eyes [on the Rover]. So I have to move my head like this [tilts her head down, rotates at the waist, tilting right hand higher than left], and I have my left eye here [pauses], and then this [swivels to the opposite side, keeping head down, with left hand higher than right] is my view from the right eye.³

Figure 1.

The body work of image planning. The camera operator uses her hands to approximate the location of the Pancam’s eyes, and her cell phone to model the location of a rock she wants the Rover to image on Mars. Drawing (from author’s photo) by Craig Sylvester, with permission.

Then, in an intricate association of talk and gesture, she demonstrates for me how she associates her body with the Rover’s, piece by piece (Figure 2):

My body by the way is always the Rover, so right here [touches chest] is the front of the Rover, my magnets are right here [touches base of her neck], and my shoulders [touches shoulders] are the front of the solar panels and that’s [leans forward, splays arms out behind to either side at 45 degrees] the rest of it. So I have all kinds of things [i.e. antennae] sticking up over here [gestures to back], um [laughs]. But when I’m taking a picture of something in the atmosphere then it helps me to kind of look up [looks up and sits up straighter], being the Rover, and this is the front of me [touches chest] and then I put my head up [puts head up, looks back and forth] wherever, to whichever vector I’m looking at … ⁴

Figure 2.

‘ … And that’s the rest of it.’ Embodying the Rover’s solar panels and bodily stance. Drawing (from author’s video) by Craig Sylvester, with permission.

Recent work in Science and Technology Studies has drawn a tight connection between visualization and embodiment. For example, Natasha Myers (2008), in a study of protein biologists, relates how a lab director, challenging a simulation model, ‘contorts her entire body into the shape of the misfolded protein. With one arm bent above her head, another wrapping around the front of her body, her neck crooked to the side, and her body twisting, she expresses the strain felt by the misshapen protein model’ (Myers, 2008: 165). Myers thus calls attention to the body work involved in simulation, and the embodied imagination that practitioners must gain about their objects of study as part of their training. Morana Alač (2008, 2011) also demonstrates the importance of taking a multi-modal semiotic approach to study the situated work of interpreting brain scans. Tracing how operators use gesture and talk to make sense of and make visible digital images on a screen, Alač (2008: 493) describes the digital image as a field for interaction, wherein ‘gesture, talk and the manipulation of the digital screen function together as techniques for managing perception.’ Anthropological work by Rachel Prentice (2005, 2012) on laproscopic surgery also describes both the embodied practices of making surgical bodies visible as well as the embodied talk surgeons commonly use to describe their laproscopic sight and interactions. While a surgeon described his work to Prentice, he discursively positioned his own body within his patient’s body at the point of the scope, saying: ‘I would say I am sitting on that piece of anatomy, or rather that you are floating around, swimming around in the [joint].’ The philosophy of science as well, renews attention to embodiment in visual practice, whether in the work of Hans Radder (2006), which recovers the role of bodily positioning in the work of ‘seeing as’; or in the postphenomenology of Don Ihde (2009) and his students, who articulate how all scientific knowledge is technologically embodied, enlisting both human bodily perceptual practices and instrumental mediation in visualization technologies. These scholars remind us that scientific seeing requires not only eyes and instruments, but hands and bodies as well.

Thus my encounter with Liz’s robotic gestures in the context of her visual work should not be surprising for an STS analyst. We might analyze her gesture as part of her embodied imagination (Myers, 2008), which supports her sense of robotic possibilities of movement; as articulating the instrumental action and interaction necessary for seeing (Prentice, 2012); or as coordinating ‘“visual” information with the world of meaningful actions and practical problem-solving’ to establish a shared semiotic space for interaction (Alač, 2008: 504). To do so would generate more fascinating examples of visual epistemology and ontological work in action. But it would also leave aside a rich and broad context in which Liz’s solitary body work takes place. For despite the fact that Liz is sitting alone at a screen, her work is in fact coordinated with 150 other technicians, engineers, and scientists across the United States, Germany, and Denmark who together work on the Rover mission team. As I will argue, visualization and embodiment are an essential part of that coordination, and cannot be fully understood outside of this organizational context.

In this paper, I seek to broaden our understanding of visualization and embodiment from individual perception or dyadic interactions to include the collective work of the laboratory in our analytical perspective. I argue that visualization practices, with their associated gestures and embodied narratives, are not only enrolled in understanding the object or instrument of analysis, but are also organizational practices that produce and maintain the social order of the laboratory. While actor epistemologies and object ontologies are certainly instrumentally mediated and are enacted through coordinating images, talk, and gesture, this work is itself situated within an organizational context. As performances of social form, visualization and embodiment together produce and enforce a particular social order.

My case study draws upon more than 2 years of laboratory ethnography with the NASA Mars Exploration Rover team. Based at one of the handful of universities affiliated with the mission, I attended more than 300 of the daily teleconferenced mission planning meetings and 60 weekly science meetings, interviewed more than 80 team members at various stages of their careers, and visited 10 of the mission’s participating networked labs.⁵ Daily work with the Rovers across these settings is suffused with image work of all kinds. Scientists at their desks work continually with image processing software to investigate the Martian terrain; engineers at the NASA Jet Propulsion Laboratory work with images to ascertain safe driving zones for the Rovers; images are constantly planned, acquired, calibrated, processed, processed again, and made available for various forms of public release. Speaking of the importance of image work on the team, Liz’s colleague Jude explained, ‘When you work with the team for a long time, you sort of learn to see like a Rover.’⁶

Borrowing Jude’s phrase, I will show how seeing like a Rover not only involves learning the ‘professional vision’ (Goodwin, 1994) skills necessary to interpret Rover images. It also involves visualization, talk, and gesture, which together draw a direct association between individual sensing and moving bodies on Earth and the body of a robot on Mars. It therefore recalls prior work by scholars who have studied the perceptual work involved in operating other remote instruments, vehicles or ships (Goodwin, 1995; Helmreich, 2009; Hutchins, 1995). But seeing like a Rover, I will show, also involves learning to see like a member of the Rover team in the context of a particular (and peculiar) social organization, and to account for robotic and human relations in particularly intimate, totem-like terms. It therefore draws a yet closer association between visualization regimes and local practices of sense-making on the one hand, and the imperatives of social order on the other. I will begin by discussing the visualization practices and ‘body work’ with associated forms of talk, gesture and embodied narratives common to the mission team. I will then explore the robot’s role as team totem in producing and maintaining a local order oriented toward collectivity.

Seeing like a Rover: Professional vision and body work

A new team member’s first introduction to seeing like a Rover usually begins with Hazcam images (Figure 3). These fish-eye lens cameras, mounted under the robots’ solar panels, are used to plan Rover drives, and thus return pictures that present the broadest possible field of view. Rover scientists often point out the distortion, frequently joking that ‘objects in the mirror are closer than they appear’ to highlight the discrepancies between human and Rover sight. Their fluency with these particular visual forms certainly demonstrates the acquisition of a local form of professional vision (Goodwin, 1994) with respect to the interpretation of Rover images: that is, they apply a discipline-specific way of seeing in their visual interpretation. However, they rarely use a corrective algorithm to digitally adjust the image to a more rectangular projection, producing a more familiar photographic frame. Instead, they learn first and foremost to acquire the robot’s own native representation of Mars, as well as its own bodily orientation and apparatus.

Figure 3.

Hazcam image of Mars, with curvature from fish-eye lens. Spirit Sol 1162. Credit: NASA/JPL/Cornell.

Visualization and robotic bodily sensitivities are elided at the level of the hardware. Each Rover’s own artificial intelligence software actively scans its images for hazardous elements of the Martian terrain in order to safely execute driving instructions. The Rover drivers and other team members adopt a parallel sensitivity to the Martian terrain as the robot encounters it. Mark, a Rover driver, explained that he drew a direct connection between how the Rover built up its ‘view of the world’ and its particular embodied sensory practice:

[t]he Rover’s view of the world when driving is very much like your view of the world if you imagine yourself trying to make your way through a dark cluttered room with nothing but a flashbulb. So you can kind of take a picture in the world and you can get a sense of where there’s a safe path and you walk a little way along that safe path and you pop the flashbulb again. … That’s one of the ways in which the Rover sees the world when it’s driving.⁷

Mark is one of the specialist engineers who must be particularly adept at identifying rocks, slippery soil, sand traps, and other potential obstacles in Rover images that would be likely to trip up a five-foot tall, six-wheeled Rover roaming the wilds of Mars. This often requires not just seeing from the Rover’s point of view, but also actively processing images to gain a sense of the terrain. For example, Mark explained the value of the three-dimensional (Figure 4) view as one that engaged his kinesthetic sense, making elements ‘pop out’ to get a ‘better sense of the size and slope’ of the terrain:

there’s something to be said for engaging your own kinesthetic sense … . If you take a look at this in 3D [three dimensions], you can see how it now kinda pops out at you, how this terrain is kind of undulating … where I could see kinda that there was a ridge here [in 2D], this is now [in 3D] giving me a much better sense of the size of that ridge and the slope of that ridge … ⁸

Figure 4.

3-D view of a crater to ‘engage your own kinesthetic sense’. Opportunity Sol 131 Navcam anaglyph. Credit: NASA/JPL.

Mark’s examples resonate with postphenomenological theories of vision (Ihde, 2009). Mark, Liz, and other team members take on their robot’s instrumentally-mediated vision as their own, making it the lens through which they experience Mars and formulate questions for continued observation. But these practices go beyond extended vision to bodily sensitivity more generally. Not only Mark’s sight, but also his ‘kinesthetic sense’, is at play. Here too, ‘engaging your own kinesthetic sense’ is not just a question of Mark’s own kinaesthesia, but his robot’s. Skilled visualization and embodiment practices are part of adopting the robot’s sensitivities and mobilities relative to its environment on Mars.

This skill is not only enacted through representational practices and talk, but is also physically performed through gestures and movements that write the Rover onto the human body. Eliding human and robotic experience begins at the level of talk about the robotic body. Although it lacks a humanoid shape, various parts of the Rover are verbally related to human body parts and actions. For example, the Panoramic Cameras are regularly referred to as the Rovers’ ‘eyes’, the hazard cameras aimed at the wheels show ‘what’s under our feet’, while the Instrument Deployment Device (IDD) is ‘the arm’ and its four instruments are sometimes described as ‘fingers’ (Figure 5). The Rovers ‘talk’ to Earth via communication antennas, ‘sleep’ at night, ‘wake up’ and ‘take a nap’, ‘stare’ or ‘look’ at targets on the surface regularly throughout the day. These active verbs describe technical activities, but also reinforce an experiential dimension of these activities consistent with human experience.

Figure 5.

Mars Rover diagram, with labeled parts. Each part is associated with a human body part (‘The Rover has … a body: a structure that protects the rover’s “vital organs” … arm: a way to extend its reach … wheels and “legs”: parts for mobility…’). Available at: http://marsrovers.jpl.nasa.gov/mission/spacecraft_surface_rover.html. Image credit: NASA/JPL (accessed 6 March 2012).

Ascribing human characteristics to machines or objects is well described in both psychology and in studies of human–robot interaction, and the Rovers are certainly anthropomorphized by team members in many common stories.⁹ The robots even have social class: during my ethnography, Spirit was frequently described to me as a ‘blue collar’ laborer who had to work for every success she earned, while Opportunity was a ‘golden girl’ who found evidence of water on Mars immediately after landing.¹⁰ But in the visual and gestural moments discussed above, the projection does not run from human to robot, with the Rovers acquiring human characteristics: the human beings on the mission must learn, imitate, and demonstrate what it is like to be a Rover on Mars. This move is thus in opposition to anthropomorphism: it is a technomorphic move in which team members take on the robot’s body and experiences as part of their practice and narrative of their work.¹¹ It is a kind of interactional work on Earth that constructs and makes present the Robot’s body-in-interaction (Alač, 2009): in this case, a body that is not co-present with its interlocutors, but located on Mars.

A key aspect of this technomorphism (or, adopting Mark’s terms, this ‘kinaesthetic sense’) is a developing sensibility to what the Rover might see, think, or feel, in relation to specific activities that must be planned. For example, Pancam operators are highly attuned to the sun’s position on Mars throughout the day, and attribute their heightened sense of Martian light and shade to knowing how to see with the Rover’s ‘eyes’, knowing where the Rover is, and whether its shadow will project onto its photographic object. A Rock Abrasion Tool operator spoke of his instrument as the Rover’s ‘sense of touch’, referring to the output graphs of drill intensity with descriptions of how the Rover ‘feels out the rock’. Mark confessed to me that, when planning a drive, ‘I have frequently tried to put myself in the Rover’s head and say, what do I know about the world … ’. He then elaborated by describing the differences between himself and the Rover: ‘ … the Rover has senses that we don’t have … the Rover sees stuff that we don’t see, it sees into wavelengths that we don’t see, it never really sees the world in color but it can see parts of the spectrum that we can’t.’¹² Jordan, a Mission Manager, related sensitivity to the Rover’s experience with his own physical bodily sensations, ‘feeling’ and ‘intuition’. For Jordan, working with the Rovers requires ‘having a feeling’ about the robot’s present and upcoming activities:

You just have more of an intuition as to, I think, I don’t know if this is a good example or not but you know as you get older you understand how your body works more and so you know the effects of if you haven’t eaten breakfast or something, you know by lunchtime you can feel, you know what it’s like to feel, you know why you feel differently right before lunch as opposed to yesterday when you had breakfast. And so operating the vehicles after a while you get an idea of well, okay, the Rover did this yesterday so I have a feeling, I know what it’s going to be like tomorrow. Or I know it did a really long drive yesterday, so I have a feeling … .¹³

Enhancing this intuitive and embodied connection are a set of practices that, taken together, become a kind of physical calculus for working through Rover motions and activities on Mars from Earth. Mark developed a set of paper weights that mimicked the degrees of force the Rover could use on Mars, so that his colleagues could get a sense of what the Rover’s experience ‘felt like’ in their own bodies. Ben, a scientist, has a piece of paper cut out in the shape of a Pancam frame that he lays over images on his screen to get a sense of what a proposed observation will capture. Jude recalls how she and her colleagues ‘used to put post-it notes on our foreheads so we could know how the [Pancam] frames would turn out’ by imagining the yellow squares projecting out from their foreheads onto their screens in the image command software, and from there projecting into the Martian world around the Rover (Figure 6).

Figure 6.

Pancam software places yellow squares onto a Navcam mosaic of the terrain to plan imaging. August 11, 2006, with permission.

Many gestures are common practice, shared across the team. When one scientist proposed a new maneuver in a planning meeting, another in the room used his wheelie chair to work through the move as it was being described. Mark also confessed that he and his colleagues ‘used to talk about how the Rover was going to go by scooting around in our chairs … ’. Team members regularly manipulate their shoulders, elbows, and wrists to mimic the robots’ range of motion, and when estimating their position they splay their arms out to either side to imitate solar panels and tilt their bodies to approximate the Rover’s pitch and yaw. One of the most common gestures I observed on the mission is that of using one’s own arm to demonstrate how the Rover deploys its IDD, colloquially called its ‘arm’: this involves lifting the right upper arm to shoulder height, dropping the forearm to 90 degrees with the hand in a fist pointed at the ground, and articulating the arm in a limited fashion across the body, mobile first from the shoulder, then from the elbow. As one Rover driver put it:

When we’re training new Rover drivers we can really tell that they get it when you start talking about moves with the IDD and they start moving their own arm to kind of show you what they mean, and they say you know we’re gonna swing this to the left and then move their elbow [moves his elbow to the left].¹⁴

This situation sets up such gestures as communicative acts. As in Alač’s analysis of the work of making brain scans legible, we might take a semiotic approach to how these gestures communicate the skill of embodied digital seeing from an experienced technician to a novice. During my fieldwork, I certainly witnessed situations in which such semiotic acts were communicative in nature, in which a wheelie chair maneuver or a skilled twist of the elbow was a central articulation in the work of coordinating action at a distance. However, the vast majority of times I witnessed these gestures, there was no one else in the room. Most frequently, scientists, engineers, and technicians alike gestured in what were clearly formal, codified, standardized ways of enacting the Rover, but they did so entirely alone, speaking to mutually invisible interlocutors on a telecon line.

Further, what are we to make of the fact that team members repeatedly spoke of or simply adopted language that drew a direct connection between their individual bodies and those of the Rovers? Countless team members I interviewed explained that their eyes have ‘become Pancam, or Navcam’;¹⁵ another put it simply: when working with the Rover, ‘I am a Rover. I am a Pancam.’¹⁶ Several team members settled on the language of the ‘proxy’, or the parent–child relationship, as the closest approximation of their embodied connection. Some analysts have taken up this language to discuss the proxy nature of the robotic team member (Clancey, 2009; Mirmalek, 2008; Schairer, 2006). Team members put this language to powerful use, particularly in public fora, to articulate the intimacy of individuals’ relationship to the vehicles. But behind the scenes, such formulations were not visible in practical everyday action. The ‘proxy’ is clearly a common member’s account shared across the team, but as such it is an explanandum, not an explanans.

Most interesting to me as an ethnographer were those backstage activities, narratives, and body work that further elucidated this inexpressible relationship. Many stories assumed a direct, even causal, somatic association between the robot’s experiences on Mars and individual bodies. As a mid-career female scientist recounted:

I was working in the garden one day and all of a sudden, I don’t know what’s going on with my right wrist, I cannot move it – out of nowhere! I get here [to the planning meeting], and Spirit has, its right front wheel is stuck! Things like that, you know? … I am totally connected to [Spirit]!¹⁷

Such matter-of-fact statements are not restricted by age or gender, but occur across the mission. Here is another example, from a young male engineer:

[I]nterestingly, I screwed up my shoulder … and needed surgery on it right about the time that Opportunity’s IDD [arm] started having problems [with a stiff shoulder joint], and I broke my toe right before Spirit’s wheel [broke], so I’m just saying, maybe it’s kind of sympathetic, I don’t know, [laughs] I mean I don’t think there’s any magic involved or anything but maybe it’s some kind of subconscious thing, I don’t know.¹⁸

Jude, too, explained to me that when something is not right with the Rover, ‘We feel it in our bodies.’ And after 6 months with the mission, I too, learned how to tell whether it was a good day or a bad day on Mars based on the sound of the Principal Investigator’s (PI’s) footfall in the hallway outside our meeting room, or the slouch or spring in my colleagues’ steps in the lab. I also started to feel the curious, different bodily experience of working with Spirit as opposed to with Opportunity. It is difficult to articulate verbally, but I can feel it in my body, like a particular kind of posture or stiffness associated with each robot’s different faults and features.¹⁹ Taking on a different bodily sensibility in this way is a disembodying experience.²⁰

This body work is certainly intimately associated with visualization practices and interpretation skills. In these stories, the immediacy and somatic relations at play involve visual interpretation associated with professional identity (Goodwin, 1994), making objects visible (Alač, 2008; Prentice, 2012), articulating a body-in-interaction (Alač, 2009), and instrumentally incorporated vision (Ihde, 2009). But there is an additional factor at play in the embodied visual practices seen here. We can glean yet further insights by panning outward somewhat from the interactional site at individual desks and placing those interactions within the local organizational context of work.

From embodiment to order

STS literatures on visualization, epistemology, and ontology are usually held somewhat distinct from the STS literature on the organization of laboratory work. Organizational currents in STS emphasize community structures and infrastructures (Knorr Cetina, 1999), collaborative practices (Shrum et al., 2007), the structural importance of funding (Salonius, forthcoming), collectivist technologies (Turner, 2005), and the epistemic politics of labor in the laboratory (Doing, 2004), but rarely treat visualization or gesture as organizational work. Similarly, studies of visualization and gestures in laboratory contexts may describe several scientists together ‘fixating’ visual evidence (Amann and Knorr Cetina, 1990), or teaching interactions in which newcomers are instructed on what to see (Alač, 2011; Myers, 2008; Prentice, 2007), but frequently refrain from placing these interactions in institutional perspective. Certainly Fyfe and Law’s volume on visuality and social relations focuses primarily on power relations in a macrosociological perspective (Fyfe and Law, 1988); Shapin and Schaffer make the foundational claim that ‘solutions to the problem of knowledge are embedded within practical solutions to the problem of social order’ (Shapin and Schaffer, 1985: 15), and show how visual technologies for ‘virtual witnessing’ are central to the production of a knowledge-making community; while Latour (1990) discusses how mobilizing inscriptions as ‘immutable mobiles’ can both muster ‘allies’ and ‘bring back’ distant objects to ‘centers of calculation’. But these early formulations aside, the broader social context of visual interactions – how visualization and gesture are organizational activities enlisted in the production of local order – remains largely unexplored.

Despite the distance separating Mars Rover team members, their community remains central to their work practices, their bodily connections to the robots, and their use of visual software tools. A few weeks after Liz performed her Rover gestures alone in her mission planning room, she described for me how the Rover’s body was ‘the glue that bonds the team together’, especially as it moved to a distant location and had to be invoked (or appresented, in Knorr-Cetina and Bruegger’s (2002) terms²¹) daily:

The hardware (i.e. a rover) is like the glue that bonds the team together while it’s being built on Earth. During that time, we can directly relate over something physical. Once that spacecraft is off the ground, that connection moves into the software realm, and also into our minds. So I’d argue that the dynamics of the team took on greater meaning once Spirit and Opportunity left the planet. Once those rovers leave Earth, the team is all we’ve got.²²

Important here is Liz’s repeated invocation of ‘the team’ as she discusses the Robot’s hardware and software systems: she discursively aligns the distant Rovers with local team dynamics. The daily work of Rover management does not simply enroll individual eyes, bodies and screens. It also produces a special relationship between a Rover and its community of caregivers, and brings that community together into a particular kind of alignment. To see how this is the case, we must learn more about this particular community’s structure and norms, as they are produced through organizational form, ritual meeting interactions, and local forms of talk.

Mission organization is a primary resource in the construction of local order. Rover team members frequently describe their organizational form as unique, with an unusual relational structure. The majority of NASA missions exhibit strict hierarchies and divisions of labor, assigning teams of scientists to single instruments with coordination between instrumental teams limited to the PI level. But the Mars Rover mission was purposefully organized differently: a single Principal Investigator (PI) leads a team with an emphatically flattened hierarchy, in a techno-scientific orientation consistent with ‘post-communitarian’ (Knorr-Cetina, 1999), ‘commons-based’ (Turner, 2005), or ‘participatory’ (Shrum et al., 2007) systems. Members of the team have clearly defined roles – Rover driver, instrument operator, scientist, engineer – and much like the operators of a naval vessel (Hutchins, 1995) they are attuned to particular sensitivities and responsibilities corresponding to those roles. Organizationally, however, they are continually reminded that no one position or individual should be considered higher than another.

Effacing distinctions and flattening hierarchies may sound utopian, but it requires considerable work and enrolls many practices and forms of talk in its continued execution. For example, certain operational duties and organizational roles rotate within the team, enlisting role distance (Goffman, 1961) to discourage personal attacks in response to an individual member’s decisions. Scientists and engineers alike are encouraged to think of the Rover as a single, unified instrument: ‘like a Swiss Army Knife’, as the PI is fond of saying. That is, any team member can use any instrument or robotic attribute (even the wheels) to conduct scientific investigations. The PI frequently emphasized to me his belief that this approach of collectivity and consensus maintained a positive spirit of collaboration, allows the team to think creatively and in an interdisciplinary fashion, resulting in what they across the board characterize as ‘the best possible science’. This narrative was repeated across the team as a central shared story: an explanation for the group’s social form and support for maintaining their collectivist attitude.²³

Within this organizational orientation, certain rituals and forms of talk proliferate across the team to produce a kind of team-wide mechanical solidarity.²⁴ One such ritual is that of consensus: generating unilateral assent at the end of every mission-planning meeting. I witnessed meeting after meeting end with the meeting Chair (a rotating responsibility) asking each representative on the teleconference line, ‘Are you happy?’ with the plan. The appropriate ‘response pair’ (Goffman, 1981), if there are no outstanding questions, is a chorus of, ‘yes, I’m happy’. Without this ritual response, the team would not even consider producing commands for the robots, but would rather return to a point where they could ensure unilateral assent.

Other members’ forms of talk cement this collectivist orientation, centered around the Rover’s-eye view: for example, the overwhelming predominance of the first person plural pronoun, ‘we’, to describe the Rovers. Planning meeting chairs frequently make statements like the following:

We expect to turn around and take images of [the target] … . We’re about four meters from the outcrop that we wanted to image and so the idea was to bump forward maybe two or three meters so that we can get better images and [thermal] observations. [Emphasis added]²⁵

Other scholars who studied this team have also noted the word ‘we’, implying a united team of both humans and the Rovers (Clancey, 2009; Mirmalek, 2008). This resonates with Ochs, Gonzales, and Jacoby’s (1996) work on physicists, which has also noted a linguistic tendency to place ‘I’ in the position of the object of study. But here, I emphasize how utterances like these, usually punctuated with Rover images that situate the entire team behind the Rover’s eyes, construct a singular subject position from which the team as a unified collective experiences Mars.

It is not the purpose of this paper to extol the virtues of one or another form of scientific organization; nor is it to verify or disprove this mission’s central narrative, or to argue that consensus is always flawlessly achieved. It is, however, important to note that this collectivist orientation suffuses the team’s actions and interactions on the level of talk, ritual, and organizational narrative.²⁶ Team members tightly adhere to these scripts. Moments when consensus is not reached or when team members attempt to exert status over one another are considered breaches. Such breaches are disciplined as ‘out of line’ in the context of mission interactions, providing evidence of their importance as ritual activities pertaining to continued community membership and maintenance.

The totem robot

Against this organizational backdrop, in which forms of talk and other ritual engagements produce collective alignment, the ordinary work of visualization, gesture, and embodiment takes on new significance as members’ practices that produce and reproduce the mission team’s particular social order. The visual modes, codified gestures and psycho-physical narratives elide Rover teammates’ bodies not only with those of their Rovers but also with each other. In this case, then, I suggest we might best understand these particular visualization practices as not only necessary for producing legible objects on Mars, but also as making the Rover present to the team in the role of team totem.²⁷

Émile Durkheim (1915 [1912]) characterized ritual, embodied actions and tribal totemic relationships as concerned with producing social order.²⁸ Relationships to totem animals, plants, or other protective forces assert and propagate the local culture’s categories and structures, such as social hierarchies or divisions between the sacred and the profane. Central to totemic relations are elaborate rituals such as gathering in effervescent assemblies and ritual dancing. Through imitative rites, members of a tribe or totem group ritually embody the object that brings them together, propagating group cohesion and adherence to social norms. It is worth quoting Durkheim at length on this topic for his uncanny resonance with the case of the Mars Rover gestures:

The totem is their rallying sign; … it is no less natural that they should seek to resemble it in their gestures, their cries, their attitude. … By this means, they mutually show one another that they are all members of the same moral community and they become conscious of the kinship uniting them. The rite does not limit itself to expressing this kinship; it makes or remakes it. … So they make themselves imitate the animal; they cry like it, they jump like it … . All these ways of representation are just so many means of ostensibly showing the end towards which all minds are directed, of telling the thing which they wish to realize, of calling it up and of evoking it. (Durkheim, 1915 [1912]: 358–359)

Approaching Durkheim through the lens of ethnomethodology (Garfinkel, 2002) or symbolic interactionism (Collins, 2004; Goffman, 1961), teammates’ visualization, talk, and gesture become visible as members’ practical actions that establish social facts. Robotic gestures make their ‘rallying sign’ present to them, unite their community in kinship and ‘show the end toward which all minds are directed’. But this does not make the robot into a simple object for mimicry, or a mere symbol of the team’s efforts. Visualization, talk, and gesture all make the extremely distant robot immediately present in members’ interactions, and simultaneously mold a group of individuals into a collective team by establishing a deeply empathetic relationship with their robots. Such activities are part of the work of enforcing and producing radical collectivity that enables the continual production of consensus. They might best be considered Interaction Rituals, as defined by Randall Collins (2004): those interactional forms which produce group solidarity through mutual focus, shared emotional energy, and engagement with symbols that represent the group.²⁹ The daily work of seeing like a Rover, then, pulls individual energies into a collective, intersubjective bodily position (in which effervescent assembly they are, notably, ‘happy’) to produce team identity. Visualization and body work together enact and produce group social form.

Rover death

The entanglement of visualization, gesture, and social order is especially visible in the threat of Rover death. The two Mars Rovers were designed to last for 90 days each: Spirit and Opportunity both outlasted their ‘warranty’ by over 2000 Martian days.³⁰ Far from producing complacency with either the inevitability of death or its consistent evasion, during my fieldwork the health, safety, and potential death of the Rovers was evoked as a constant concern. All team members were enculturated into the understanding that the Rovers were ‘finite resources’, and that their short lives could end at any time. This produced a particular urgency for whatever was being planned immediately. But it was also invoked as a way of bringing team members into alignment with expectations and interactional norms.

For example, in the summer of 2007, two faulty commands were accidentally uploaded to Spirit. The mission ground to a halt immediately: the Project Manager required all team members to take a 4-day holiday over the coming weekend. On the Monday of their return, the first item on the agenda was an ‘All-Hands Meeting’: reportedly the first since the mission began in 2004. Clearly the faulty commands signaled a significant breach in social order, and the All Hands Meeting was an opportunity to restore this order: as the PI put it, ‘to return to first principles.’³¹

First the Project Manager exhorted his team to discuss ‘what went wrong’ with an eye to how to fix things procedurally, behind the scenes, and prevent another problem. Both he and the PI used the opportunity to reinforce the team ideal that all members are equal. The Project Manager in particular invoked a story about quality control in the Japanese auto industry in the 1980s. In his account, the Japanese ascendance in the 1980s over the American car companies was due to the ‘cord at every station’ in the Japanese sites that could ‘stop the entire process,’ compared with the American Fordist division of labor which disempowered individuals down the chain.³² The moral of his story was that, ‘We all share … a responsibility to the health and safety of the Rovers, and we should all be willing and motivated to ask that question, to raise our hand and say I don’t know, I don’t understand that … .’ The PI reiterated, ‘if you see something that looks funny you are empowered just like everyone else to pull the cord, to ask questions … it applies to everybody as part of the process.’ As the managerial component of this otherwise flattened group, such talk resonated strongly with individuals’ senses of the importance of collective responsibility, self-discipline and community management, and reasserted the relationship between those values and the health and safety of the robotic totem.³³

A few minutes into the meeting, an engineer who had helped to build the cameras explained how a ‘return to first principles’ and collectivity required more than willingness to speak up and ‘pull the cord’. Rather, he explained that new recruits were not working with Rover images in quite the way that they should:

[At the beginning of the Mission] there was sort of this culture of curiosity combined with paranoia and everyone was on their game … . As people have been cycled in and out of [the team] … we have new people and I kind of get the feeling that they don’t have the fear [we had]. … It’s more of a video game for a lot of people, it’s kind of cool … it’s sort of abstracted a little bit … . They may not be as connected to the fact that the Rover is only one day away from we’re never going to hear from it again … any thing we could potentially do could end the whole game … .³⁴

According to this engineer, the sense of virtual presence that Rover imagery inspires among members of a younger generation of engineers produces a kind of affective disconnect similar to the activity of playing a video game. In both cases, the visual and bodily apparatus are engaged. But, in the case of the Rovers, he emphasizes not only the consequences of inconsiderate play (loss of the vehicle), but also the connectedness that these visual interactions should inspire between the individuals in the team and the Rovers. The right way to approach these images, he suggests, is as embodied and consequential expressions for the ‘we’ of the team, and not as distanced and abstract pictures of a robot on Mars. The implication that the Rover is always ‘only one day away from we’re never going to hear from it again’, is, for this engineer, imposed through a specific visual orientation. The correct way to see these images, he argues, is to see like a Rover, with all the social relations that orientation implies.

In this moment of breach surrounding the possible ‘death’ of the Rover, the visual, the embodied, the organizational, and the totemic are all simultaneously invoked to produce a ‘correct’ team orientation. Social order is very much in the foreground here, evoked through the mutual entanglement of embodied connection, organizational structure and visual modes. The totemic quality of the robot is enlisted in reinforcing individuals’ positions in a collectivity, wherein members have responsibilities to their teammates as well as to their robots. Repeated emphasis on robotic fragility reminds them that there are not as many Rovers as there are team members; but there is only one single robotic body that team members must collectively inhabit and preserve to experience Mars.

Conclusion

The embodied visual practices of seeing like a Rover are not only a question of individuals’ interpretative activities at their screens. Like any visualization practice, this is an essential part of team dynamics. The visual skill, gestures, and forms of talk associated with visualization and interaction among the Rover team draw a connection between team members’ individual bodies and the Rover’s body, cementing collective social ties between team members on Earth. Imaging that places the observer behind the Rover’s eyes builds empathy and intimacy between team members and their distant robots, just as gesture evokes the robot’s body-in-interaction and makes Mars available to visual interpretation. But these activities also bring team members together in the body of the Rover: a totemic object to which and through which they are all committed, realizing the team’s collective solidarity and complicity through the Rover’s activities.

From the point of view of small groups or individual observers, then, gesture, visualization, and associated forms of talk certainly do make scientific objects visible, legible, accountable, mobile, and actionable, as much scholarship in STS suggests (Alač, 2011; Amann and Knorr-Cetina, 1990; Latour, 1990; Lynch, 1990; Myers, 2008). In addition, visualization and gesture may build affective and embodied relationships between individual humans and robots, whether one-on-one or in groups, and produce local distinctions between robot and human as well (Alač, 2009; Suchman, 2007, 2011). But as much as seeing, interacting and pointing are situated in time and space – they are organizationally situated too. When Mars Rover team members see like a Rover, they not only take on a phenomenological perceptual frame, acquire skilled professional vision, or render Mars legible, but they also assume a role in a team. In some teams, this requires coordinating work with others across disciplinary divides (Goodwin, 1995) or through hierarchical and organizational roles (Hutchins, 1995); in this team, talk and action are oriented towards producing a radically collective orientation. Seeing from the Rover’s position, with all the gesture and talk involved, is a question of ordering subjects and aligning embodied subjectivities, alongside a question of ordering objects in the world.

This point resonates in many ways with other STS scholarship. It reminds us that as both imaged objects and subjects are disciplined in the process of scientific seeing and representing (Daston and Galison, 2007; Lynch, 1985), these subjects are organizationally disciplined too. The Rover case also speaks to how a community adopting a singular view may adopt one that is very clearly situated in time, body, and space, but without approaching the ‘objectivist’ perspective of the ‘God’s eye trick’ (Haraway, 1999: 176). After all, seeing like a Rover is very much a question of seeing from somewhere, not adopting a view from nowhere, but still maintaining the power of a singular perspective.³⁵ Even situated views from somewhere may enroll multiple observers in complex and binding power relations. Seeing like a Rover, therefore, resembles Seeing like a State (Scott, 1998) in its mutual entanglement of ordered visuality, institutional agency, and power relations. But, this perspective can also be produced from the ground up, instead of being imposed as a view from above, as in maps or orbital images. Such interrelations of visuality, power, and organization are all at play in the laboratory, even at the isolated, singular, computer console. We must remain especially attuned to these arrangements in our studies of contemporary scientific work, which is frequently multi-sited, networked, and enmeshed in cyberinfrastructure.

This discussion of seeing like a Rover may help us to understand a particular (and, admittedly, peculiar) team of scientists and engineers, but it also offers a contribution to STS more generally. It is likely that visualization and gesture are not always a question of building a totemic relationship between users and their instruments. But, as I have argued, visualization and gesture are always, also, about the work of ordering. Much STS work has shown how scientific actions and interactions order the world or discipline the individual, yet (like Shapin and Schaffer’s (1985) original formulation of literary technologies) they produce, reflect, and maintain a social order as well. In addition to being enlisted in representation and interaction, embodied visualization practices constitute the activities through which social order is continually performed.

When examining images in scientific practice across settings, then, it is important to bear in mind that the scientists and other individuals crowded around a screen not only possess disciplinary identities, but that they also are members of an organization with its own norms and interactional regimes. Each interacts with an image in a way appropriate to his or her position, consistent with both disciplinary heritage and with the organizational protocols of their laboratory, shop floor, or teleconference line. Image-making and interpretative practices and techniques recreate and reinforce these organizational norms. On the Rover mission, the organizational culture emphasizes the flattening of distinctions; other cases no doubt betray different organizational arrangements and preferences. Thus, investigating visual politics and power is not only a question of examining an image’s ‘principles of inclusion and exclusion’ (Fyfe and Law, 1988: 1), but also of inquiring into the social relations that are part and parcel of an image’s execution and legibility; of examining how those social relations are enacted through the everyday practices of visual work. Ultimately, the visualizations that the Rovers produce represent the team as much as they represent Mars.

Footnotes

Notes

Biographical note

Janet Vertesi is a Link-Costen Postdoctoral Fellow at the Society of Fellows in the Liberal Arts at Princeton University, where she also holds a Lectureship in the Department Sociology. She completed her PhD in 2009 at Cornell University’s Science & Technology Studies program. Her manuscript, Seeing Like a Rover: Images and Interaction on the Mars Exploration Rover Mission, based on over 2 years of ethnography with the Mars Rover team, examines the use of digital images in a variety of contexts across the mission. Vertesi has also published on images in 17th century astronomy, the relationship between the London Underground Map and users’ representations of urban space, and a variety of topics in human–computer interaction.

References

Alač

(2008) Working with brain scans: Digital images and gestural interaction in fMRI laboratory. Social Studies of Science 38(4): 483–508.

Alač

(2009) Moving android: On social robots and body-in-interaction. Social Studies of Science 39(4): 491–528.

Alač

(2011) Handling Digital Brains: A Laboratory Study of Multimodal Semiotic Interaction in the Age of Computers. Cambridge, MA: MIT Press.

Amann

Knorr Cetina

(1990) The fixation of (visual) evidence. In: Lynch

Woolgar

(eds) Representation in Scientific Practice. Cambridge, MA: MIT Press, 85–121.

Bell

Squyres

Herkenhoff

Maki

Arneson

Brown

. (2003) Mars Exploration Rover Athena Panoramic Camera (Pancam) investigation. Journal of Geophysical Research 108 (E12), 8063, doi 10.1029/2003JE00207010.1029/2003JE002070.

Collins

(2004) Interaction Ritual Chains. Princeton, NJ: Princeton University Press.

Clancey

(2009) Becoming a Rover. In: Turkle

(ed) Simulation and Its Discontents. Cambridge, MA: MIT Press, 107–127.

Daston

Galison

(2007) Objectivity. New York: Zone Books.

DiSalvo

Gemperie

(2003) From seduction to fulfillment: The use of anthropomorphic form in design. Proceedings of DPPI 2003 (23–26 June, Pittsburgh, PA, USA). New York: ACM Press, 67–72.

10.

DiSalvo

Gemperie

Forlizzi

Kiesler

(2002) All robots are not created equal: The design and perception of humanoid robot heads. Proceedings of DIS 2002. New York: ACM Press, 321–326.

11.

Doing

(2004) Lab hands and the ‘Scarlet O’: Epistemic politics and (scientific) labor. Social Studies of Science 34(3): 299–353.

12.

Durkheim

(1915 [1912]) The Elementary Forms of the Religious Life (trans. Swain

J.W.

). London: Allen & Unwin.

13.

Fyfe

Law

(1988). On the invisibility of the visual. In: Fyfe

Law

(eds) Picturing Power: Visual Depiction and Social Relations, Sociological Review Monograph 35. London: Routledge, 1–14.

14.

Garfinkel

(2002) Ethnomethodology’s Program: Working out Durkheim’s Aphorism. Lanham, MD: Rowman & Littlefield.

15.

Goffman

(1961) Encounters: Two Studies in the Sociology of Interaction – Fun in Games & Role Distance. Indianapolis: Bobbs-Merrill.

16.

Goffman

(1981) Forms of Talk. Philadelphia: University of Pennsylvania Press.

17.

Goodwin

(1994) Professional vision. American Anthropologist 96(3): 606–633.

18.

Goodwin

(1995) Seeing in depth. Social Studies of Science 25: 237–274.

19.

Haraway

(1999) Situated knowledges: The science question in feminism and the privilege of partial perspective. In: Biagioli

(ed.) The Science Studies Reader. New York: Routledge, 172–188.

20.

Helmreich

(2009) Alien Ocean: Anthropological Voyages in Microbial Seas. Berkeley, CA: University of California Press.

21.

Hess

(1995) Science and Technology in a Multicultural World. New York, NY: Columbia University Press.

22.

Hutchins

(1995) Cognition in the Wild. Cambridge, MA: MIT Press.

23.

Ihde

(2009) Postphenomenology and Technoscience: The Peking University Lectures. Albany, NY: SUNY Press.

24.

Knorr Cetina

(1999) Epistemic Cultures: How the Sciences Make Knowledge. Cambridge, MA: Harvard University Press.

25.

Knorr Cetina

Bruegger

(2002) Global microstructures: The virtual societies of financial markets. American Journal of Sociology 107(4): 905–950.

26.

Latour

(1990) Drawing things together. In: Lynch

Woolgar

(eds) Representation in Scientific Practice. Cambridge, MA: MIT Press, 19–68.

27.

Latour

(1993) We Have Never Been Modern. Cambridge, MA: Harvard University Press.

28.

Lynch

(1985) Discipline and the material form of images: An analysis of scientific visibility. Social Studies of Science 15(1): 37–66.

29.

Lynch

(1990) The externalized retina: Selection and mathematization in the visual documentation of objects in the life sciences. In: Lynch

Woolgar

(eds) Representation in Scientific Practice. Cambridge, MA: MIT Press, 153–186.

30.

Mirmalek

(2008) Solar discrepancies: Mars exploration and the curious problem of interplanetary time. Unpublished doctoral thesis, University of California, San Diego, La Jolla, CA.

31.

Myers

(2008) Molecular embodiments and the body-work of modeling in protein crystallography. Social Studies of Science 38: 163–199.

32.

Ochs

Gonzales

Jacoby

(1996) ‘When I come down I’m in the domain state’: Grammar and graphic representation in the interpretive activity of physicists. In: Ochs

Schegloff

Thompson

(eds) Interaction and Grammar. New York: Cambridge University Press, 328–370.

33.

Prentice

(2005) The anatomy of a surgical simulation: The mutual articulation of bodies in and through the machine. Social Studies of Science 35(6): 837–866.

34.

Prentice

(2007) Drilling surgeons: The social lessons in embodied surgical learning. Science, Technology, & Human Values 32(5): 534–553.

35.

Prentice

(2012) Bodies of Information: An Ethnography of Anatomy and Surgery Education. Chapel Hill, NC: Duke University Press (in press).

36.

Radder

(2006) The World Observed/The World Conceived. Pittsburgh, PA: University of Pittsburgh Press.

37.

Salonius

(forthcoming) Social organization of work in biomedical labs in leading universities in Canada: Socio-historical dynamics and the influence of research funding. Forthcoming in Social Studies of Science.

38.

Schairer

(2006) Diffused embodiment, extended visions: The prosthetics of Martian geology. Presentation at Society for Social Studies of Science Annual Meeting, Vancouver BC (2 November).

39.

Scott

(1998) Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed. New Haven, CT: Yale University Press.

40.

Shapin

Schaffer

(1985) Leviathan and the Air-Pump: Hobbes, Boyle and the Experimental Life. Princeton, NJ: Princeton University Press.

41.

Shrum

Genuth

Chompalov

(2007) Structures of Scientific Collaboration. Cambridge, MA: MIT Press.

42.

Suchman

(2007) Human–Machine Reconfigurations: Plans and Situated Actions, 2nd edn. Cambridge: Cambridge University Press.

43.

Suchman

(2011) Subject objects. Feminist Theory 12(2): 119–145.

44.

Tollinger

Schunn

Vera

(2006) What changes when a large team becomes more expert? Analyses of speedup in the Mars Exploration Rovers science planning process. Proceedings of the 2006 Cognitive Science Society Conference, Vancouver, BC. Available at: www.cogsci.rpi.edu/CSJarchive/Proceedings/2006/docs/p840.pdf (accessed 2 March 2012).

45.

Turner

(2005) Where the counter culture met the new economy: The WELL and the origins of virtual community. Technology and Culture 46: 485–512.

46.

Wales

Bass

Shalin

(2004) Requesting distant robotic action: An ontology of work, naming and action identification for planning on the Mars Exploration Rover mission. Available at: http://ti.arc.nasa.gov/m/pub/888h/0888%20(Wales).pdf (accessed 20 October 2008).