What makes a robot ‘social’?

The HRI context

Human-robot interactions have been investigated systematically since at least the 1990s. By the mid-2000s, this research track had increased in size and scope to become recognized as a specialist field in its own right (Goodrich and Schultz, 2007). It encompasses the whole of robotics, since interactions between humans and robots underpin any use of robots. Even autonomous service and industrial robots are used by humans directly or indirectly. Social robots feature minimally in Goodrich and Schultz’s agenda-setting review – mentioned in a single sentence as a subset of assistive robotic systems: ‘wheelchairs, mobile robots with manipulators, animal-like robots, and humanoids’ (p. 214). The First Annual Conference on HRI took place in March 2006 in Salt Lake City, USA. Successive HRI conferences show a progressive widening of what is perceived to be within the remit of HRI. By 2010, the Fifth HRI conference in Osaka, Japan was themed ‘Grand Technical and Social Challenges’ and centred on addressing issues in anticipation of a future in which ‘[r]obots may become our co-workers in factories and offices, or maids in our homes. They may become our friends’ (HRI 2010, n.d.). The organizers of the Eighth Annual Conference, which took place in Tokyo in March 2013, invited papers under themes of distinctively sociological flavours alongside the staple diet of engineering-focused themes. As they put it: ‘Robotic solutions are increasingly applied to real world problems. … These societal problems require a holistic approach to the design and development of robots that meet human needs, address technical challenges, and foster acceptance in everyday settings’ (HRI 2013, n.d.). However, engineers’ recognition of social and societal issues often serves as an invitation for technological ‘fixes’ (cf. Šabanović, 2010).

Three years after the inaugural HRI conference, Diocaretz and van den Herik (2009) reflected that ‘what so far did not enter the agenda and curricula is the personal and intimate relational dimension between a human and a robot’ (p. 205). They give the example of assistive robots in individualized healthcare: since a robot-patient relationship requires face-to-face interaction, it is important to know what psychophysical and neurological mechanisms are activated in the human body, so as to create a perception and experience of presence. In addition, the personal relationship involves ‘inherent trust in order to build love and friendship, including emotional attachment’ (p. 206). Whilst they underline the epistemological necessity of knowing what ‘activates’ affective reactions in human bodies, the reference to inherent trust and to building love and friendship sets in motion dynamics of discourse whereby the loved and trusted robot is extricated from the material domain and is installed in the social. The talked-about robot ceases to belong to the category of machines we trust to function reliably and the category of objects we love, personal things to which we are sentimentally attached; instead, it is talked about as we ordinarily talk about someone we love and trust, and who may love and trust us in return. The robot as an object formed in discourse is thus imagined into a dialogical space.

Goodrich and Schultz (2007: 231) articulate a concept of dynamic interaction as the HRI field’s conceptual cornerstone – a concept that ‘places the emphasis on shaping the types of interactions that can and will emerge as humans and robots interact’, and that they pitch in opposition to the traditional conception of pure teleoperation or supervisory control (static interaction). Their notion of a dynamic interaction directs attention to interactional patterns that evolve in a self-regulatory way: the operator becomes more skilful and the robotic system, too, learns and adapts to its user. The actions of both operator and machine flow together in synergy to effect and perfect the performance of the task. Goodrich and Schultz (2007) define interaction as ‘the process of working together to accomplish a goal’ (p. 217). But ‘working together’ in this context does not mean a co-construction of meaning and a sharing of goals, as human co-workers might do. Goodrich and Schultz highlight cases of humans and robots performing tasks that are external to both (rescuing survivors, exploring a terrain, firing on military targets, etc.). The dynamic interaction is prerequisite to improving the efficient performance of the task. In social robotics, however, the focus is on interactions that are themselves the task being performed. The assistive or companion robot does something for the human with whom it interacts. This is closer to the relationships between service providers and their clients, tutors and their students or pets and their owners, than to the relationships between machine operators and the machines they operate. In this context, a reference to working together imaginatively places these robots in social spaces populated with interlocutors and tacitly positions the robot as an interlocutor, again implying dialogicality.

The pragmatic challenge in engineering is to design robots that have at least some human social characteristics and can engage people in interactions that ‘feel’ natural. A common strategy is to identify what humans typically do in face-to-face interactions in order to define what robots should be able to do: e.g., recognize the presence of humans by means of vision, touch and sound, perceive and express emotions, and participate in conversations (Li et al., 2011). The rationale that extant scientific knowledge of the mechanisms of interpersonal interaction in humans can be harnessed towards designing machines that achieve similar effects cuts across biologically inspired and functional design approaches. The former seek to create robots that internally mimic social cognitive capacities; for instance, apropos robots, Cog and Kismet: ‘We want to build … a robot capable of learning how to imitate facial expressions from simple imitative games played with a human, using biologically inspired mechanisms’ (Breazeal et al., 2005: 31). In contrast, designers opting for the functional approach ‘do not necessarily need to understand how the mind really works’, since the objective is to create robots that outwardly give the impression of being intentional agents (Fong et al., 2003: 148). As Duffy (2006) put it, ‘[i]f the fake is good enough, we can effectively perceive that they [robots] do have intentionality, consciousness and free-will’ (p. 34). His statement implies the emergence of a theme pertaining to ways in which robots could be inserted into an ‘I-you’ system. The theme becomes explicit in his collaboration with a sociologist (Zawieska and Duffy, 2014). Defining the self in accordance with symbolic interactionism, their narrative presents the ontological clarification of the self as epistemologically a necessary step before tackling the technical problem of how to put a self in a machine. A decade earlier, Fong et al. (2003) identified a gamut of issues to do with creating a good-enough fake, requiring design decisions about physical appearance, personality as expressed in behavioural traits and how to give the robot the skills necessary for ensuring optimal engagement of human interaction partners. Most of these remain central in robotics at the time of my writing; in addition, themes of robot personhood – machines with ‘selves’ and entitlement to human-like rights (see Coeckelbergh, 2010) – have also entered this field.

Shifts of emphasis intersect conflicting ideologies within HRI. Van den Herik et al. (2011: 107) describe two camps at loggerheads, hotly debating how to steer the technology. A society-driven camp takes ‘Safety, Security, and Supervision’ as its measures and ‘opines that the world is driven and run by social aspects. The society (of human beings) dictates the governance.’ In contrast, a technology-driven camp measures progress by means of ‘Interaction, Intelligence, and Imagination’, maintaining that ‘the world is driven and run by technological developments, and that robots are here for further enhancements and new applications. It means no less than that technology dictates the governance.’ Van den Herik et al. (2011: 109) predict that, since technological acceleration is imminent, ‘we will cross the limits of human understanding [into] the area of robot understanding’. In his roadmap for roboethics, Veruggio (2007) forecasts that within the present century ‘humanity will coexist with the first alien intelligence we have ever come into contact with – robots’ (p. 5). Thus, alongside (and interwoven with) the pragmatic concerns of R&D, marketing, applied ethics and legislation, there unfolds a technology-driven metanarrative of a future in which human lives merge with smart technology at every level of human existence (e.g. Saadatian et al., 2013; van den Herik et al., 2011). The semiotic robot that comes into being through technology-driven discourse implicates qualitatively different moral questions – as well as challenges for conceptualizing human-robot relationships – that take the field beyond engineering.

Seeking to align social robotics with sociology, Meister (2014) equates it with a specialism concerned chiefly or solely with humanoid robot companions, hence different from both ‘service robotics’ and HRI. Arguably, these categorical boundaries are not only difficult to maintain but also are obscure in the engineering literature, within which the designation ‘social’ is applied to a wide range of robots, including biologically inspired designs modelled on insects (Fong et al., 2003). Dautenhahn (2007: 684) identifies a variety of terms applied to robots – socially evocative, socially situated, sociable, socially intelligent and socially interactive – with correspondingly diverse concepts. For instance, whereas socially evocative robots are defined by the responses they elicit in humans, socially interactive robots possess ‘skills to interact and communicate, guided by an appropriate robot control and/or cognitive architecture’ (p. 685). A robot can be socially interactive by means of teleoperation, but to make it socially intelligent requires inbuilt architecture emulating the primate brain. In general, the machines that are most commonly talked about as ‘social’ are definable as physically embodied intelligent systems that enter social spaces in community and domestic settings (Jones, 2016). This definition excludes disembodied AIs or bots, such as automated response systems and search engines. Specifying ‘social spaces’ avoids merely listing settings in which robots could be placed (hospitals, schools, shops, etc.) and prompts taking people’s experiences of the robot as the salient criterion. For instance, a robot nurse might enter patients’ social spaces, whereas a robot surgeon or a robot janitor in the same hospital won’t. Strictly speaking, the above definition excludes robots designed as labour-saving appliances; but researchers investigating the introduction of robot vacuum cleaners into American households found that participants came to regard the robot as a social entity – ascribing lifelike qualities to it and giving it a name, gender and personality – and concluded that even a robot vacuum cleaner can mediate and enhance interpersonal relationships (Sung et al., 2010). People often anthropomorphize their cars, computers and other objects. Nonetheless, it seems a peculiarity of talking about robots that the machine leaps out of the material environment and into the social.

Pinpointing robot sociality

Among several open questions in robotics that Fong et al. (2003) identify is ‘What are the minimal criteria for a robot to be social?’ (p. 160) It remains open in 2017. There is a gradient, or are degrees, of sociality, as well as qualitatively different ways of approaching its conceptualization. Kahn et al. (2006b) differentiate ontological and psychological criteria: The ‘strong’ ontological claim is that future humanoids will actually become human, and the ‘weak’ claim is that humanoids will remain fully artificial despite appearing to be human. Psychological claims focus on what people attribute to fully humanlike humanoids. The ‘strong’ psychological claim is that people will regard the humanoid as human. The ‘weak’ claim is that people will continue to regard it as a machine. Cutting across the kinds of claims identified by Kahn and his co-authors, there are phenomenological criteria, to do with the extent to which people experience the interaction with a robot as an interaction with a sentient being.

An early definition of social robots, originally formulated in 1999 by Dautenhahn, highlights a functional similarity with human and animal individuals within a group:

Social robots are embodied agents … able to recognize each other and engage in social interactions, they possess histories (perceive and interpret the world in terms of their own experience), and they explicitly communicate with and learn from each other. (In Fong et al., 2003: 144)

This posits sociality as a trait or a set of competences that certain organisms possess, and that can be built into machines. Since the mid-1990s, Dautenhahn has drawn upon the social intelligence hypothesis in biology, which posits that complex cognition and enlarged ‘executive brains’ evolved in humans and other animals in response to challenges associated with social complexity. She contends that a robot’s social skills ought to form an important part of its ‘cognitive skills and the degree to which it exhibits intelligence’, thus challenging a prevalent notion that social skills are merely a necessary ‘add-on’ for improving its appeal to users (Dautenhahn, 2007: 683). This nevertheless posits sociality as biologically hardwired in species that evolved that way, and therefore as something that can be potentially reverse-engineered. Whether or not her recommendation is technologically feasible, and whether the social intelligence hypothesis is tenable in biology, it seems indisputable that our capacities for sociality have an evolutionary origin and biological basis (evident, for example, in mirror neurons). But having a capacity for engaging in social interactions does not mean that everything we do is social. There remains the issue of defining sociality.

During the past decade, the widening applications of robots in everyday settings have brought issues concerning long-term human–robot relationships to the fore – an exigency evident in a spate of publications that could be viewed as a nascent relational turn (Jones, 2013) or what Šabanović and Chang (2016) term a ‘social turn’ in robotics. It is expressed in research interest in users’ subjective experiences, social and cultural factors affecting people’s perceptions and societal concerns about robots. A subtle ‘paradigm shift’ can be detected along with these widening interests (Jones, 2013, 2016). Since the mid-2000s, some roboticists have begun to assert that a robot becomes social by virtue of how people regard it. For instance, ‘A social robot is a robot plus a social interface. A social interface is a metaphor which includes all social attributes by which an observer judges the robot as a social interaction partner’ (Hegel et al., 2009: 174). Redefining the robot’s social character as contingent on people’s judgments signals a shift away from construing sociality as a trait that enables individuals to interact in a social manner, and towards construing it as an emergent property of the interaction itself. Consequently, the question about the minimal criteria for a robot to be social raises a question about the minimal conditions for a social interaction.

Could a robot be classed as a social agent if it elicits a ‘social’ reaction in people? Recalling her visit to Rodney Brooks’ lab in MIT, Turkle (2011: 84) describes her response to the robot Cog as ‘visceral. Cog had a face, it made eye contact, and it followed my movements. … I had to fight my instinct to react to “him” as a person.’ Her point is that humans are hardwired to respond to social cues, and robots can be designed so as to create a powerful illusion. However, this illusion is not the same as being involved in an actual interaction.

Could an interactive robot in teleoperation scenarios be classed as social? Having placed an android replica of roboticist Hiroshi Ishiguro in an Austrian café and documented visitors’ interactions with it, Straub (2016) found that minimal signs of reactiveness (e.g. turning its head towards approaching people) spurred visitors to behave differently than when the android was ‘idling’. When it was teleoperated so as to be fully interactive, visitors ‘appeared to ascribe a distinct identity to the robot itself and forget about the teleoperating person as the source of the interactive performances’ (p. 565).

It might seem self-defeating to investigate dynamics of social interactions involving humans and robots when the robot is in fact another human. What is known as ‘Wizard of Oz technique’ is defensible in the context of HRI: observing how people respond to robots that seemingly display behaviours more sophisticated than what current technology allows could help robot designers to identify directions for building machines that may eventually achieve that degree of sophistication. But if the social scientific aim is to analyse how the specific technological object has entered the given social space, using the Wizard of Oz technique raises the question of exactly what we are seeing.

One way to look at these cases is in terms of simultaneous simulation and dissimulation. ‘To dissimulate is to pretend not to have what one has. To simulate is to feign to have what one doesn’t have’ (Baudrillard, 1994: 3). Applied to stage magic, dissimulation means that effects are achieved by an apparatus that prevents spectators from knowing the methods and mechanisms producing the effect. Smith (2015: 322) points to an overlap of the concept with Suchman’s (2007) reference to the ‘erasure of human labours’ around technological performances and, citing Alač et al. (2011), draws a comparison with ‘recent productions of computerized life forms, especially those of social robotics’. In both stage magic and social robotics, the experiential effect is produced by ‘the combination of simulation and dissimulation: creating an effect known by all to be contrived, while simultaneously erasing signs of its contrivance in machinery and method’ (p. 336). Yet, whilst positing the semiotic as inseparable from the materiality of artefacts and apparatus, Suchman (2007: 238) draws attention also to rhetorical (my emphasis) ‘modes of erasure of human labours and nonhuman alignments’, whereby an artefact is brought into existence as an autonomous agent. Alač et al. (2011: 918), reflecting that they ‘have often heard remarks that compare social roboticists with magicians’ – remarks insinuating that the robot’s performance is controlled by the roboticists – sum up their own paper as showing instead that ‘the flow of the interaction … is a process of participation and co-construction’, since, in their study, children’s spontaneous actions played a significant role in establishing the social character of robots installed in the classroom. This argument may parallel some variant of psychological claims, but likely will fail to persuade proponents of ontological claims (cf. Kahn et al., 2006b).

Could a robot be classed as social only if it autonomously participates in ‘meaningful’ exchanges with people? Arguments for and against insisting on this condition often cite the Eliza Effect, referring to evidence that people interacted with the computer programme ELIZA as if it were a person (e.g. Wolfe, 1991; Zhao, 2006). The question of whether an AI could ever be accepted as another human is often articulated in terms of whether the AI will pass the Turing Test and deceive people about its artificial nature. This raises philosophical issues concerning the extent to which the artificial system would require bodily engagement with the world in order to acquire the kind of interactional expertise (ways of speaking) that would allow it to pass as human (e.g. the debate between Dreyfus and Collins; see Selinger et al., 2007). The possibility of deception may be ruled out when people actually see the robot, but there remains the question of whether being fully autonomous would make the robot truly social. Dominey and Warneken (2011) sought to simulate how humans coordinate actions with each other in a study that dispensed with a humanoid appearance (the visible part of their ‘Cooperator’ system was a utilitarian robot arm with a two-finger gripper). They extrapolated a theory of shared intentionality championed by Tomasello, according to which our species has evolved a unique capacity to share goals and intentions, and our minds learn to process social information through forming ‘dialogic cognitive representations’ (Moll and Tomasello, 2007). The Cooperator engaged with a human partner in a game-like activity that involved taking turns in placing moveable pieces of a wooden puzzle in response to the partner’s spoken commands. Since the robot performed the task as the human did, Dominey and Warneken have presented the study as proof that the hypothesized mental process can be translated into algorithms and be put in a machine. Not everyone is impressed. Thompson et al. (2013) contend that Dominey and Warneken (along with Tomasello) have misunderstood the concept of shared intentionality: they seek a causal explanation (what makes it possible to cooperate) whereas philosophers who talk about shared intentionality seek to explain what makes an act social.

Suffice it to quote G. H. Mead’s (1934) definition of the social act: ‘a dynamic whole … something going on – no part of which can be considered or understood by itself – a complex organic process implied by each individual stimulus and response in it’ (p. 7). HRI studies often analyse human-robot dyads as if in a social vacuum, but increasingly there is also attention to how persons’ judgments of robots are influenced by others’ interactions with the robots. Lab experiments indicate that people’s appraisal of an android is likely to be influenced by seeing another person making eye contact with it (Shimada et al., 2011). Since robots are becoming commercially available, HRI researchers too are leaving the lab – or take the lab to the ‘wild’ – and opportunities arise to observe the dynamics of multiparty interactions. Studies investigating how Paro (a baby seal robot developed by AIST in Japan) was used in elderly care homes found that the presence of the robot encouraged residents to communicate with each other, on which basis the researchers surmised that the robot’s presence strengthened the residents’ interpersonal ties (Wada and Shibata, 2006, 2009; see also Šabanović and Chang, 2016). However, when observational data are quantified so as to demonstrate behavioural probabilities, the social act in Mead’s sense vanishes from sight.

In contrast with the above, Alač’s ‘bodies in interaction’ approach echoes Mead’s holistic definition of the social act. Apropos my title question, she provides the compelling answer that ‘any conception of the robot as an interlocutor must take into account the dynamics of interaction’, and further asserts:

[T]he robot’s social character extends beyond its physical body, to include multimodal interaction within everyday routines. The robot’s social character thus includes its positioning in the space and the arrangement of other actors around it, as well as its interlocutors’ talk, prosody, gestures, visual orientation, and facial expressions. (Alač et al., 2011: 894)

This is diametrically opposed to the premise underpinning an assertion made elsewhere by her co-authors: ‘An electronic circuit with a few switches, resistors and capacitors can be used to provide an index of what appears to be an abstract concept, the ongoing quality of social interaction’ (Tanaka and Movellan, 2006: 1). To Mead, the ongoing quality of the social act is irreducible to people’s reciprocal behaviours. It is the difference between looking at someone and making eye contact, or between an actual conversation – an observable event that can be recorded and analysed – and the dialogical space that opens up between people in conversation, which is unobservable and unanalysable. This quintessential phenomenological quality of a social act, its dialogicality, cannot be translated into algorithms and built into a few switches, etc., that may enable a robot to take turns in a conversation.

We may query the extent to which roboticists’ objectives are served by the insinuation that a robot’s social character does not reside in its design. Meister (2014: 129) points to tensions between the position articulated in Alač et al. (2011) and what engineers are trained to do, tensions that render Alač’s claim counterintuitive in the context of HRI. My position is more sympathetic to Alač’s oeuvre. Meister calls for developing robot architecture on grounds that, in Alač’s (2016) terms, ‘to be deemed social, a robot needs to possess mental configurations such as frames and scripts so that it can, for example, identify intentions of others and execute (mentally anticipated) actions’ (p. 531). Alač argues (and I concur) that the ‘internalist’ proposal makes it ‘difficult, if not impossible, to see participants’ actions as sensible’. Her own analyses of classroom interactions vividly demonstrate how a robot lacking the frames and scripts called for by Meister nonetheless engages children and their teachers in meaningful interactions. To Alač, this evidences the robot’s agential and social character. To me, those interactions reveal human dialogicality, including the capacity to co-construct make-believe with a robot – even when the robot is switched off, as will be seen in the next section.

Dialogical action in a give-and-take game with RUBI

RUBI was a three-foot tall, plump robot clad in bright yellow cloth, with a head, two arms and a touch screen – a design that proved to be pleasing and non-threatening to young children (Movellan et al., 2007). It was developed for a longitudinal study carried out by the University of California San Diego’s Machine Perception Laboratory in the mid-2000s. Along with Sony’s QRIO, RUBI was placed in the university’s Early Childhood Education Center for six months. Its teleoperated performances were achieved by means of an elaborate apparatus, involving hardware, software and human resources, spread across at least two rooms. Publications and conference papers generated from the project indicate that it has yielded a wealth of information relevant for robot design. Alač et al. (2011) present Alač’s argument through detailed analyses of two scenes and reflections on additional observations. I have selected an episode in which RUBI was deactivated, since the dramatic goings-on can illustrate not only the contingency of a robot’s social character on its assimilation into spatially organized human activities (Alač’s point) but also my point that such assimilation implicates a dialogical dimension that challenges a construal of robots as social agents.

The episode opens with the teacher and the principal investigator (PI) sitting on the floor facing each other and engaging two female toddlers who are standing next to them. RUBI, switched off, seems to have been forgotten behind the PI’s back. The teacher tries to get the girls interested in plastic toast and pizza. An older infant, two-year-old Greg, enters the room. Silently approaching the group, he observes their activity and glances at RUBI. Reaching the teacher, he tries to take the toy pizza she is holding. The teacher asks him whether he’d like to have it. Greg removes his hand and looks again at RUBI. The teacher gives the pizza to the PI. Greg grabs it from the PI and, ignoring the teacher’s prompt to thank the PI, moves towards RUBI. The PI leans over, touches RUBI’s hand and waves it. Greg offers the pizza to RUBI, and the PI says ‘Thank you!’ in a high-pitched voice. For the rest of the session, the PI ‘continues to “lend” his body to the machine’ while Greg and the two other toddlers place toys in the robot’s hand (Alač et al., 2011: 906).

The dialogical action unfolding in this scene does not mean simply taking turns in a dialogue. I use the italicized term in Taylor’s (1995: 64) sense, denoting the constitution of the self-knowing subject, the ‘I’, as ‘an articulate identity defined by its position in the space of dialogical action’. A two-year-old has yet to grow into this space of dialogical action. Adults scaffold the child’s inculcation into it. From the moment that Greg tries to take the toy from the teacher, she attempts to make him take ownership of the intentional states she imputes into his behaviour: ‘Did you wanted [sic] it, Greg? Greg wanted it? Yeah?’ (Alač et al., 2011: 902). He says nothing, but nonetheless asserts his autonomy by removing his hand from the toy (resisting her imposition). The teacher then attempts to draw him into a give-and-take game (pp. 902–903; transcription style altered):

When Greg does that, the teacher is the first one to say ‘Thank you’. The PI then chimes in with the high-pitched ‘Thank you’. The adults thus collude in bringing the toddler into their game of saying thank-you. Greg is not completely silent, but his vocalizations do not amount to intelligible speech. Without hearing the prosody and seeing the body language (was he smiling?) it is difficult to tell what exactly was going on there. Based on the transcript alone, the scene could be read as a power play. Greg seizes an object in which the adults appear interested (taking their power). He seems interested in the robot. The PI quickly follows his lead, becoming a puppet master – performing affiliation (playing together) and simultaneously retaining power (he controls the object of Greg’s interest). The teacher retains her position of power by continuing to articulate Greg’s intentions on his behalf (‘You wanna give it …’). RUBI may be the centre of attention and activity but it has no say whatsoever in the dialogical dynamics of this social act. The robot remains a voiceless thing.

Alač asserts: ‘This act of turning an object into an agent is not a metaphoric process but an achievement that involves the materiality of the robot’s body-in-interaction…. Through the co-participation of the group’s members, the robot talks, while its body moves’ (Alač et al., 2011: 905). It could be argued that the transformation was entirely ‘metaphoric’. RUBI talks and moves only in make-believe. Even when activated, the ghost in the machine is a human operator. In the focal episode, the group’s co-participation in the pretence did not make RUBI a social being – someone with a voice (a point of view) – any more than the toy Greg had offered it became real pizza through the gesture. Nevertheless, Alač challenges us to consider:

Who is talking? Who is waving the hand? Is the agent the PI (who physically moves the robot’s hand, and talks in a high-pitched voice), or is it Greg (upon whose attention orientation and movement through the space the PI built his actions), or the robot? (p. 905)

The scene in focus reveals ‘a deep-seated tension’ between the give-and-take activity which ‘attributes the agency to single individuals,’ and the extent to which the ‘participants’ gestures, talk, and actions configure them as multiparty, situated achievements’ (pp. 905–906). Arguably, the deep-seated tension exists only in academia. It is manifested in perennial controversies and in scholars’ construal of a ‘problem’ with folk psychology (ordinary folk’s tendency to attribute actions to the actors’ mental states). Outside certain intellectual enclaves, extending agency to a robot still goes against the grain – to echo Wittgenstein’s (1953: 178) reflection apropos language games: ‘“I believe that he is suffering.” – Do I also believe that he isn’t an automaton? It would go against the grain to use the word in both connexions.’ In the language game played out through the text in focus, it does not go against the grain to say that ‘the robot’s attainment of social agency is grounded in the specificities of the preschool routine’ and to observe that when the PI moved RUBI’s head and arm to receive the toy from Greg, the robot ‘functions as an interlocutor in the historically shaped interactional dynamics of which it is part’ (Alač et al., 2011: 906; my italics). The added emphasis highlights rhetorical moves that disrupt the ordinary language game. The word ‘interlocutor’ means a speaker; and yet RUBI does not speak. Adults playing with toddlers frequently act as mouthpieces for dumb toys, as the PI did with RUBI on this occasion, but the voice (literally and figuratively) is the adult’s.

To speak of a robot’s attainment of social agency parallels the standpoint attributed to Latour, according to which ‘our interaction with technological artefacts is … a reciprocal exchange between actors’ (Khong, 2003: 695). A common critique of ANT is that it ‘flattens and squeezes out any psychological complexity from its notion of agency’ (Ivakhiv, 2002: 392). Similarly, the endorsement of human-nonhuman symmetry and positing social agency as an emergent property of a ‘hybridization of human flesh, plastic, and wires’ (Alač et al., 2011: 905) runs the risk of throwing away – like the proverbial baby and bathwater – the scope for theorizing the psychological significance of interacting with technological artefacts. Such theorizing need not commit us to ‘black box’ mentalism. As I see it, the question of what makes either a human or a robot social (an agent) is inextricably interwoven with the question of what makes an act social (since agency is a property of social acts). I tend to follow Mead. Yet, as Taylor (1995: 64) contended, ‘the very impoverished behaviourist ontology’ that Mead allowed himself as a scientist fails to capture the constitution of the self-knowing subject, the ‘I’ whose identity is defined by its positioning in the space of dialogical action. Alač’s sophisticated analysis of semiotic engagements with the robot nonetheless reproduces an ontology that fails to capture dialogical action. The social act I see unfolding in the provided transcript involves people in dialogical entanglements that are played out around and through a robotic prop. Where Alač et al. (2011: 905) see the robot turning ‘from a nonfunctioning object into an actor in the give-and-take activity’, I see the robot turning into an instrument of power for the infant – who ‘engages the two adults in a series of local moves’ and ‘by situating his silent action in the interactional work of the group, manages to redirect the group’s attention toward the robot’ (p. 905) – and for the adults, who maintain their control of the situation by steering the activity.

What kind of agent is RUBI?

We may start by eliminating what kind of agent RUBI definitely wasn’t. It was unlike the androids conjured in Ramey’s (2005) address to roboticists: ‘A human and android in a normal social relationship must equally regard each other as capable of disclosing a new world through each other’s perspective, … for each is capable of acting as selves’ (p. 143). His passionate plea for conceptualizing future androids as embedded in a person-person relation is at bottom deontological, to do with propriety of conduct. Ramey’s thesis pivots on a categorical separation of the ‘I-you’ and ‘I-it’ – subject-subject and subject-object respectively. His rationale for wanting to include androids in the subject-subject category differs from Latour’s argument against the subject-object dichotomy, though comparable with what Latour (1999: 202) describes as a literal crossover, ‘the swapping of properties’, from human to nonhuman. Ramey is not merely saying that competences associated with human agency are potentially transferrable to androids. He takes it for granted that future technology will deliver those machines – and this inevitability raises the ethical issues he points out. In contrast, Latour draws a picture of present-day sociotechnical reality in which nonhumans are ‘folded’ into the social world by virtue of regulating our everyday conduct. He gives the example of the speed bump, the so-called ‘sleeping policeman’ (Latour, 1999: 186). This nonhuman plays an active role in enforcing a speed injunction: ‘we are no longer justified in viewing the speed bump as mere object, inert and incapable of action’, since its constitution of concrete, paint and gravel contributes to ‘the meaning of “law enforcer”’ (Khong, 2003: 695). Elaborating upon the example, Latour (1999) notes that ‘the shift is “actorial”: the “sleeping policeman” … is not a policeman, does not resemble one in the least’ (p. 188). In Ramey’s thesis, the shift is in the opposite direction: androids resemble humans, and therefore ought to be recognized as artificial persons. The difference between these discourses thus lies in whether they prompt us to judge a robot as social on the basis of how closely it simulates a human actor or prompt us to consider its ‘actorial’ efficacy.

An agent in this sense is something that has an effect. RUBI had an effect. The robot was a unique fixture in the preschool classroom. It ‘invited’ interactions that would not have taken place otherwise, thereby organizing the goings-on in definite ways. Its effects on the people interacting with it could be described without anthropomorphic projections, for instance, in terms of Kurt Lewin’s concept of Aufforderungscharakter (roughly translatable as ‘invitation qualities’, though his translators have rendered it ‘valences’). ‘Objects are not neutral to the child, but have an immediate psychological effect on its behaviour. … These imperative environmental facts – we shall call them valences – determine the direction of the behaviour’ (Lewin, 1935: 77). A Lewinian construal might depict Greg, the PI and the teacher as operating upon their environment (which includes each other) as if they are separate from it. This view is quite different from the Latourian one in which ‘operating’ only takes place in networks of actors that cannot be separated.

The subject-object dichotomy that Latour attempts to disrupt could be regarded as a legacy of construing the person as ‘a bounded, unique, more or less integrated motivational and cognitive universe’, a concept that is peculiar in the world’s cultures (Geertz, 1974: 31). The peculiar concept posits the psychological interior as prerequisite for agency, since human agency presupposes intentionality, forethought, self-regulation and self-reflectiveness (Bandura, 2006). With particular reference to ANT, Sayes (2014) identifies several ways of construing nonhuman agents, none of which implies intentional states in the artefact. A step up from regarding artefacts as merely instrumental is to understand them as mediators. Understood thus, nonhumans are not actors in the ‘proper sense of the word’ but are seen as necessarily ‘adding something to a chain of interactions’ by ‘continually modifying relations between actors,’ and therefore cannot be treated simply as substitutes for human actors (p. 137). Nonhumans ‘are both changed by their circulation and change the collective through their circulation. They act and, as a result, demand new modes of action from other actors’ (p. 138). Goodrich and Schultz’s (2007) aforementioned concept of dynamic interaction in HRI can be placed here.

The role of nonhumans as mediators comes into sharper focus when we consider the synergy between human behaviour and the machines humans operate. For example, cars demand new modes of action from people who wish to drive them. Although Greg offers the pizza to the robot, not to a peer or an adult (he takes it away from them), he does not do anything that cannot be done with another person or indeed a traditional doll. It is difficult to attribute Greg’s action to the fact that RUBI was a technologically sophisticated artefact. To be sure, children would respond to a socially interactive robot differently than to an inanimate object. A study comparing how 2½-6 year-olds interacted with the robot dog AIBO and a stuffed toy dog reports similar affectionate behaviour towards both objects but significant differences that reflected reactions to AIBO’s reactiveness (Kahn et al., 2006a). With AIBO, the children were more often apprehensive (e.g. startled when it moved) and attempted reciprocity with it; and they mistreated the stuffed toy more often. By age four, most children make clear distinctions between prototypical living and non-living kinds, and tend to designate robots to the inanimate group; yet research investigating children’s attributions of aliveness to robot pets with which they actually interact suggests the emergence of a new ontological category that disrupts current animate/inanimate distinctions (Kahn et al., 2006a, 2011, 2013; Severson and Carlson, 2010). AIBO was discontinued in 2006, but robot companions for children are increasingly ‘pushed’ by the industry, targeting busy parents who can afford the gadgets. For instance, in autumn 2016 Avatar Mind introduced humanoid iPal as ‘a great companion robot for kids. With its cute cartoon outlook, fine craftwork, latest natural language understanding technology, and cloud apps, it will be your child’s best friend’ (Avatar Mind, 2016). Whether children growing up with robots might suffer psychological damage (Sharkey and Sharkey, 2010) or benignly acquire a new ontology, a robot companion would not become a ‘significant other’ for the child unless the robot’s voice (position of consciousness) becomes differentiated in the child’s experience from those of other persons with whom the child regularly interacts, thus entering processes of the child’s dialogical self (cf. Hermans and Hermans-Konopka, 2010). Present-day robotic products could affect child development in subtle ways. For instance, a litterbin that runs after children urging them to pick up their litter (Yamaji et al., 2011) sounds like a good idea, but enticing children to comply with a robot is not the same as teaching children to obey their parents. Such gadgets potentially reorganize power relations within the family, and thus the space of dialogical action into which children are entered as ‘selves’.

In Sayes’s (2014) classification, the next step up from nonhumans as mediators posits nonhumans as members of moral and political associations. He cites examples given by Latour: an annoying bleeping noise sounds when a driver’s seatbelt is not on, or the ignition does not work unless the seatbelt is worn (which eliminates the possibility of dissent). The bleep and ignition control constitute ‘a new type of moral or political actor’ whereby ‘the very fabric of our moral and political associations’ has changed (p. 139). RUBI’s membership in a moral or political association is less direct. Instead of changing the fabric of morality and politics within the classroom by making the children behave in a desirable manner, this robot appears to be on standby for the teachers’ agenda. As seen, the teacher seized the opportunity to foster etiquette through the give-and-take game. Tanaka et al. (2007) report that early on some children cried when robot QRIO fell. The researchers advised the teachers to tell the children not to worry, since the robot couldn’t be damaged, but the teachers had another lesson in mind. Ignoring the roboticists’ advice, they ‘taught the children to be careful; otherwise children could learn that it is acceptable to push each other down’ (p. 17956). The teachers treated the robots as instrumental, an aid to socializing. Tanaka subsequently explored this potential in his concept of a ‘care-receiving robot’ (Tanaka and Kimura, 2010). Further testing the concept, Tanaka and Matsuzoe (2012) describe how 3-year-old Japanese children in an English class spontaneously acted as a teacher to NAO (Aldebaran Robotics) when the teleoperated robot, placed there as a classmate, appeared to get vocabulary wrong. Tanaka attributed the robot’s care-eliciting affordances to its design (a small humanoid). However, the caring was spontaneously demonstrated by some of the children some of the time.

Emotional attachment to robots (as to any nonhuman) need not reflect the artefact’s capacity to reciprocate. The case of US soldiers holding funerals for their broken drones, machines that had helped to keep them alive (Carpenter, 2016), demonstrates ‘bonding’ with robots that are not designed for social interaction. As one blogger commented apropos funerals for AIBO in Japan, ‘robot funerals reflect our humanity’ (Hsu, 2015). In such scenarios, robots seem to hold affiliative powers in terms of ‘the ways in which objects are not innocent but fraught with significance for the relations that they materialize’ (Suchman, 2005: 380). Put another way, the robot has affiliative affordances for humans who have feelings towards it.

Judging a robot by its affordances

The human body that functions as the fulcrum of sociological analysis is produced as a ‘whole’ object at the intersection of multiple semiotic systems: the body-without-organs, which ‘presents its smooth, slippery, opaque, taut surface as a barrier’ (Deleuze and Guattari, 1984: 9). An added twist in the present context is that the bodies in interaction may include entities that are literally without organs. The humanoid robot is discursively produced in parallel to the human body as an AI-without-algorithms insofar as technical details of its computational platform are tangential for describing people’s behaviour, attitudes and semiotic designations vis-à-vis the robot. This framing of the subject matter can be contrasted with the body-with-organs (albeit often reduced to a brain) and its analogous robot-with-algorithms in cognitive science and robotics. MacDorman (2007: 143) articulates the scientific problem thus: ‘How can human bodies – and perhaps robot bodies – construct themselves into persons by attuning to patterns and norms in their social environment?’ The cognitive-scientific problematic inverts the sociological problematic – namely, how do patterns and norms in the social environment construct human bodies into persons – and its extension into STS, in which context the ‘how’ question invites descriptions of how artefacts alter social spaces.

James Gibson’s ecological approach to sensory perception concerns organismic bodies in interactions with their material environment. During the 1960s and 1970s, Gibson eschewed mentalism and articulated his theory of information pickup in opposition to the information processing models that dominated cognitive science of the day. His concept of affordances (a word coined by him) modifies Lewin’s aforementioned concept of valences. Whereas valences can be positive or negative, and change according to the perceiver’s momentary state, affordances are action opportunities and constraints that inhere in invariant characteristics of an animal and its environment, and are therefore neutral (Gibson, 1979). A critique emerged in the late 1980s among pro-Gibson psychologists who problematized issues that Gibson hadn’t, such as cultural relativism and the role of language (e.g. Costall and Still, 1989; Heft, 1989). Through its original and subsequent formulations, the concept of affordances epitomizes a principle of reciprocity that some scholars (e.g. Lombardo, 1987; Wells, 2002) regard as Gibson’s key insight. The reciprocity of affordances is not reciprocal causality. Two interacting entities can have causal effects on each other (turning the ignition key causes a car to start) but that is not the same as constitutive complementarity; e.g. the distinguishable yet mutually supportive realities of car mechanics and driver behaviour. Such a potential complementarity exists even for able-bodied people who have not learned to drive (thus invariant). After Gibson, we may posit a driver-car hybrid and investigate how this symbiotic entity negotiates a path of safe travel (Dant, 2004; Gibson and Crooks, 1938) – a view that could be taken as concurring with Latour’s human-nonhuman symmetry.

There have been some attempts to integrate Gibson’s ecological psychology with ANT or post-ANT frameworks (Ivakhiv, 2002; Lorimer, 2007), but the difference between the two frameworks is germane here. Each framework has a blind spot with respect to something that the other framework brings to the fore. Imagine a road that is closed off by a barrier. Following Latour, we may comment that the barrier enforces access restrictions. A ‘no entry’ sign serves the same purpose with respect to drivers, though less effectively, and it won’t stop pedestrians or animals from walking on. A natural obstacle, such as a fallen tree, does not have the same sociological significance, but it has the same psychophysical significance as does the barrier. Following Gibson, we observe that large sighted animals pick up the impassability of the barrier and tree alike. ‘It is a mistake to separate the natural from the artificial as if there were two environments; artefacts have to be manufactured from natural substances’ (Gibson, 1979: 130). Today, Gibson’s ideas inform also robotics (see reviews in Chemero and Turvey, 2007; Horton et al., 2012). Clearly, an autonomous robot – e.g. a self-driving car – approaching a road barrier or a fallen tree should be able to pick up the obstacle’s impassability. The affordances of a robot for people are a different matter.

Gibson understood the social realm in a profoundly biological sense. To individuals, other animals afford complex interactions, such as ‘sexual, predatory, nurturing, fighting, playing, cooperating, and communicating’; and ‘what other persons afford, comprises the whole realm of social significance for human beings’ (Gibson, 1979: 128). To Gibson, the significance of the social realm is evident in the fact that people pay the closest attention to optic and acoustic information that discloses what others afford them. For a robot to enter this realm of social significance would mean more than people paying close attention to sensory information about what it affords them. They will care whether the robot likes or dislikes them, feel they are being rude if they don’t thank it, be offended if it is rude to them, and so forth. In short, the robot would be assimilated into a dialogical space.

A construal of ‘social’ that is closer to social scientific uses of the word enters the post-Gibsonian critique in its attention to culturally specific affordances, such as the eating-with affordance of a fork (Heft, 1989) and chopsticks (Jones, 1999). Costall (2012: 90) has articulated a concept of canonical affordances: ‘the conventional, normative meanings of things, notably in relation to human artefacts’. He proposes to view artefacts’ canonical affordances as part of wider constellations of artefacts, settings and the practices that brings them together. His concept of constellation seems to me convergent with the premise of Latour’s theory. Latour gets a footnote mention in Costall (1995: 478) but Costall does not make the leap that Latour does in attributing agency to objects and settings. Drawing upon Vygotsky and Leont’ev, Costall avers that ‘we experience objects in relation to the community within which they have meaning’ (p. 472). His argument roughly parallels Alač’s argument apropos the semiotic aspect of humans’ interactions with robots. She highlights the teachers’ semiotic training of the children; e.g., before Greg enters, the teacher instructs the toddlers on what the toys represent: ‘Look it’s a pizza it looks like a pizza but it’s not a real pizza’ (Alač et al., 2011: 905). As coined in Jones (1999), indexical affordances pertain to the denotative properties of utterances, gestures and images, a concept that can apply also to artefacts. We may say here that in Alač’s example the teacher instructs the children about the toy’s indexical affordance – the fact that it can be used to represent a pizza although it lacks the material affordances of edibility. RUBI facilitated the playful activity through its indexical affordance as a humanoid. As seen, the teacher also trains Greg in making ‘indexical’ commitments to his own intentions by articulating what she assumes he wants to do and inviting him to confirm, thus bringing the child into consciousness of his self. The teacher’s interpretations connote mental states in the child. Participants in the give-and-take game are positioned as a giver/taker, polite/impolite – actions which connote individual agency within human symbolic interactions (see Jones, 1999, on symbolic affordances). Such reasoning clearly moves away from the materiality of artefacts and their invariant affordances for human bodies.

The word ‘affordances’ has become common parlance and detached from its Gibsonian origin. Suchman (2004) speaks of the ‘indexical affordances of machine “speech”, and … the ways that subjects and objects together can perform interesting new effects’, without reference to the Gibsonian theory. Her use of the phrase also differs from Jones’s (1999) post-Gibsonian concept of indexical affordances, although both denote the open-endedness of language as structuring conversational interactions. In the cited extract, Suchman comments on designing automated systems for open-ended conversations with people. As a step towards remodelling the artificial interlocutor, she urges ‘a re-imagining of humans as contingently divisible participants in sociomaterial collectives, who live out their particular histories in uniquely inflected ways’ as opposed to the entrenched ‘image of the monadic, rational actor’ (Suchman, 2004). Contemporary psychology has long jettisoned the contested image and replaced it with models similar to the reimagining called for by Suchman, such as Hermans’s dialogical self theory and other frameworks articulated since the 1980s. Nonetheless, it could be argued that they do not attend enough to people’s situated embeddedness in sociomaterial collectives. The converse could be said in the present context: the dialogical tends to drop out of sight when the scholarly gaze is fixed on sociomaterial collectives.

Whereas Gibson’s affordances and Costall’s canonical affordances alike refer to potentialities that are present in persons’ environments, the affordances to which I want to draw attention are autopoietic, self-creating and bring into existence the dialogical space that we inhabit as selves. The give-and-take with RUBI indirectly provides a vivid illustration of how human actions can have affordances in their own right. Alač et al. (2011) describe how the PI orients bodily to what the toddler is attending to: ‘As he follows the direction of Greg’s looking, the PI extends his upper body to the left, touches the robot’s hand, and waves it’ (p. 906). In classic Gibsonian terms, RUBI has a ‘make-me-wave’ affordance by virtue of having a moveable hand. This affordance exists for any able-bodied human, whether or not individuals realize it. With Costall, we may observe that RUBI has a make-believe affordance akin to the canonical affordances of dolls or puppets. Canonical affordances exist as potentialities within cultural scripts. A further ‘layer’ of affordances becomes apparent in the observation that the PI’s ‘gesture builds upon Greg’s orientation to create a referent of joint attention’ (Alač et al., 2011: 906). The PI’s action of waving RUBI’s hand – itself a response to Greg’s action of looking at RUBI – fixes the robot as a referent of joint attention and creates a new affordance for action: ‘Greg can now perform the action deemed desirable when the robot “works properly”: he can give the toy to the robot’ (p. 906). In turn, the toddler’s act invites the PI’s ‘Thank you!’ While RUBI and the pizza are indispensable for the make-believe, it is the participants’ actions – not the artefacts – that create affordances for dialogicality. Within this autopoietic chain of affordances, a robot can be semiotically designated as having a social character.

Conclusion

The epigraph at the beginning of this article quotes a statement that Lacan made in a 1954 seminar, in which he urged his audience to read La Mettrie’s L’Homme Machine. Reading both Lacan and La Mettrie, De Vos (2011: 71) attributes to Lacan the realization that La Mettrie was ‘one of the first to understand that with the emergence of science we also see the emergence of the symbolic, mathematized body’ with zero subjectivity. This symbolic body epitomizes a paradox already present in La Mettrie’s thesis. There is inevitably subjectivity in imagining oneself as a being without subjectivity. The subjectivity paradox acquires a new twist in the discourse of AI and robotics, which endorses the technical feasibility of artificial minds. We don’t have to imagine ourselves as being machines devoid of subjectivity. We can imagine machines with subjective states. Locating social agency in the ‘hybridization of human flesh, plastic, and wires’ (Alač et al., 2011: 905) performs a further twist that squeezes out the residual ghost. Hybridization is a process that produces a hybrid. Hence, instead of imagining ourselves like machines devoid of subjectivity or alternatively imagining machines endowed with subjectivity, we are cued to imagine ourselves as existing symbiotically within a multi-bodied cyborg hybrid. And yet behind this posthuman parlance there is inevitably someone, a person with a point of view, who engages in dialogues that seek to eradicate the Cartesian ghost. The subjectivity paradox prevails.

Dialogism hardly resolves the intractable mind-body problem; but making a case for considering it apropos social robots may nudge the problem space in this context towards a re-conceptualization of robot sociality. Arguably, a robot will become truly social only if and when it autonomously and inescapably partakes in dialogical action. Irrespective of whether this criterion can serve as an achievable benchmark for near-future robotics, its absence in technology-oriented discourses is significant. It reveals something about the intellectual and imaginal landscapes within which particular expectations about robots unfold.

Social robotics could be construed as a technology that strives to convert qualities of interpersonal relationships (companionship, caregiving) into commodities that can be stored and then supplied on demand. This statement loosely paraphrases Heidegger’s description of modern technology as storing energy, although the analogy is limited. Suffice it to cite his example of a plane on the runway: we see the object but its purpose is not apparent; ‘it conceals itself as to what and how it is … it stands on the taxi strip only as standing-reserve’ (Heidegger, 1993: 322). Complex machinery, which is not visible to onlookers, prepares the plane to take off. He talks of ‘the way in which the actual reveals itself as standing-reserve’ (p. 329). Alač et al. (2011) show that even an inanimate humanoid, a deactivated RUBI, can reveal itself as standing-reserve for sociality if and when people include it in their social activities. However, whereas the plane reveals its purpose by mechanically performing its function regardless of any observer, RUBI invited Greg’s action through its humanoid appearance and the history of its placement in the classroom. When ‘Greg hands the toy to the robot, bodily-scaled his action is lodged in a series of prior encounters with the robot and the RUBI team’ (p. 907). Given his previous experiences, the child might well have expected RUBI to become animated as if of its own accord, though he wasn’t put off by the fact that it was visibly manipulated by the PI. In this instance, the technology-in-action seems to have mattered less than did the human action that the artefact elicited. In instances of interaction with autonomous robots, the artefact’s capacity for responsiveness and the kind of responses of which it is capable clearly do matter. The appearance of such robots in everyday settings challenges the sociological imagination to consider not only whether robotising social spaces induces new ontologies, but also to consider whether scholarly descriptions of these emerging sociomaterial collectives ‘forget’ fundamental aspects of human personhood.