Human perception of intrinsically motivated autonomy in human-robot interaction

1.2. Overview

First, section 2 outlines the background on intrinsic motivation, its computational approaches, and its relation to autonomy, insofar it relates to the present work. Section 3 then introduces time-local predictive information (TiPI), the information theoretic formalism we use to implement intrinsic motivation in our studies. We outline the concrete approximations (and their assumptions) to compute PI. In particular, we highlight how to make this approach suitable for deployment on an actual robot and why it is a good candidate for our research questions.

Section 4 presents the materials and methods of our within-subjects study (N = 24). The study consists of two conditions with the same robot platform: the intrinsically motivated behavior in one condition is generated using predictive information maximization, the behavior in the other condition is a reactive baseline behavior. The focus is on the interplay between the robot and the human participant. We designed a study where the participants interact and observe the robots in order to understand their behavioral differences, but they are not given information about the robot’s task and they cannot order the robot to do something. Instead, the robot and the human participants explore their behavior towards each other. We call this “robotologist” study design (for details, see section 4.3). The study design is motivated by a preliminary study by Scheunemann et al. (2019) which has been conducted and published prior to this work. We outline the learned lessons from this previous study where needed.

Section 5 presents the results, concentrating on our two main hypotheses: one focusing on the perceived Warmth of the intrinsically motivated robot behavior, and the other on the lack of difference in Perceived Intelligence and Competence between the robot scenarios. We found that our study design makes both robot behaviors appear similarly competent. This is important in order to focus solely on the effect on the Warmth dimension, without interfering by the Competence rating. Most importantly, the study provides evidence that the intrinsically motivated robot displays behavior that is perceived as more warm compared to the reactive baseline behavior. Section 6 discusses the implications of those findings, and how they can be applied to other projects. Section 7 summarizes the study and concludes the article.

2.1. Intrinsic motivation

A common definition of intrinsic motivation (IM) in psychology is “doing […] an activity for its inherent satisfactions rather than for some separable consequence” (Ryan & Deci, 2000a). An intrinsically motivated agent is moved to do something, for the enjoyment of the activity itself, for the “fun or [the] challenge entailed rather than because of external products, pressures, or rewards” (Ryan & Deci, 2000a, pg. 56). Since intrinsic motivations have been considered an instrumental ingredient in the development of humans (Oudeyer et al., 2007), there has also been a great interest in developmental robotics to produce formalized models that can be used to imbue robots with drives for competence and knowledge acquisition. Although concepts like “fun” and “challenge” are presented as crucial for the definition of IM in psychology, the literature lacks consensus on what these concepts are (Oudeyer & Kaplan, 2009). This missing consensus and the resulting vagueness of the definition make it impossible to transfer it directly onto a robotic system. Oudeyer & Kaplan (2008) characterize intrinsic motivation in the following, broadly accepted way:

An activity or an experienced situation, be it physical or imaginary, is intrinsically motivating for an autonomous entity if its interest depends primarily on the collation or comparison of information from different stimuli […]. […] the information that is compared has to be understood in an information theoretic perspective […], independently of their meaning. As a consequence, measures which pre-suppose the meaning of stimuli, i.e., the meaning of sensorimotor channels (e.g., the fact that a measure is a measure of energy or temperature or color), do not characterize intrinsically motivating activities or situations.

Nowadays, there is a range of formal models that roughly fall under the header of intrinsic motivation, such as the autotelic principle (Steels, 2004), learning progress (Kaplan & Oudeyer, 2004), empowerment (Klyubin et al., 2005), predictive information (Ay et al., 2008; Der et al., 2008), the free energy principle (Friston, 2010), homeokinesis (Der & Martius, 2012), and others. These models have a range of commonalities: they are free of semantics, task-independent, universal and can be computed from an agent’s subjective perspective. Most of the work related to IMs focuses on how they create reasonable behavior (in some suitable sense) for simulated agents. There has been some work in the domain of computer games that focuses more explicitly on the relationship between intrinsically motivated agents and humans, and how an intrinsic motivation could generate more believable Non-Player Characters (NPCs) (Merrick & Maher, 2009), or produce generic companions (Guckelsberger et al., 2016) or antagonist behavior (Guckelsberger et al., 2018). So far, IMs have been deployed on simulated and physical robots (e.g., Der & Martius, 2012; Martius et al., 2014; Oudeyer et al., 2007), but, as far as we know, there has been no human-robot interaction study yet evaluating the perception of intrinsically motivated robots from the perspective of humans. In this work, we use predictive information maximization to implement an autonomous, intrinsically motivated robot. We describe the formalism in more detail in section 2.3.

2.2. Autonomy

The term autonomy is used with multiple meanings (Boden, 2008). When we talk about autonomous robots, we merely mean robots that are not directly controlled by a human operator, autonomy just being a dimension of the experimental design (Huang et al., 2004; Stubbs et al., 2007). In SDT, however, autonomy refers to being in control of one’s own life, which can be seen as a close enough analogy for living systems (Paolo, 2004). SDT also assumes that there is a drive to maintain this state of autonomy, which we do not see in general with autonomous robots. We might see autonomy used as the idea that a robot should strive to maintain operational autonomy, that is, not be in need of external help, but it usually does not refer to a robot striving to not be controlled by a human. Finally, autonomy might also be referring to the concept of self-making or self-law-giving, which is closely related to autopoesis (Froese & Ziemke, 2009; Maturana & Varela, 1991). In robots, this is currently only a theoretical idea (Smithers, 1997), but it is often considered necessary for true intrinsic motivation. Any heteronomy during the development or creation of an agent would ultimately make them extrinsic and hence undermine their very nature; that is, computational models of intrinsic motivations on robots are usually put on those robots by humans and are thus actually extrinsic. Computational models of intrinsic motivation are an attempt to merely reproduce the behavior or functionality of genuinely intrinsic motivation in organism. This is also the reason that we talk about perceived agency and perceived autonomy. One idea behind this is that by using those models for the robots to pretend to be intrinsically motivated, humans might indeed perceive the robot as thus. In the following, when we talk about intrinsic motivation on the robot we exclusively refer to the initial, technical meaning, the computational model that aims to mimic intrinsic motivation. The more philosophical underpinnings of autonomy are highly relevant to the larger context of this work and indicate that this approach is useful even if we develop robots with more extensive autonomy, making it a robust approach, even for more self-directed robots in the future. Here, the main purpose of this section was to clarify that there are different levels of autonomy—so it is clear that when we talk about intrinsically motivated autonomy, we do not just speak of a robot that can move by itself, but one that can self-directly change its behavior, based on a goal that is at least aligned with its own agency.

2.3. Predictive information

This section describes predictive information (PI) maximization, the intrinsic motivation model used for the robot behavior generation in our experiments. PI has been described as early as 1986, termed effective measure complexity (Grassberger, 1986) or excess entropy (Crutchfield & Young, 1989). Previous work with PI-driven robots in simulation demonstrated its applicability to a large range of different robot morphologies (Der et al., 2008; Martius et al., 2013; Martius et al., 2014; Zahedi et al., 2013). A range of existing videos from experiments in simulation showcase apparent exploratory, playful, and open-ended behavior of individual robots and robot collectives (see Research Network for Self-Organization of Robot Behavior, 2015). The PI-induced behavior in the videos suggests PI as a promising immediate candidate measure to test our core idea.

Conceptually, when this measure is transformed into a behavior-generating rule, the resulting dynamics essentially fall into a family of learning rules related to the reduction of the time prediction error in the perception-action loop of a robot (see especially the book The Playful Machine, Der and Martius, 2012). The aforementioned book also shows how these approaches can be computed from the robot’s perspective alone. Additionally, the variety of different robots and their behaviors presented there shows how different behaviors arise from the same formalism due to the sensitivity towards the agent’s specific embodiment.

The predictive information formalism consists of computing a specific learning rule that aims to maximize the mutual information between a robot’s past and future sensor states (Ay et al., 2008); that is, PI quantifies how much information a history of past sensor states contain about future sensor states. More generally, predictive information is defined as the mutual information between the past and the future of a robot’s sensor input. A high amount of predictive information requires two things: First, past sensor states should make future sensor states more predictable. This should lead the robot to act so that its actions have predictable consequences. Furthermore, the robot also needs to create a high variety of sensor input. If the robot would always perceive the same sensor input, then there is either insufficient information in the past to predict future sensor states, or an insufficiently varied future for which there is not much to predict. In both cases, an impoverished sensor input reduces the predictive information. Alternatively, if there is strong variation in the sensor input but little structure in the sensor data stream; that is, the past has little to do with the future, that also leads to low predictive information. Vice versa, a high value for predictive information requires a high entropy in future sensor states, that is, a richly varied future (a robot motivated to excite its sensors to reach a rich variety of different states) which at the same time depends on the observable past (i.e., which the robot can predict well based on the past). The behavioral regime is created by these two counterpoised requirements: predictability and variety. This yields a robot wanting to act so that its future is highly predictable, while exploring and experiencing new sensor states. The PI literature argues that this balancing act produces rich exploratory behavior that is sensitive to the robot’s embodiment and argues that predictive information is “the most natural complexity measure for time series” (Bialek et al., 2001; Martius et al., 2013).

A robot which acts depending on the maximization of PI only compares its sensors channels on an information theoretic level, without the need of pre-defining any meaning to the sensors. The quantity, therefore, falls in the characterization of IM presented by Oudeyer and Kaplan (2008) (see section 2.1), and it is a candidate measure to enable intrinsically motivated autonomy in a robot.

Der et al. (2008) and Ay et al. (2008, 2012) presented derivation rules for PI, which allows for computing the model directly for linear systems with stationary dynamics. The next section discusses an extension of their work by Martius et al. (2013) for the use in nonlinear and nonstationary systems—such as physical robotic systems.

3.1. Deriving update rules

Martius et al. (2013) present estimates of the time-local predictive information (TiPI) for general stochastic dynamical systems. For systems with Gaussian noise and with gradient ascent on the TiPI landscape, they derive explicit expressions for exploratory dynamics. This subsection introduces the derivation of the explicit expression. The derivations are kept short providing only the basic concepts of the quantity and introduce the underlying main approximations and assumptions that need to be considered when applying the algorithm to a robot in an HRI scenario. For a detailed treatment, the reader should refer to Ay et al. (2008) and Martius et al. (2013).

Assume a robot has n sensors and the sensor readings are polled in constant time steps (Δt = 1). Combine now the result of all sensor values in a vector s ∈ ℝ n . A series of those sensor readings between points of time a and b (with a < b) can be described as a time-discrete process { S t } t = a b , where both boundaries are included. Let the past be defined by the points of time a, …, t − 1 and the future by t, …, b. Bialek et al. (2001) define the PI for some point in time t for the time series S as the mutual information between the past and the future. Intuitively, the mutual information measures the shared information of two random variables, here S_past and S_future; that is, it measures how much knowledge of the past S_past reduces the uncertainty of the future S_future. The predictive information, expressed as mutual information, is thus defined as follows

I ( S future ; S past ) = 〈 ln p ( s future , s past ) p ( s future ) p ( s past ) 〉 = H ( S future ) − H ( S future | S past )

(1)

with the average taken over the joint probability density distribution p(s_past,s_future).

The first essential simplification proposed by Martius et al. (2013) is applying the Markov assumption to equation (1). If { S t } t = a b is a Markov process, all past information relevant to the future is stored in the very last state of the system, that is, S_past = S_t−1.

The predictive information in this case reduces to

I ( S t ; S t − 1 ) = ∑ s t − 1 ∈ S t − 1 ∑ s t ∈ S t p ( s t , s t − 1 ) ln ( p ( s t , s t − 1 ) p ( s t ) p ( s t − 1 ) ) = H ( S t ) − H ( S t | S t − 1 ) .

(2)

Equation (2) is a quantity derived for the whole process. However, to create an actual behavior rule that reacts to the current situation, it necessary to compute a local quantity, specific to the current situation. Therefore, instead of computing the probability distribution p(s _t ) over the whole process, we additionally condition the PI on a state s_t−2. The new quantity derived is then

I ( S t ; S t − 1 | s t − 2 )

(3)

s ^ t − 2 = s t − 2

(4)

s ^ t − 1 = ψ ( s t − 2 , θ t − 2 )

(5)

s ^ t = ψ ( ψ ( s t − 2 , θ t − 2 ) , θ t − 1 )

(6)

s t = ψ ( s t − 1 , θ t − 1 ) + ξ t

(7)

We denote the deviation of the actual dynamics (equation (7)) from the deterministic prediction (equation (6)) as

δ s t ′ = s t ′ − s ^ t ′

(8)

For very small prediction errors, the dynamics of δs (equation (8)) can be linearized as an approximation:

δ s t ′ = L ( s t ′ − 1 ) δ s t ′ − 1 + ξ t ′ + O ( ‖ ξ t ‖ 2 )

(9)

L i j ( s ) = ∂ ψ i ( s , θ ) ∂ s j

(10)

If we further assume that the prediction error ξ is white Gaussian, the entropy can be expressed as covariances (Cover & Thomas, 2012). The resulting explicit expression of TiPI on δS becomes

I ( δ S t ; δ S t − 1 | s t − 2 ) = 1 2 ln | Σ t | − 1 2 ln | D t |

(11)

We now give the algorithm used to drive a robot’s behavior towards increasing TiPI. Martius et al. (2013) derive it explicitly for the gradient ascending neural network presented in equation (6). They argue that the prediction error ξ is essentially noise and does not depend on the parameter of the controller and that therefore the term ln|D| of equation (11) can be omitted when computing the gradient. Based on equation (11), the resulting gradient step executed at each time t is

Δ θ t = ϵ ∂ I ∂ θ = ϵ ∂ ∂ θ ln | Σ t |

(12)

Applying equation (9) to above equations results in explicit gradient step

Δ θ = ϵ 〈 δ u t T ∂ L ( s t − 1 ) ∂ θ δ s t − 1 〉

(13)

δ s t − 1 = s t − 1 − ψ ( s t − 2 , θ t − 2 ) δ s t = s t − ψ ( ψ ( s t − 2 , θ t − 2 ) , θ t − 1 ) δ u = Σ t − 1 δ s t Σ t = 〈 δ s t δ s t T 〉

(14)

Δ θ = ϵ δ u t T ∂ L ( s t − 1 ) ∂ θ δ s t − 1

(15)

As per Der et al. (2008) and Martius et al. (2013), equation (15) is the equation by which the (approximate) TiPI maximization is ultimately implemented. We remark that increasing |Σ| corresponds to an increase of the norm of δs. In other words, this reflects the amplification of small fluctuations in the motor dynamics, that is, an increase of the instability of the system dynamics.

3.2. Approximations and assumptions

Along with the above derivation, several approximations and assumptions have been made. When the measure is applied to a real robot in a real-world human-interaction scenario, this requires a careful consideration of the assumptions and approximations, which we do in the following.

3.3. Applicability for an HRI study

Martius et al. (2013) apply the above maximization of TiPI to simulated robots. As a result, those robots show complex behavior (Martius et al., 2013). One example is a humanoid robot with 17 degrees of freedom (DoF) controlled by a single high-dimensional controller implementing the PI optimization principle from equation (15). Importantly, despite using the same rules, the formalism produces different behavioral regimes of the simulated humanoid, depending on the environment it is exposed to. Its universality for different embodiments and nonstationary settings makes it a good candidate for applying it to a robot without concerning oneself too much with the environment or the robot’s particular embodiment. However, this formalism on its own does not result in high-level behavior, such as walking or serving a human, which may be directly usable in traditional HRI scenarios. It is not a general control algorithm which may generate extrinsic motivation, such as helping a human to get its battery charged. This raises the question of how to align this kind of behavior generation with behavior a designer of, for example, an assistance robot would want? Both issues can be addressed by combining IM-based behavior generation with scripted behavior, or behavior based on extrinsically rewarded reinforcement learning. In the study presented in the next section, we deliberately leave these things out and focus on an empirical study of the effect of intrinsic motivation on its own. In other words, we intend to investigate the effect of intrinsically motivated autonomy in isolation from additional criteria and methods for behavior generation. This evaluation is completely missing from the existing body of work on TiPI or intrinsically motivated autonomy in general. This is the gap this article aims to fill.

4.1. Robot

We use the off-the-shelf spherical robot from the company Sphero, specifically, the BB8 platform (Sphero, Inc., 2020), as depicted in Figure 1. BB8 is a character from the Star Wars movies (Lucasfilm Ltd., 2015). A magnet keeps the head in the driving direction, which gives the user a sense of the robot’s direction. We believe that this helps the human participants to interact with the robot.OPEN IN VIEWER

A humanoid robot platform may raise expectations of advanced social capabilities in participants (Dautenhahn, 2004; Hayashi et al., 2010). For example, humans would expect the robot to have similar eyesight as themselves, or be able to speak and gesture similarly. This in turn may interfere with the investigation of how IM is being perceived. We wanted to reduce as many factors as possible to maximize the focus on the effects induced by TiPI solely, which is why we decided to use a non-humanoid platform with few degrees of freedom.

The robot’s on-board hardware is proprietary which prevented us from flashing it or running our code. However, it is possible to communicate with the robot using Bluetooth Low Energy (BLE). We can, for example, request a stream of sensor information from the robot, or control the robot by either using (i) the robot’s balance controller or (ii) directly setting the speed of each servo.

The balancing controller receives speed and heading as input values. The heading is globally initialized to zero degrees when the robot is started. This means, on sending 20° to the controller, the robot always sets its heading towards 20° on top (clockwise turn) of the initial heading. This controller is a closed-loop controller. If the robot gets nudged or turned, it tries to keep the previously set heading constant. This is a closed-loop controller since the resulting servo speed depends on the readings of the robot’s inertial measurement unit (IMU).

It is also possible to directly set the speed of the left and right servo. This happens in an open-loop fashion, always setting the speed without any further observations.

As for sensors, the robot offers raw sensor information from a 3-axis accelerometer, a 3-axis gyrometer and the actual motor speed of each servo measured as voltage of the back electromotive force (back EMF). The robot can stream data from an IMU represented in quaternions or Euler angles. Additionally, it offers velocity information along a plane in the x and y directions, and also positional data (i.e., odometry) estimated from its starting position. For our studies, we use sensor data from the IMU, the accelerometer, the gyrometer, and the speed of the wheels (see section 4.6).

We built a custom API to control the robot. The API is based on C++ and can run on embedded hardware. It communicates with the robot using BLE to send commands and to read sensor data (see Scheunemann, 2018).

4.2. Measures

To investigate whether human participants perceive an intrinsically motivated robot as a social other, we employed measures from social cognition. There is a long history in the field of social cognition to understand human impression formation of other peers based on two central dimensions (Rosenberg et al., 1968; Wojciszke et al., 1998). Wojciszke et al. (1998) found that two central dimensions explain on average 82.25% of the variance of impression formation. A popular model for human impression formation is the stereotype content model (SCM) introduced by Fiske et al. (2007). According to Fiske et al. (2007), people perceived as warm and competent elicit uniformly positive emotions, are in general more favored, and experience more positive interaction by their peers. The opposite is true for people scoring low on these dimensions, meaning they experience more negative interactions. Warmth and Competence, together, almost entirely account for how people perceive and characterize others (Fiske et al., 2007).

Highly simplified, perceived Warmth leads to positive social bias, referred to as active facilitation (Cuddy et al., 2007). The Competence dimension mostly moderates this effect. High Warmth and high Competence result in admiration, while high Warmth and low Competence result in pity (Judd et al., 2005). The corresponding effects for low Warmth are envy and contempt. As a result, Warmth can be considered the primary factor for predicting the valence of interpersonal judgments (Abele et al., 2016; Fiske et al., 2007). This means, it primarily predicts whether an impression is positive or negative.

Recent research in HRI has shown that human participants prefer to interact again with the robot when they perceive its behavior as more warm compared to another robot behavior (Oliveira et al., 2019; Scheunemann et al., 2020). Our expectation, which would need to be confirmed in future work, is that the behavior of a robot that is perceived as more warm is a good candidate to facilitate long-term interaction.

In our present study, we use the Robotic Social Attributes Scale (RoSAS) designed by Carpinella et al. (2017), which tests for the two aforementioned central dimensions Warmth and Competence. We accompany the scale with the Godpseed scale (Bartneck et al., 2009) because (i) it is a popular scale which may allow to compare the results to other studies and (ii) it encompasses the dimension Perceived Intelligence, which is highly related to the Competence dimension (Carpinella et al., 2017). The latter is important because we designed our study in such a way that participants should perceive one robot as similarly competent as the other robot since the perception of Competence can influence the perception of Warmth (Fiske et al., 2007). However, finding evidence for that, that is, for a small or no effect on the Competence dimension, can be prone to errors. We therefore investigate whether two related dimensions, Perceived Intelligence and Competence, yield similarly small effects. In case they do support each other, it is unlikely that we missed an effect.

The Godspeed scale uses a 5-point semantic differential scale and investigates the dimensions Anthropomorphism, Animacy, Likeability, Perceived Intelligence and Perceived Safety. The RoSAS tests for the dimensions Warmth, Competence and Discomfort. Carpinella et al. (2017) do not recommend a specific size for the Likert-questions, but recommend including a neutral value, for example, by having an uneven number of possible responses. Our questionnaire consists of 7-point Likert-type items.

This article focuses on the evaluation and discussion of the dimensions Warmth, Competence and Perceived Intelligence to answer the research questions (see section 4.5). However, our questionnaires encompass all the dimensions offered by the RoSAS and the Godspeed scale. The idea is (i) to hide the intent of our questionnaire and (ii) to allow future studies to compare their results with our study. We report the effects of all dimensions. However, we only discuss our target dimensions and any other statistically significant effect which was revealed. This allows us to discuss possible implications of unexpected effects (warranting the formulation of hypotheses and conducting a new study).

Note that we continue to capitalize the dimensions to indicate that we refer to, for example, the questionnaire dimension Competence, as opposed to true competence. At times, however, we use the adjectives, for example, competent, where it is clear that we refer to the dimension.

4.3. Robotologist study design

To evaluate the effect of intrinsically motivated behavior generation on human perception of the robot, we had to develop a new study design that could fulfill the following criteria:

1. Encourage the human to pay attention to the robot.

The aim of the study design is to engage the human participant and to direct their attention towards the robot, so that participants can judge the robot behavior based on their interaction with the robot. We also want physical interaction, so we can test that the effect we are looking at is robust in regards to being perturbed by human interaction. This is because it would be counterproductive to identify some robot behavior that could encourage a human to physically interact with the robot and then have this behavior destroyed by the resulting interaction.

In general, there are a lot of task-based study designs that can fulfill items 1 and 2, but the challenge was to find a design that can fulfill items 3 and 4 at the same time. Items 3 and 4 are necessary because our research questions focus on how warm the participants’ perceive the robot, something that can be biased by the perception of the robot’s Competence. Perceived competence in turn tends to be influenced by how well a robot fulfills a stated task—or even just an implicitly assumed task.

For example, in the preliminary study (Scheunemann et al., 2019), we had asked the participants to use their hands to keep the robot from falling off the table, to encourage physical interaction. However, this likely led to an implicit assumption that the robot should stay on the table, and resulted in those robots which were more likely to fall off the table being considered less competent. In this study, we wanted to minimize the explicit and implicit task assignment to evaluate the effect of IM behavior generation on Warmth in the least biased way.

In order to achieve the aforementioned requirements, we designed our study so the participants effectively would learn to become robotologists in analogy to how anthropologists or naturalists would study animals or humans. The participants’ task was to determine if the two presented robot behaviors behave the same or different. We hypothesized that this encourages interaction with the robot because participants usually want to perform well in order to satisfy the perceived needs of the researcher (Orne, 1962). At the same time, however, this task does not influence the participants’ expectations of the robot directly and it does not create the expectation that the robot will help or hinder this assessment. We hypothesized that this robotologist study design encourages the human to interact with the robot, while at the same time reducing their preconceptions.

4.4. Environment and interaction

Figure 2 shows the table that the robot locomoted on. It is circular, with 91 cm in diameter and 72 cm in height. A foam wall of 2.5 cm in height and with 4 cm in width surrounds the border of the table. We decided on these measurements in such a way that the robot cannot fall off the table, even with a very high velocity. Three blankets of a total height of 3–4 mm cover the surface (including the walls). This applied some friction and made it easier for the robot to locomote on the otherwise smooth and slippery surface of the wooden table top. The table’s distance to the surrounding wall of the room was at least 60 cm, which allowed participants to freely move around the table.OPEN IN VIEWER

This study design is different from our preliminary study (Scheunemann et al., 2019). There, we positioned the participant at one particular side of the table which had no border and therefore permitted the robot to fall off the table. We hypothesized that this encourages interaction with the robot and that participants would see that the intrinsically motivated robot adapts to their input. Furthermore, we hypothesized that an intrinsically motivated robot, which explores the area and the interaction with the participant would be perceived as having higher agency and competence. This design was not entirely successful. The intrinsically motivated robot could not sense the edge. Therefore, when it approached participants they did not consider it as approaching them, but judged the robot as rather suicidal. In consequence, the participants judged the robot as less competent, despite its ability to adapt (cf. Scheunemann et al., 2019, sec. 5). This is a good example of why it is critical to consider the participants’ expectations when choosing a robot and designing a study.

In this study, we decided to keep the robot hardware, but change the design of the environment, so that all borders of the interaction environment are enclosed. If the participant decides to be passive, the robot cannot fall off the table. In addition, the round shape of the table and its position allows participants to reach all borders. We do not assign a specific position to the participants. This further reduces our instructions to the participants, aiming to further reduce our influence on their implicit task assignment for the robot. This design of the interaction environment is key to enable the robotologist study design and supports the requirements outlined above in section 4.3.

Figure 2 shows the first author of this article interacting with the robot using the HRI tool referred to as a wand that was developed specifically for this study. Participants were asked to use the wand to touch and nudge the robot with the white end. The wand is 50 cm long and weighs 78 g. It consists of a 40 cm long aluminum tube with a diameter of 10 mm. The end is a round, softer sphere. It is made of an off-the-shelf table tennis ball with a diameter of 40 mm.

The style of interaction is another difference to the preliminary study (Scheunemann et al., 2019) where participants used their hands to interact with the robot. The participants in this study were asked to use the wand for interacting. We assumed that this would help to ease the interaction, as some participants in the preliminary study felt uncomfortable with the idea of using their hands for means of interactions. We further hypothesized that the mere existence of a tool would make the participants want to use it and therefore encourages the interaction.

4.5. Research questions

The study concerned itself with two main research questions. First, we wanted to understand if participants perceive an intrinsically motivated robot as a social other and are interested in interacting with such a robot. As discussed in section 4.2, we employed the dimension Warmth to measure participants’ perception. We are interested in a change of the perception because there is not enough research to understand what absolute value of Warmth would be a good indicator. In this study, we first want to understand whether IM-driven behavior has a positive effect:

R1

Is an intrinsically motivated, adaptive robot behavior perceived as more warm compared to a robot with a reactive baseline behavior?

We hypothesized that we would find evidence that this is the case. We did not expect any other strong effects, but we still report and discuss the statistically significant main effects of all other dimensions.

In the preliminary study (Scheunemann et al., 2019), we saw evidence that participants perceive an intrinsically motivated robot as more warm. This, however, needed confirmation because (i) the effect was not statistically significant and because (ii) the study design made participants perceive one of the robots as more competent than the other. This can act as a confounding factor because perceived Competence can influence perceived Warmth. In this study, we aimed to produce a fairer set-up to allow an evaluation of Warmth that is not biased by perceived differences of the robot’s competence. The above subsections presented what we call the participants’ task to become a robotologist and the environment and the interaction we employed in order to achieve that goal.

Drawing a conclusion from not finding an effect for Competence is prone to the risk that we simply might have missed an effect. To be certain, we therefore used the dimension Perceived Intelligence, a dimension related to Competence (see section 4.2), and expected both of them to show no effect. Our second question is therefore:

R2

Does the robotologist study design help to make the two robot behaviors appear to the human participants as similarly competent and intelligent?

We hypothesized that the answer to the question is “yes.” This means that for the dimensions Competence and Perceived Intelligence we would expect to see no evidence for an effect.

4.6. Conditions

This experiment consists of two conditions of behavior generation with the following characteristics:

ADA

The robot is adapting continuously, based on maximization of TiPI and directly applies the resulting IMs as servo speed.

REA

The robot uses its balanced mode for locomotion, the network controlling the robot has been pre-adapted using PI and it remains constant.

The reactive robot in the REA condition uses the same binary and starts with the same networks as the robots in the preliminary study (cf. Scheunemann et al., 2019, sec. 3.1). The weights are received based on pre-trial adaptation. This determines how it reacts to sensor input, but it does not further update its internal network during the experiment. There are two reasons for taking the REA robot from our preliminary study (Scheunemann et al., 2019). Firstly, the behavior is a good baseline behavior. The robot was interesting to the participants and the behavior showed enough variety for them to not see any patterns (Scheunemann et al., 2019). Secondly, keeping the baseline constant, but changing other variables, allows for a better comparison to the previous findings and the previous intrinsically motivated, adaptive robot.

The intrinsically motivated robot in the ADA condition realizes behavior motivated by TiPI maximization, and it continuously updates its internal networks based on that gradient during the experiment. In contrast to the preliminary study, the robot changed the speed of its two servos directly, instead of using the balancing controller (cf. Scheunemann et al., 2019, sec. 3.3). This way the robot’s behavior is only influenced by its IMs, unconstrained by additional software such as the closed-loop balancing controller. In particular, this balancing controller might have added meaning to the robots output (i.e., staying upright), which may not yield an intrinsically motivated robot in accordance to the definition presented in section 2.1. Therefore, this change allowed to further focus the analysis on the perception of intrinsically motivated autonomy.

The robot sensor input is again the linear acceleration for the forward/backward and left/right axis from the accelerometer, and the angular velocity around the upright axis received by the gyrometer. Instead of using the absolute position of the robot received via its pitch and roll angles from the IMU as in the preliminary study (Scheunemann et al., 2019), we now input the speed of the two servos. This allows us to directly couple the output of the controller changing the servo speed and the actual measured servo speed.

We wanted both robots to behave similarly at the beginning of the two conditions, to avoid the formation of very different first impressions. For example, one robot starting off smooth and slow, and the other accelerating very fast and bumping into the wall may form a first impression in the participant which influences their overall responses. Therefore, we tweaked the starting weights of the network of the intrinsically motivated robot by hand. As there is direct coupling between the servo speed readings and the controller output, that is, the set speed for the servos, the weights were set in such a way that a reading on the left servo would amplify the output for the left servo, and vice versa. This way we could create a slow-pace forward movement for the first few seconds, which looks similar to the reactive baseline robot. An example video of the two conditions of one session accompanies the article (see Scheunemann, 2021b). As seen in the video, the resulting two robot behaviors look overall very similar and are hard to distinguish visually. We believe this is a strength of the experimental set-up, as it allows us to rule out other, incidental reasons for the observed change in human perception. This makes it more likely that it is the adaptation to stimuli received by the robot, driven by TiPI maximization, that is responsible for the differences in human perception.

4.7. Procedure

Participants are welcomed to the experimental room and were then handed an information sheet. They were encouraged to discuss concerns related to their participation. If they were happy to proceed with the study, they were asked to sign an informed consent form. Then the environment and the robot are presented and briefly described. It was then emphasized that they could leave the study whenever they feel uncomfortable, stressed, or bored. Participants then complete a pre-questionnaire. This gathers information regarding their gender, age, and background.

We then formulated the task for the participants, namely that they should find out whether the two presented robots are any different. For understanding differences in the robots’ behaviors, they can use the HRI tool: the wand. They are allowed to nudge the robot or block it. Both of these actions are demonstrated to the participants. However, no other information is provided.

Next, the two conditions are presented to the participants in a randomized but counterbalanced order, each lasting approximately 5 min. They complete a post-questionnaire containing the two scales after each condition. The entire experiment takes 50–60 min per participant.

4.8. Participants

We recruited 24 participants (10 female; 14 male) mostly from university staff and students, between the ages of 18 and 64 years (M = 31.7, SD = 12.6). The participants were undergraduates or post-graduates from the university, but all naïve towards the objectives of the experiment. Eight participants had a background in HRI, whereas nine participants never participated in any prior HRI study. All were asked how familiar they are with interacting with robots, programming robots and the chosen robot platform. 5-point Likert-questions were chosen with the value 1 for “not familiar” to 5 for “very familiar.” The self-assessed experience for interacting with robots showed an average of 3.5 (Mode = 5). The average familiarity with programming robots 3.2 (Mode = 5) and the experience with the chosen robot platform was rated an average 2.1 (Mode = 1). The familiarity with the movie series Star Wars was rated an average of 3.2 (Mode = 4).

These data were collected primarily to understand what types of participants were attracted to the study. We did not form any hypotheses, but we expected that many participants may be aware of the Star Wars movies or the characters and their impression of the robot’s behavior may vary depending on their expectations. However, in this study we are interested whether participants perceive one robot behavior as more warm than the other to understand their perception of intrinsically motivated autonomy (cf. section 4.5). This is why we decided for a within-subject design and presented both conditions to each participant. This way, we can investigate the participants’ change in their responses, rather than comparing responses of participants with different expectations of the robot’s behavior and capabilities. This allows us to concentrate on the changes of their responses without considering their expectations of the robot’s behavior.

The study was conducted on the premises of the University of Hertfordshire and was ethically approved by the Health, Science, Engineering & Technology ECDA with protocol number aCOM/PGR/UH/03018(3). The anonymity and confidentiality of participants’ data are guaranteed.

4.9. Data analysis

Our research questions asked whether the responses to a questionnaire dimension of a participant differ between conditions. In particular, we want to know whether participants perceived one robot behavior as more warm than the other (RQ1). Since we are interested in the change of participants’ perception, we designed the study so that the robot behavior generation is a within-subjects variable; that is, all its conditions are presented to each participant. This has the benefit that we can answer the research questions with the use of statistical tests for pairwise comparisons of the participants’ responses.

Since our study consists of exactly two conditions, we decided to employ two-sample location test of matched pairs: the Wilcoxon signed-rank test. In contrast to the paired t-test, it is a non-parametric test which is both: (i) robust for small sample sizes and (ii) usable for the assessed Likert-scale data. We decided to use the two-sided version of the test to investigate for effects in both directions. This way, effects contrary to our hypotheses are revealed too. We use wilcox.test(ADA, REA, paired=TRUE) to compute the test, a method that is part of R’s built-in stats package.

4.10. Data preparation

In order to use the Wilcoxon signed-rank test, we first had to analyze the data from participants’ responses in two steps: (i) we analyzed the scale reliability in order to amend scale items if needed and (ii) we tested whether the condition responses are independent of their presented order.

4.10.1. Scale reliability

To prepare the data for analysis, we tested the score reliability of the scales of both standardized questionnaires using Cronbach’s α. Table 1 presents all test results for the used dimensions. We found that the item quiescent–surprised of the dimension Perceived Safety of the Godspeed scale is negatively loaded on the dimension. Even if reversed, the reliability is poor with α = .54. We decided to remove Perceived Safety altogether. All other dimensions reveal a good score reliability ranging from .75 to .85, or acceptable reliability for the dimension Anthropomorphism: α = .67. This is evidence that the scale dimensions are reliable and can be used for further investigation.

OPEN IN VIEWER

Table 1.

Internal consistency reliability measured with Cronbach’s α.

	Dimension	Items	α
RoSAS	Warmth	6	.80
	Competence	6	.85
	Discomfort	6	.79
Godspeed	Anthropomorphism	5	.67
	Animacy	6	.74
	Likeability	5	.82
	Perceived intelligence	5	.84
	Perceived safety	3	.37

4.10.2. Interaction effects

We then analyzed whether the randomized and counterbalanced assignment of the order of conditions to the participants was successful in that the data does not show interaction effects between the condition responses and the order of the condition.

An analysis of variances (ANOVA) is commonly used for investigating interaction effects. Following our above argumentation for employing non-parametric tests, we use a non-parametric ANOVA-type test (Noguchi et al., 2012). For computing the ANOVA-type test, we used the R package nparLD. The study can be expressed as F1-LD-F1 Model with the one within-subjects variable behavior generation (levels: REA, ADA) and the one between-subjects variable order (two levels). The nparLD package offers the function f1.ld.f1() for computing such models. Table 2 shows the results. For a 5% significance level, there is no statistical significance and there is not enough evidence for an interaction effect for any of the dimensions.

OPEN IN VIEWER

Table 2.

ANOVA-type test results for interaction effects.

	Dimension	F	df1	P
RoSAS	Warmth	0.001	1	.976
	Competence	1.473	1	.225
	Discomfort	1.787	1	.181
Godspeed	Anthropomorphism	0.164	1	.685
	Animacy	0.665	1	.415
	Likeability	1.455	1	.228
	Perceived intelligence	<0.001	1	.984

The randomized but counterbalanced ordering of conditions resulted in no evidence for interaction effects. Therefore, we can safely investigate the main effects independently of their order. This means we can use the Wilcoxon signed-rank test and compare the responses to both conditions independently of whether the participants were exposed to, for example, ADA, at the beginning of the experiment or at the end.