User-centered Social Interaction Design in Intelligent Personal Assistants and Social Robots

IPA and Social Robot Social Interactions

IPAs, such as Amazon’s Alexa, Google’s Assistant, and Apple’s Siri, and social robots, such as Embodied Inc.’s Moxie, Amazon’s Astro, and Miko, utilize voice-based social interactions as their primary means of communication between the user and the technology. Due to the centrality of social interactions in their use, each user’s perception of the social attributes exhibited by the devices is a key driver of the artificial social relationship the user develops with the IPA or social robot. For example, Pitardi and Marriott (2021) demonstrated that two unique drivers of the trust developed between users and IPAs were social presence and social cognition. In the context of IPAs and social robots, social presence is the degree to which a technology can make a user feel that they are interacting with a social agent (Van Doorn et al., 2017). Similarly, perceived social cognition, derived from human-human interaction theories of social cognition (Fiske & Macrae, 2012), reflects the mental model a user has about an IPA’s or social robot’s social processing capabilities. These two constructs, along with other important drivers of an individual’s perception of social “others,” reflect the importance for developers to carefully design the social interaction of their devices and virtual agents.

User-centered Design of Social Interactions

A primary tenet in the design of computer-based systems is the concept of user-centered design. User-centered design extols an approach to design in which products are made to cater to the needs and desires of the user (Chandran et al., 2020; Salinas et al., 2020). In general, developers will test their prototypes with a representative sample of users to establish a consumer model of their product (Kujala & Kauppinen, 2004; Salinas et al., 2020). This approach assumes the sample who tested the product is a good representation of the final user base of the product, implying both good sampling practices and a strong understanding of who the final user base will be. While product testers generally make a good faith effort to meet these requirements, samples may fail to meet the appropriate levels of user diversity (Kujala & Kauppinen, 2004).

A second assumption is that catering to the average user will provide the best outcomes and a user-centered baseline. However, as with normal distributions in statistics, the mean is only representative of the population if the sample has adequate distribution, and the sample and population do not have excessive variability. In terms of user-centered design, the users may be part of a multi-modal distribution where there are multiple user preferences which may not match the mean (most common) user preference.

User-centered Design of Social Interactions: The Case of Personality

One approach developers take to imbue social cognition into their IPAs or social robots is by creating the appearance of a personality (Andriella et al., 2021; Lambert et al., 2020; Lopatovska, 2020). In human-human interaction, personality is a critical factor for relationship formation (Izard, 1963). When examining relationship formation using the Big-Five Factor personality model (John & Srivastava, 1999), agreeableness, introversion/extroversion, neuroticism, and openness are all associated with relationship formation outcomes (Harris & Vazire, 2016). Given the centrality of social interaction in their design, apparent personality is critical in social robots and IPAs. Users both perceive a personality from these devices and have preferences as to what they want that personality to be (Andriella et al., 2021; Joosse et al., 2013; Robert, 2018). When designing social interactions with social robots, developers may try to ground their device’s social personality on what is assumed to be the baseline personality users will most like. For example, Amazon’s intent for Alexa’s personality is that it is perceived as approachable (i.e., welcoming, fun, imaginative, respectful), efficient (i.e., concise, patient, helpful), trustworthy (i.e., reliable, honest, factual, safe), responsive (i.e., cooperative and attentive), and natural (i.e., authentic, polite, relevant; Amazon, 2024).

While a user-prefered personality helps cater to the preferences of most users, personality preferences are person, relationship stage, and task dependent (Aly & Tapus, 2016; Duck & Craig, 1978; Robert, 2018). In other words, while it is important for social interaction and personality designers to consider the most commonly preferred traits, it is equally important for designers to find ways to adapt their IPA and social robot personalities to those of individual user preferences.

A Framework for True User-centering

Catering to a representative target user’s preferences and needs should be a first step toward making social technology user-centered. Given today’s capabilities with machine learning and advanced algorithms, social technology should use the representative target user as a baseline and when possible, adapt its features, such as personality, to the preferences and needs of individual users. Such social interaction will be more similar to the varying social nature of humans and can provide more unique IPA and social robot user experiences.

Figure 1 above shows a simplified framework for catering to individual users based on user-research data. Step 1 involved collecting sample data based on commonly accepted user research practices, but it should be emphasized that qualitative research is key for this approach to work adequately as it provides the most unique perspectives on user needs and preferences (Schirr, 2013; Weiss, 2004). Step 2 relates to using the collected data to establish the target user baseline, which the product will employ as its social interaction “out-of-the-box” approach. Finally, step 3 adds machine learning, algorithm, or artificial intelligence elements that help adapt this baseline to meet the unique preferences and needs of the target user.

OPEN IN VIEWER

Figure 1.

Framework for true user-centered social interactions.

The Present Study

Personality is an important context in which IPA or social robot developers would benefit from matching the social capabilities of the agent to the individual (and not target) users. Personality varies widely among users (John & Srivastava, 1999) and these personality variations drive important changes in the way users interact with technology (Ghafurian & Dautenhahn, 2023; Robert, 2018). The capability to self-adapt user interfaces (UI) to meet the varying user personalities has been explored extensively. Previous research has looked at ways to adapt UIs to different personalities of users (e.g., Capuano et al., 2015), adjust responses to user personalities (e.g., Ahmad et al., 2022), and adjust personality to match the users (e.g., Yorita et al., 2019). This is important, as differences in user personalities are tied to differences in user preferences in IPA and social robot personalities (Montalvo, 2024; Robert, 2018).

The present study utilizes qualitative research to examine how user personality social robot preferences vary from the target user baseline approach and how the framework presented above might be used to meet the needs and preferences of these target users.

Participants

Study participants included 250 undergraduate students (59.2% female, M_age = 19.33, SD_age = 3.34), as well as 100 older adults from the surrounding community (64.0% female, M_age = 74.68, SD_age = 6.18).

Apparatus and Questionnaires

Embodied Inc. Moxie

Participants interacted with a Moxie social robot (see Figure 2) capable of listening to user commands and equipped with a camera for facial recognition and orienting to the user. Moxie utilizes natural language processing to communicate and displays social cues through a graphical child-like face.

OPEN IN VIEWER

Figure 2.

Embodied Moxie social robot.

Procedure

After providing informed consent, participants completed the demographic and use experience questionnaire, followed by the user’s Big-Five personality inventory. Participants then interacted with Moxie during a 25-min interaction. For consumer experience validity, the interaction represented the first 25-min a typical user would experience when interacting with Moxie out-of-the box. After completing this timed interaction, participants would assess the perceived personality of the robot and desired personality characteristics question.

Perceived Robot Personality and User Personality

Participants rated Moxie to be high in extraversion (95% CI = 34.04, 34.84), very high in agreeableness (95% CI = 37.86, 38.56), high on conscientiousness (95% CI = 37.46, 38.51), low on neuroticism (95% CI = 17.44, 18.29), and high on openness (95% CI = 35.56, 36.84). Moxie was rated significantly higher than participants self-rated themselves in all personality characteristics, except neuroticism, which was rated lower (all p < .001, except openness, p = .02).

Step 1: Gather User Data

Participant preferences for robot personality were assessed through a grounded theory approach (Strauss & Corbin, 1994). In this approach, the qualitative themes are driven by participant responses rather than a priori hypotheses. Participants presented 161 distinct social robot personality themes, which were then grouped into the five personality factor categories.

Step 2: Preferred User Personality Characteristics

In regard to the Big five personality factors, the main personality factor users focused on (44.35% of participants) was extroversion-introversion. This was followed by conscientiousness (18.93%), agreeableness (11.86%), openness to experience (11.30%), and neuroticism (7.34%). Participants desired a robot that was friendly (42.37%), outgoing (29.66%), cheerful (24.29%), helpful (20.62%), and attentive (15.25%). In general, these qualities match with the industry approach to commercial robots and IPA personality.

Step 3: Individual User Preferences

While these prevailing themes can be used to design a user-centered product, they did not match the specific user preferences of most participants. Only 11.4% of participants preferred a social robot personality that matched the target-user approach in Step 2.

Deconstructing these themes into individual user preferences shows that users had 161 individual thematic responses which could be grouped into 49 themes. These themes include a range of 1 to 15 subthemes, which provide a large number of combinations of personality characteristics that users may prefer. Even grouping the user-desired personality characteristics further into 13 prevailing themes produces 101,376,600 possible combinations across all subthemes demonstrating the theoretical diversity in participant preference combinations. This number is significantly higher than the 32 combinations that can be produced from choosing low/high characteristics on the Big 5 personality dimensions. In other words, given the diversity of combinations of user preferences, there can be a significant mismatch between individual and target user preferences. While still challenging, ML/AI methods should be used to bridge the gap and improve presented personality to match individual user needs.

Figure 3 below highlights how individual user preferences vary across participants for a single personality factor. In step 1, participant response are analyzed by theme and combined into prevailing themes. The prevailing themes of kind, helpful, and diligent indicate that users have a strong preference for a robot which is high in conscientiousness (Step 2). However, in Step 3, it is evident that individual user responses to preferences in conscientiousness vary considerably. For some users, high conscientiousness is about the traits of kindness or caring, while others would like robots which are supportive or help solve problems. If a single characteristic in social interaction design was utilized for the design of conscientiousness in the robot, some users would be satisfied, while others would be left wanting differences in the robot’s social design. The user-centered robot would not be truly user-centered.

OPEN IN VIEWER

Figure 3.

Demonstration of conceptual framework using the personality trait of conscientiousness.

Limitations

The study did not test the final outcome of both average and individual user personality preferences in a modified interaction. Therefore, the study is limited in its ability to determine if individual user preferences will lead to higher intent to use the technology and an improved user experience. Additionally, participants’ expressed a total of 161 personality traits, which cannot all easily be mapped into the Big Five-factor model.

Future Directions

Future research should focus on identifying techniques for matching the social interaction of robots and IPAs to individual user preferences. Additionally, user research should explore how differences in these adaptive technologies impact usability, usefulness, user experience, and intention to use social robots and IPAs.