Facial Recognition Patterns of Children and Adults Looking at Robotic Faces

Abstract

The present study investigates whether adults and children exhibit different eye-fixation patterns when they look at human faces, machinelike robotic faces, and humanlike robotic faces. The results from two between-subject experiments showed that children and adults did have different facial recognition patterns; children tended to fixate more on the mouth of both machinelike and humanlike robotic faces than they do on human faces, while adults focused more on the eyes. The implications of notable findings and the limitations of the experiment are discussed.

Keywords

facial recognition eye-fixation eye-tracking robotic face

1. Introduction

Do children and adults share the same or similar patterns when they interact with robots? Are the ways in which children and adults view and form impressions of robots different? These and related questions have become particularly relevant in recent years because R-Learning for children has gained much attention and popularly lately. The present study argues that we may answer these questions by examining whether children and adults look at robotic faces differently when forming first impressions.

When we communicate with others, we exchange our emotions, feelings and thoughts by using various communication modalities. In addition to verbal communication, we also use non-verbal communication modalities such as body movements, hand gestures and facial expressions [1, 2, 3, 4, 5, 6, 7]. Gross and Ballif suggested that facial expressions are one of the most efficient ways for understanding others' emotions and feelings. In particular, children do not have sufficient social communication skills to understand others' minds or feelings when compared to adults [8], suggesting that the ways in which children recognize and see robotic faces may be different from that of adults.

Therefore, the aim of this paper is to identify different recognition patterns between children and adults when looking at robotic faces.

2. Literature Review

2.1. Essential Facial Components for Effective

Human-Robot Interaction

To find the main communication channels in human robot interactions, DiSalvo analysed the facial components of various robots, including academic, commercial and entertaining robots as well as imaginary robots from popular films. After examining 48 robots, DiSalvo found that 81% and 62% of the robots had eyes and a mouth respectively, suggesting that eyes and mouths are crucial facial components needed for effective human-root interactions [9, 10].

Kim and Han found that robots' eyes were the most important facial component affecting children's preferences for robots [11]. In addition to eyes, Bassili compared a robot that moves its mouth while speaking to a robot with an immovable mouth, and found that the movement of the mouth was key to effective communication and successfully facial recognition [12]. Furthermore, the nose is another important core component because it is located between the eyes and the mouth; people are required to look at the nose when they try to look at the eyes and the mouth. Therefore, these three components – eyes, mouth and nose – are considered the key facial components [13].

2.2. Children's Communication Skills

Children's social communication skills develop gradually. In early childhood, they try to read and understand different thoughts, feelings and perspectives of others. For instance, children attempt to integrate social cues for social cognition, including not only verbal sources but also non-verbal information such as gestures, facial recognition and tone of voice. Therefore, seeing others' faces is an important social activity to gather such crucial information [8, 14].

2.3. Cognitive Robots

Many cognitive robots have been developed in order to improve children's social communication skills [15, 16, 17, 18]. For example, Keepon was designed for engaging in emotional communications with children and improving their communication skills. Although Keepon was not capable of making emotional expressions and it delivered its emotions only though body movements and rotations, the robot was still able to convey diverse emotions through its eyes and mouth, which was particularly effective for autistic children with limited social skills [18].

As R-Learning (i.e., robot assisted learning) for children has recently gained much attention and earlier studies primarily focused on the adult population [19, 20], it is important to examine whether children interact with robots in different ways.

Therefore, the present study intends to examine the facial recognition patterns of adults and children when they are exposed to human and robotic faces. Study 1 investigated whether adults and children show different eye-fixation patterns when they look at human faces and machinelike robot faces. Study 2 further explored the eye-fixation patterns when looking at humanlike robotic faces [13].

3. Study 1

3.1. Design

Study 1 was a between-subjects experiment with four conditions: 2 (participants' age: children vs. adults) × 2 (type of face: human vs. machinelike robotic).

3.2. Participants

80 participants (40 females, 40 males) were recruited from a large private university and a kindergarten in Seoul, South Korea. The age of the children ranged from 6 to 11, with a mean age of 9.0 years (SD=1.70). The age of the adults ranged from 23 to 32, with a mean age of 28.1 years (SD=2.22). All of the participants had normal eyesight. Each condition had 20 participants (10 females, 10 males).

3.3. Apparatus and Stimulus

Photos of human and machinelike robotic faces were prepared. For the human faces, we hired twelve actors from various age groups: two teenagers (1 male, 1 female), four in their twenties (2 males, 2 females), four in their thirties (2 males, 2 females), and two in their forties (1 male, 1 female). They were instructed not to express any emotions or feelings during the photo-shot. We took eight photos from each actor. 48 photos from six actors were then selected after excluding 48 photos that included facial expressions.

The validity of the photos was supported through a pre-test by twelve adults. They were asked to respond to a 7-point Likert scale measuring the degree to which they were able to identify the facial expressions of the actors depicted in the photos (1=“I found strongly clear expression in the photo,” 7=“I did not find any definite expression in the photo”). Based on this procedure, 24 photos (average ratings from 6.1 to 6.9) were finally selected as stimulus materials for the experiment.

For the machinelike robotic face, 64 images of 16 existing robots were collected. We selected robots that had eyes, a nose and a mouth. From a similar pre-test, 24 photos were selected as stimulus materials for the experiment. The robots were Kaspar, Kobian, EveR-2 Muse, and Jules [21, 22, 23, 24]. All of the photos were prepared in 1200 dpi and 900×1200 pixels resolution.

3.4. Experimental Environment

A 24-inch LCD monitor and Tobii x120 eye-tracker (Fig. 1) were prepared in a room with a comfortable chair [25]. A laptop computer was placed remotely from the monitor because participants might be distracted with other computer components, such as the mouse. A desk was placed between the monitor and the participants so as to maintain a proper distance. Two experimenters were present in the room for instructing participants and controlling the eye-tracking software.

Figure 1.

Tobii x120 eye tracker

3.5. Procedure

As the participants entered the room, the experimenter provided the participants with a brief overview of the experimental procedure. Before the main-task, the participants were asked to look at the photos for the calibration of the eye-tracker.

As the main session began, the eye-tracker began to record participants' eye movement patterns and fixation time. The participants were instructed to look at the monitor carefully. 24 photos were randomly displayed on a black background for 4 seconds. To remove the effects of previous images, a black-screen was displayed for a second between each image.

3.6. Measurement

Eye-tracker recorded and calculated the eye fixation time on the eyes, nose and mouth of the faces depicted on each photo (Tobii x120). Because there was a few milliseconds gap between the photos (presented times ranged from 3.991 to 4.007 seconds), we used the percentage-results of the fixation time for the entire presented time for an accurate comparison between the conditions.

3.7. Results

Table 1 showed the results of the average fixation time of the core features.

3.7.1. The Effects of Human vs. Machinelike Robotic Faces

An analysis of variance (ANOVA) was conducted to examine the effects of face type on the percentage of fixation time on the sum of core features. Also, a multivariate analysis of variance (MANOVA) was conducted so as to examine the effects of face type on the percentage of fixation time of each core feature (eyes, nose and mouth). The results of the ANOVA indicated that the face type did not have a notable effect on fixation time in the sum of core features (p=.09). However, the results from the MANOVA indicated that participants who looked at the machinelike robotic faces (M=24.00%, SD=17.10) reported a longer fixation time on the mouth than those looking at the human faces (M=15.45%, SD=9.74), F(1,76)=16.81, p<.001, while the participants who looked at the machinelike robotic faces (M=26.88%, SD=16.54) reported a longer fixation time on the nose than those looking at the machinelike robotic faces (M=21.49% SD=9.68), F(1,76)=4.10, p=0.046.

Table 1.

Mean and standard deviation of the percentage of fixation time on core facial features

	Facial Components	Human Face (%)	Machinelike Robotic Face (%)
Adults	Eyes	49.09 (17.81)	48.07 (13.27)
	Nose	17.45 (7.89)	22.18 (9.87)
	Mouth	11.60 (6.70)	9.96 (7.72)
Children	Eyes	26.30 (18.25)	22.25 (8.48)
	Nose	36.35 (17.66)	20.81 (9.68)
	Mouth	19.34 (10.89)	38.05 (11.19)

3.7.2. Effects of Age Level

The age of the participants also had a significant effect on the percentage of the fixation time on the eyes, mouth and nose. Although the age of the participants did not have a significant effect on the percentage of the fixation time on the sum of core features, the results from the MANOVA found that adults (M=48.58%, SD=15.52) had a longer fixation time on the eyes than children (M=24.27%, SD=14.19), F(1,76)=52.57, p<.001, while the children (M=28.68%, SD=14.45) had a longer fixation time on the mouth than the adults (M=10.78%, SD=7.18), F(1,76)=73.60, p<.001. Also, the results indicated that the children (M=28.56%, SD=16.10) had a longer fixation time on the nose compared with the adults (M=19.81%, SD=9.14), F(1,76)=10.81, p=.002.

3.7.3. The Interaction between Face Type and Age Level

There was an interaction between face type and age, such that children looked at the machinelike robot's mouth (F(1,76)=23.89, p<.001) for relatively longer than the nose (F(1,76)=14.45, p<.001) while adults showed no difference in their eye-fixation patterns due to the face type (Fig. 2, 3, 4, 5).

Figure 2.

Eye-tracking image of one of robotic faces; the left shows the adults' and the right the children's

Figure 3.

Mean and standard error of the percentage of the fixation time on eyes

3.8. Discussion

In Study 1, we examined the differences in facial recognition patterns of children and adults looking at human and machinelike robotic faces. While the adults showed no difference in their eye-fixation patterns, the children tended to pay more attention to the mouth than the eyes or the nose of machinelike robotic faces. Interestingly, the children's facial recognition patterns did not show any difference when they looked at the human faces.

One of the major limitations of this study was that it did not consider other characteristics of the face, such as face shape and colour. In other words, if the area of the core features is bigger, participants may be more likely to look at the core features.

4. Study 2

4.1. Design

Study 2 was a between subject experiment with four conditions: 2 (participants' age level: children vs. adults) × 2 (Terminator face: human vs. humanlike robotic).

4.2. Participants

The participants from Study 1 participated in this study.

Figure 4.

The mean and standard error of the percentage of the fixation time on the nose

Figure 5.

The mean and standard error of the percentage of the fixation time on the mouth

4.3. Apparatus and Stimulus

100 photos of Arnold Schwarzenegger [26] depicted as the Terminator and 40 photos of a robotic version of the Terminator were collected from the Terminator series movies, TV shows and Internet websites. After the identical procedure used in Study 1, three photos of Arnold Schwarzenegger depicted as the Terminator and three photos of the mechanical version of the Terminator were finally selected as the stimulus materials of the experiment (Fig. 6).

4.4. Experimental Circumstances

The experimental environment, procedure and measurements were identical with Study 1.

Figure 6.

A sample photo of the Terminator used in Study 2 (left: robotic face, right: human face)

4.5. Results

Table 2 shows the results of the average fixation time of the core features.

Table 2.

The mean and standard deviation of the percentage of the fixation time on the core facial features

	Facial Components	Human Face (%)	Humanlike Robotic Face (%)
Adults	Eyes	59.15 (13.58)	67.05 (12.04)
	Nose	20.25 (11.73)	14.45 (7.03)
	Mouth	11.25 (4.12)	8.75 (3.89)
Children	Eyes	34.60 (14.20)	36.10 (8.03)
	Nose	31.90 (11.11)	23.08 (11.12)
	Mouth	20.65 (8.30)	28.39 (9.86)

4.5.1. The Effects of Human vs. the Robotic Face of the Terminator

A multivariate analysis of variance (MANOVA) was conducted to examine the effects of face type on the percentage of the fixation time on each core feature (eyes, nose and mouth). The results of the analysis indicated that those participants exposed to the humanlike robotic faces of the Terminator (M=18.77%, SD=10.17) reported a longer fixation time on the nose than those exposed to the human faces of the Terminator (M=26.08%, SD=12.73), F(1,76)=9.84, p<.001. However, there was no significant effect on the eyes (p=.089) or mouth (p=.100)

4.5.2. The Effects of Age Level

The results from the MANOVA showed that the adults (M=63.10%, SD=13.28) had a longer fixation time on the eyes than the children (M=35.35%, SD=11.41), F(1,76)=103.44, p<.001, while the children (M=24.52%, SD=9.81) had a longer fixation time on mouth than the adults (M=10.00%, SD=4.15%), F(1,76)=85.08, p<.001. Also, the results indicated that the children (M=27.49%, SD=11.85) had a longer fixation time on the nose than the adults (M=17.35%, SD=9.99), F(1,76)=18.95, p<.001.

4.5.3. The Interaction between Face Type and Age Level

There was an interaction between the face type and age. Consistent with the findings of Study 1, the children looked at the mouth of the humanlike robotic faces (F(1,76)=10.58, p=.002) longer than nose while adults showed no difference in their eye-fixation patterns due to the face type (Fig. 7, 8, 9).

Figure 7.

The mean and standard error of the percentage of the fixation time on the eyes

4.6. Discussion

Study 2 examined the effects of humanlike robotic faces on facial recognition patterns. By using the same areas of the core features and the same face shape, the existence of different fixation tendencies between adults and children was confirmed. Specifically, the data showed a higher tendency in the fixation time of the eyes than that of Study 1. It may be due to the effect of luminous materials. The eyes, especially among robotic faces, looked like an electric bulb. This may have attracted the participants' gaze (Fig. 6).

5. General Discussion

Recently, many studies have shown increasing interest in the psychological effects of interactive robots. In this study, we conducted two experiments in an attempt to identify and analyse differences in facial recognition patterns between children and adults. Also, we focused on the different face searching patterns between robot faces and human faces. A number of previous studies indicated that social communication skills were strongly associated with and linked to the humans' gazing behaviour [27].

Therefore, this study considered facial recognition as an important behaviour when people communicate or interact with others. Also, we focused on the differences in the participants' ages, because childhood is a critical period during which our social communication skills are developed. Therefore, adults and developing children took part in Study 1 with the photos from actors and machinelike robots. Study 2 used the photos from humans and humanlike robots which were extracted by the same facial forms. The participants' eye movement and fixation patterns were recorded by an eye-tracker and then analysed for age-specific patterns. The present study found that children and adults have different eye-fixation patterns when recognizing human and robotic faces.

In Study 1, all of the participants tended to see core features in the face. However, the adults gazed longer on the eyes than on the mouth or the nose while seeing either the robot photos or human photos, while children gazed for a shorter time on the eyes than the mouth in robot photos or the nose in human photos. In particular, in comparison with Study 2, these tendencies were strongly shown in Study 1. This may be due to three reasons. First, the robots in Study 1 included the mouth, which occupied a big part of the robots' faces relative to the case of the humans. Second, it may be that children tend to focus more on dynamic features, such as the mouth, rather than static features such as the eyes. Third, other characteristics of the faces in the photos, such as the face shape or other features, might easily affect the children's attention or gaze.

To explain the first and third reasons, we controlled other possible features of the robots' and humans' faces, by using the same person's face frame. The results of Study 2 confirmed the findings of Study 1.

The results are interesting in that the children had different patterns and manners of fixation in comparison with those of the adults. Although the rate of the children's eye fixation time on the core features was similar to that of adults, the patterns and manners are different. This raises the important implication that the dynamic features - such as the mouth - of an interactive robot should be carefully designed if it is to be used for children. This means that the design guidelines of interactive robots for children should be different from those of other robots'. It especially suggests the needs of the design guidelines of robots for education or other services for children. Also, many other considerations for differences between humans and robots may be in existence.

Figure 8.

The mean and standard error of the percentage of the fixation time on the nose

Figure 9.

The mean and standard error of the percentage of the fixation time on the mouth

6. Limitations and Future Studies

There exist a number of limitations in the present study. First, future studies should consider other modes of communication, including facial expressions, because facial expressions are one of the easiest ways of delivering emotions or states of mind [28]. Second, future studies may also consider other characteristics of participants. Age and the cultural differences of the participants may affect their fixation patterns and manners. Several studies indicated that different social interaction styles could come from cultural differences [29, 30, 31]. Third, the present study used the facial images of humans and humanoid robots. Therefore, future studies might include the images of entire body of humans and robots. Finally, other characteristics of humans and robots in photos, such as gender, may affect respondents' preferences and fixation patterns. Some previous studies reported different fixation patterns based on the different genders of humans in photos [32, 33].

Footnotes

7. Acknowledgments

A preliminary version of this study was presented at the IRoA 2011 conference [13]. This version includes a more extensive literature review and additional data analyses in order to revisit notable limitations discussed in the preliminary version of the paper. This study was supported by a grant from the World-Class University programme (R31-2008-000-10062-0) of the Korean Ministry of Education, Science and Technology via the National Research Foundation.

References

Smalley

Scott

, For better or for best. Zondervan (Grand Rapids, Mich.), 1988.

Mundy

Sigman

Ungerer

Sherman

, “Defining the Social Deficits of Autism: The Contribution of Non-verbal Communication Measures,” Journal of Child Psychology and Psychiatry, vol. 27, pp. 657–669, 1986.

Argyle

, Ed., Non-verbal Communication in Human Social Interaction, (Non-verbal communication). Oxford, England: Cambridge University Press, 1972.

Jones

N. G. Blurton

, Ed., Non-verbal Communication in Children, (Non-verbal communication). New York: Cambridge University Press, 1982, pp. 271–295.

Nowicki

Duke

M. P.

, “Individual differences in the nonverbal communication of affect: The diagnostic analysis of nonverbal accuracy scale,” Journal of Nonverbal Behavior, vol. 18, pp. 9–35, 1994.

Kacperek

, “Non-verbal Communication: The Importance of Listening,” British Journal Nursing, vol. 5, pp. 275–279, 1997.

Graham

J. A.

Argyle

, “A Cross-Cultural Study of the Communication of Extra-Verbal Meaning by Gesture,” International Journal of Psychology, vol. 10, pp. 57–67, 1975.

Gross

A. L.

Ballif

, “Children's understanding of emotion from facial expressions and situations: A review,” Developmental Review, vol. 11, pp. 368–398, 1991.

DiSalvo

C. F.

Gemperle

Forlizzi

Kiesler

, “All robots are not created equal: The design and perception of humanoid robot heads,” in Proceedings of the 4th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, 2002, pp. 321–326.

10.

Blow

Dautenhahn

Appleby

Nehaniv

C. L.

Lee

D. C.

, “Perception of robot smiles and dimensions for human-robot interaction design,” in Proceedings of the 15th IEEE International Symposium on Robot and Human Interactive Communication, 2006, pp. 469–474.

11.

Kim

Han

, “Quantitative Analysis on Children's Perception for Designing Tutoring Robots' Heads,” Journal of the Korean Association of Information Education, vol. 10, pp. 769–775, 2006.

12.

Bassili

J. N.

, “Emotion recognition: The role of facial movement and the relative importance of upper and lower areas of the face,” Journal of Personality and Social Psychology, vol. 37, pp. 2049–2058, 1979.

13.

Park

Kim

K. J.

del Pobil

A. P.

, “Do Children See Robots Differently? A Study Comparing Eye-Movements of Adults vs. Children When Looking at Robotic Faces,” Lecture Notes in Electrical Engineering, vol. 107, pp. 421–427, 2011.

14.

Felleman

E. S.

Carlson

C. R.

Barden

R. C.

Rosenberg

Masters

J. C.

, “Children's and adults' recognition of spontaneous and posed emotional expressions in young children,” Developmental Psychology, vol. 19, pp. 405–413, 1983.

15.

Dautenhahn

Werry

, “Towards interactive robots in autism therapy: Background, motivation and challenges,” Pragmatics & Cognition, Cognition, vol. 12, pp. 1–35, 2004.

16.

Pioggia

Igliozzi

Ferro

Ahluwalia

Muratori

De Rossi

, “An android for enhancing social skills and emotion recognition in people with autism,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 13, pp. 507–515, 2005.

17.

Robins

Dickerson

Stribling

Dautenhahn

, “Robot-mediated joint attention in children with autism: A case study in robot-human interaction,” Interaction Studies, vol. 5, pp. 161–198, 2004.

18.

Kozima

Michalowski

M. P.

Nakagawa

, “A playful robot for research, therapy, and entertainment,” International Journal of Social Robotics, vol. 1, pp. 3–18, 2009.

19.

Han

, Ed., Robot-Aided Learning and r-Learning services, ch. 17, (Human-Robot Interaction). Croatia: Intech, 2010, pp. 247–266.

20.

Han

Jones

J. H.

, “Comparative study on the educational use of home robots for children,” Journal of Information Processing Systems, vol. 4, pp. 159–168, 2008.

21.

Dautenhahn

Nehaniv

C. L.

Walters

M. L.

Robins

Kose-Bagci

Mirza

N. A.

Blow

, “KASPAR-a minimally expressive humanoid robot for human-robot interaction research,” Applied Bionics and Biomechanics, vol. 6, pp. 369–397, 2009.

22.

Zecca

Mizoguchi

Endo

Iida

Kawabata

Endo

Itoh

Takanishi

, “Whole body emotion expressions for KOBIAN humanoid robot -preliminary experiments with different Emotional patterns-,” in Proceedings of the 18th IEEE International Symposium on Robot and Human Interactive Communication, 2009, pp. 381–386.

23.

Lee

D. W.

Lee

T. G.

Choi

Shin

E. C.

Yang

K. W.

Back

Kim

H. S.

Lee

H. G.

, “Development of an Android for Emotional Expression and Human Interaction,” in Proceedings of the 17th IFAC World Congress, 2008, pp. 4336–4337.

24.

Bristol Robotics Laboratory. (2011). Available: http://www.brl.ac.uk/projects/empathy/empathy.html

25.

Tobii x120 Eyetracker. (2011). Available: http://www.imotionsglobal.com/Tobii+X120+Eye-Tracker.344.aspx

26.

Arnold Schwarzenegger. (2011). Available: http://en.wikipedia.org/wiki/Arnold_Schwarzenegger

27.

Riby

D. M.

Hancock

P. J. B.

, “Viewing it differently: Social scene perception in Williams syndrome and autism,” Neuropsychologia, vol. 46, pp. 2855–2860, 2008.

28.

Craig

K. D.

, “The facial expression of pain Better than a thousand words?” APS Journal, vol. 1, pp. 153–162, 1992.

29.

Ekman

, Universals and cultural differences in facial expressions of emotion. University of Nebraska Press Lincoln, 1971.

30.

Salili

, “Age, sex, and cultural differences in the meaning and dimensions of achievement,” Personality and Social Psychology Bulletin, vol. 20, pp. 635–648, 1994.

31.

Matsumoto

, “American-Japanese cultural differences in the recognition of universal facial expressions,” Journal of Cross-Cultural Psychology, vol. 23, pp. 72–84, 1992.

32.

Lorigo

Pan

Hembrooke

Joachims

Granka

Gay

, “The influence of task and gender on search and evaluation behavior using Google,” Information Processing & Management, vol. 42, pp. 1123–1131, 2006.

33.

Jernajczyk

Sobanska

Czerwosz

Szatkowska

, “The influence of age and gender on the latency of eye movement in healthy humans,” Journal of physiology and pharmacology, vol. 56, pp. 93–97, 2005.