Robot Emotion and Performance Regulation Based on HMM

Abstract

This paper discusses the transference process of emotional states driven by psychological energy in the active field state space and also builds a robot expression model based on the Hidden Marov Models(HMM). Facial expressions and behaviours are two important channels for human-robot interaction. Robot performance based on a static emotional state cannot vividly display dynamic and complex emotional transference. Building a real-time emotional interactive model is a critical part of robot expression. First, the attenuating emotional state space is defined and the state transfer probability is acquired. Secondly, the emotional expression model based on the HMM is proposed and the performance transference probability is calculated. Finally, this model is verified by a 15 degrees of freedom robot platform. Moreover, the interactive effects are analysed by a statistical algorithm. The experimental results demonstrate that the emotional expression model can acquire an expressive performance and remove the mechanized interactive mode when compared to traditional algorithms.

Keywords

Human-Robot Interaction Emotional Expression Active Field State Space Dynamic Emotional Regulation HMM State Transference

1. Introduction

Human emotion is complex and multi-interrelated and affective computing is gradually developing alongside psychological research. The OCC cognitive-affective model is generated by the affective situations (including events, objects and agents) that not only induce cognitive processes but also trigger a subjective emotional experience. Therefore, emotion can be regarded as a kind of psychological energy to drive human behaviours ^[1-3]. Free mental energy and constrained mental energy are the two main kinds of psychological energy form. Free mental energy guides spontaneous emotional state transference and is dependent on personal interest and character ^[4]. Constrained mental energy is an emotional state suppressed by an objective purpose or prerequisite ^{[5, 6]}. The internal manifestation of psychological energy is emotional state transference and the external is behaviour.

Facial expressions and behaviours are two important channels for human-robot interaction. Psychology research has shown that in the emotional expression process language only accounts for 7% and tone for only 35%, the remaining 55% comes from facial and behavioural expressions ^{[7, 8]}. Researchers pay more attention to facial expressions and robot behaviour in human-robot interactions.

This paper discusses the transference process of emotional states driven by the psychological energy in the active field sate space and builds a robot expression model based on the HMM. The attenuating emotional state space is defined and the state transference probability is acquired. This model is verified by a 15 degrees of freedom robot platform in the experiment and the interactive effects are analysed by a statistical algorithm.

2. Dynamic Emotional Regulation in the Active Field State Space

2.1. Description of the Emotional State Space and Regulation Process

The robot's own emotional state space is defined by S = {s₁,s₂,…,s_n}. Here, n is the number of emotional states. The space S, where the robot emotion is freely transferred, includes all the available emotional states. The robot input is W = {w₁, w₂,…,w_m}. m is the emotional stimulus from the facial expression input. In this paper, the facial expression of an interactive object is mapped into 7 basic emotional states in order to simplify the experiment. Based on Ekman's emotional theory, human emotional states can be divided into 6 basic emotions: anger, disgust, fear, happiness, sadness and surprise. Calming is included in the emotional state space to make the robot more similar to humans. Stimulus emotional state space is:

W = {anger, disgutst, fear, happiness, sadness, surprise, calming}

As a result of the external stimulus, the robot's own emotion is aroused and expressed by facial expressions and behaviour. As shown in Figure 1, the robot emotional expression model is built in terms of the above description.

2.2. State Transference Probability in Emotion Space

2.2.1. Active Field State Space

Based on Kismet's emotion space ^[9], each external stimulus from the expression input is looked upon as a source in the active field state space. In this space, shown in Figure 2, the intensity of the external stimulus decides the active degree of the field. According to basic theory of the active field, when there is only one field source in the space, point u(x,y,z)‘s field strength and direction vector ${\vec{e}}_{u}$ are:

E_{i} (u) = \frac{μ \cdot y (t)}{| {\vec{E}}_{i} - \vec{u} |^{2}}

(1)

{\vec{e}}_{u} = \frac{{\vec{E}}_{i} - \vec{u}}{| {\vec{E}}_{i} - \vec{u} |}

(2)

Here, ${\vec{E}}_{i}$ and $\vec{u}$ are the vectors from the origin of source E_i and point u respectively and i = {1,2,…,7}. y(t) is field source's attenuation function, which changes with time t, ${\vec{e}}_{u}$ is the direction vector of u(x,y,z) and μ is an impact factor. E_i(u) is the field strength of emotion i to point u.

2.2.2. Definition of Attenuation Function

The robot's correct emotional state is regarded as a point charge in the active field. The electric quantity of each field source is Q₀ without an external stimulus. Because the emotion is a reactive and movable process, emotional state changes with surroundings, character and satisfaction ^[10-12]. As a result of the external stimulus, the robot's emotional state is transferred according to subjective psychological processing. Because emotional intensity weakens gradually with time, the attenuation function is defined. As is shown in Figure 3, at time t₀ emotional intensity begins to decline and the attenuation function is:

Figure 1.

Robot emotional expression model.

Figure 2.

Emotional state space in the active field.

Figure 3.

The attenuation function curves of 7 basic emotions.

y (t) = \begin{matrix} λ Q_{o} \cdot (\arg \tan \frac{1}{t}) \cdot \exp [- C (t - t_{0})] & t > t_{0} \end{matrix}

(3)

Here, λ = {1,2,3,4,5} is the intensity factor of the external stimulus. In addition to the type of cognition, the emotional essence also influences the intensity of attenuation, so factor C is defined. Each emotion has its own attenuation speed. The progression from slow to fast occurs in the following order: sadness, fear, happiness, disgust, anger and surprise. The external stimulus attenuation factor of calming c₇ is zero. The external stimulus attenuation factor of the other 6 emotions c_i is calculated by the Analytic Hierarchy Process. First, disgust, surprise, happiness, anger, fear and sadness are expressed as w₁, w₂,…,w₆ respectively. The comparative value between emotions i and j is shown in Table 1.

Table 1.

The comparative value of the attenuation speed between emotion i and j

Attenuation Speed	Fast 1	Fast 2	Fast 3	Fast 4	Fast 5
Value a_ij	10	8	6	4	2
Attenuation Speed	Slow 1	Slow 2	Slow 3	Slow 4	Slow 5
Value a_ij	1/10	1/8	1/6	1/4	1/2

Secondly, a_ij is the comparative result between emotion i and j and a_ij = 1/a_ji. The comparative matrix is:

\begin{array}{l} A = (a_{i j})_{6 \times 6} = (\begin{matrix} 1 & \frac{1}{4} & 2 & \frac{1}{2} & 4 & 6 \\ 4 & 1 & 6 & 2 & 8 & 10 \\ \frac{1}{2} & \frac{1}{6} & 1 & \frac{1}{4} & 2 & 4 \\ 2 & \frac{1}{2} & 4 & 1 & 6 & 8 \\ \frac{1}{4} & \frac{1}{8} & \frac{1}{2} & \frac{1}{6} & 1 & 2 \\ \frac{1}{6} & \frac{1}{10} & \frac{1}{4} & \frac{1}{8} & \frac{1}{2} & 1 \end{matrix}) \\ a_{i j}^{'} = \frac{a_{i j}}{\sum_{i = 1}^{6} a_{i j}} \end{array}

(4)

A‘ is A normalized by formula (4).

A^{'} = {(a_{i j}^{'})}_{6 \times 6} = (\begin{matrix} 0.126 & 0.117 & 0.145 & 0.124 & 0.186 & 0.194 \\ 0.505 & 0.467 & 0.436 & 0.495 & 0.372 & 0.323 \\ 0.063 & 0.078 & 0.073 & 0.062 & 0.093 & 0.129 \\ 0.253 & 0.233 & 0.291 & 0.247 & 0.279 & 0.258 \\ 0.032 & 0.058 & 0.036 & 0.041 & 0.047 & 0.065 \\ 0.021 & 0.047 & 0.018 & 0.031 & 0.023 & 0.032 \end{matrix})

The external stimulus attenuation factor C = [0.15 0.43 0.08 0.26 0.05 0.03 0]is calculated by formula (5).

c_{i} = \frac{\sum_{j = 1}^{7} a_{i j}^{'}}{\sum_{i = 1}^{7} \sum_{j = 1}^{7} a_{i j}^{'}}

(5)

2.2.3. Emotional State Transference Probability

As mentioned in Section 2.2.1, the field strength influences the direction of emotional state transference in the active field space. The transference probability is proportional to the electric quantity of the field source and is inversely proportional to the distance from the stimulus emotion to the current emotion state. Based on formula (1-3), the stimulus intensity matrix of 7 basic emotions is E = [E₁(u) E₂(u) … E₇(u)]. The transference probability from the current emotional state to the next along ${\vec{e}}_{i}$ is:

p ({\vec{e}}_{i}) = \frac{E_{i} (u)}{\sum_{i = 1}^{7} E_{i} (u)}

(6)

Here, ${\vec{e}}_{i}$ is the direction vector from u(x,y,z) to emotion i.

3. Emotional Expression Model based on HMM and Implementation

3.1. Emotional Expression Model based on HMM

Human emotion can be divided into two parts, the primary psychology and the secondary emotional feeling. Primary emotional feeling is an unconditioned reflex in the external environment. Secondary emotional feeling is produced when the relation between the primary emotion and the present perception is built ^{[13, 14]}. Therefore, the HMM stimulation-transference model is proposed for realizing emotional regulation and expression, as shown in Figure 4.

The emotional state transference probability matrix A=[p( ${\vec{e}}_{1}$ ) p( ${\vec{e}}_{2}$ ) … p( ${\vec{e}}_{7}$ )] is obtained from formula (6). The state space of robot facial expressions and behaviours is V = {v₁,v₂,…v_n}. In the HMM transference model, the first stochastic process forms an emotional state transference sequence and the second forms a robot performance sequence based on emotion. The performance transference probability matrix is:

B = [\begin{matrix} b_{11} & b_{12} & \dots & b_{1 n} \\ b_{21} & b_{22} & \dots & b_{2 n} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ b_{n 1} & b_{n 2} & \dots & b_{n n} \end{matrix}]

When the robot emotional state is s_k, b_ij is the transference probability from performance v_i to v_j.

b_{i j} = p ({\vec{e}}_{k}) \cdot p (v_{j} | v_{i}) \cdot p (v_{i})

(7)

Performance transference is full state transference. So the sum of the probability from one performance state to all the others in V is p( ${\vec{e}}_{k}$ ) under emotional state s_k. Thus, the constraint of matrix B is:

\sum_{j} b_{i j} = p ({\vec{e}}_{k})

(8)

According to Gross emotional response regulation, the expression suppression strategy is proposed in the emotional regulation process for controlling facial expression and behaviour. The degree of facial expression and behaviour is Ω in theory and the expression suppression degree is η(0 < η < 1). Therefore, the performance degree is Ω' = (1 – η)Ω. The magniloquent degree of robot performance is adjusted by the factor η.

3.2. Implementation of Emotional Performance

The robot mechanical structure is designed according to research on the proportion relationship of the torso and limbs of Chinese adults ^[11]. As shown in Figure 5, there are 15 degrees of freedom and 100 kinds of facial expression. Therefore, the robot can achieve real time control for an abundance of behaviours and facial expressions. The robot's upper-body joint is composed of 13 motors and many connecting pieces. The mechanical structure is shown in Figure 6. Basic movements of a human arm can be completely simulated by this structure. The specific distribution of degrees of freedom is shown in Table 2.

Figure 4.

HMM transference model.

Figure 5.

Robot's structure and typical expressions.

Figure 6.

Robot's upper-body mechanical structure.

Table 2.

The specific distribution of degrees of freedom.

Part name	Degrees of freedom	Number	Action Description
Motor 1(Shoulder)	1	2	swing arm
Motor 2(Shoulder)	1	2	lay down or spread arm
Motor 3(Shoulder)	1	2	rotate arm
Motor 4(Elbow)	1	2	left up or lay down forearm
Motor 5(Elbow)	1	2	rotate forearm
Motor 6(Waist)	1	1	rotate waist
Motor 7(Waist)	1	1	sideways torso
Motor 8(Waist)	1	1	bend down
Neck	2	-	nod or shake head
Total degrees of freedom	15	-	-

The current emotional state is composed of external stimuli and previous emotions and has a certain randomness. When the emotion is produced, it will trigger a series of behaviours. The robot determines the current behaviour via the production rule: IF Emotional State THEN Behaviour Plan. Because of the influence of personality, randomness and expression suppression, there are different behaviour plans for one person in different surroundings or at different times. Some typical emotional performances from the experiments are shown in Table 3. In the HMM stimulus transference process mentioned in Section 3.1, dual stochastic processes for emotional state transference and behaviour performance transference are implemented to shake off the formulaic interactive mode. Thus, the randomness of the robot's emotional state and behaviour displays a sharp increase and the performance is more varied and hominine.

Table 3.

Typical emotional performance in the experiment.

Emotional state	Typical performance
Anger	shake fist or back hands or surround arms, straighten waist, raise head, mouth tight close, press eyebrows, etc.
Disgust	droop hands, open elbows, head-twist, close eyes, lift lip, mouth corner up, etc.
Fear	raise hands, lower head, elbows close to body, lift lip, wrinkle eyebrows, etc.
Happiness	clap or raise hands, straighten waist, raise head, mouth corners up, narrow eyes, etc.
Sadness	stoop, droop hands, lower head, weep, mouth tight close, droop mouth corners, etc.
Surprise	raise hands or surround arms, open elbows, head forward, open mouth, brow up; lift eyelid, etc.

Figure 7.

Real time expression recognition results: fear, sadness, anger, disgust, surprise, calming, happiness.

Table 4.

Statistical contrast results of human-robot interaction.

	Facial expressive degree	Facial expression accuracy	Behaviour expressive degree	Behaviour accuracy
Stimulation-response model	3.97	4.40	4.10	4.27
HMM model	4.53	4.50	4.63	4.50
	Interactive fluency	Friendliness	Facial expression exaggeration	Behaviour exaggeration
Stimulation-response model	3.93	3.60	3.87	4.17
HMM model	4.80	4.53	4.03	4.20

4. Experiment

Emotion is a high-level intelligent behaviour that reflects human thinking activity and is represented as an implied state in the HMM model. Emotional performance is the external manifestation and is represented as an observable state in the model. The emotional performance model is used in the real human-robot interactive process for proving this impact. First, the feature of the facial expression is extracted by Gabor wavelet and the SVM is used as a classifier for obtaining the stimulus emotional state. The recognition result is shown in Figure 7. There were 30 volunteers in this experiment. The accuracy rate of the expression recognition is approximately 78.9%. Second, the emotional state model is built in the active field via the calculation of matrix A. Third, the performance transference matrix B and the expression suppress factor η are determined and then the robot performance model based on HMM is built. When the object's facial expression is recognized and mapped into the emotion space, the robot produces the facial expression plan and behaviour plan via the HMM model. Finally, the robot implements the performance via control of the hardware devices. A typical robot emotional expression in the human-robot interactive experiment is shown in Figure 8.

The robot facial expression and behaviour is an observable state for the emotional regulation process mentioned in Section 2. If the robot expression corresponds with interactive context and its manifestation is flexible, the emotional expression model is believable. 30 volunteers evaluated the interactive process based stimulation-response model and HMM model via the Likert scale. The scale includes involved with the advantageous projects, such as interactive fluency, facial expressive degree, facial expression accuracy, the plenty degree of behaviours/expressions, behaviour accuracy and friendliness. And itIt also includes involved with disadvantageous projects, such as the exaggeration degree of facial expressions exaggeration, the exaggeration degree of behaviours exaggeration. Advantageous projects from very satisfied to discontented is divided into five levels, which is given a mark as 5, 4, 3, 2, 1 credit respectively. Disadvantageous project from very agreement to disagreement is also divided into five levels, which is given a mark as 1, 2, 3, 4, 5 credit respectively. The statistical contrast results of the human-robot interaction are shown in Table 4.

Figure 8.

6 typical emotional performances.

The experimental evaluation results show that compared with the Stimulation-response model, robot performance is more expressive and friendly in the HMM model and the accuracy and exaggeration degree is not raised obviously. It can be seen that the robot expression mode is more interesting due to the introduction of the emotion and performance probability space. Under the control of this model, a robot can regulate its own emotion autonomously and the mechanical interactive mode is improved to a certain extent. A robot can dynamically regulate emotion and performance in real time and the interactive quality improves significantly.

5. Conclusion

Based on the OCC cognitive-affective model and the psychological energy theory, the emotional state transference process is described in the active field state space. The HMM emotional expression model, combined with the robot's own emotional effect from the interactive object, human emotional attenuation and expression suppression, is used in the human-robot dynamic interaction. Because the model has double stochastic processes, the robot performance model is expressive and the interaction is more natural. The model is applied to a real robot platform and psychological surveys from 30 volunteers are analysed. The survey results show that the robot communicates with objects by means of facial expressions and behaviour and removes the mechanized interactive mode to improve the effect of the human-robot interaction.

There are numerous effects in emotional regulation, including internal psychology elements and external surroundings. In this paper, the model and implementation only considers a few obvious effects. Continuing further research in infinite-dimensional continuous space is very important.

Footnotes

6. Acknowledgments

This work is supported by National Natural Science Foundation of China (No. 61170115; No. 61170117; No. 61105120), and the 2012 Ladder Plan Project of Beijing Key Laboratory of Knowledge Engineering for Materials Science (No. Z121101002812005).

References

Ortony

Clore

G. L.

Collins

The Cognitive Structure of Emotions [M]. London: Cambridge University Press, 1988.

Picard

R. W.

IEEE Transactions on Affective Computing. 1, 11, 2010

Atsuhiro

Hiroshi

Koichi

Object Recognition Based on n-gram Expression of Human Actions. The 20th International Conference on Pattern Recognition, 2010, Istanbul, Turkey

Lee

W. H.

Park

J. W.

Kim

W. H.

Kim

Ju Chang J. C.

Chung

M. J.

Robot's Emotion Generation Model for Transition and Diversity using Energy, Entropy and Homeostasis Concepts[C]. Robotics and Biomimetics, 2010 IEEE International Conference on, 2010, 555–560.

Vami

Volpe

Camurri

A System for Real-Time Multimodal Analysis of Nonverbal Affective Social Interaction in User-Centric Media[J]. Multimedia, IEEE Transactions on, 12, 6, 2010, 576–590.

Lun

Xie

Lhiliang

Wang Z.

J. M.

Mechanical Design and Affective interaction of Bionic Robot Head[J]. Advanced Science Letters, 4, 4-5, 2011, 1337–1341.

Di Lorenzo

Pinelli

Pereira

F. C.

Biderman

Ratti

Lee

An Affective Intelligent Driving Agent: Driver's Trajectory and Activities Prediction [C]. Vehicular Technology Conference Fall, 2009 IEEE 70^th, 2009, 1–4.

Park

Jong-Chan J. C.

Song

Hyunsoo H. S.

Koo

Seongyong S. Y.

Kim

Young-Min Y. M.

Kwon

Dong-Soo D. S.

Robot's Behavior Expressions According to the Sentence Types and Emotions with Modification by Personality. IEEE Workshop on Advanced Robotics and its Social Impacts, 2010, Menlo Park, USA

Breazeal

Function Meets Style: Insights From Emotion Theory Applied to HRI [J]. IEEE Transactions on Systems, Man, and Cybernetics-Part C: Applications and Reviews, 34, 2, 2004, 187–194.

10.

Haresh

Mani

P. P.

Jain

Satish C.

Simulation-Transactions of the Society for Modeling and Simulation International, 83,188, 2011

11.

Laling

Yi F.

Jiao

Guo

Qongqiu

Tu Y.

Jin

Yang

. Research on Traditional Chinese Medicine Human Body Structure and Function. International Conference on Technologies and Applications of Artificial Intelligence, 2010, Taoyuan, Taiwan

12.

Breazeal

Function Meets Style: Insights From Emotion Theory Applied to HRI [J]. IEEE Transactions on Systems, Man, and Cybernetics-Part C: Applications and Reviews, 34, 2, 2004, 187–194.

13.

Kulic

Croft

E. A.

Affective State Estimation for Human-Robot Interaction [J]. Robotics, IEEE Transactions on, 23, 5, 2007, 991–1000.

14.

Liu

Xie

Dynamic Regulation Process of Facial Expression Robot[J]. Control Theory and Applications, 28, 7, 2011, 936–946.