Abstract
This brief paper sets out arguments for the introduction of new technologies into telecare and lifestyle monitoring that can detect and monitor the emotive state of patients. The significantly increased use of computers by older people will enable the elements of emotive computing to be integrated with features such as keyboards and webcams, to provide additional information on emotional state. When this is combined with other data, there will be significant opportunities for system enhancement and the identification of changes in user status, and hence of need. The ubiquity of home computing makes the keyboard a very attractive, economic and non-intrusive means of data collection and analysis.
Introduction
Telecare strategies aim to detect and respond to changes in behaviour using data derived from sensors distributed around the home. This means that a significant factor in the determination of behaviour, an individual's emotional state, is not accounted for. Significant developments have occurred recently in what is referred to as emotive or affective computing. 1 These are systems that recognize, interpret, process, simulate and respond to human emotions with the goal of expressing the emotional state of an individual and hence adapting the computer system's behaviour in response. 2 We believe that in telecare, and indeed within the wider field of e-health generally, there is now an opportunity to supplement sensor and observational data by evaluating the way in which a person carries out specific tasks, and use the information generated to detect subtle changes in their emotions linked to their behaviour. Interesting developments which are currently taking place include emotional avatars in dementia care, technology to enhance social inclusion for the elderly in their homes and wearable systems aimed at detecting emotional state.
Emotional intelligence in humans has been defined as ‘… the ability to monitor one's own and others' feelings and emotions, to discriminate among them and to use this information to guide one's thinking and actions’.
3
Meanwhile, intelligent behaviour by machines has been studied by the artificial intelligence community for decades, and has found wide ranging applications such as in:
Planning and prediction for businesses; Medical diagnosis; Natural language processing; Character recognition; Transportation; Communications; Computer games.
Until the late 1990s, the ability of a computer to act on an emotional level with humans had not been addressed. In 1997 Picard coined the phrase ‘affective computing’ to define a machine's ability to recognize, express and potentially to have, emotions. Essentially, opportunities are presented for any system or environment that could benefit from what Pentland
4
refers to as ‘affect-sensitive’ technology and a recent article by Muir
5
suggests that ‘emotion detectors … are coming’ with special reference to ‘mood detectors’ as wearables for medical purposes.
For non-verbal communication, research has tended to focus on the face and on the recognition of the six standard emotions of fear, anger, happiness, sadness, surprise and disgust. Research is now progressing beyond the single mode of facial expression to include other modalities such as the voice and the consequent fusion of two modalities. 6 Others have fused expression data from body movement in the head and shoulders in recognition that 70% of emotional communication when talking comes from body language, from the role of the body shape in enhancing the believability of animated characters and from the full body skeletal movements in humans to predict emotional displays. The importance of adopting physiological pattern recognition into medical applications was made explicit by Picard's wide review of the potential of machine emotional intelligence. 7
Within the wider e-health field, an ability to detect, and respond to, the emotional state of an individual has potential including:
Speech The application of speech-based emotional analysis to regular communication could add significantly to the understanding, and hence interpretation, of individual behaviour patterns.
8
It is also possible that certain illnesses could be identified in this way, enabling an appropriate response to prevent possible escalation. Facial expression More than one modality can be combined or fused, as for instance facial expression and speech prosody
9
or facial expressions and hand gestures to provide an estimation of the subject's emotional state. While accessing visual information is likely to be more difficult than accessing speech, the possibility of using a visual link for communication (i.e. closed-circuit TV or webcams) with, say, family and friends may provide the necessary source data. Gestures This relates to the positioning, and changes in position, of the body.
10
In telecare, the capture of gestures presents similar problems to the capture of facial expression, i.e. a camera is required, something which users have not always been receptive to. Physiological monitoring All of the above then support the potential integration of physiological data with more conventional data forms associated with telecare.
11
Conclusions
The potential to detect behavioural change using computer-based systems is gaining credence in areas such as activity analysis, recognition of emotions and security. Thus, the way in which an individual carries out a task can be used to support the identification of the individual and their emotional state. This leads to an ability to detect and respond to a range of conditions such as dementia and Parkinson's disease, along with potentially more subtle behaviour. 12 In the context of telecare, this could support individuals who are already keyboard users, whilst being adapted to link into regular and routine tasks. While there are few applications of emotive or adaptive computing in telecare and e-health at present, we believe that there are significant opportunities to take advantage of changes to behavioural profiles. Specifically, the increased use of computers by older people will enable the integration of elements of emotive computing with features such as keyboard and webcam use to provide additional information on emotional state which, when combined with other data, affords significant opportunities for system enhancement and the identification of changes in user status, and hence of need.
Finally, there is the potential for much wider, and more general, applications within health care. For instance, could facial recognition systems linked to emotive computing be used to assess discomfort, in support of remote therapies to improve the quality, safety and effectiveness of such therapies? Could Parkinson's disease be monitored through the monitoring of speech and hand tremor? These and other related questions highlight the potential for emotive computing in telecare.