Abstract
Humanoids have increasingly become the focus of attention in robotics research in recent years, especially in service and personal assistance robotics. This paper presents the application developed for humanoid robots in the therapy of dementia patients as a cognitive stimulation tool. The behaviour of the robot during the therapy sessions is visually programmed in a session script that allows music to play, physical movements (dancing, exercises, etc.), speech synthesis and interaction with the human monitor. The application includes the control software on board the robot and some tools like the visual script generator or a monitor to supervise the robot behaviour during the sessions. The robot application's impact on the patient's health has been studied. Experiments with real patients have been performed in collaboration with a centre of research in neurodegenerative diseases. Initial results show a slight or mild improvement in neuropsychiatric symptoms over other traditional therapy methods.
1. Introduction
One field of growing interest in robotics is humanoids. Prototypes such as the Honda Asimo or the Fujitsu HOAP-3 are the basis for many research efforts, some of them designed to replicate human intelligence and manoeuvrability. Their appearance, being similar to people, facilitates their acceptance and natural interaction with humans as a personal assistant in, for instance, the field of service robotics. As a representative sample, the functionality achieved in the Asimo humanoid has progressed significantly in recent years, allowing it to run, climb stairs, push carts and serve drinks.
Neurodegenerative dementia is a disease that progressively deteriorates brain functionality. One of the most common symptoms of dementia is memory loss. In addition, patients usually lose the ability to solve problems or control their emotions and present changes in personality and normal behaviour. Over time, people with dementia are unable to properly perform the basic activities of daily living such as maintaining personal hygiene or food. Estimates point to the fact that by 2016 there will be 26.6 million people worldwide with Alzheimer's disease, and this figure will be three times bigger by 2050 when Alzheimer's will affect 1 in 85 people of the total world population. In addition, 40% of them will be in an advanced state of disease, requiring a level of care that involves high consumption of resources [1].
While there is no causal cure for the disease, palliative medication and nonpharmacological therapy are the only ways patients can improve symptoms and slow down its progression.
Nonpharmacological therapy focuses on strengthening activities mentally, physically and emotionally. Such actions seek to maintain the functional capacity of the person, while ensuring her levels of quality of life and autonomy. Animal therapy has also shown good results, especially with the elderly living alone. However, this is not always possible as sometimes the entrance of animals into residences for the elderly is not allowed for health and safety reasons. Other times it is the cost of maintaining these animals and the care they need which precludes their presence in the residence. Another issue to consider is that the therapeutic interaction at the cognitive level needed in older people with dementia is not resolved with the presence of animals in the patient's environment [2].
Regular therapy includes several sessions per week with a human therapist monitoring a group of patients. The therapist asks patients simple questions, tells them stories, talks to them, interacts with them, hugs them, suggests games, riddles or guides them while doing physical exercises. These activities pursue the cognitive, affective and physical stimulation of the patients.
In this paper we describe the use of a humanoid robot as a cognitive stimulation tool in the therapy of dementia patients. Several software modules have been programmed to generate the robot behaviour in the therapy sessions. Three types of robotherapy sessions have been developed: physiotherapy, music and logic-language sessions. The robot and the developed software have been used in a pilot study with real patients to evaluate the feasibility and usefulness of robots in dementia therapy.
The remainder of this paper is organized as follows. The second section presents some works with social robots and their use in dementia therapies. The third section explains all the software developed for this humanoid application, including some tools designed to visually program the robot behaviour for the robotherapy sessions. The fourth section describes the experiment performed with real patients to measure the impact of this robotic tool on their health and some preliminary results are presented. Finally, some conclusions are summarized.
2. Related Works
Interest in social robots is growing as one of the upcoming application fields of next generation robots, for instance, gaming platforms [3], personal assistants, nursing robots [4] or assistive tools in rehabilitation [5].
In the past five years, several projects have been initiated with the therapeutic use of social robots [6] as reasonable substitutes for animal therapy in people with dementia. Robots do not involve the responsibility or the need for an animal facility and their sensors can respond to environmental changes (movements, sounds, etc.) simulating an interaction with the patient [7]. At the same time, they provide the opportunity to monitor patients and perform cognitive therapy, unlike animal therapy [8]. Other potential benefits of therapy with robots are that it has no secondary effects (like drug therapy) and does not require specially trained personnel (as opposed to the other therapies such as music therapy, pet therapy, etc.).
The seal-shaped Paro robot has been used in dementia therapies [9] with positive results.
Broekens et al. published in 2009 [10] a systematic review analysing the literature on the effects of social robots in the health care of the elderly, especially in the role of company for the patient. It is noteworthy that all studies were undertaken after 2000, indicating the novelty of this research area. Most studies have been conducted in Japan, Southeast Asia and the U.S. [11]. The main results of analysis of these studies are:
Most of the elderly like robots.
The shape and material of the robot influence the acceptance and the effect of the robots.
Improved health by lowering stress levels (measuring stress hormones in urine) [12] and increased immune system response [13].
Improvement of humour (through surveys and the evaluation of facial expressions).
Decreased sense of loneliness (using scales measuring loneliness).
Increased communication (measured by the frequency of contact with robots and family).
Remembering the past (especially with a robot shaped as a baby).
Some studies indicate that the use of robots helps to reduce the severity of dementia in some patients.
3. Robot software for robotherapy
We created a programming framework, named Behaviour-based Iterative Component Architecture (BICA) [14], to develop autonomous applications for our humanoid robots. It has been used in research for several years around the RoboCup environment, in teaching at robotic courses at Universidad Rey Juan Carlos and it has also been used for robotherapy.
We have developed several software pieces for the use of humanoids in therapy. The robot behaviours in therapy sessions are described mostly as a sequence of basic movements, music or text playing and light turning on-off operations. File format syntax has been developed to store these behaviour descriptions - they are called session scripts.
Some specific components inside BICA have been developed, like one that runs session scripts or another that provides access to robot lights from the application software. In addition, some tools have also been created: a session script generator that allows easy and visual “programming” of robot behaviour in therapy sessions, and the session monitor tool that helps the human therapist to control the session progress. They are all described in this section.
3.1 BICA Architecture
The software of our humanoid robot is organized with a behaviour-based architecture. It is implemented in component-oriented software architecture, BICA, programmed in C++ language. Components are independent computation units which periodically execute control iterations at a pre-configured frequency.
Every component has an interface to modulate its execution and to retrieve the results of its computations.
Behaviours in BICA are defined by the activation of perception components and actuation components. Actuation components take movement decisions, send commands to the robot motors, or locomotion system, or activate other actuation components. They run iteratively to periodically update their outputs. Perception components take data from the robot sensors or other perception components and extract information. They basically provide information to the actuation components. The output of a perception component is refreshed periodically and can be read from many other components in the system.
Not all the perception capabilities of the robot must be active at the same time, consuming computing resources. Even more, the whole set of behaviours that the robot is able to perform is not suitable for dealing with the current situation. Only a subset of those behaviours and perception units are relevant to the current situation. In BICA, each component can be activated and deactivated at will, so it remains inactive until the situation demands it - when another component activates it. Typically an actuation component activates the perception components it requires and activates the child actuation components (if any) that implement its control decisions. This activation chain creates a dynamic component tree to cope with the robot's current situation. Figure 1 shows a component activation tree with both perception and actuation components.
Behaviour in BICA composed by actuation and perception components
Beyond being a framework to integrate perceptive and actuation capabilities for autonomous behaviours, the BICA architecture also includes components that provide access to the basic sensors and actuators of the robot, a Hardware Abstraction Layer (HAL) for robot applications. BICA is built on top of Naoqi, the manufacturer middleware, and offers this HAL as a set of object method invocations. For instance, the Body component provides access to the motion capabilities, both the legged locomotion and the arm movements. The Perception component provides access to the camera images. The LED component provides access to several lights on the robot head and chest, which can be turned on and off from the application software. The Head component provides access to the neck of the humanoid, allowing the head to be rotated horizontally or vertically. The Music component provides the capability of playing sound files which has been specifically developed for the robotherapy application. The stories, questions, songs involved in therapy sessions are stored as sound files and played back using this BICA component.
The behaviour-based organization of the software of the robot in BICA allows a modular development of robot functionalities, with new components to accomplish new robot tasks or to perceive new associated stimuli.
Beyond the humanoid behaviour in the robotheraphy application, this architecture has also been used for the programming of humanoid behaviours in other scenarios like the RoboCup competition. We have developed several perceptive and actuation components for the robot soccer player inside the RoboCup Standard Platform League. Some actuation components were programmed as PID reactive controllers and others as Finite State Machines, depending on the complexity of the behaviour.
3.2 Session script generator
The robot behaviour set required for robotherapy application is smaller than for other fully autonomous applications like the robotic soccer in RoboCup. In essence, the robot behaviours in therapy sessions are described mostly as a sequence of basic movements, music or text playing and light turning on-off operations. They are usually launched together as the robot may be, for instance, playing a song and dancing at the same time.
A high level language has been developed to store these behaviour descriptions. They can be stored in text files following a given syntax and read from them - they are called session scripts. The language includes three basic instructions: move, music and light. Two or three basic operations of different type can be grouped together, in group instructions, to be executed simultaneously. The robot behaviour is a sequence of these basic or group instructions. In the script some synchronization points can be included to wait for the termination of all the basic instructions inside a group. In addition, the wait instruction causes the robot to stop execution until the human therapist provides the continue order, striking one button on the robot body or by using any monitoring tool. This allows the human therapist to control the session progress.
The scripts are generated and stored in text files. Their contents are designed by medical doctors and health assistants, attending to the desired stimulation in the dementiar's patients. At the beginning they were created by directly editing text files, however more recently we have developed a graphical tool, the session script generator (Figure 2), that allows a fast and visual creation of these scripts.
Session script generator
3.3 Movie component
One specific component has been developed inside BICA for the robotherapy application, the Movie component. It accepts session scripts as input and runs the corresponding orders to robot motors and actuators, at the proper timing, unfolding the specified robot behaviour. It uses several HAL components available in BICA, like the Body, LED, Music and Head components, as shown in Figure 3.
Movie component in BICA runs session scripts on board the robot
For dancing the robot has previous descriptions of its movements. They are stored as single files following a given syntax, and they can be referenced in the scripts run by the Movie component. Those movement files include the position of all robot joints and the right time for each one. For singing or story telling the corresponding song and text are stored as sound files, and they can also be referenced in the session script.
3.4 Session monitor
The therapist needs a way to communicate with the robot, for instance, to start a robotherapy session, to stop its execution while the patients answer one of the robot questions, to repeat any script step, among other tasks. The basic interface with the real robot was the set of buttons on its feet and chest. At the beginning these buttons were used, but we developed two session monitor applications to allow an easier way to control the robot.
The first session monitor is an application running on a regular computer. It offers a Graphical User Interface with sliders, selectors, visual buttons, etc., as shown in Figure 4. This allows the teleoperation of the robot body and head, so that the robot can approach the patients at the beginning of the sessions, for instance. It can be operated from an external computer or used in conjunction with a Wiimote. This Wiimote device is more convenient than the regular screen, keyboard and mouse configuration. In this case the session monitor reads the therapist's orders from the Wiimote buttons and accelerometers using Bluetooth.
Session monitor at a regular computer
In order to improve the tool usability, a second session monitor has been created that runs on mobile devices like Android tablets or smartphones (Figure 5). In using this neither an extra computer nor Wiimote is required, just the robot and the tablet or smartphone. With this monitor, the human therapist has full control of the progress of the therapeutic session.
Session monitor at a tablet
Interaction between different BICA components is performed as a method of invocation of other component objects. A specific module has been developed for communication between BICA and the software outside the robot, for instance, the communication between these session monitor tools and the Movie component on board the robot. This module and the session monitor use ICE as communication middleware.
4. Experiments
The platforms available for this project were initially three: the robot seal Paro, the Aibo robot dog and the Nao humanoid. One of them needed to be selected for the real experiments.
The mobility of the robot seal Paro is mainly confined to its head (its eyes move). It also produces sounds that simulate those of a baby seal.
The robot dog Aibo and the Nao humanoid offer much more functionality: they are both also nice to look at, they walk, move their head, have lights and make sounds. The larger size of humanoid makes it more visible than the robot dog. In addition, the humanoid robot is most useful in physical therapy, as it can perform physical exercises that can be directly mimicked by patients. This is a key issue.
From a technical point of view, the development of software for the robot seal Paro is complicated because it is a closed platform. Our group has extensive experience in the programming of the other two platforms in the RoboCup environment. One difference in favour of the humanoid robot Nao is the availability. Although we have several Aibo robots and it was a bestselling commercial robot, since 2008 its manufacturing has been discontinued. Currently our group is participating in the RoboCup with the Nao humanoid, and the software architecture developed to control the robot in this environment, BICA, is general enough to host the robot software for the robotherapy application.
The Aibo and humanoid robots were tested with real patients to see their acceptance (Figure 6). The Nao robot was presented to a group of 20 patients with differing severity of dementia. The humanoid robot was accepted by most of them: 80% showed a very positive attitude, 15% did not react and 5% (one person) showed aggression towards the robot (and also to therapists and psychologists). Most patients identified him as a child and tried to talk to him.
Aibo robot acceptance test with dementia patients
For all these reasons, the Nao humanoid was the selected platform. Nao robot is a fully programmable humanoid robot. It is equipped with a x86 AMD Geode 500 MHz CPU, 1 GB flash memory, 256 MB SDRAM, two speakers, two cameras (non-stereo), Wi-Fi connectivity and Ethernet port. It has 25 degrees of freedom. The operating system is Linux 2.6 with some real-time patches. The robot is equipped with an ARM 7 micro-controller allocated in its chest to control the robot's motors and sensors, called DCM.
4.1 Therapy sessions
The therapy session contents have been designed by therapists specialized in the disease of dementia. The robotics work focused on developing the software required so that these sessions can be carried out with maximum similarity to how they were conceived. We developed all the required software components, sounds and robot movements, and proposed new tools to be used and evaluated.
We performed cognitive therapy and physiotherapy sessions, 2 days/week during one month with a humanoid robot in a group of 13 patients (Figure 7). Evaluation at baseline and follow-up was carried out with scales to detect apathy, quality of life and dementia severity. Most of the patients had moderate-severe dementia (Figure 8), mean age 83.2 years (range: 74–91) and 92% were women.
Real session with dementia patients using a humanoid robot
Global deterioration scale of the group involved in the first phase
Each session took between 30 and 45 minutes, and was recorded by two cameras. We designed four types of sessions: language, music therapy, storytelling and physiotherapy. Cognitive therapy included music therapy, playing activities and language sessions. In the language sessions the robot asked about numbers, days of the week and set riddles and questions aimed at cognitive activation. In the music therapy sessions the robot combined basic questions related to popular songs. Physiotherapy sessions consisted of a set of exercises that the robot explained and performed: movements of arms, head, torso and walking exercises. In storytelling sessions the robot told a story, moving itself and turning its lights on at the same time, there was no direct interaction with patients.
The sessions for patients with severe dementia cannot be structured as those for people with moderate dementia because they are unable to maintain attention long enough to be effective. For them we designed a set of activities to be carried out by the robot: walking towards a patient, “looking at” her face, making sounds of animals, etc. - these actions seemed to improve their apathy. Some of these activities (walking towards a patient and “looking at” her face) were also applied during the sessions with the rest of the group, also improving their responses. To carry out these activities, we extended the software and robotic tools to be easily managed by the therapists. The Wiimote extension of the session monitor and the tablet-based session monitor has been developed after observing in the experiments the autonomy needs of the therapists.
Some preliminary medical results have been presented in medical forums and are better explained in [15]. All scales showed a trend to improvement in neuropsychiatric symptoms, apathy and quality of life, although Wilcoxon test showed no significant statistical differences between baseline and follow-up. Patients accepted well the robot and participate as actively in therapy sessions with robot as in the regular sessions.
This pilot study showed that a clinical study using robots for cognitive therapy in dementia institutionalized patients is possible. Currently we are using robots as a new tool for dementia therapists in a pilot clinical assay to discover the effect of this new non pharmacological treatment compared to habitual treatment. We involve more than a hundred patients, use a control group to compare and the evaluators are blind to the therapy.
5. Conclusions
In this paper we have presented a cutting edge application of humanoid robots in the therapy of dementia patients.
We have developed several software pieces to support this application. First, our BICA software architecture integrates all robot perceptive and actuation capabilities. Second, a software module helps to visually generate “session scripts”. These session scripts are simple descriptions of robot behaviour sequences during the therapeutic sessions. They involve music playing, movements and light generation capabilities onboard the humanoid. They have been created with the knowledge and support of medical experts, and are stored in single files. Third, a software module inside BICA runs the session scripts on the real robot. Fourth, a monitor module allows the human therapist to control the script execution, proceeding with the next behaviour, repeating steps, etc., and so modulating the session development. Two different monitors have been developed, one running on a regular PC and another one running on an Android tablet. The Wiimote device support has also been incorporated for easy robot control by the therapists, improving their autonomy.
Four kinds of sessions have been prepared and performed: storytelling, music therapy, physiotherapy and logic-language therapy. In music sessions, the robot plays songs from the years when the patients were young, trying to stimulate their emotions. In physiotherapy sessions, the robot performs several physical exercises with the intention of being repeated by the patients. Logic-language therapy is based on several simple questions to cognitively stimulate the patient responses.
The preliminary medical results on real patients with moderate dementia are promising. Their neuropsychiatric symptoms tend to improve over those of patients following classic therapy methods, but further research is required. Surprisingly the robot captures the attention of the elderly due to its human shape, its movements and music capabilities.
We are working on extending the direct interaction between the patients and the humanoid robot. For instance, the real patient showing coloured cards to answer questions set by the humanoid. Also we are programming the robot with more autonomous behaviours like face tracking or people following.
Footnotes
6. Acknowledgments
This work was supported by the project S2009/DPI-1559, RoboCity2030-II, from the Comunidad de Madrid, by the project PI10/02567 from the Spanish Ministry of Science and Innovation and project 231/2011 from IMSERSO. The authors also want to acknowledge the collaboration of the patients, their relatives, the therapists and evaluators, in particular Luis I. Casanova, Sara Saiz, Cynthia Prez, Emma Osa, Elena Ortega, Ana Casarrubio, Cristina Martín and people from the UMA group.