Abstract
We studied the feasibility of using a humanoid robot as an assistant in the monitoring of nursing home residents. The robot can receive alarms via its wireless Internet connection and navigate independently to the room where the alarm originated. Once it has entered the room, the robot can transmit near real time images to the staff and also open a voice connection between the resident and the remote caregivers. This way the remote caregiver is able to check the situation in the room, and take appropriate actions. We tested the prototype robot in three private nursing homes in the Finnish county of South Ostrobothnia. During the testing, 2–4 alarms were produced by each participant and there were 29 alarms in total. The robot was able to navigate correctly to the room from which the alarm was sent and open the speech connection, as well as transmit images via the wireless Internet connection. The experiments provided evidence of the feasibility of using autonomous robots as assistants to nursing home staff in remote monitoring. The response from the nursing home residents was uniformly positive.
Introduction
A nursing home is a place for people who do not need hospital care but cannot be cared for at home. In a nursing home there is a high demand for personal monitoring of the residents, especially in dementia units. 1 The rooms of the residents usually have telecare equipment, which may include personal alarm buttons or telephones that the residents can use to request assistance or to raise an alarm. When any of the sensors in the room is activated, either by the resident's action or automatically, members of staff must check the situation in the room. However, especially in the smaller nursing homes, it is difficult to arrange 24-hour service to respond to the alerts, and there is a clear requirement for systems to carry out remote monitoring. It is obvious that some alerts require the personal involvement of caregivers, although many alerts could be handled remotely if a suitable means for communication was available.
In previous research, assistive robots 2 have been used in elderly care to support nursing home staff in their daily activities. 3 The tasks carried out by the robots include improving drug compliance, 4 increasing physical exercise adherence, 5 acting as a substitute for a companion animal 6,7 or assisting in different kinds of rehabilitation tasks. 8 Cesta et al. have presented a monitoring approach in which a robot works as part of an intelligent home environment. 9
We have studied the feasibility of using a humanoid robot as an assistant in the monitoring of nursing home residents. The robot can receive alarms via its wireless Internet connection and navigate independently to the room where the alarm originated. Once it has entered the room, the robot can transmit near real time images to the staff and also open a voice connection between the resident and the remote caregivers. This allows the staff of the monitoring centre to check the resident's condition and take appropriate action. In the same way, the robot can be sent automatically to check the door area of the nursing home after a door sensor alarm, or to monitor the movements in the corridor.
Robot-based monitoring system
We used a NAO H25 robot manufactured by Aldebaran Robotics.
10
The NAO is a programmable humanoid robot 57 cm tall (Figure 1) and is capable of autonomous movement using its electric motors and actuators. It contains a variety of sensors and devices, including two cameras, four microphones, two hi-fi speakers with voice synthesizer and sonar sensors for distance estimation. Tactile sensors and the ultrasound system make it possible for the robot to avoid obstacles when walking. The robot has a wireless connection (WLAN) for communication and information transmission.
The NAO H25 humanoid robot which is equipped with camera sensors, microphones, speakers and distance sensors
Alarms
In most nursing homes there are different kinds of sensors and alarm buttons installed in the rooms of the residents. When any of the sensors in the room is activated, either by the resident's action or automatically, a radio signal is sent to a central unit in the nursing home. If there is no staff member available in the nursing home, the central unit automatically calls a monitoring centre elsewhere. In our system, the goal is to enable the caregivers, e.g. in a monitoring centre, to open communication with the resident in the room where the alert was activated. Thus the robot has access to the alert data in the central unit. When an alert is received, the robot reads the code of the room from the alert message and navigates to this target room.
Autonomous movement and navigation
To be able to move autonomously inside the nursing home or hospital, the robot uses map-based navigation. With an accurate map, the robot can move around precisely in a defined indoor environment. Provided that the robot knows its initial location, it can find its way to a particular room that needs to be checked. The doors of the rooms can be opened either by using a special tool, or by using an electrical unlocking system.
An example map of a building is shown in Figure 2a. The stand-by position of the robot is labelled A (source), and other points represent vertices B-AX. The map is transferred to a graph that contains the source (A), vertices (B-AX) and edges between them (Figure 2b). The weight of each edge on the graph is proportional to the physical distance between the points. The path finding system is based on Dijkstra's graph search algorithm
11
that solves the single-source shortest path problem. In practice, this means that the path chosen is the one whose combined weight is as small as possible. All the available paths are saved in the robot's memory as a graph-type data structure. Thus as soon as the robot receives an alarm from one of the rooms, it can automatically select the shortest path and navigate to this destination. After completing the task, the robot automatically returns to its initial position. In case of failures or errors, the robot sends an error message to the monitoring centre.
(a) A map from a nursing home in which the robot can be used. The points represent vertices in the robot's path (b) A graph based on the map. The weight of each edge corresponds to the physical distance between the vertices
Equipment
For monitoring purposes, we employed the built-in cameras of the robot. The robot has two cameras of VGA resolution (640 × 480 pixels) in its head. Once the robot has entered the room to be checked, it captures either still images or video sequences from the room interior, and transmits this information to a web server via the WLAN connection. The robot uses an internal buffer to store the voice and video to be sent, which causes a delay of not more than 5 s. This makes it possible for the caregivers to view the images in near real-time from their computer screen or mobile devices, such as a smart phone equipped with a web browser.
The microphones and speakers located in the head of the robot make it possible to establish an interactive voice connection between the patient and a remote caregiver. The caregiver can open the remote voice connection to the robot when needed.
The robot is connected to the Internet via the WLAN connection. Thus the robot can continuously send and receive data such as alerts, images and voice. A sufficient bandwidth for the WLAN connection is provided by the 802.11b protocol, which means a connection speed of 11 Mbit/s.
Alarm signal transmission
An alarm signal from the sensors or alarm buttons installed in the residents' rooms is transmitted to the central unit and thence to an alarm server in the monitoring centre. Finally the alarm server sends the alarm to the robot via email.
Image and video signals
Once an image or video sequence is captured and saved to the internal memory buffer of the robot, the robot transmits it to a web server using an FTP (File Transfer Protocol) connection. Under normal conditions, the robot captures images continuously. A JavaScript application updates the most recent image automatically on the server, and consequently a near real-time view of the robot's camera is shown on the appropriate web page. This way the images from the robot can be viewed with any terminal that is connected to Internet, and there is no need for special application software in these terminals.
Audio signals
Packet format is a standard method for audio transmission over IP networks. The voice from the robot's microphones is buffered and sent over the wireless network as an RTP (Real-time Transport Protocol) data stream. The signal can be received by any network-connected terminal that is able to decode streamed audio, such as a PC or smartphone. Similarly the voice from the terminal is sent as an RTP data stream to the robot, which receives it and plays it via its loudspeakers.
Methods
The prototype robot system was tested in two stages: first in a complex office environment and then in nursing homes.
Office environment
We carried out initial tests in a relatively complex office environment consisting of corridors and small rooms. The objective was to make the indoor navigation accurate enough for testing in real environments. The main challenges for the indoor navigation were found to be the accuracy of door position detection, and the ability of the robot to maintain its direction in long corridors. Utilizing the ultrasound sensors of the robot solved both of these problems, because these sensors could be used to detect the edges of the doors and to measure the distance of the robot from the walls of the corridor. As soon as the indoor navigation was working well, the robot was able to receive alarms from the rooms and navigate to the target room independently. Also image transmission and the voice connection worked well, provided that the WLAN connection covered the area of interest.
Nursing homes
We tested the prototype robot in three private nursing homes (A, B and C) in the Finnish county of South Ostrobothnia in January–May 2012. The homes were relatively small ones with 10–22 residents each. The average age of the residents was over 85 years, and many of them were suffering from cognitive problems related to memory, or had a very limited ability to walk. For this reason, one of the most important tasks for the staff was the monitoring of the residents. Because the testing required a member of the research group to be present in the nursing home to check that the system worked correctly, we arranged four testing days in each nursing home. A member of the nursing home staff supervised the testing, assisted by a member of the research group.
Details of the tests in nursing homes
Results
During the testing, 2–4 alarms were produced by each participant and there were 29 alarms in total. The robot was able to navigate correctly to the room from which the alarm was sent and open the speech connection, as well as transmit images via the Internet connection. A member of the nursing home staff was present in the monitoring centre, and communicated with the resident in the room via the robot. A limitation in the voice connection was found to be the background noise, which sometimes disturbed the discussion between the resident and caregiver. To reduce the effect of this noise, the robot could move close to the resident in the room. In addition, filtering of the audio signal was tested and found to reduce the effect of the noise.
Discussion
Robotic applications are being adopted for several kinds of assistive tasks in elderly care. Socially assistive robots can be applied in a wide range of tasks from social interaction and companionship to drug delivery and monitoring. We have investigated a robot-assisted monitoring application for elderly care. Obviously the tasks described in the present study could also be carried out by using fixed cameras and telephone systems installed in the rooms of the residents. However, that approach would impinge on the privacy of the residents, the preservation of which is a key aim in the home-like living environments provided by nursing homes. Even if the fixed cameras were only used in case of an alarm, their existence might be ethically problematic. Moreover, the field of view of a fixed camera in a room is limited, and it cannot cover bathrooms, for example. Consequently, a ‘friendly’ looking robot that comes to the room to check the situation and provides a connection to a caregiver is preferred. In our experiments in the nursing homes, the residents showed a very positive attitude towards the robot.
The robotic monitoring system was implemented by using a commercially available robot. The programming of the robot is relatively straightforward, and it can be taught to perform different tasks in the nursing home, not only monitoring as described in the present study. The other parts that make up the monitoring system consist of standard telecare devices that are already in routine use in elderly care. The present study shows that it is possible to make them work together with the robot. In general, we tried to design the system to be largely device independent. For this reason, the voice connection and image transfer were based on standard Internet protocols. This makes it possible to use any network-connected terminal to establish the voice connection to the robot and view the images provided by it.
Our experiments revealed that there are several areas in which the humanoid robot needs improvement. One of them is battery life: in the case of our robot, it was about one hour before recharging was required. It should be possible for the robot to connect itself to a charging station automatically whenever it is in standby mode. Currently our robot does not have this possibility. Another drawback is the robot's relatively small size, which may cause some problems in accessing rooms and avoiding obstacles like thresholds. The robot may fall if it collides with an obstacle, although it can stand up by itself. The walking speed of the humanoid robot is also quite slow. Hence the time required to reach distant rooms may be too long in case of an emergency. This limits the working area of the robot in larger nursing homes.
Conclusions
The present study shows that a commercially available humanoid robot can be used as a tool in remote monitoring. By using its onboard instrumentation, it can autonomously perform the checking of the resident's room and provide a remote caregiver with near real-time images and a voice connection to the room. The robot system can operate with existing telecare equipment installed in the nursing home, and therefore its future deployment would not require additional equipment. Alarm transmission for the prototype system was relatively straightforward, although for future commercial work, reliable software for communication would be required.
The prototype system was successfully tested in both a complex office environment and three nursing homes, where the robot successfully performed monitoring tasks. The experiments provided evidence of the feasibility of using autonomous robots as assistants to nursing home staff in remote monitoring. The response from the nursing home residents was uniformly positive.
Footnotes
Acknowledgments
The authors thank the Technology Development Centre of Finland (TEKES) for financial support.