Network Distributed Monitoring System Based on Robot Technology Middleware

1. Introduction

Many service robotic systems have been developed in order to improve care cost and the QoL of the elderly people in the population-aging society. Our HARSP (Human-Assistance Robotic System Project) project has been developing a network distributed Human-Assistance Robotic System since 2000 (Jia, S.; Hada, Y. & Takase, K. 2004; Jia, S. & Takase, K. 2001). We developed hardware base, key technologies, and implemented CORBA (Common Object Request Broker Architecture) application servers to provide some basic services to aid the aged or disabled. Improvements of the developed system has been doing from practical viewpoint. In order to enable a remote users to get a better understanding of the state of the robots working and the state of the aged or disabled, we developed network distributed monitoring system to transmit media streams in real time to improve interaction between a remote user and local robot system. Home robot integration system and network monitoring system using QuickCam Orbit cameras were developed and demonstrated from June 9 to June 19 at the 2005 World Exposition, Aichi, Japan. Robot Technology Middleware (RTM) was used in the developed system. RT component has standard interfaces and hides its implementation, and has network transpararency and efficient communication channel. RT components of the system can be implemented by different programming languages, run in different operating system, or connected in different networks to inter-operate. We develop robotic functional elements as “RT software component”, which makes it easy to create new robot system by re-using existing modules and to lower the cost of development of new robot application. In order to enhance the performance of the network distributed monitoring system, we developed IEEE-1394 cameras RT control components in order to make application and system integration easier. For different vendors of IEEE-1394 cameras, system supplies classes method easy to implement them as RT components. Being different from QuickCam Orbit cameras, IEEE-1394 cameras were widely used in robot field and it has high performance and high resolution. This paper introduces the architecture of the developed system and the results of experiments.

2. Robot Technology Middleware

Robot Technology Middleware was developed by Agency of Industrial Science and Technology of Japan (AIST, RTM) to promote application of Robot Technology (RT) in various fields. RTM aims to modularize robotic functional elements as “RT software component”, which enables the system to be extended and integrated easier for a new system or new applications. In order enable system to be language independence, operating system independence, RTM has been developed based on CORBA. RTM used OminORB 4.0.5 to implement framework. OmniORB is a robust, high-performance CORBA 2 ORB, developed by AT & T. It is one of only three ORBs to be awarded the Open Group's Open Brand for CORBA. This means that omniORB has been tested and certified CORBA 2.1 compliant. omniORB implements the specification 2.3 of CORBA.

RTM includes Standard RT Components, Standard RT Services, RT Components Framework and RT Library (RTM::RtcBase, RTM::InPortBase, RTM::OutPortBase and so on). RT component includes RT body object, Inport object and Outport object. RT component's features are such as it has activity, interface for connecting output and input data stream and administrative function of component-objects. RT component Manager manages all RT component, controls activity thread, adds or deletes the object of RT components and does instance of component. Registering all RT components' names and object references to name server using CORBA name service enables the other host to access them in different network (Ando, N. & Suehiro, T. et al. 2004).

3. Network Monitoring System Using QuickCam Orbit Cameras

Home robot integration system and network monitoring system using QuickCam Orbit cameras were developed and were demonstrated from June 9 to June 19 at the 2005 World Exposition, Aichi, Japan (Hada, Y.; Jia, S.; Takase, K.; Gakuhari, H.; & Ohnishi, T. 2005). In order to enable a remote user to get a better understanding of the local environment of mobile robot working, it is necessary to receive and transmit media streams in real time to improve interaction in network distributed service robotic system. Network monitoring system using QuickCam Orbit cameras was developed to visualize home robot integration system working, the state of the age and disabled, and to enable the remote user to know what is going on in local environment. The QuickCam Orbit cameras were used in our system with high-quality videos at true CCD 640×480 resolutions and automatic face-tracking and mechanical Pan, Tilt and face tracking feature. It mechanically and automatically turns left and right for almost a 180-degree horizontal view or up and down for almost 90-degree top-to-bottom view. Its maximum video frame rate is 30 fps (frames per second) and works with both USB 2.0 and 1.1.

We implemented video stream RT component based on RT Middleware, and OmniCORBA IIOP is employed as message communication protocol between RT component and requester. Booth for demonstration of our developed robotic system is approximately 4.5 × 5 m², so two QuickCam Orbit cameras were set up in the environment, and they can overlook the area the omnidirectional wheelchair and service robot moves in by adjusting the mechanical Pan and Tilt of the cameras. The structure of the developed RT video stream monitoring system using QuickCam Orbit cameras was shown in Fig. 1. In addition, we developed a graphic user interface (GUI) for the video stream system that provides a remote video stream, a camera zoom and pan-tilt adjustment, and a chat function that allows a remote user to communicate with a local user.

OPEN IN VIEWER

Fig. 1.

RT video stream functional component

In order to enhance the performance of the network distributed monitoring system, we developed IEEE-1394 cameras RT control components in order to make application and system integration easier. For different vendors of IEEE-1394 camera, system supplies method easy to implement RT component. A new robot system application can re-use existing modules thus lowering the cost of development. The developed IEEE-1394 cameras RT control components allow other applications in different network to communicate directly with them. Being different from QuickCam Orbit cameras, IEEE-1394 cameras were widely used in robot field and it has high performance and high resolution. Fig. 2 illustrates the configuration of the enhanced network distributed monitoring system.

OPEN IN VIEWER

Fig. 2.

Configuration of the enhanced system with IEEE 1394 cameras

A desktop personal computer running the Windows XP operating system (2.4GHz, Pentium4) with IEEE-1394 buses was used in the system for cameras control. IEEE-1394 cameras (Flea) as examples were used in the network monitoring system to take video of the environment and multi-cameras can connected to one computer to be controlled. IEEE-1394 cameras' RT components (RTM-0.2.0) were implemented and were controlled to start to take video or to set up the configuration of cameras according to the commands from network GUI. Network GUI was also developed and was run on the other computer (Windows XP operating system 2.4GHz, Pentium4) to enable the user to operate the real-time video stream from IEEE-1394 cameras. User can start, stop the IEEE-1394 cameras' RT components thus starting or stopping the display of video stream and setting up the configuration of cameras. OmniCORBA 4.0.5 is employed as communication architecture between the network GUI and cameras control computer.

2.2. IEEE-1394 Camera Control RT Component

Figure 3 illustrates the structure of the IEEE-1394 camera control RT component. It has one Inport for camera property for camera's setting, and Outport1 for image data and Outport2 for status data for IEEE-1394 camera control.

OPEN IN VIEWER

Fig. 3.

IEEE-1394 camera control RT component

IEEE-1394 camera control function includes: 1).

IEEE-1394 camera “image get” function: get the image the user selected by IEEE-1394 camera and change the format (RGB) to display in Windows, output it to the OutPort1 of RT component.

2).

IEEE-1394 camera “set property” function: set the camera's property such as size, frame rate, and mode of the image selected by the user.

3).

IEEE-1394 camera “output status” function: output the status of camera to the OutPort2 of the RT component, inform the user this information via GUI. IEEE-1394 camera status includes IEEE-1394 camera's property, IEEE-1394 camera's initialization success or failure, and IEEE-1394 camera control error.

2.2. Structure of classes of IEEE 1394 camera control RT component

Fig. 4 illustrates the structure of the classes of IEEE 1394 camera control RT component. The IEEE-1394 camera's control function classes includes:

1).

RtcBase: OpenRTM-aist-0.2.0 component base class.

2).

InPortBase: OpenRTM-aist-0.2.0 InPort base class.

3).

OutPortBase: OpenRTM-aist-0.2.0 OutPort base class.

4).

InPortAny<TimedUShortSeq>: InPort template class.

5).

OutPortAny<TimedUShortSeq>: OutPort template class.

6).

RtcManager: RT component management class.

7).

IEEE1394CameraRTC: IEEE1394 camera control RT component class.

8).

IEEE1394CameraComp: IEEE1394 camera control RT component main class.

9).

FleaCameraControl: IEEE1394 camera operation class.

OPEN IN VIEWER

Fig. 4.

Classes of IEEE 1394 camera control RT component

2.3. GUI for Network Distributed Monitoring System with IEEE 1394 cameras

Graphic User Interface was designed for the network distributed monitoring system that provides a remote video stream, camera selection, capture type (static or stream), frame rate (30fps, 15fps, 7. 5fps and so on) of camera component, format (640 × 480, 320 × 240) of camera component's image data. The status data such as the starting time, which camera was connected or the other message of the system was displayed in log part. GUI also displays the current frame rate (from 0fps to 40fps) of video stream.

Fig. 5 illustrates the developed GUI for IEEE 1394 camerea montoring system. User pushes “Load Button”, selects a room name, camera name, a frame rate and a format the user hopes, then pushes “Start Button”, the video stream the user selected will start. When user want to stop display or to change the camera, he or she just pushes “Stop Button”, and selects the other camera's name, the system will stop the video stream transition of the camera now and start the new camera the user selected. Because we implemented IEEE1394 cameras as RT components and registered them to naming server using CORBA name service in advance, the user can access any cameras to get video stream even cameras are connected in the different computer or in different network.

OPEN IN VIEWER

Fig. 5.

GUI for IEEE 1394 camerea montoring system

Home integration robotic system was demonstrated from June 9 to June 19 at the 2005 World Exposition, Aichi, Japan. Network distributed monitoring system was also demonstrated using QuickCam Orbit cameras in order to know the state of robotic system's working and the state of the aged or disabled. The video server was run on Windows 2000 Professional (1.9GHz, Pentium4), and the video components were run on Windows XP (2.4GHz, Pentium4). The average frame rate is approximately 16.5fps (320×288). In order to extend the function of the network distributed monitoring system, we also developed IEEE-1394 cameras RT components. Being different from QuickCam Orbit cameras, IEEE-1394 cameras were widely used in robot field and it has high performance and high resolution. Some performance experiments have been done using the improved monitoring system to monitor the service robotic system working. IEEE-1394 cameras for monitoring the environment were connected to the computer running on Windows XP (2.4GHz, Pentium4), and GUI was run on the other same specification Windows XP (2.4GHz, Pentium4). Two computers were connected in a LAN (1Gbps), the average frame rate of video transmission is approximately 15.6fps.

Fig. 6 illustrates some images of experiments using the improved monitoring system with IEEE-1394 cameras to monitor the service robot system. By using designed GUI, user can push “Load Button” to select a room name and camera name, a frame rate, a format of image, then push the “Start Button” to start video display. The user can access any cameras to request video stream without their information about the language of implementation, operating system or network. Fig. 6(a), (b) are different video from different camera in the same room. Fig. 6(c), (d) are the video from the camera in corridor. When the user push “Stop Button”, the display of the video will stop.

OPEN IN VIEWER

Fig. 6.

Experiments of network monitoring system with IEEE 1394 cameras.

This paper presented the architecture of the developed network distributed monitoring system for human-assisted robot to improve interaction between remote users and local robotic system. Home integration system and network monitoring system using QuickCam Orbit cameras were demonstrated from June 9 to June 19 June at the 2005 World Exposition, Aichi, Japan. In order to enhance the performance of the network distributed monitoring system, we also used IEEE-1394 cameras to develop IEEE-1394 cameras RT components, which makes system applied widely. Some experiments have been done for the improved network distributed monitoring system, and experimental results verified that the enhanced system with IEEE1394 cameras improved the flexibility of system. Because Robot Technology Middleware was used in the developed system, the system has high scaling and inter-operating ability that allows the users access RT components written in different languages, run in different operating systems, and connected to different networks. For future work, we will develop the other functional robot system components as RT components such as RFID RT object recognition component facilitating object recognition for robot or RT localization component for localizing mobile robot.