Abstract
The paper presents a control system for mobile robots in distributed life science laboratories. The system covers all technical aspects of laboratory mobile robotics. In this system: (a) to get an accurate and low-cost robot localization, a method using a StarGazer module with a number of ceiling landmarks is utilized; (b) to have an expansible communication network, a standard IEEE 802.11g wireless network is adopted and a XML-based command protocol is designed for the communication between the remote side and the robot board side; (c) to realize a function of dynamic obstacle measurement and collision avoidance, an artificial potential field method based on a Microsoft Kinect sensor is used; and (d) to determine the shortest paths for transportation tasks, a hybrid planning strategy based on a Floyd algorithm and a Genetic Algorithm (GA) is proposed. Additionally, to make the traditional GA method suitable for the laboratory robot's routing, a series of optimized works are also provided in detail. Two experiments show that the proposed system and its control strategy are effective for a complex life science laboratory.
1. Introduction
In recent years, there have been major advances in laboratory automation for life science processes [1]. The developed advanced systems used in life science automation can be classified into three main kinds, referring to their purposes: complex robot-based systems [2–3]; automated analytical and measurement systems [4–5], and laboratory information management systems [6]. Among these systems, the most widely known are laboratory robots. An example of a laboratory robot from the Centre for Life Science Automation (Celisca), Germany, is shown in Figure 1.
A laboratory robot system for life science automation [1]
Within laboratory robotics, there are also two sub-categories: desk robots and mobile robots. Generally, the desk robots are developed to aspirate/dispense liquid samples from and to plates or else to move plates between instruments while the mobile robots take charge of the necessary long-distance transportation for the distributed workbenches. The desk robots are used for specific tasks, and so their movement paths are relatively fixed. Compared to the desk robots, the mobile robots are more complicated due to their dynamic working environment. Nowadays especially, more and more mobile robots are being employed in life science laboratories [7], and their cooperation with other existing automated systems and facilities is a considerable issue. Indeed, in considering this issue, the construction of a smart common control system between the mobile robots and the higher laboratory Process Management System (PMS) is an effective option. Distinct from other independent control systems, this kind of system will focus on integrated performance with other existing automatic modules. It will determine the final path for a transportation task from a higher PMS and distribute the selected path to an available mobile robot. At the same time it also should meet the technical requirements of indoor localization [8], a communication network [9], path-planning [10] and collision avoidance [11]. From the previous references, it can be seen that many systems have been proposed for any given upper aspect, but there is still no published paper which directly demonstrates a mobile robot control system for a big life science laboratory including all of the upper aspects. In this study, a completed control system for mobile robots in a real life science laboratory is presented.
2. Architecture of the System
The architecture of the proposed system in this study is shown in Figure 2. The system comprises a laboratory Process Management System (PMS), a Robot Remote Centre (RRC), mobile robots (DrRobot H20), Robot Onboard/Boarding Centres (RBCs) and Robot Arm Components (RACs).
The architecture of the proposed system
The control strategy of this proposed transportation system can be explained as follows: (a) the highest level is the Process Management System (PMS), which takes charge the life science process control. It will present mobile transportation tasks while considering the schedules of different life science modules. Once a real-time mobile task is needed, the PMS will send a remote command to the robot control system; (b) there are two executing centres in the robot control system, the RRC and the RBC. The RBC will be run in the robot on-board PC, which controls all the hardware modules inside the robots (including motion, the arm, power, localization, the image, etc.). The RRC will be installed in an independent super PC that manages robot path planning, robot status monitoring and communication with both the PMS and the RBCs. Once the RRC receives a distributed transportation task from the PMS, it will select an appropriate mobile robot and give it an executable path. When the RBC of a robot gets a path from the remote RRC, it will control the robot hardware modules to execute the received path; and (c) the arm manipulation is controlled by the RACs in this transportation system. When the robots arrive at the expected positions (including the starting positions and the destination positions), the RBCs on board will activate the integrated RAC components to execute the relevant arm actions (including the grasping and the placing). Based upon this strategy, the data flows in the system complex, as given in Figures 3, 4 and 5 [12].
Dataflow among the robots, the RBCs and the RRC [12]
Dataflow between the RRC and the PMS
Dataflow between the RBC and the RAC
As shown in Figure 3, the dataflow among the robots, the RBCs and the RRC can be explained as follows: (a) the hardware data of the robots will be measured by their corresponding RBCs, which include the parameters of their robots' indoor positions, powers, ultrasonic sensors, IR sensors, motor encoder information, camera video and arm and head joint coding values (see Figure 3, Tag Robot Hardware Data). This data will be displayed in the sensor monitoring GUI in every RBC automatically if the connections between the robots and their RBCs are available; (b) the original hardware data from the robots will be filtered and sent to the RRC. The filtered data only retains the parameters of the robot positions and powers (see Figure 3, Tag RBC Data) which will be adopted to do transportation path planning and robot selections by the RRC; (c) when the RRC is given a transportation task by the PMS, the RRC will actively send the relevant commands (see Figure 3, Tag RRC Commands → Waypoints) to get the required filtered data from all of the RBCs; (d) after the necessary computational steps, the RRC will select the most suitable H20 robot and send the RBC of the chosen robot the best path with a sequence of waypoints (see Figure 3, Tag RRC Data); (e) when a RBC receives a path from a RRC, it means that the RBC has been asked to control the corresponding mobile robot to execute the transportation task. The RBC will map the received path numbers to the parameters and commands, which the robots can understand. After the fast mapping process, the RBC will send the results to the robot hardware modules for transportation movements (see Figure 3, Tag RBC Commands); (f) when the mobile robots reach the starting and destination positions of the transportation tasks, they will drive the right arm movements using the prepared XML-based arm control files.
The dataflow between the RRC and the PMS in Figure 4 can be specified as follows: (a) the PMS proposes a transportation task when it is required by the laboratory process. The PMS commands include the transportation parameters of the laboratory rooms, the starting devices/positions and the ending devices/positions. The PMS does not consider how the presented transportation tasks will be executed (see Figure 4, Tag PMS Commands); and (b) when the RRC receives a distributed task, it should continue to report the latest status to the PMS during the task implementation (see Figure 4, Tag RRC Data).
Similarly, the dataflow between the RBC and the RAC in Figure 5 can be illustrated as follows: (a) the RBC distributes an arm task (see Figure 5, Tag RBC Commands) to the RAC when it is required to finish a transportation process; and (b) when the RAC receives a given task, it will drive the arm servo hardware and report the latest status to the RBC during the arm movement implementation (see Figure 5, Tag RAC Data).
3. Robot Indoor Localization
Accurate indoor localization is the basis for the latter form of robot path planning. In the last decade, a number of robot indoor localization methods have been proposed. However, most of them are unsuitable for application in this study due to the special requirements of the life sciences environment. S.Y. Hwanget al. [13] presented a vision-based indoor localization using corners, lamps and doors. Unfortunately, this method cannot be applied in a poor lighting environment. A. Yazici et al. [14] developed an accurate ultrasonic indoor navigation system (SESKON). It is unsuitable for large laboratories because of its complicated system installation and expense. S.H. Park et al. [15] proposed a RFID indoor robot localization method. Based on the board RFID module and passive floor landmarks, the system can get an accurate localization without external sensors. However, it cannot be used for multiple robots at the same time because this kind of floor landmark does not have enough identification information.
In this study, a measure module called “StarGazer localization” from the company Korea Hagisonic using an infrared radio camera and a number of ceiling passive landmarks is used, as shown in Figure 6. The StarGazer module can capture a series of continuous images, which are reflected from their ceiling passive landmarks with independent IDs. The maximum ID of this kind of landmark is 4,095, which is good for a large life science laboratory.
The StarGazer's location
4. Communication Network and Protocol
In this study, mobile robots will work in different rooms and so a wireless communication network is needed. After the detailed comparison of different wireless communication methods, the standard IEEE 802.11 network has been adopted. The other methods cannot be considered - for example, the Bluetooth method does not have enough data width [16] and the GPRS/GSM method needs to pay data transmission fees [17], etc. The GPRS/GSM method can be used for global communication through mobile phones if necessary. Actually, a pure IEEE 802.11 network still cannot meet the requirements of huge data transmission and long distance connection in this application, so one kind of 2.4 GHz wireless marine bridge is applied for the connection between the router and the mobile robots. The detailed parameters of this kind of bridge can be found in [18].
The architecture of the final communications network is demonstrated in Figure 7. From Figure 7, it can be seen that: (a) in this system every module is connected together by the same IEEE 802.11 network with an independent IP address; (b) the communication between different modules is all based on a TCP/IP protocol; (c) a pair of wireless bridges is used to guarantee the stable connection between the robot board centre and the remote centre. In the system, a XML-based command protocol is designed for both the remote and board sides to understand the transportation commands accurately. For example, the command ‘Transportation’ is demonstrated in Figure 8.
The architecture of the chosen network in the system
A XML-based ‘Transportation’ command
5. Collision Avoidance
Collision avoidance means that the moving mobile robots will recognize dynamic obstacles in front of them and select an alternative avoidance route automatically [19]. This function is one kind of local path planning. To realize it in this study, a Microsoft Kinect sensor has been utilized. This Kinect sensor is a recent motion sensing input device developed by the company Microsoft which can capture the distances and images of moving obstacles with a high resolution of 1cm at a distance from 0.5m to 5m. Here, the Kinect sensor is mounted on the robot body, as demonstrated in Figure 9.
A H20 mobile robot equipped with a Kinect sensor
As for the avoidance path selection, the artificial potential field algorithm (APF) is adopted. The details of the APF model can be found in reference [20].
6. Multi-robot Path Planning
In this study, all of the interesting positions of transportation tasks are stored in a map in advance. To determine the shortest paths for mobile transportations effectively, a new hybrid path planning method using a Floyd algorithm and a GA is proposed. This hybrid method has two computational processes.
6.1 Floyd Planning
The Floyd algorithm is one of the most popular graph algorithms for finding the all-pairs shortest paths in a positive or negative weighted graph, as proposed by a computer scientist Robert Floyd. Due to its robust performance and simple computation, it is popular in the area of shortest path planning. Here, the Floyd algorithm is adopted for mobile robots as follows: (a) once the remote centre receives all the prepared waypoints from the board centre, the Floyd algorithm will be executed to calculate all the shortest paths for any two waypoints; (b) once the remote centre receives a transportation task from a PMS system, the remote centre will select a shortest path from the stored path matrix automatically and send it to the board centre. This planning process is a kind of offline path planning mode. A completed Floyd searching process can be found in [11].
6.2 Genetic Planning
Besides the offline path planning, there is another online path issue be prepared. When a robot is implementing an attributed Floyd path but the next waypoint in this path has just been occupied by another robot, how should it find an alternative online path quickly? For this problem, there are two simple solutions: (a) one is to stop the moving robot so as to wait until the next waypoint becomes available again; and (b) the other is to re-run the Floyd algorithm to update the existing shortest path matrix. Obviously, the first of these is not effective. As for the second one, it is also not preferred, since once the Floyd algorithm is re-started, it will update all the shortest paths for any two waypoints, including those passed because of their characteristics. In the case of large waypoints, the Floyd method will cost a considerable amount of time.
For this online planning issue, in this application the GA is utilized. GA is a global optimal algorithm which imitates the principles of biological evolution [21-22]. The main calculation steps of the GA include making codes on chromosomes, generating initial populations, defining the fitness function and applying the evolutionary process, including selection, crossover and mutation operations. A completed GA computational process is demonstrated in Figure 12.
A completed process of the GA calculation
1) Chromosome coding
The traditional binary coding [23] and real-number coding [24] in GA theory are not suitable for graph map, and so a new sequence coding is proposed.
An example of the new chromosome encoding from Node 1 (Starting point) to Node K (Destination point) is shown in Figure 13. The chromosome will be a list of nodes along the constructed path.
A robot path and its encoding format
2) Generation of initial populations
There are two ways to generate the initial populations: random initialization and heuristic initialization [25]. Although the first of these can guarantee the diversity of the populations, it is easy to cause a searching failure and it needs more training time. To avoid obtaining a dead-loop or a broken path in the graph map searching, here a heuristic rule is proposed as follows: (a) only the connected waypoints can be included in the initial populations; (b) no waypoint can be selected and included in the initial populations more than twenty times, except for the starting point and the destination point; (c) all the initial populations are generated at random by selected the connected middle points between the starting point and the destination point. As for the size of the populations, this will be decided considering both of the model performance and the path searching time.
3) Fitness function
To simplify the GA search process, a fitness function is defined as follows:
where fi is the fitness value of the ith chromosome, Li is the length of the ith chromosome, gi(j) represents the gene (waypoint) of the jth path in the ith chromosome, and D is the distance between points.
4) Selection operation
The purpose of the selection operation is to improve the average quality of the population by giving the high-quality chromosomes a better chance to be copied into the next generation [25]. Here, the proportionate method using a roulette wheel algorithm has been adopted to realize the selection operation. It selects the chromosomes based on their fitness results relative to the average fitness of the other chromosomes in the population.
5) Crossover operation
The crossover operation means examining the current chromosomes to find better ones, which will guarantee the diversity of the solution space. To avoid the waypoint linkage of the parent populations caused by the crossover operation, two chromosomes selected for the crossover operation should have at least one common point, except for the starting and destination points. The detailed steps of the crossover operation are proposed as: (a) find the common points between parent chromosomes except for the starting and destination points; select any point randomly among the common points as a crossover position; (c) execute a crossover operation. An example of the new crossover operation is demonstrated in Figure 14.
An example of the crossover operation
6) Mutation operation
To increase the diversity of the populations and prevent the premature convergence phenomenon during the GA iterative search process, a new rule for the mutation operation is proposed as follows: (a) select a random gene (waypoint) from the parent chromosome as the selection position; (b) find all the waypoints that are connected to the selection waypoint as the potential mutation points; (c) choose a point from the discovered potential mutation points randomly as the official mutation points; (d) based upon the calculated results by the Floyd algorithm, find two shortest paths between the starting point and the mutation point, and the mutation point and the destination point, respectively; (e) check if the two new sub-paths have a common waypoint (the purpose of the check is to avoid a possible dead-loop problem): if no, merge them as the new child chromosome to replace the parent chromosome; if yes, reselect an official mutation point. An example of the use of the new mutation operation for a graph map (see Figure 15) is demonstrated in Figure 16.
A topology graph with nine waypoints
An example of the mutation operation
7. Experiments
7.1 Algorithm Performance Experiment
An algorithm experiment is provided to demonstrate the effective performance of the GA in this application. In this experiment, the designed GA will be executed to find a robot path in a circle map with 82 different waypoints. The task of the GA is to find the shortest path, which should pass through all these waypoints. Obviously, the best path is exactly the circumference of the circle. The path searching process and the GA fitness changing results in every iterative step are shown in Figures 17 and 18, respectively. From Figures 17 and 18, it can be seen that: (a) at the beginning the GA generated path is random (see Figure 17-a); (b) during the search process, the shortest path is selected (see Figure 17-b/c and Figure 18); (c) when the iterative step is equal to 400, the best path is finally found (see Figure 17-d and Figure 18). This experiment proves that the robot path planning based on the GA is appropriate and feasible.
The shortest path searching process using the GA:
The GA fitness changing results
7.2 Robot Movement Experiment
A robot transportation experiment is provided to demonstrate the effective performance of the developed control system. The experiment is executed as follows:
Step 1: Environment Initialization
Use the ceiling landmarks to set up an indoor positioning environment for a big life science laboratory at Celisca, Germany (see Figure 19). Suppose there is a transportation task required by the laboratory PMS to transfer a chemical bottle from a desk robotic workbench to another workbench. The sketch map of the experimental environment is provided in Figure 20. In this transportation experiment, a H20 mobile robot will be expected to pick up the chemical bottle on an automated workbench, take outside by passing through an automated door to reach an expected position, and then go back through the same door to place the bottle on another indoor workbench.
Ceiling landmark-based environment for the robotic transportation at Celisca, University of Rostock, Germany: (a) the indoor laboratory where the robot picks up the chemical bottle; (b) the outdoor corridor area where a supposed outdoor position is located; and (c) the automated door area which the robot needs to pass through twice for this experiment
Sketch map of the experimental environment
Step 2: Transportation Map Definition
By using the map definition GUI of the developed RBC in the system, a transportation map can be built in a short time, including a starting position, a destination position and a series of in-between positions. In the study, two H20 mobile robots are adopted in this transportation experiment and a transportation map with ten positions is established for them, as shown in Figure 20. From Figure 20, it can be seen that: (a) Points ① and ⑩ are the starting position and the destination position of the movement, respectively. In those two points, besides the standard motion function, they are also proposed for the arm manipulation. This means that when a mobile robot reaches either of them, a corresponding re-prepared arm action will be executed subsequently; (b) Point ⑥ is an outside position which is proposed deliberately to check whether the robot can go outside effectively; (c) Points ②, ③, ④, ⑤, ⑦ and ⑨ are all in-between transportation positions, which are proposed by considering the working environment. In the transportation system presented in this study, the number of middle positions is unlimited; and (d) as explained in Section 6.1, the Floyd algorithm is utilized as the default method to perform the shortest path planning computation if the number of total waypoints is not huge. As demonstrated in Section 7.1, the GA-based intelligent planning strategy only executes when the number of the transportation waypoints is huge. To use the Floyd algorithm, a definition of a matrix of connection needs to be decided in advance, as shown in Figure 20.
In this study, the convenience of building a new transportation map has been fully considered. The users do not need to input the coordinate values of the excepted positions to form a transportation map manually; they only need to place the robots in the correct positions which the robots need to recognize and then operate the map definition GUI of their RBC software to record the required coordinates into the transportation map. The positioning data will be measured by the robot on-board the StarGazer modules automatically. For instance, the starting and the destination positions in Figure 20 are measured by the robot on-board StarGazer module, as given in Figures 21 and 22, respectively. From Figures 21 and 22, it can be seen that in the transportation map that every position is localized by a corresponding ceiling landmark (for example, the number of the ceiling landmark to the starting position is 2720 in the experiment).
The starting position ① in the experiment
The destination position ⑩ in the experiment
Actually besides the positioning coordinates for the path planning computation, more control parameters (e.g., the moving velocities of the robots, the moving directions of the robots, etc.) are also required in the lower hardware control for organizing a transportation task. In this study, all of those kinds of hardware describing the parameters can also be decided in the map definition GUI of the RBC during the mapping process.
Step 3: Best Robot Selection
There are two steps for the selection of the best robots for a transportation task. Firstly, the RRC is run to connect the available robots' RBCs so as to read their defined transportation maps and finish the path planning calculation for all of them. As explained in Section 2, the real-time power and positioning status of all the connected robots will be sent from the running RBCs to the RRC. Secondly, the RRC selects the right mobile robot to execute a special transportation task. In the RRC, there are two standards for selecting the robots: the power standard and the distance standard. The former means that the robot with the highest power status will be chosen for a task, while the latter uses a rule such that the robot which is the closest to the starting position of a task will be selected. In this experiment, the first standard is adopted for the robot selection. Based upon the power rule, in the experiment the mobile robot named “Robot 4D” is finally chosen, because its power is higher than that of the other robot (see Figure 23).
The power status: (a) Robot 3C; and (d) Robot 4D
Step 4: Transportation Execution
Since Robot 4D has been selected for the experimental transportation due to its higher power, the RRC will send the corresponding shortest path to the RBC of that robot. Once the RBC receives the provided shortest path sequence, it will drive the motion module of the corresponding robot to execute the transportation. When the robot reaches the starting position ①, the on-board RBC will activate its related RAC component to implement the action of arm grasping. Similarly, when the robot arrives at the destination position ⑩, the RBC will control the RAC to execute the action of arm placing. In this experiment, Robot 4D is ultimately distributed a path sequence as 1->2->3->4->5->6->5->4->9->10. Figure 24 shows the real-time moving process of Robot 4D after receiving the provided path sequence in the map definition GUI. From Figure 24, it can be seen that Robot 4D is moving between Point ② and Point ③. Furthermore, the real movement of Robot 4D is demonstrated in Figure 25, whereby the real arm grasping of the robot at the starting position (the Point ②) is given in Figure 26 and the real arm placing of the robot at the destination position (the Point ③) is displayed in Figure 27.
GUI of the map definition and execution monitoring
Robot 4D is executing the transportation movement based on a given path: (
H20 robot arm grasping at the starting position ①
H20 robot arm placing at the destination position ⑩
To check the performance of the presented transportation system within the function of collision avoidance, in the experiment a test is also provided for. As shown in Figure 28, Robot 4D successfully avoids a member laboratory staff who is occupying position ④ of the transportation experiment.
Transportation Collision Avoidance at position ④: (a) the robot is detecting a human who makes its next position unavailable; (b) and (c) the robot is starting to avoid the human and is trying to find an alternative path; and (d) when the human leaves the occupied position, the robot is moving towards to the expected position.
9. Conclusions
In this paper, a system for mobile robot management in a real life science laboratory is proposed. In the system, and so as to be suitable for a large working environment, an IEEE 802.11g wireless network is adopted for data communications for all kinds of hardware and software; to enable every robot work independently even the communication is broken, a robot board control centre is established on the robots besides a remote multiple control centre; to avoid any misunderstanding of a given command between the remote side and the board side, a XML-based control protocol is designed; to achieve a low-cost but effective indoor robot localization, a method using ceiling landmarks is utilized; to realize the function of dynamic collision avoidance, an artificial potential field method based upon a Microsoft Kinect sensor is presented; to find the best paths for the robot transportation tasks, a hybrid Floyd-Genetic Algorithm method is proposed. Compared to other planning methods (such as Artificial Neural Networks), the hybrid planning method in this system has integrated the performance of map theory and artificial intelligence. Besides, all of the control software is developed in the Microsoft C# .Net framework, which can realize a seamless interface with any other programming.
To validate the effectiveness of the proposed application, various experiments are performed in this paper. From the experimental results, it can be concluded that: (a) as distinct from some other independent robotic applications, the proposed system in this study is integrated with the laboratory process management of the life sciences environment to organize transportation activities. The RRC of the proposed system will present the best transportation solution to the desired transportation tasks, considering both of the real-time status of the available robots and the speciality of the life sciences environment; (b) due to the transparent C/S architecture, the proposed application can be initialized conveniently for another life science laboratory. A new mobile robot or sensor component can also be integrated easily into this system; (c) the proposed application not only considers all of the key aspects of indoor mobile robotics for laboratory transportation, including transportation organization, arm grasping, indoor positioning and path planning computation, but also provides a series of demonstrational experiments to prove the designs, which can provide a direct reference to the relevant engineers and scientists.
Footnotes
10. Acknowledgments
This study was funded by the Federal Ministry of Education and Research (03Z1KN11). The authors would like to thank the Canadian company DrRobot for their technical support of the H20 mobile robots in this study. The authors also would like to thank the reviewers for their precious comments, which have improved the quality of this submission considerably.