Ten Years of Cooperation Between Mobile Robots and Sensor Networks

1. Introduction

Nowadays, in most robotics applications, robots need to live in scenarios that are highly dynamic and quite rich in terms of information. In order to deal with such a wide range of different sources of information, homogeneous systems do not provide the best solution. In most cases, the use of heterogeneous technologies allows systems to improve their performance. Thus, the tendency is to deploy heterogeneous entities that complement each other and cooperate in order to achieve more complex goals.

The combination of sensor networks (SNs) with mobile robots is a clear example of this kind of cooperation. Merging them can be advantageous in several ways. On the one hand, mobile robots can help SNs to perform their tasks in a more efficient and robust manner; for instance, by deploying new sensors, replacing failing ones and acting as mobile sensors, data collectors or communication relays. Mobile robots can also help static sensors to recharge. On the other hand, robots can also benefit from SNs, since they allow them to widen their sensory capacities, which is useful for example for mapping, localization and navigation in unknown environments.

Integration and cooperation between robots and SNs has been a trending topic in the last few years. For instance, there are some studies on algorithms to guide mobile nodes that react to sensorial stimuli, such as the diffusion-based technique [20] and the random walk algorithm. The so-called probabilistic navigation algorithm is used to guide a mobile robot assuming that neither the map nor GPS measurements are available [21]. Other works study how quality metrics in an SN with mobile nodes are influenced by the motion strategies of those mobile nodes [22]. An algorithm based on distributed path planning is proposed in [23] to navigate a mobile robot in an SN field, and a potential field method is presented in [24] for planning paths of several underwater vehicles in an environment with multiple sensors and obstacles. The NAMOS Project at USC also integrates marine robots and wireless sensor networks [25]. Moreover, the deployment of sensor nodes for coverage and exploration is proposed in [26] by means of the algorithm LRV (least recently visited). There are also many works on systems that bring together SNs and unmanned aerial vehicles (UAV). For instance, UAVs are used to deploy and repair nodes of a wireless sensor network in [27, 28]. Besides, the ANSER Project [29] merges autonomous aerial and ground sensors to perform decentralized data fusion and simultaneous localization and mapping (SLAM).

The Robotics, Vision and Control Group (GRVC)¹ is a research group led by Prof. Anibal Ollero at the University of Seville. The GRVC has been working for more than ten years on a number of research projects on the cooperation between sensor networks and mobile robots, focusing mainly on outdoor scenarios. These projects usually involve complex environments where the exploitation of synergies between heterogeneous systems becomes essential in order to achieve the assigned missions. In particular, we exploit the collaboration between robots, camera networks and wireless sensor networks (WSN) comprised of light and low-cost sensor nodes. We combine both technologies together for several purposes. First, robots are used to facilitate the deployment of the light WSN nodes. Those inexpensive WSN nodes make the whole system more versatile, while the robots still act as data collectors and perform complex processing tasks. Besides, mobile robots with on-board sensors collaborate with static sensor nodes in order to improve the perception of the surroundings, for instance in localization, tracking and mapping.

The GRVC started working on robot-SN cooperation with the EU-funded project EMBEDDED WISENTS (FP6-2003-IST-2), where the first requirements for the integration of both technologies were analysed. The idea of the self-deployable sensor network was introduced in the EU-funded project AWARE (FP6-IST-2006-33579) and the project AEROSENS (DPI2005-02293), funded by the Spanish government. In those projects, UAVs were used to deploy a WSN for search and rescue operations. The information coming from ground camera networks, WSNs and the aerial robots was fused in order to perform cooperative tracking and localization. Similar techniques for cooperative tracking were proposed in the EU-funded project URUS (FP6-IST-045062), but this time using ground robots, WSNs and a surveillance camera network for people guidance in a urban scenario. Currently, a step forward is being taken in the EU-funded projects ARCAS (FP7-ICT-2011-287617) and EC-SAFEMOBIL (FP7-ICT-2011-288082), where SLAM techniques are proposed. In this case the robots, aerial and ground, use the information from a WSN to localize themselves and the sensor nodes (mapping) at the same time. Finally, the deployment of large-scale systems was the focus in the EU-funded projects CONET (FP7-INFSO-ICT-224053) and PLANET (FP7-ICT-257649), where we proposed techniques to collect the data from WSNs whose nodes have been deployed sparsely in a large-scale environment. In a similar line, the EU-funded project MUAC-IREN (FP7-PEOPLE-295300) and the project CLEAR (DPI2011-28937-C02-01), funded by the Spanish government, are developing techniques to enhance operation endurance in large-scale systems.

This paper summarizes the main research carried out by the GRVC in robot-sensor network cooperation. Table 1 shows our main works derived from the aforementioned projects, which will be explained throughout the paper. The organization is as follows: Section 2 introduces some relevant challenges that need to be addressed in order to combine SNs and mobile robots; Section 3 presents our works on self-deployable SNs; Section 4 presents our techniques and results for cooperative localization and tracking; Section 5 summarizes some techniques and results in SLAM with SNs; Section 6 briefly presents our works on the deployment of large-scale systems; Section 7 summarizes the main lessons learnt; and Section 8 concludes and gives future perspectives.

OPEN IN VIEWER

Table 1.

Summary of works by the GRVC on cooperation between mobile robots and sensor networks. Publications are classified according to topics and projects.

R&D challenges	Projects	Publications
Autonomous self-deployment	AWARE, AEROSENS	[1, 2]
Cooperative localization and tracking	AWARE, URUS	[1, 3–9]
SLAM	ARCAS, EC-SAFEMOBIL	[10–15]
Large-scale applications	CONET, PLANET, MUAC-IREN, CLEAR	[16–19]

The synergies between mobile robots and SNs, and the advantages of combining both technologies, seem to be clear. However, this integration is not straightforward. Some major issues need to be addressed before robots and SNs can cooperate in an efficient manner. In this paper, we analyse some challenges that researchers should face if they want to exploit the benefits of this combination.

Sensor networks can benefit from the mobility, sensing and actuation capabilities of mobile robots. Using mobile robots, a static SN can be transformed into a mobile network, which improves the flexibility and adaptability of the system. Moreover, mobile robots can deploy sensors, allowing the SN to become self-deployable. However, in order to exploit all their functionalities, robots should know the positions of the different sensors in the network. In general it is necessary to know where the other nodes are if we want to ensure network connectivity, to collect data from the sensor nodes, or even to deploy new nodes. This is even more relevant in outdoor systems, where nodes may be located sparsely or in unknown areas.

Besides, it is also necessary to have functionalities to localize new nodes after deployment. It is not always straightforward to deploy a sensor with a mobile vehicle and know with high accuracy the deployment position. Moreover, it is necessary to have a map of the environment in order to place new nodes at accessible and suitable areas.

Robot localization, and not only nodes localization, is also important. SNs can be used to help robots to improve localization and to navigate in unknown or GPS-denied environments. However, the integration of all the information available is not straightforward. Usually, data from heterogeneous sources need to be fused, coming from local sensors on the robots and from nodes of the SN. Moreover, many sensor nodes provide only range measurements, which are less informative than others and have additional constraints.

Finally, many of the above issues become even more challenging when operating in large-scale spaces. In applications where the SNs are deployed in large areas, additional constraints should be considered mainly due to the sparsity and number of the nodes deployed and the long-endurance requirements imposed.

In the following sections we analyse the above challenges and present our works and solutions for each of them. We discuss our methods and place them in the related work of the literature. We also summarize some of the most relevant results.

3. Autonomous self-deployment of SNs with mobile robots

The development of wireless communication technologies in the last ten years makes possible the integration of autonomous vehicles with WSNs in the environment. In contrast to the traditional static WSNs, the inclusion of these mobile robots within a WSN provides more flexibility to the system, which achieves more complex functionalities. This integration can be beneficial in different manners:

Mobile robots carry on-board sensors that can complement the information collected by the WSN and even help that WSN with the calibration process.

The technologies of static ubiquitous sensor networks have important limitations as far as the required coverage and the communication range between the nodes are concerned. The use of mobile sensor nodes brings significant improvements, however. They can provide the ability to dynamically adapt the network to environmental events, and improve the network connectivity in case of static node failure. Node mobility for ad-hoc sensor networks has been studied by many researchers [30, 31]. Moreover, mobile nodes with single-hop communication and the ability to recharge batteries (or re-fuelling) are proposed in [32] as data collectors of the network. They can move to be near the static nodes and collect their data, reducing the energy consumed by communications. The coordinated motion of a small number of nodes in the network to achieve efficient communication between any pair of other mobile nodes is also proposed.

In many scenarios, such as civil security or disaster, the motion of the mobile sensor nodes installed on ground vehicles or carried by persons is very constrained, due to the characteristics of the terrain or the hazardous conditions involved. Therefore, the cooperation of aerial vehicles with ground WSNs offers many potentialities. For instance, the use of UAVs as data sinks has been proposed by several authors in the WSN community. They fly over fixed sensor networks following a predictable pattern in order to gather data from them. In [33], an algorithm for path computation and following is proposed and applied to guide the motion of an autonomous helicopter flying very close to the sensor nodes deployed on the ground.

Nonetheless, it should be noticed that the flight endurance and range of the currently available low-cost UAVs are very constrained [34]. In addition, reliability and fault-tolerance in terms of communication are major issues for the operation of UAVs. Therefore, these autonomous vehicles need an existing communication infrastructure in order to cooperate or to be tele-operated by humans, e.g., in emergency situations. When this infrastructure is not available, or the required communication range is too large for the existing technologies, the UAVs could be used to deploy a WSN that fulfils those requirements. The solution is to mount devices onto UAVs to carry and deploy nodes.

Sometimes, during deployment in hazardous scenarios or even during network operation, the network infrastructure may be partially damaged. In those cases, the UAVs can also be used to repair the network coverage or connectivity by deploying new nodes. For instance, in [28], the application of an autonomous helicopter for the deployment and repair of a WSN is proposed. We also follow a similar approach in the AWARE project [1], whose platform has self-deployment and self-configuration capabilities for operation in sites without sensing and communication infrastructure. We deployed wireless sensors with a special device installed on board the UAVs, as depicted in Fig. 1. Besides, in AWARE we also accomplished the deployment and transportation of heavier loads (e.g., a static camera with a pan & tilt system), which necessarily requires tight coordination between several UAVs [2].

OPEN IN VIEWER

Figure 1.

Sensor deployment from an autonomous helicopter in the experiments of the AWARE project

In complex and dynamic environments, the localization of robots is an issue that can still become challenging. This is mainly due to the limited range of on-board sensors or the high computational requirements of many visual-based techniques. It is true that there are methods for accurate indoor and outdoor localization, such as beacon systems or differential GPS. The problem is that these methods are usually expensive, rely on satellite visibility, or require the pre-installation of costly infrastructures.

Sensor networks embedded in the environment can play an important role when robots need to localize in unknown scenarios or need to track any moving target. These sensors provide ubiquitous and rich information for the robots working in the area. In many cases, robots can take advantage of already existing infrastructures, like surveillance camera networks; in other cases, WSNs made up of low-cost devices can be easily deployed.

The idea is to allow the mobile robots to use the information provided by the sensor networks in order to improve localization and tracking tasks. However, even though this cooperation can be advantageous, there are some challenges to solve. First, the robots need to access heterogeneous devices, which usually provide uncertain information. Second, issues such as communication delays and decentralization should be considered to tackle the problem in a robust and efficient manner. Therefore, in this section we propose some techniques to fuse information coming from heterogeneous sources. In particular, we have developed decentralized data fusion algorithms that can deal with uncertainties and communication issues. Then, we apply them to cooperative localization and tracking with SNs and mobile robots.

Although our approach is more general, we present results mainly in two relevant scenarios. First is a rescue mission in a disaster management application, where the SNs are deployed by autonomous robots (the AWARE project). Second is an urban scenario where the robots interact with ubiquitous SNs already embedded in the environment (the URUS project). In both scenarios the SNs help the robots to localize themselves, or to localize some person or object to be tracked.

The fusion of data gathered from a network of heterogeneous sensors is a highly relevant problem in robotics. Most approaches model the sensors as uncertain sources and use Bayesian techniques [35, 36], which provide a sound mathematical framework to deal with uncertain sources of information. Moreover, Bayesian approaches are easy to decentralize [37, 38], which is essential to achieve robust and scalable solutions.

The first option would be to have a central node fusing all the information received from the heterogeneous sensors. We propose a centralized extended Kalman filter (EKF) [3] for fire monitoring and firefighter tracking. Within the framework of the AWARE project, ground static cameras and a WSN are used to help UAVs to localize and track fire sources and firefighters in a rescue mission. The WSN is deployed by the UAVs and can measure high temperatures to detect potential fire alarms. The firefighters also carry a WSN node and the RSSI (received signal strength indicator) measurements are used by the WSN to track these mobile nodes, see [4]. Besides, vision-based algorithms are applied to the images from the static cameras (with known positions) in order to provide information about the fires or the firefighters.

As mentioned above, in real-time systems decentralized approaches are more adequate, since they alleviate the bandwidth requirements and improve the reliability of the system. Therefore, we also propose for the same application in AWARE a decentralized delayed-state extended information filter [5, 6]. In this version, each entity only employs local information (data from local sensors, i.e., the sensors on board the robot), and then shares its estimation with the others.

We show that the decentralized filter can obtain the same estimation as the centralized filter as long as the common information exchanged between the sources is maintained by a separate filter called the channel filter [39]. This is achieved by using filters over the full trajectory of the state instead of just considering distributions over the state at time t. The main drawback of channel filters is that they enforce a tree-shaped network topology in order not to double-count information, which may be a strong constraint for dynamic systems outdoors. We overcome this issue by using conservative fusion rules such as the covariance intersection algorithm [40] to avoid information double-counting regardless the network topology (the estimation is no longer necessarily equal to the centralized version). Another advantage of using delayed states, i.e., the full trajectory of the state, is that the systems can receive and correctly fuse information from the past. This is especially relevant in applications with communication delays and failures.

The results included in [6, 5] are for cooperative tracking of firefighters. Again, a WSN is used together with static ground cameras. An experiment is depicted in Fig. 2, where the estimated XY trajectory provided by two ground cameras and the WSN are plotted together with the centralized estimation. It can be seen how all estimations converge to the same solution with errors to the order of one metre.

OPEN IN VIEWER

Figure 2.

XY estimation provided b y two ground cameras and the W SN, and centralized estimation. A sensor network was deployed into the experimental area, pink squares denote the position of each sensor node.

Additional results of the same technique are shown in [1], but mobile cameras on board the UAVs are also considered to track several firefighters in the scenario. The way-points of the UAVs are selected with a technique that aims to reduce uncertainty on the target estimation all the time. This is achieved by pointing the UAVs’ cameras perpendicularly to the axis of higher uncertainty of the target estimation. Moreover, the results of our decentralized filter applied to fire monitoring are included in [7].

Similar techniques for decentralized data fusion were applied within the framework of the URUS project. In this case, the scenario consists of a urban environment where the robots provide services for humans, such as, for example, person guiding. Thus, ubiquitous SNs are integrated within the buildings and available for the robots through 3G or WiFi communication.

In [8], we show how to apply our decentralized delayed-state extended information filter in the scenarios of the URUS project for tracking and guiding missions. In particular, a WSN, a surveillance network of cameras, and a mobile robot with an on-board camera are used to perform the experiments. We demonstrate how combining the information from the robot and the ubiquitous sensors allows the whole system to overcome tracking failures due to occlusions, clutter or lighting changes. In this application the robot needs to find and track a person that is to be guided. That person also carries a mobile WSN node. Figure 3 depicts the urban scenario for the experiments, with the location of the surveillance cameras and the WSN.

OPEN IN VIEWER

Figure 3.

Urban scenario in the URUS project. The dimension is 100 by 100 metres, approximately. Cameras in green and WSN nodes as black dots.

Additional results for person guiding are shown in [9]. We use a Particle Filter for integrating all the RSSI measurements coming from the WSN in order to track a mobile node. Then, that estimation of the person position (who carries the mobile node) can be fused with those obtained using the cameras in the building or on board the robot. Figure 4 shows some examples of the tracks obtained by a camera on board the mobile robot and some of the surveillance cameras installed in the building. As it can be seen, the robot travels aside the person for guidance.

OPEN IN VIEWER

Figure 4.

(a) Tracks obtained by the camera network. (b) Tracks obtained by the camera on board a mobile robot.

Localization of sensor nodes (i.e., building a map of the sensor network) is also a critical problem in robot-SN cooperation. For instance, it is necessary in applications that require spatial mapping of the sensed measurements. It has been pointed out in [41] that, when the number of sensors is large, the manual deployment and position recording is error-prone, and in many applications hand-placing the sensors is not even an option. Thus, for example, if the sensors are scattered from an aeroplane, automatic localization methods should be employed. This is particularly true for networks deployed in emergency response scenarios without pre-existing infrastructure.

GPS-based systems provide an immediate solution to the problem of localizing a node in outdoor scenarios. However, this solution is not always feasible due to its cost, energy consumption and inapplicability to different scenarios. In [10], we propose the use of a mobile node, i.e., a robotic vehicle with GPS, for the localization of the other nodes of the network by using the RSSI measurements from incoming messages. Although the proposed method can be used with any autonomous vehicle, the implementation with autonomous helicopters is proposed in the paper. These vehicles have higher accessibility than ground vehicles, which can be a relevant constraint in some scenarios [42]. The technique described in [10] uses particle filters (PF) to process the RSSI measurements that static nodes receive from a mobile robot equipped with GPS and a WSN node. The method computes the mean and standard deviation of the localization of each static node. The node on board the robot is used to recover the information from the deployed static WSN nodes and, at the same time, it is employed as a beacon node for network localization. The method is probabilistic and takes into account the uncertainty in RSSI measurements.

Once an initial estimation of the position of the static nodes has been obtained, this localization can be refined using information filters as in [11]. Each sensor node locally refines its own position by using the information received from the mobile robot and from its neighbouring nodes (which also have an initial estimation of their positions). The messages exchanged among the nodes contain the estimated position of the emitter. By using measurements from several neighbours or from the mobile robot, the position of the node can be further refined.

Although PF-based localization schemes showed good results with small WSNs, these approaches have two main disadvantages. First, estimating when the PF has converged to a single hypothesis is computationally demanding. Second, it is not possible to integrate the estimation of the filter into more complex localization architectures such as SLAM until the PF has converged to a single hypothesis.

The problems related to multiple hypotheses in the early steps of the estimation in range-only localization approaches were addressed in [43]. The paper describes an algorithm that allows delayed initialization of the node position by tracking the most probable two hypotheses. After integrating a number of measurements, the wrong hypothesis was discarded and the correct one was included into the SLAM filter. We generalise this approach in [12] by means of undelayed node position initialization and extension to n-hypotheses. This work allows integrating measurements from the very beginning using a weighted Gaussian Mixture Model (GMM) to represent the non-Gaussian prior distribution of the node position in a way similar to the approach presented in [44].

Figure 5 shows an example of the multi-hypothesis mapping framework. The different stages of the mapping approach can be seen easily. At the beginning, all the hypotheses are placed around the vehicle according to the range information, which is very noisy at such distances. Later, the localization of the hypotheses in front the vehicle is improved because the range-RSSI relation is more accurate as the distance decreases. In the next steps, the hypotheses behind the robot are removed from the filter because their weights are too low. Next, only the hypotheses on the sides of the vehicle remain. This is because the robot is moving in a straight line with respect to the node, so both hypotheses are possible if range-only measurements are considered. Finally, after the robot moves to a different position, the wrong hypothesis is removed and the filter converges to the correct node position.

Using GMMs to model the probability distribution of the sensor node position opens the door to new localization approaches able to consider robot motion in order to improve position estimation (see Fig. 6). Thus, we extend the method to consider active sensing strategies in order to map the nodes [13]. Entropy variation is used as a measurement of information gain, prioritizing the control actions of the robot. However, as there is no analytical expression for the entropy of a GMM, upper bounds of the entropy, for which closed-form computation is possible, are used instead. The paper describes simulations that show the feasibility of the approach.

OPEN IN VIEWER

Figure 5.

Evolution of the multiple hypotheses for the localization of one sensor node. The blue circle denotes the beacon position, the red square represents the robot position and the yellow ellipses are the multiple hypotheses for the localization.

OPEN IN VIEWER

Figure 6.

Two examples of range-only localization. The robot (red triangle) receives range data from the beacon (green square) at three different positions. Yellow areas denote possible localizations of the beacon (the more intense the yellow colour, the more likely this localization is). (a) Result of the node localization using a straight robot trajectory (there are two possible solutions for the localization). (b) Results of the node localization if the robot trajectory is adapted from active sensing considerations (the localization converges to a single correct solution).

Range-only SLAM (RO-SLAM) is an emerging approach that aims to localize a mobile system at the same time that it maps the position of a set of range sensors using only range measurements. In contrast with other SLAM approaches based on cameras or LIDAR, RO-SLAM has the advantage of integrating non-line-of-sight measurements when radio-based range sensing is used. In addition, the data association problem is intrinsically solved by using unique identifiers for each range sensor into the system. However, RO-SLAM poses serious challenges, mainly related to its low-informative measurements (distance between two sensing elements), which lead to multiple localization hypotheses that do not fit well with the usual linear/Gaussian approximations. The GRVC has been very active in RO-SLAM in recent years. Two works that illustrate the research being developed are presented in the following.

The research work we present in [14] extends the 2D method in [12] to 3D RO-SLAM. The proposed method is based on a centralized EKF-SLAM, which includes the position of the robot and the position of all range sensors (landmarks), which allows the integration of sensor-to-sensor measurements and not only robot-to-sensor, a key issue in RO-SLAM as concluded in [45]. It also uses a state vector parametrization that allows the reduction of the required computational load, especially in the correction stage of the EKF: the multi-modal belief of the azimuth and elevation angles of a range sensor is integrated efficiently in a EKF employing two independent Gaussian mixtures (Fig. 7 shows an example including two aerial vehicles). The method allows the integration of measurement information from the very first measurement, and also initializes the position estimation and parametrization of the Gaussian mixtures automatically with only one range measurement.

OPEN IN VIEWER

Figure 7.

Example of multi-hypothesis RO-SLAM. The picture shows the position estimation of two aerial vehicles (blue quad-rotors) together with their ground-truth position (red quad-rotors) and the uncertainties associated with the different position hypotheses for each sensor node whose localization is being estimated into the RO-SLAM filter.

Most existing RO-SLAM techniques consider beacons as passive devices, disregarding the sensing and computing capabilities they are actually endowed with. In [15], we propose a RO-SLAM scheme based on sparse extended information filters (SEIF) that exploits those capabilities of sensor nodes. The proposed scheme integrates all robot-beacon measurements into the SEIF, but also the inter-beacon measurements that involve at least one beacon within the robot sensing area, avoiding repeated measurements. Compared to traditional schemes this method reduces the uncertainty of the map estimation (>40%) and also indirectly improves robot localization accuracy (>20%). Moreover, it inherits from the SEIF its efficiency and scalability, significantly reducing the robot computational burden and enabling its implementation in robots with lower computer capabilities.

The continuous miniaturization of everyday devices, as well as the convergence of communication, computing and control, provide the ability to build large-scale, heterogeneous, pervasive, networked systems that can be deeply embedded in the physical world. Cooperation between mobile robots and sensor networks open huge fields for research in unprecedented applications, but of course imposes significant challenges.

In the general case, the deployment of very large-scale complex systems consists of a series of heterogeneous devices such as unmanned vehicles, sensor networks, etc., that work together and integrate with the pre-existing infrastructure in a transparent manner. For this reason, a solution applicable to a wide variety of scenarios requires the development of new distributed architectures and integration platforms with the following characteristics:

Ability to adapt to the changing conditions of the network and the application itself in order to always choose a solution that is as optimal as possible.

We address these challenges in the PLANET project. The main objective of PLANET is the design, development and validation of an integrated platform to enable the deployment, operation and maintenance of large-scale complex systems of heterogeneous networked devices, including wireless sensors and mobile robots. The platform supports adaptive and optimal deployment and operation by means of mobile cooperating robots, i.e., vehicles networked with static nodes. Moreover, we validated the platform in two complementary scenarios: the monitoring of the Doñana Biological Reserve, which has a remarkable ecological value and is highly sensitive to the impact of pollution; and an automated airfield.

The approach adopted in PLANET can be seen in Fig. 8. Different stages where mobile robots and sensor networks exchange information and explicitly cooperate can be distinguished: sensor node deployment, sensor network repairing and healing, etc. Of course, the technological constraints and heterogeneity impose significant requirements that have to be addressed from different perspectives. For instance, from an integration and architectural point of view, one of the main challenges is to integrate in a seamless fashion all heterogeneous components and functionalities.

Another relevant technological constraint derives from the fact that in large-scale systems devices are usually deployed sparsely in the environment. In this sense, the higher mobility of UAVs is usually preferred against unmanned ground vehicles (UGVs). Besides, most cooperation schemes are based mainly on explicit communication, so sparsity is a significant issue if we recall that WSNs are energy limited and have low communication ranges. From this point of view, tasks that can be trivial in short-scale deployments, such as data collection, become tougher when the nodes are deployed in a larger scenario.

Data collection using UAVs is a very relevant topic in large-scale deployments of sensor networks. No matter how many nodes are deployed or how many measurements are generated, they are useless if they cannot be collected and transmitted to the rest of the modules.

In the baseline data collection approach, the deployed nodes gather and buffer the readings. When a UAV flies near the nodes it sends a beacon message, and the nodes send their readings in reply. This approach has been tested in works such as [46] and [47].

In [16], we propose a scheme that improves the scalability of the above basic approach by grouping the deployed nodes. All of the nodes from one group send messages with their readings in response to the UAV beacon specific for that group. The groups and their collection zones are pre-computed, taking into account the node locations and radio coverage, among others. However, in this scheme, as in the rest of the previous works, the operations of UAVs and WSN are independent of each other. They essentially consider WSN and UAVs as independent units that do not influence each other. Consequently, they often lack reactivity to unexpected events.

OPEN IN VIEWER

Figure 8.

The PLANET approach for large-scale cooperation of highly heterogeneous networked systems

Our work in [17] describes the approach adopted in the PLANET project for UAV-based data collection. Sensor nodes are organized into clusters scheduled by the cluster head. The cluster head is the only one that communicates with the UAVs, while the others can be in sleep mode during inactive periods. The UAV plans its trajectory to fly over the coverage regions of the heads of each cluster. Of course, the energy consumption of the cluster head nodes is significantly higher than others, and the cluster head role should be rotated among the cluster members. The scheme comprises: (1) a dynamic clustering method that is executed by deployed nodes and takes into account the UAV trajectory for cluster head rotation; (2) a UAV path planning method that solves the travelling sales problem considering the current location of cluster heads.

The main novelty of our method is that UAVs and WSN cooperate to increase data collection performance and robustness. The proposed method presents two main cooperative behaviours. First, the results of the WSN operation are used to update the UAV flight plan. Second, the UAV trajectory is considered in the operation of the WSN in order to improve data collection performance. This method outperforms non-cooperative UAV-based collection approaches. In particular, our approach and techniques were validated in field experiments in an airfield in Seville (Spain), as can be seen in Fig. 9.

OPEN IN VIEWER

Figure 9.

(Left) Estimated flight plan and actual UAV trajectory (black line) obtained in field experiments. Right) Picture of the embedded PC104 in one Piper airframe used in the experiments.

Assuring the long endurance and operation of the system is another challenge in unattended large-scale heterogeneous systems. Traditionally, energy consumption efficiency has attracted significant research efforts in sensor network fields. Research is still needed in applications that require a high node longevity. Another challenge is to extend the flight operation of mobile robots, and more particularly of UAVs.

Currently, we are researching two main approaches for improving UAV flight endurance. The first approach is mimicking the performance of species like vultures, which perform extraordinarily long flights with negligible energetic cost. In the MUAC-IREN project, together with researchers from the Australian Center for Field Robotics, UAV planning strategies for benefiting from thermal currents are being researched. For instance, a glider UAV that gains altitude and energy when inside a thermal and uses that energy to fly ‘jump’ to another thermal. A second research line followed in the CLEAR project is the use of UAVs to recharge batteries.

After all of these years of research on mobile robots and SNs, as well as field experiments involving real and heterogeneous devices, we have learnt some practical lessons that we try to summarize in this section.

One of the first lessons learnt when using information from heterogeneous networked systems is the relevance of time synchronization. We solve this issue by using a network time protocol (NTP), where a server distributes timing information among the different cooperative systems.

The selection of an adequate communication system also turns out to be crucial, since many of the communication problems are caused by interferences in the wireless channels. Due to the current abundance of WiFi devices, selecting WiFi channels at 5 GHz proved to be less problematic, since those channels are usually less flooded.

In general, fusing information from heterogeneous sensors has proved to be very advantageous to achieve more reliable systems. For example, when tracking or localizing in dynamic scenarios, a single source of information is not usually enough to deal with occlusions, illumination changes, clutter and coverage issues. On the contrary, the combination of complementary sensors is required. Thus, WSNs can provide coarse initialization for localization tasks, but camera networks provide more accurate measurements. Sensors on mobile robots also allow the system to cover places occluded for the static sensor networks.

However, the above fusion from heterogeneous sources has also proved to be difficult, mainly due to communication latencies, mobility and connectivity issues. In this sense, decentralized approaches are mandatory, since centralized approaches do not scale well in this kind of system. Moreover, the use of filters with delayed states is very useful to cope with communication delays and dropouts. Our techniques using full trajectories of the state allowed the entities to accumulate information during communication drop-outs to fuse it later, once the communication was recovered.

RO-SLAM techniques allow the integration of robots with their environment, enabling them to gather information from the environment at the same time that they refine their localization, a critical task in robot autonomy. Robot localization based on range measurements has proved to be an excellent method for coarse estimation, simplifying robot localization methods and improving the reliability of the whole system. Nevertheless, our experiments showed that results of RO-SLAM are closely linked to the state of the art in radio-based range estimation: the better the range estimation, the more accurate the robot localization. Recent advances in radio-based range sensing allow an estimating range with sub-metre accuracy at a reasonable cost (around 200 euros per node), which is a good trade-off between accuracy and investment.

Integration of multiple-hypotheses in RO-SLAM is also a key feature identified by the GRVC together with other state of the art researchers. Approaches based on a single hypothesis do not scale well with real scenarios due to the uncertainties induced by the environment and by the measuring process. Thus, the development of new efficient multi-hypothesis approaches for RO-SLAM is an interesting and challenging problem that will probably gain more attention in the next years.

Although many RO-SLAM techniques have been developed, most of them consider sensor nodes as simple beacons for range measurements disregarding the rest of their capabilities. We have developed a number of techniques that exploit sensor network capabilities in order to significantly improve map and robot localization estimations, exploiting the computing capability of sensor nodes and hence reducing the robot's computational effort.

Finally, we have learnt the advantages of using robot-SN testbeds, which allow testing and evaluation of techniques in realistic conditions. Using these kinds of facilities is a current trend. For example, a survey of the growing number of testbeds for robot-SN cooperation can be found in [18]. Moreover, the GRVC developed and maintains the CONET Integrated Testbed² [19], a public remote tool to assess and compare methods on the cooperation and integration of robot and sensor networks.

8. Conclusions and future challenges

This paper presented a summary of the main contributions of the GRVC to the state of the art in the field of mobile robots and sensor network cooperation. The document analysed the main results obtained by the research group and some of the lessons learnt during these ten years.

Two main conclusions are derived from the research and experimental results obtained by the GRVC. First, the merging of two different technologies such as robots and SNs has proved to be fruitful for many applications. Therefore, the current tendency of integrating heterogeneous teams where mobile robots cooperate with SNs is opening unprecedented possibilities in an increasingly wide field of applications. However, our second conclusion is that this integration is not straightforward, and several issues have to be addressed carefully. The connectivity and communication among the different parts should be ensured; the localization of the entities is desirable and necessary in most cases; heterogeneous sources of information should be fused, etc. If we consider outdoor scenarios and we deploy our systems in larger and larger scenarios, the previous issues are not trivial at all.

In the paper we have detected some of the above main issues and have classified our works according to them. We have also proposed feasible solutions for the different problems presented. However, there are still other open issues for cooperative mobile robots and SNs, and those explained below will guide our future research lines.

One of the strongest trends in robot-SN cooperation is its gradual application to real problems. This poses additional issues, some of them related to the robustness required for the systems to operate in realistic conditions. From this point of view, the validation of techniques using testbeds with different degrees of realism is critical. This fact has been clearly understood by the community, and increasing efforts are being devoted to develop testbeds that focus on particular problems and applications. Other critical problems are related to the integration of these robot-SN systems in human environments. Co-existence with humans opens very interesting issues in terms of social robotics, i.e., how interactions of humans with robots should be addressed. Finally, in certain applications, such as inspection and repairing operations in industrial plants, the robots involved require a very high accuracy that could be provided by additional sensor networks.