Leader-Follower Formation for Nonholonomic Mobile Robots: Discrete-Time Approach

1. Introduction

The formation and cooperation of mobile robots are found in different fields: for example, in the aerial environment for navigation, exploration tasks, recognition of hostile terrain, aerial photography, terrain mapping or in the search and capture of ground targets [1, 2]. They are also considered for tracking and obstacle avoidance, patrolling an environment to prevent intruders in arbitrary topologies, 3D mapping, archaeological exploration and terrain recognition [3, 4]. In general, in order to perform such tasks, it is necessary to have a measuring device on board as in [5], where encoders and a compass on board are used to maintain the formation. In some cases, multiple devices are required to accomplish recognition of the environment: usually a laser system or a video camera [6, 7]. A special case is the detection of a leader robot or a particular pattern mounted on the robot [8].

The leader-follower formation problem has been analysed by considering several control strategies, mainly by means of the feedback linearization approach [9–13]. In [9], an on-board camera is used to estimate the orientation and the distance between the robots, and in [11], three different controllers are proposed and a camera is used for measuring the distance and angles between the vehicles, verifying the controller's performance by numerical simulation. Lyapunov-based control strategies have also been considered; for instance, in [14], two different controls are proposed using a camera fixed to the ceiling, while in [5, 15], the strategy is complemented by a receding horizon approach together with a single camera mounted on the ceiling. In addition, backstepping strategies have been implemented in the solution of the leader-follower problem; for example, in [16], the leader-follower formation is assured using Cartesian coordinates to avoid singularities that appear in polar coordinates. Backstepping and cascade system theory is used in [17] to control multiple robots and in [18], the method is complemented by a bioinspired dynamic to avoid impractical velocity jumps. The backstepping is merged with a fuzzy control strategy in [19] to perform an obstacle avoidance strategy. Game theory is used in [4] for terrain patrolling tasks, modelling the problem as a set of two players developing an optimal strategy for the solution of the problem. In [20], the leader-follower formation is analysed by measuring the relative distance between the vehicles from a mobile frame located on the follower robot, which is allowed to adopt a variable position under an input-constraints strategy.

One important aspect of the problem is the real-time implementation of the mentioned solutions. In this sense, there are two main approaches. The first one is based on a camera mounted on the ceiling of the working space as in [5] or [15], and the more general approach that considers an on-board camera used to estimate the orientation and the distance between the robots as in [9] and [21], in which observer-based solutions are proposed. In [22], the solution is based on an entropy segmentation that provides the relative-position estimation of the robots.

The objective of this paper is to analyse the leader-follower formation problem using a discrete-time approach: an approach that has not been considered in the literature. The consideration of a discrete-time strategy is based on the necessity of a practical implementation of the solution of the leader-follower problem stated in this work, which should be carried out, in general, by means of microprocessors or computer-based platforms. In addition, the discrete-time nature of the vision system seems to be more suitable for the approach proposed. In particular, a leader-follower formation based on relative distances between the robots is suitable for outdoor environments where an absolute localization system is difficult to implement. An exact discretization strategy based on direct integration, such as the one used in [23], is considered here.

The control of nonholonomic mobile robots of the type considered in this work is a challenging problem due to the fact that they do not satisfy the well-known Brocket's rank condition [24], which prevents the existence of a smooth time-invariant feedback for the solution of the stabilization problem. This problem is related to the control of the vehicle's posture. The standard formation problem is addressed in a discrete-time context assuming that the leader robot follows a free trajectory; it is intended that the follower robot tracks the leader within a specified distance using only a camera on board as a sensor. In this form, the solution does not depend on positions and velocities measurements, and only considers relative measurements between vehicles. To solve this problem, and following [16], a coordinate transformation is considered in which the relative time evolution of the position and angle between the robots is used.

Instead of a standard video camera, a Kinect device is used to estimate the relative distance and the angle between the leader and the follower robot. The solution of the problem is based on a non-standard discrete-time feedback that is retarded by one sampling period of time, which produces a closed-loop perturbed system. It is formally proven that the follower tracks the leader by using bounded tracking errors. The proposed formation strategy is evaluated by numerical simulation as well as real-time experiments by means of two differentially driven mobile robots.

The rest of the work is organized as follows. Section 2 describes the discretization procedure and the mathematical preliminaries that are used throughout the document. In Section 3, the leader-follower formation problem is geometrically stated and the control objective based on the relative errors is described. In Section 4, the solution to the leader-follower formation problem (the formation-error analysis and the main procedure to obtain the control law) is shown. The numerical and experimental evaluation of the proposed formation strategy is described in Section 5 and some conclusions are given in Section 6.

2. Discrete-time Model of the Mobile Robots

A differentially driven mobile robot (type (2.0)), such as the one depicted in Figure 1, is considered. The kinematic model of this robot is given by [25],

OPEN IN VIEWER

Figure 1.

Representation of a mobile robot (2,0) in the plane

x · = V cos ( θ ) y · = V sin ( θ ) θ · = W

(1)

where x and y are the positions along the axes X and Y, respectively, and θ corresponds to the orientation angle of the vehicle. V and W are the linear and angular velocities of the robot.

The kinematic evolution of the point α, located at the front of the robot along the longitudinal axis, can be described by considering the changes to the following coordinates,

α x = x + l cos ( θ ) α y = y + l sin ( θ ) α θ = θ

(2)

where l > 0 is the constant distance between point P and α. In these coordinates, the kinematic model of the mobile robot takes the form,

α · x = V cos ( α θ ) − l W sin ( α θ ) α · y = V sin ( α θ ) + l W cos ( α θ ) α · θ = W .

(3)

The linear and angular velocities of each robot are related to the angular velocity of the right wheel, w_r, and the angular velocity of the left wheel, w_l, by means of the relation,

[ w r ( t ) w l ( t ) ] = 1 r [ 1 m 1 − m ] [ V ( t ) W ( t ) ]

(4)

where r is the wheel ratio and 2 m is the distance between the wheels. This transformation is non-singular for all r, m > 0 .

The control of nonholonomic mobile robots of the type considered in this work is a challenging problem due to the fact that they do not satisfy the well-known Brocket's rank condition [24], which prevents the existence of a smooth time-invariant feedback for the solution of the stabilization problem. The use of Lyapunov techniques has provided solutions for a single robot by means of discontinuous feedback [26] or by means of time-varying control laws [27, 28]. Instead of considering the posture of the vehicle, in this work, controlling the Cartesian coordinates of the point α was preferred, while obtaining simultaneously the boundedness of the orientation angle.

2.1. Exact discrete-time representation

Taking into account the discrete-time nature of a real time implementation, an exact discrete-time representation of the kinematic equations (3) is considered. The non-linear kinematic model (3) can be discretized in an exact manner by considering a sampling period T > 0 , a time interval t k = { t ∈ [ k T , k T + T ) } with k = 0,1,2,… , and assuming that the control signals between the time intervals t_k are constant, this is, v ( t ) = v ( k T ) , w ( t ) = w ( k T ) , ∀ t ∈ [ k T , k T + T ) . For the sake of simplicity, the compact notation ζ ± n = ζ ( k T ± n T ) , ζ ± = ζ ( k T ± T ) and ζ 0 = ζ for any positive integer n, is considered in all the subsequent developments.

Following [23], the time integration of the dynamics (3), with the initial condition evaluated at t = k T and the final condition t = k T + T , allows us to obtain the discrete-time representation for both the leader ( i = L ) and follower ( i = F ) robot in the form,

[ α x i + α y i + α θ i + ] = [ α x i α y i α θ i ] + T ψ i ( W i ) [ cos ( γ i ) − l sin ( γ i ) sin ( γ i ) l cos ( γ i ) 0 1 ψ i ( W i ) ] [ V i W i ]

(5)

where ψ i ( W i ) = sin ( T 2 W i ) T 2 W i and γ i = α θ i + T 2 W i .

Remark 1 It should be noted that ψ i ( W i ) is a well-defined function of W_i and, in particular, of lim W i → 0 ψ i ( W i ) = 1 . In addition, note that model (5) depends directly on the sampling period T and, thus, the consideration of different values of this variable does not affect the properties of the model.

In addition to (5), the following output signals are considered:

y i = [ y i 1 y i 2 ] = [ α i 1 α i 2 ] .

(6)

Model (5)–(6) is used in the sequel to study a leader-follower formation problem.

3. Leader-follower Formation Problem

The configuration of a leader-follower formation is shown in Figure 2. The complexity of this formation is due to the difficulty of measuring the different relative positions and velocities that exist between the leader and the follower robot. The problem can be stated as follows.

OPEN IN VIEWER

Figure 2.

Representation of the leader-follower formation of mobile robots type (2,0) in the plane

Control objective Suppose that the leader robot follows a free path that is subject to the nonholonomic restrictions of the vehicle. The objective of the leader-follower formation is that the follower robot tracks the leader robot while maintaining a constant distance l ˉ 0 . With respect to Figure 2, it is necessary to design a feedback law such that,

lim k → ∞ l 0 ( k T ) = l ˉ 0 ∈ R .

(7)

Remark 2 Model (5) considers the evolution of the frontal point α described by (2). In fact, since the leader robot has a free displacement in the working space; thus, it may not be necessary to consider model (5) in order to describe its dynamics (for example, a different discrete-time dynamic model obtained from (1) could be used). Here, model (5) is considered for the sake of simplicity of notation in the subsequent developments.

From Figure 2, it is possible to obtain the geometrical relationship between the leader and the follower robot. Note first that the distance l₀ between the points ( α x F , α y F ) and ( α x L , α y L ) can be decomposed on the local Cartesian components l_x, l_y that describe the position of the point ( α x F , α y F ) with respect to the moving frame X m − Y m located at the point ( α x L , α y L ) of the leader robot. With respect to the fixed frame X − Y , the distances l_x and l_y can be written as:

l x = − ( α x L − α x F ) cos α θ L − ( α y L − α y F ) sin α θ L l y = ( α x L − α x F ) sin α θ L − ( α y L − α y F ) cos α θ L .

(8)

It is clear that the control objective (7) can be rewritten in terms of the evolution of l_x and l_y. That is, it is required that, lim k → ∞ l x ( k T ) = l ˉ x , lim k → ∞ l y ( k T ) = l ˉ y , for l ˉ x , l ˉ y ∈ R , such that, l ˉ x 2 + l ˉ y 2 = l ˉ 0 2 .

The time evolution of l_x and l_y can be obtained by considering a direct forward shift of equation (8), this is:

l x + = − [ α x L + − α x F + ] cos α θ L + − [ α y L + − α y F + ] sin α θ L +

(9)

l y + = [ α x L + − α x F + ] sin α θ L + − [ α y L + − α y F + ] cos α θ L + .

(10)

Substituting α x L + , α x F + , α y L + , α y F + and α θ L + from equation (5) into (9), leads to:

l x + = − [ α x L + T V L ψ L cos ( γ L ) − T l W L ψ L sin ( γ L ) − ( α x F + T V F ψ F cos ( γ F ) − T l W F ψ F sin ( γ F ) ) ] cos ( α θ L + T W L ) − [ α y L + T V L ψ L sin ( γ L ) + T l W L ψ L cos ( γ L ) − ( α y F + T V F ψ F sin ( γ F ) + T l W F ψ F cos ( γ F ) ) ] sin ( α θ L + T W L ) .

After a number of manipulations and substitutions of some trigonometric identities, one finds that the dynamics of l_x can be written as:

l x + = − ( α x L − α x F ) cos ( α θ L ) cos ( T W L ) − ( α y L − α y F ) sin ( α θ L ) cos ( T W L ) + ( α x L − α x F ) sin ( α θ L ) sin ( T W L ) − ( α y L − α y F ) cos ( α θ L ) sin ( T W L ) − T V L ψ L cos ( T 2 W L ) − T l W L ψ L sin ( T 2 W L ) + T V F ψ F cos ( e θ + ( W F − 2 W L 2 ) T ) − T l W F ψ F sin ( e θ + ( W F − 2 W L 2 ) T )

(11)

where the orientation error between the vehicles e_θ, is defined as e θ = α θ F − α θ L .

Considering equation (8), it is possible to rewrite equation (11) in the form:

l x + = l x cos ( T W L ) + l y sin ( T W L ) − T V L ψ L cos ( T 2 W L ) − T l W L ψ L sin ( T 2 W L ) + T V F ψ F cos ( γ ) − T l W F ψ F sin ( γ )

(12)

where γ = e θ + T ( W F − 2 W L 2 ) .

Applying a similar procedure to l y + in equation (10), one obtains the dynamics:

l y + = − l x sin ( T W L ) + l y cos ( T W L ) + T V L ψ L sin ( T 2 W L ) − T l W L ψ L cos ( T 2 W L ) + T V F ψ F sin ( γ ) + T l W F ψ F cos ( γ ) .

(13)

The leader-follower formation problem can be alternatively described in terms of relative position errors, defined as e l x = l x d − l x , e l y = l y d − l y , where l x d and l y d are the desired relative distances between the vehicles. From Figure 2, it is easy to see that l x d and l y d can be expressed as:

l x d = l ˉ 0 cos ( φ d ( k T ) ) , l y d = l ˉ 0 sin ( φ d ( k T ) )

(14)

for a desired time-varying angle φ d . Note that equation (14) satisfies the relation ( l x d ) 2 + ( l y d ) 2 = l ˉ 0 2 .

The time evolution of e l x can be obtained by considering, again, a direct forward shift; this is:

e l x + = l x d + − ( l x cos ( T W L ) + l y sin ( T W L ) − T V L ψ L cos ( W L T 2 ) − T l W L ψ L sin ( W L T 2 ) + T V F ψ F cos ( γ ) − T l W F ψ F sin ( γ ) ) .

which can be rewritten in the form:

e l x + = l x d + + e l x cos ( T W L ) + e l y sin ( T W L ) + T V L ψ L cos ( T 2 W L ) + T l W L ψ L sin ( T 2 W L ) − T V F ψ F cos ( γ ) + T l W F ψ F sin ( γ ) − l x d cos ( T W L ) − l y d sin ( T W L ) .

(15)

Following a similar procedure for e l y , one obtains the dynamics:

e l y + = l y d + − e l x sin ( T W L ) + e l y cos ( T W L ) − T V L ψ L sin ( T 2 W L ) + T l W L ψ L cos ( T 2 W L ) − T V F ψ F sin ( γ ) − T l W F ψ F cos ( γ ) + l x d sin ( T W L ) − l y d cos ( T W L ) .

(16)

4. Delayed Feedback Solution to the Leader-follower Problem

According to the definition of the leader-follower problem as stated in Section 3, the follower robot must be able to follow any path described by the leader robot while maintaining a constant distance with respect to it. Therefore, the design of the control strategy should be based on the follower's control signals V_F, W_F. In order to simplify the developments of the required feedback, equations (15) and (16) can be rewritten in the compact form:

[ e l x + e l y + ] = A 1 ( W L ) [ e l x − l x d e l y − l y d ] + T ψ L A 2 ( W L ) [ V L W L ] + T ψ F A 3 ( γ ) [ V F W F ] + [ l x d + l y d + ]

(17)

where,

A 1 ( W L ) = [ cos ( T W L ) sin ( T W L ) − sin ( T W L ) cos ( T W L ) ] , A 2 ( W L ) = [ cos ( T 2 W L ) l sin ( T 2 W L ) − sin ( T 2 W L ) l cos ( T 2 W L ) ] , A 3 ( γ ) = [ − cos ( γ ) l sin ( γ ) − sin ( γ ) − l cos ( γ ) ] .

To obtain a solution to the leader-follower problem, the virtual control signals, ξ 1 = T ψ F V F and ξ 2 = T ψ F W F are considered. Note also that, by defining e l = [ e l x , e l y ] T , l d = [ l x d , l y d ] T , ξ = [ ξ 1 , ξ 2 ] T and u L = [ V L , W L ] T , equation (17) can be rewritten as:

e l + = A 1 ( W L ) e l − A 1 ( W L ) l d + T ψ L A 2 ( W L ) u L + A 3 ( γ ) ξ + l d + .

(18)

A feedback law can now be synthesized based on the virtual controls ξ as:

ξ = A 3 ( γ ˉ ) − 1 [ v − A 1 ( W L ) e l + A 1 ( W L ) l d − T ψ L A 2 ( W L ) u L − l d + ]

(19)

where the new input signal v is selected as:

v = K e l = [ k 1 0 0 k 2 ] [ e l x e l y ]

(20)

where k 1 , k 2 ∈ R , with | k 1 | < 1 , | k 2 | < 1 and γ ˉ = e θ + T 2 ( W F − − 2 W L ) .

The original control signals V_F, W_F can be derived from ξ₁ and ξ₂, more precisely:

V F = ξ 1 T ψ ( W F ) , W F = 2 T sin − 1 ( ξ 2 2 ) .

(21)

Remark 3 It should be pointed out that control signals V_F, W_F cannot be synthesized directly from equation (17) due to the dependence of the matrix A 3 ( γ ) on the signal W_F, since γ = e θ + T ( W F − 2 W L 2 ) . For this reason, equation (19) is defined in terms of the delayed function γ ˉ that allows us to synthesize the virtual control signal ξ as a function of W F − = W F ( k T − T ) .

Let us now consider the formation error dynamics (18), together with the feedback (19); this is:

e l + = A l ( W L ) e l − A l ( W L ) l d + T Ψ L A 2 ( W L ) u L + A 3 ( γ ) A 3 ( γ ˉ ) − 1 [ v − A 1 ( W L ) e l + A 1 ( W L ) l d − T Ψ L A 2 ( W L ) u L − l d + ] + l d + .

(22)

Note first that,

A 3 ( γ ) A 3 ( γ ˉ ) − 1 = [ cos ( γ − γ ˉ ) − sin ( γ − γ ˉ ) sin ( γ − γ ˉ ) cos ( γ − γ ˉ ) ]

where γ − γ ˉ = T 2 ( W F − W F − ) . Since cos ( x ) = 1 − 2 sin 2 ( x 2 ) , it is possible to write:

A 3 ( γ ) A 3 ( γ ˉ ) − 1 = I − Δ ( γ − γ ˉ ) = I − [ ∂ 1 ( γ − γ ˉ ) ∂ ( γ − γ ˉ ) − ∂ ( γ − γ ˉ ) ∂ 1 ( γ − γ ˉ ) ]

(23)

where I ∈ R 2 is an identity matrix and ∂ ( γ − γ ˉ ) = sin ( γ − γ ˉ ) , ∂ 1 ( γ − γ ˉ ) = 2 sin 2 ( γ − γ ˉ ) . Hence, equation (22) takes the form:

e l + = K e l − Γ

(24)

with Γ = [ Γ x Γ y ] T being a non-linear term of the form:

Γ = h 1 + h 2

(25)

where

h 1 = Δ K − A 1 ( W L ) [ e l ] , h 2 = Δ A 1 ( W L ) l d − T ψ ( W L ) A 2 ( W L ) u L − l d + .

It is possible to show that the terms h₁ and h₂ are appropriated bounded functions. The term h₁ can be majored as:

‖ h 1 ‖ = ‖ Δ K − A 1 ( W L ) e l ‖ ≤ ‖ Δ ‖ [ ‖ K − A 1 ( W L ) ‖ ] ‖ e l ‖ .

Then, due to the boundedness of matrices Δ, K and A 1 ( W L ) , there exists a non-zero real positive C₁, such that:

‖ h 1 ‖ ≤ C 1 ‖ e l ‖ .

(26)

The term h₂ can be majored as:

‖ h 2 ‖ ≤ ‖ Δ ‖ [ ‖ A 1 ( W L ) l d − T ψ L A 2 ( W L ) u L − l d + ‖ ] ≤ ‖ Δ ‖ [ ‖ A 1 ( W L ) ‖ ‖ l d ‖ + T ‖ ψ L ‖ ‖ A 2 ( W L ) ‖ ‖ u L ‖ + ‖ l d + ‖ ] .

Since u L = [ V L W L ] T , l d and l d + are bounded by design, it is possible to proof that the function ψ_L and matrices Δ, A 1 ( W L ) and A 2 ( W L ) are also bounded. Then, it is possible to write:

‖ h 2 ‖ ≤ C 2

(27)

for a non-zero positive constant C₂. Hence, from (26) and (27), it is obtained that:

‖ Γ ‖ ≤ C 1 ‖ e l ‖ + C 2 .

(28)

4.1. Formation error analysis

Based on the previous developments, it is now possible to state the main result on the evolution of the formation error, as follows.

Lemma 4 Let us consider a pair of differentially driven mobile robots, each of which is described by (5), together with the discrete-time feedback (19)–(21) where the real parameters k₁ and k₂ are such that | k i | < 1 for i = 1,2 . Then, if the condition:

‖ K ‖ + C 1 < 1

(29)

is satisfied, then the formation error e l = [ e l x e l y ] T is ultimately bounded.

Proof. The time evolution of the closed-loop formation error e can be determined by considering the solution of equation (24); this is:

e l ( n ) = K n e l ( 0 ) − ∑ m = 0 n − 1 K ( n − 1 − m ) Γ ( m )

(30)

which can be majored by using bound (28) as follows:

‖ e l ( n ) ‖ ≤ ‖ K ‖ n ‖ e l ( 0 ) ‖ + ∑ m = 0 n − 1 ‖ K ‖ ( n − 1 − m ) { C 1 ‖ e l ( m ) ‖ + C 2 } .

(31)

Since the eigenvalues of the matrix K are all within the unit circle, one finds that:

lim n → ∞ ( ‖ K ‖ n ) = 0 and lim n → ∞ ( ∑ m = 0 n − 1 ‖ K ‖ ( n − 1 − m ) ) = 1 1 − ‖ K ‖ .

Therefore, from equation (31), the ultimate bound can be obtained as:

lim n → ∞ ‖ e l ( n ) ‖ ≤ C 2 1 − ‖ K ‖ − C 1 .

(32)

Then, if condition (29) holds, the formation error e l = [ e l x e l y ] T is ultimately bounded.

Remark 5 If the approximation of the angular acceleration as a function of the angular velocity W_F is considered, this is a W F ≈ ( W F − W F − ) T , then γ − γ ˉ ≈ T 2 2 a W F . Therefore, consideration of a low angular acceleration, together with a sufficiently small sampling period, allows us to ensure that γ − γ ˉ ≈ 0 .

Remark 6 Note that the magnitudes of constants C₁, C₂ in the ultimate bound (32) directly depend on the norm of matrix Δ in equation (23). It is clear that when ‖ Δ ‖ → 0 , one has C 1 , C 2 → 0 , ensuring in this way that lim n → ∞ ‖ e ( n ) ‖ = 0 . Since the magnitude of ‖ Δ ‖ depends on γ − γ ˉ , a small ultimate bound is obtained in terms of Remark 5.

Remark 7 Note that in the special case in which the angular velocity is constant, as it is in the case of a straight or a circular path, the perturbation term Γ becomes a vanishing perturbation. In this case, the closed-loop formation error asymptotically converges to zero.

The result stated in Lemma 4 assures the boundedness of the formation error but it cannot determine the evolution of the orientation of the follower robot. This is an important issue since the control of the follower vehicle is carried out by considering the point α F = ( α x F , α y F ) , which is located at the front of the longitudinal axis of the vehicle. A good reference for the orientation of the follower vehicle is the orientation of the leader vehicle; that is, the error e θ = α θ F − α θ L , which has a time evolution that can be obtained by considering its corresponding forward shift, more precisely:

e θ + = e θ + T ( W F − W L ) .

(33)

Since W_L is an input to the leader robot that generates the tracking trajectory, it can be assumed to be bounded. Also, from (21) it is clear that W_F is also a bounded function. Therefore, the evolution of the formation error is bounded. The evolution of α θ L is determined by the trajectory described by the leader robot. To determine the evolution of α θ F , the first two equations of the discrete time model (5) are considered in order to obtain:

( α x F + − α x F ) sin ( γ F ) − ( α y F + − α y F ) cos ( γ F ) + l ψ F T W F = 0.

Then, by considering the last equation in (5) and the definition of γ_F, one obtains:

( α x F + − α x F ) sin ( α θ F + + α θ F 2 ) − ( α y F + − α y F ) cos ( α θ F + + α θ F 2 ) + 2 l sin ( α θ F + − α θ F 2 ) = 0.

(34)

Equation (34) is the discrete-time counterpart of the continuous time nonholonomic restriction:

α · x sin θ − α · y cos θ + l θ · = 0

associated with the original system (3). The importance of equation (34) is that since the proposed leader-follower control strategy determines the evolution of the point ( α x F , α y F ) , the evolution of the orientation of the follower robot α θ F must satisfy restriction (34). This fact shows the boundedness of α θ F .

5. Experimental Evaluation

The solution proposed for the leader-follower formation problem (21) is initially evaluated by numerical simulations and by real-time experiments carried out using a pair of differentially driven wheeled mobile robots of the type Pioneer 3-DX from MobileRobots Inc.

5.1. Numerical results

A numerical experiment (NE) was carried out by considering that the leader robot follows a circular path with a radius of 1.8 m centred at ( x , y ) = ( 3 m ,3 m ) . The trajectory described by the leader robot is generated by considering a linear velocity V L = 1 m / s and an angular velocity W L = 0.5 r a d / s . It is considered that the follower robot remains at a constant distance of one metre behind the leader robot, this is l ˉ 0 = 1 m and φ d = 0 r a d (equivalently, l x d = 1 m and l y d = 0 m ). The initial conditions considered in the experiment are x L ( 0 ) = 3 m , y L ( 0 ) = 1 m , θ L ( 0 ) = 0 r a d and x F ( 0 ) = 2.5 m , y L ( 0 ) = 2.5 m , θ L ( 0 ) = 0 r a d . A sampling period of T = 0.1 s was used, together with gains k 1 = 0.4 and k 2 = 0.4 .

Figure 3 depicts the evolution of the leader-follower formation on the Cartesian plane. It can be observed that the trajectory tracked by the follower robot is different from the one developed by the leader robot. This is a consequence of the control strategy that considers the control of the relative distance between the robots and not a specific path.

OPEN IN VIEWER

Figure 3.

NE. Time formation evolution on the Cartesian plane.

The tracking errors e l x and e l y , associated with the desired relative distance l ˉ 0 , are shown in Figure 4 where it can be observed that e l x and e l y converge to the origin due to the constant angular velocity considered in the experiment (see Remark 7).

OPEN IN VIEWER

Figure 4.

NE. Cartesian relative errors e l x , e l y .

5.2. Experimental platform

As mentioned above, two differentially driven mobile robots of the type Pioneer 3-DX that support loads of up to 21 k g were used. For these robots, l = 0.1 m in equation (2), and r = 0.0973 m , m = 0.34 m in equation (4). To carry out the experiment, the leader robot carries a recognition graphical pattern placed at the midpoint of the wheel's axis. The pattern consists of three white rectangles within a black rectangle (see Figure 6). The follower robot has a Kinect device positioned at the midpoint of the wheel's axis and an on-board laptop in which image processing and calculations of the control law programmed in C++ are performed. An RGB camera in the Kinect device captures images that are smoothed and then binarized. Then, a program in the laptop searches the recognition pattern and detects the centroids of each of the three rectangular contours within a larger rectangular contour.

OPEN IN VIEWER

Figure 6.

Relative distance and orientation recognition using the graphical pattern and Kinect device

5.2.1. Vision-based localization system

This section provides an overview of the major steps of our method for detecting the leader robot from the visual information provided by the Kinect device that is on-board the follower robot for feedback purposes. The Microsoft Kinect device contains a depth sensor, an RGB camera and a four-microphone array. The Kinect device has the capabilities to perform 3D motion capture, and face and voice recognition [29]. The depth sensor consists of an infrared projector to differentiate depth by infrared vision. The infrared projector and a monochrome CMOS sensor in the Kinect device is combined to capture video data in 3D under any ambient light conditions. The vision sensor provides a maximum of 30 frames per second with a resolution of 640 × 480 pixels for each camera. Due to its limitations, the depth sensor can be used for indoor navigation only.

The Kinect depth sensor can detect the distance of an object so that an autonomous robot can navigate while avoiding obstacles in an indoor environment [30]. The three-dimensional space is reconstructed by merging the two images obtained by the RGB and the depth camera. The RGB camera obtains the image of the environment while the depth camera obtains the depth information of each pixel in the image. A detailed analysis of the accuracy and resolution of the Kinect can be found in [31].

Before its use, the Kinect has to be calibrated to merge the data of the images obtained by the cameras and to process the raw data of the depth sensor. This calibration was done as in [32], following the procedure developed in [33]. The depth information obtained is linearly perpendicular to the longitudinal axis sensor. Thus, all the objects that are over the same line will have the same depth value, regardless of whether they are at the edge or at the centre of the image; this distance is denoted as l x r .

Figure 5 shows how the distance between the centre of the image and a centroid of the recognition pattern board is calculated; first by obtaining the depth information of the central point of the image and then by determining the number of pixels that exist between these two points at the same depth. The distance covered by each pixel is obtained experimentally using the approximate relation c m p p i x e l = 0.169 ∗ d e p t h + 0.016 obtained for a distance between 0.9 m and 3.5 m , where the first constant depends on the standard geometric characteristics of the cameras and the free constant term is a correction factor. See [32, 33] for more information.

OPEN IN VIEWER

Figure 5.

Distances between the centre of the image to the centroid of the recognition pattern

Knowing the number of pixels N P between the centre of the image and a centroid of the recognition pattern, the length l y r is obtained as l y r = N P ∗ c m p p i x e l . If the centroid is on the right of the image this length will be positive, and negative if it is on the left.

Figure 6 shows how the recognition pattern on the leader robot is used to obtain the orientation error e_θ. Once the distance of the three centroids are determined from the Kinect information, the distances A and B (see Figure 6) are used to compute the relative orientation angle, e_θ, between the robots. This error is calculated using basic trigonometry; since the distance between the external centroids on the graphical pattern ( C = 11.5 c m ) is known, and the distances A and B are known, it is a simple matter to obtain the orientation error as e θ = sin − 1 C A − B . The tracking distances l_x, l_y and the orientation error e_θ are calculated in real-time at each iteration of the control.

From the geometric relation between the robots, as shown in Figure 7, it is clear that the relative distances l_x, l_y used to obtain the solution to the formation problem can be obtained as: l x = l x r + l x α , l y = l y r + l y α where l x α = l cos ( e θ ) , l y α = l sin ( e θ ) .

OPEN IN VIEWER

Figure 7.

Geometrical representation of the alpha point and the middle point of each robot

5.3. Real-time results

Two real-time experiments were carried out in order to evaluate the proposed solution. The first one considered the tracking of a straight path by the leader robot, and a sinusoidal path for the second experiment. In both cases, the follower robot must track the leader vehicle at a constant distance of one metre l ˉ 0 = 1 m with an angle φ d = 0 r a d (that is, l x d = 1 m and l y d = 0 m ). A sampling time of T = 0.05 s was considered for the control loop. The Kinect device obtained the image and the depth data of all the pixels. Then, using the software OpenCV, the image was smoothed and binarized so it was possible to recognize the pattern board on the leader. The centroids of each rectangle were obtained knowing the pixels of the centroids, and the linear distances were obtained from the depth data; in this way, the measurement of the relative distances l_x, l_y were obtained, so that it was possible to compute l₀ and angle φ Then, the control variables were calculated and sent to the follower robot every 0.3 s . It is important to point out that even when the control loop was carried out with a faster sampling time, the new control signal in the follower robot changed until a new relative distance measurement was obtained. For the real-time experiments, it is not considered to be a local absolute localization system and, therefore, it is not possible to show the evolution of the formation on the Cartesian plane.

5.3.1. Straight path experiment (StPE)

Even when, in the presented solution to the problem, internal or external disturbances have not been considered, a disturbance was introduced on the straight path experiment as a practical evaluation of the robustness of the discrete-time strategy. At approximately t = 11 s , the follower robot was manually shifted from its current position, thus introducing instantaneous tracking errors and causing the vision system to lose the tracking reference pattern; incorrect estimations of the relative distances and angles between the robots thus appeared. The straight line was generated by considering V L = 1 m / s and W L = 0 r a d / s . As an initial condition, the follower robot was placed at l 0 ( 0 ) = 1.5 m and φ ( 0 ) = 3.3685 r a d (that is, l x ( 0 ) = − 1.438 m and l y ( 0 ) = − 0.425 m ) from the leader robot.

The evolution of the relative errors e l x , e l y is depicted in Figure 8, which shows their convergence to a neighbourhood of the origin until the mentioned disturbance appeared. After the new relative position was measured, the convergence of e l x , e l y was recovered. The evolution of the non-controlled error signal e_θ is shown in Figure 9, where it can be appreciated that it remained constant for the period that the disturbance was not affecting the experiment. Finally, Figure 10 shows the time evolution of the control signals.

OPEN IN VIEWER

Figure 8.

StPE. Behaviour of the relative errors e l x , e l y .

OPEN IN VIEWER

Figure 9.

StPE. Evolution of the non-controlled error e_θ.

OPEN IN VIEWER

Figure 10.

StPE. Control signals V_F, W_F.

Note that the effects of the external disturbance are more evident in Figure 9, where e_θ becomes an unbounded signal and, because of the physical characteristics of the experiment, this signal should have a magnitude between ± π rad. The control signal W_F is shown in Figure 10. It should be pointed out that during the interval of time when the disturbance was acting, the control signal (based on measured distances) was not acting on the mobile robot.

5.3.2. Sinusoidal path experiment (SiPE)

The sinusoidal path described by the leader robot was obtained by considering: V L = 0.16 m / s and W L = − 0.647 sin ( k T ) r a d / s for k = 0,1,2,3,… and T = 0.1 s . The initial conditions of the experiment were set to l 0 ( 0 ) = 0.8 m and φ ( 0 ) = π r a d .

The relative errors e l x , e l y are shown in Figure 11; it can be noted that they converge to a neighbourhood of the origin, evolving in a gap of ± 5 c m for e l x and ± 8 c m for e l y . Note that this gap depends on the constant associated with the norm of the perturbation term Γ, as well as on the magnitude of the relative error e_l. The main disadvantage in our experiment is the low refresh rate of the vision system, which induces an increment on the relative error e_l and consequently on the magnitude of the neighbourhood's convergence. The evolution of the non-controlled signal e_θ is shown in Figure 12 and it is bounded, as expected, from equation (34), oscillating in a gap of ± 0.3 r a d . The time evolution of the control signals is shown in Figure 13.

OPEN IN VIEWER

Figure 11.

SiPE. Behaviour of the errors e l x , e l y .

OPEN IN VIEWER

Figure 12.

SiPE. Evolution of the non-controlled error e_θ.

OPEN IN VIEWER

Figure 13.

SiPE. Control signals V_F, W_F.

Finally, the disturbance terms Γ x and Γ y , in equations (24)–(25) are depicted in Figure 14, where it can be seen that they are appropriated bounded, as it is required in order to obtain a small ultimate bound for the relative errors e l x and e l y .

OPEN IN VIEWER

Figure 14.

SiPE. Perturbation signals evolution Γ x , Γ y .

6. Conclusions

The leader-follower formation problem for nonholonomic differentially driven mobile robots is addressed in this work by considering the discrete-time representation of the kinematic models of the robots. To deal with the associated kinematics constraints, a control located at the front of the robot is considered. A solution to the problem is obtained by considering a single sampling time delayed feedback, based on relative distances and angles between the robots that are measured by an on-board camera mounted on the follower robot, together with a graphical pattern mounted on the leader robot.

It is formally proven that the relative Cartesian errors with respect to a desired relative distance between the robots are ultimately bounded. The main advantage of the proposed discrete-time strategy lies in its ease of implementation on microprocessors or computer-based platforms, and the consideration of the vision system that is of a discrete nature. The main disadvantage of the solution proposed is the closed-loop practical stability property of the convergence errors, since they converge to the origin only for constant angular velocities. The proposed strategy was evaluated via numerical and real-time experiments in a laboratory environment, using a low-cost measurement device. The use of relative distances between the leader and the follower robot has been previously used in the literature in a continuous time context; it is shown in this work that a discrete-time approach can also be exploited in the solution to the problem.