Stochastic Consensus of a Class of Continuous-Time Multi-Agent Systems with a Leading Agent

1. Introduction

Over the last decade, multi-agent systems have been a popular subject in various research fields, because of their extensive applications in areas such as formation control, flocking, synchronization and distributed inference. In a multi-agent system, each agent interacts with the others in a coupling topology, which usually varies with time. One of the key problems of multi-agent systems concerns consensus, which refers to all agents asymptotically reaching a common state.

For multi-agent systems with an arbitrary switching coupling topology, [1] and [2] provided consensus results using Lyapunov functions and a spinning tree method, respectively. Consensus conditions were presented in [3] and [4] for multi-agent systems with a stochastic switching coupling topology driven by a Bernoulli distribution and a Markov process, respectively. The coupling functions considered in [1–4] were linear. As many applications of multi-agent systems involve nonlinear couplings, consensus with nonlinear couplings has recently attracted enormous attention. For multi-agent systems with directed nonlinear couplings, the consensus problem has been investigated using contraction analysis [5]. For multi-agent systems with undirected nonlinear couplings, [6] and [7] presented frequency-domain consensus criteria using an absolute stability approach [9,10]. Absolute stability theory has also been applied to the problem of directed nonlinear couplings [8], which resulted in linear matrix inequality (LMI) consensus conditions with a fixed coupling topology. The coupling topologies in [5–7] had arbitrary switching conditions.

The above research on nonlinear couplings has not considered stochastic switching coupling topologies. In fact, the couplings among agents usually have certain stochastic properties. In particular, in many multi-agent systems, the conditional probability distribution of future switching of the coupling topology depends only upon the present coupling topology. Hence, the stochastic switching can be described by a Markov process. In addition, [5–8] assumed that the coupling functions among agents were continuous, or even differentiable. This assumption imposes some restrictions on the applicability of discontinuous nonlinear coupling functions. This paper explores consensus conditions in the form of LMIs for continuous-time multi-agent systems with a Markov switching coupling topology, a leading agent and unknown directed nonlinear couplings. Our results can be applied to discontinuous nonlinear coupling functions.

The remainder of this paper is organized as follows: Section 2 introduces the model and the problem description. A stochastic consensus condition is developed in Section 3. Section 4 presents a numerical example and the paper is concluded in Section 5.

2. System Modelling

Let ℜ denote the field of real numbers, e_i the ith real coordinate vector, I n × n the identity matrix of dimension n and 0 m × n the m × n zero matrix. The matrix A T is the transpose of the matrix A. For a square symmetric matrix A, λ min ( A ) denotes the minimal eigenvalue of A and A<0 indicates that A is a negative definite matrix. E [ ⋅ ] defines the expectation. ⊗ is the Kronecker product of matrices.

A directed graph, denoted by D = ( M , S ) , consists of a node set M = { 1 , 2 , ⋯ , m } and an arc set S = { ( i , j ) : i , j ∈ M , i ≠ j } . The coupling topology of a multi-agent system can be described through a directed graph: node i represents agent i and arc (i, j) represents a directed coupling from agent j to agent i. For a directed graph D, the adjacency a i j is given by

a i j = { 1 if ( i , j ) ∈ S 0 otherwise

and the Laplacian matrix L is defined as

L = [ ∑ j ∈ V \ { 1 } a 1 j − a 12 … − a 1 m − a 21 ∑ j ∈ V \ { 2 } a 2 j ⋯ − a 2 m ⋮ ⋮ ⋮ ⋮ − a m 1 − a m 2 ⋯ ∑ j ∈ V \ { m } a m j ] .

This paper considers multi-agent systems in a stochastically switching coupling topology. For the multi-agent system considered, we have p Laplacian matrices L 1 , ⋯ L p . The multi-agent system has a time-varying Laplacian matrix

L θ ( t ) = [ ∑ j ∈ M \ { 1 } g 1 j ( t ) − g 12 ( t ) … − g 1 m ( t ) − g 21 ( t ) ∑ j ∈ M \ { 2 } g 2 j ( t ) ⋯ − g 2 m ( t ) ⋮ ⋮ ⋮ ⋮ − g m 1 ( t ) − g m 2 ( t ) ⋯ ∑ j ∈ M \ { m } g m j ( t ) ] ∈ ℜ m × m

(1)

θ ( t ) is driven by a continuous-time homogeneous Markov process and takes values in { 1 , ⋯ , p } with transition probabilities

p r o b ( θ ( t + δ t ) = q | θ ( t ) = k ) = { π k q δ t + ο ( δ t ) i f q ≠ k 1 + π k k δ t + ο ( δ t ) i f q = k

where

π k q ≥ 0 k ∈ { 1 , ⋯ , p } , q ∈ { 1 , ⋯ , p } , k ≠ q

are the known switching rates,

π k k = − ∑ q = 1 q ≠ k p π k q k ∈ { 1 , ⋯ , p }

and δ t > 0 , lim δ t → 0 o ( δ t ) δ t = 0 .

The continuous-time multi-agent system studied in this paper is modelled as

{ X · 1 ( t ) = − ∑ j e M \ { 1 } g 1 j ( t ) F 1 j [ X 1 ( t ) − X j ( t ) ] + B 1 ( X 1 ( t ) − X ˉ ) X · i ( t ) = − ∑ j e M \ { i } g i j ( t ) F i j [ X i ( t ) − X j ( t ) ] i ∈ M \ { 1 }

(2)

where time t ∈ [ 0 , ∞ ) ; X 1 ( t ) ∈ ℜ n is the state of agent 1 at t; X i ( t ) ∈ ℜ n is the state of agent i at t; X ˉ ∈ ℜ n is a prescribed target point; B 1 ∈ ℜ n × n is a known matrix; and the nonlinear maps F 1 j : ℜ n → ℜ n and F i j : ℜ n → ℜ n represent couplings among agents. From model (2), it can be seen that only agent 1 is made aware of target point X ˉ . Accordingly, agent 1 is called the leading agent. For any i ∈ M and any j ∈ M \ { i } , the nonlinear map F i j [ ] is unknown, but subject to

{ F i j [ 0 n × 1 ] = 0 n × 1 Y T Φ i j Y ≤ Y T F i j [ Y ] ∀ Y ∈ ℜ n F i j T [ Y ] F i j ≤ Y T Ψ i j F i j [ Y ] ∀ Y ∈ ℜ n

(3)

where Φ i j ∈ ℜ n × n and Ψ i j ∈ ℜ n × n are known positive definite matrices with Φ i j − Ψ i j < 0 . Map F i j [ ] may be discontinuous.

REMARK: When n=1 in condition (3), we have

{ F i j [ 0 ] = 0 Φ i j Y T Y ≤ Y T F i j [ Y ] ≤ Ψ i j Y T Y ∀ Y ∈ ℜ

which is the sector-shaped restriction in the absolute stability approach. In other words, condition (3) can be viewed as a multi-input–multi-output version of sector-shaped restriction.

The multi-agent system is said to reach stochastic consensus if ∀ i ∈ M , ∀ X i ( 0 ) ∈ ℜ n , ∀ θ ( 0 ) ∈ { 1 , 2 , … , p } ,

lim t → ∞ E [ ( X i ( t ) − X ˉ ) T ( X i ( t ) − X ˉ ) ] = 0.

In the next section, we derive conditions for stochastic consensus.

3. Stochastic Consensus Conditions

In system (2), we first define

Z i ( t ) = X i ( t ) − X ˉ

(4)

After the offset, (2) is transformed into the equivalent

{ Z · 1 ( t ) = B 1 Z 1 ( t ) − ∑ j e M \ { 1 } g 1 j ( t ) F 1 j [ Z 1 ( t ) − Z j ( t ) ] Z · i ( t ) = − ∑ j e M \ { i } g i j ( t ) F i j [ Z i ( t ) − Z j ( t ) ] i ∈ M \ { 1 }

(5)

From (3), we know that Z 1 = Z 2 = ⋯ = Z m = 0 n × 1 is an equilibrium point of system (5). Denote

Z ( t ) = [ Z 1 T ( t ) ⋯ Z m T ( t ) ] T ∈ ℜ m n ;

B = d i a g ( B 1 , 0 n × n , ⋯ , 0 n × n ) ∈ ℜ m × m n ;

C = [ I n × n − I n × n I n × n − I n × n ⋮ ⋱ I n × n − I n × n − I n × n I n × n I n × n − I n × n ⋮ ⋱ I n × n − I n × n ⋮ − I n × n I n × n − I n × n I n × n ⋱ ⋮ − I n × n I n × n ] ∈ ℜ ( m 2 n − m n ) × m n

For any W 1 , W 2 , ⋯ , W m ∈ ℜ n and W = [ W 1 T W 2 T ⋯ W m T ] T ∈ ℜ m n , denote

F [ W ] = [ F 12 [ W 2 ] F 13 [ W 3 ] ⋮ F 1 m [ W m ] F 21 [ W ] F 23 [ W 3 ] ⋮ F 2 m [ W m ] ⋮ F m 1 [ W 1 ] F m 2 [ W 2 ] ⋮ F m , m − 1 [ W m − 1 ] ] ∈ ℜ m 2 n − m n

By erasing the diagonal entries of L θ ( t ) in (1), we obtain

G θ ( t ) = [ − g 12 ( t ) − g 13 ( t ) ⋯ − g 1 m ( t ) − g 21 ( t ) − g 23 ( t ) ⋯ − g 2 m ( t ) ⋮ ⋮ ⋮ ⋮ − g m 1 ( t ) − g 2 m ( t ) ⋯ − g m , m − 1 ( t ) ] ∈ ℜ m × ( m − 1 )

Q θ ( t ) = d i a g ( e 1 T G θ ( t ) , e 2 T G θ ( t ) , ⋯ e m T G θ ( t ) ) ⊗ I n × n ∈ ℜ m n × ( m 2 n − m n )

Taking advantage of Z ( t ) , B, C, F[] and Q θ ( t ) , multi-agent system (2) is expressed concisely as

Z · ( t ) = B Z ( t ) + Q θ ( t ) F [ C Z ( t ) ]

(6)

with an equilibrium point Z = 0 m n × 1 .

THEOREM 1: For multi-agent system (2), suppose that there exist positive definite P 1 , P 2 , ⋯ P m ∈ ℜ m n × m n , positive definite diagonal U 1 ∈ ℜ ( m 2 − m ) × ( m 2 − m ) and positive definite diagonal U 2 ∈ ℜ ( m 2 − m ) × ( m 2 − m ) , such that ∀ k ∈ { 1 , 2 , ⋯ , p } ,

[ ∑ q = 1 p π k q P q + B T P k + P k B − 2 C T U 1 Φ C P k Q k + C T U 1 + C T Ψ U 2 Q k T P k + U 1 C + U 2 Ψ C − 2 U 2 ] < 0

(7)

with

Φ = d i a g ( Φ 12 , Φ 13 , ⋯ , Φ 1 m , Φ 21 , Φ 23 , ⋯ , Φ 2 m , ⋯ , Φ m 1 , Φ m 2 , ⋯ , Φ m , m − 1 )

Ψ = d i a g ( Ψ 12 , Ψ 13 , ⋯ , Ψ 1 m , Ψ 21 , Ψ 23 , ⋯ , Ψ 2 m , ⋯ , Ψ m 1 , Ψ m 2 , ⋯ , Ψ m , m − 1 )

Then, system (2) reaches stochastic consensus.

PROOF. From condition (3), ∀ Z ( t ) ∈ ℜ m n ,

[ Z ( t ) F ( C Z ( t ) ) ] T [ 2 C T U 1 Φ C − C T U 1 − U 1 C 0 ( m 2 n − m n ) × ( m 2 n − m n ) ] [ Z ( t ) F ( C Z ( t ) ) ] ≤ 0

(8)

[ Z ( t ) F ( C Z ( t ) ) ] T [ 0 m n × m n − C T Ψ U 2 − U 2 Ψ C 2 U 2 ] [ Z ( t ) F ( C Z ( t ) ) ] ≤ 0

(9)

Let

H 1 = [ 2 C T U 1 Φ C − C T U 1 − U 1 C 0 ( m 2 n − m n ) × ( m 2 n − m n ) ]

H 2 = [ 0 m n × m n − C T Ψ U 2 − U 2 Ψ C 2 U 2 ]

H 3 ( k ) = [ ∑ q = 1 p π k q P q + B T P k + P k B − 2 C T U 1 Φ C P k Q k + C T U 1 + C T Ψ U 2 Q k T P k + U 1 C + U 2 Ψ C − 2 U 2 ]

Construct the stochastic Lyapunov function V ( Z ( t ) , θ ( t ) = k ) = V ( Z , k ) = Z T ( t ) P k Z ( t ) t ∈ [ 0 , ∞ ) .

We have ∀ Z ( t ) ∈ ℜ m n , ∀ θ ( t ) = k ∈ { 1 , 2 , ⋯ , p } , the weak infinitesimal operator (a stochastic analogue [11–13] of the deterministic derivative)

D ˜ V ( Z , k ) = lim δ t → 0 E [ V ( Z ( t + δ t ) , θ ( t + δ t ) ) | Z ( t ) , θ ( t ) = k ] δ t − lim δ t → 0 V ( Z ( t ) , θ ( t ) = k ) δ t

From this,

V ( Z ( t + δ t ) , θ ( t + δ t ) ) = Z T ( t + δ t ) P θ ( t + δ t ) Z ( t + δ t ) = ( Z T ( t ) + Z · T ( t ) δ ( t ) ) P θ ( t + δ t ) ( Z ( t ) + Z · T ( t ) δ ( t ) ) = Z T ( t ) P θ ( t + δ t ) Z ( t ) + Z · T ( t ) P θ ( t + δ t ) Z ( t ) δ ( t ) + Z T ( t ) P θ ( t + δ t ) Z · T ( t ) δ ( t ) + Z · T ( t ) P θ ( t + δ t ) Z · T ( t ) ( δ ( t ) ) 2

Thus, we see that

D ˜ V ( Z , k ) = Z T ( t ) lim δ t → 0 E [ P θ ( t + δ t ) | θ ( t ) = k ] − P k δ t Z ( t ) + Z T ( t ) lim δ t → 0 E [ P θ ( t + δ t ) | θ ( t ) = k ] Z · ( t ) + Z · T ( t ) lim δ t → 0 E [ P θ ( t + δ t ) | θ ( t ) = k ] Z ( t ) = Z T ( t ) ∑ q = 1 p π k q P q Z ( t ) + Z T ( t ) P k ( B Z ( t ) + Q F [ C Z ( t ) ] ) + ( B Z ( t ) + Q k F [ C Z ( t ) ] ) T P k Z ( t ) = Z T ( t ) ( ∑ q = 1 p π k q P q + B T P k + P k B ) Z ( t ) + Z T ( t ) P k Q k F [ C Z ( t ) ] ) + F T [ C Z ( t ) ] Q k T P k Z ( t )

(10)

Combining (7)–(10), we have that, for any nonzero Z ( t ) ∈ ℜ m n , ∀ k ∈ { 1 , 2 , ⋯ p } ,

D ˜ V ( Z , k ) = [ Z ( t ) F [ C Z ( t ) ] ] T ( H 1 + H 2 + H 3 ( k ) ) [ Z ( t ) F [ C Z ( t ) ] ] ≤ [ Z ( t ) F [ C Z ( t ) ] ] T H 3 ( k ) [ Z ( t ) F [ C Z ( t ) ] ] < 0

The positive definite Lyapunov function V ( Z , k ) with D ˜ V ( Z , k ) < 0 means that ∀ Z ( 0 ) ∈ ℜ m n , ∀ θ ( 0 ) ∈ { 1 , 2 , ⋯ , p } ,

lim t → ∞ E [ Z T ( t ) P θ ( t ) Z ( t ) ] = lim t → ∞ E [ V ( Z , k ) ] = 0

(11)

Applying

μ = min k ∈ { 1 , 2 , ⋯ , p } λ min ( P k ) > 0

μ Z T ( t ) Z ( t ) ≤ Z T ( t ) P θ ( t ) Z ( t )

(11) implies that lim t → ∞ E [ Z T ( t ) Z ( t ) ] = 0 , i.e., system (2) reaches stochastic consensus.

4. Simulation

In this section, an example is developed to illustrate Theorem 1. The example considers a multi-agent system consisting of six agents with a Markov switching coupling topology. The dimension of each agent is one. At any time, the time-varying coupling topology is one of the four coupling topologies displayed in Fig. 1. Therefore, m=6, n=1 and p=4 in the example. In addition, let B 1 = − 0.8 and

OPEN IN VIEWER

Figure 1.

Four coupling topologies

π 12 = 4.0 , π 13 = 0.6 ; π 14 = 2.1 π 21 = 1.3 , π 23 = 0.0 ; π 24 = 1.8 π 31 = 0.0 , π 32 = 4.9 ; π 34 = 3.6 π 41 = 0.7 , π 42 = 1.5 ; π 43 = 2.7

For the unknown nonlinear couplings, set the coefficients of the bounds as

Φ 14 = 0.81 , Ψ 14 = 0.90 ; Φ 21 = 0.10 , Ψ 21 = 0.12 ; Φ 25 = 0.28 , Ψ 25 = 0.32 ; Φ 31 = 0.16 , Ψ 31 = 0.18 ; Φ 34 = 0.42 , Ψ 34 = 0.46 ; Φ 41 = 0.79 , Ψ 41 = 0.86 ; Φ 43 = 0.66 , Ψ 43 = 0.72 ; Φ 53 = 0.04 , Ψ 53 = 0.06 ; Φ 56 = 0.68 , Ψ 56 = 0.73 ; Φ 61 = 0.74 , Ψ 61 = 0.79 ; Φ 64 = 0.39 , Ψ 64 = 0.42 ; Φ 65 = 0.17 , Ψ 65 = 0.20 ;

The other Φ i j and Ψ i j are set to zero.

According to Theorem 1, an LMI is constructed for the example. The MATLAB LMI solver can be used to compute the LMI. The computation obtains positive definite P 1 , P 2 , P 3 , P 4 , P 5 , P 6 ∈ ℜ 6 × 6 and positive definite diagonal U 1 , U 2 ∈ ℜ 30 × 30 , which satisfy the LMI of the example. In other words, our example meets the condition of Theorem 1 with the above coupling topologies, transition probabilities and sector-shaped restriction bounds. Hence the multi-agent system of the example can achieve stochastic consensus.

As a simulation case, we give the directed coupling from agent j to agent i as

F i j [ Y ] = { ( Φ i j + Ψ i j ) Y + ( Φ i j − Ψ i j ) Y sin Y 2 Y ∈ ∪ d ∈ ℤ [ d , d + 0.5 ) ( Φ i j + Ψ i j ) Y − ( Φ i j − Ψ i j ) Y sin Y 2 Y ∈ ∪ d ∈ ℤ [ d + 0.5 , d + 1 )

which satisfies condition (3). Figure 2 illustrates the graph of F 65 [ Y ] . Clearly, the coupling is discontinuous. With the initial state

OPEN IN VIEWER

Figure 2.

Graph of coupling F 65 [ Y ]

X 1 ( 0 ) = − 3.24 , X 2 ( 0 ) = − 2.19 , X 3 ( 0 ) = 3.78 X 4 ( 0 ) = 3.74 , X 5 ( 0 ) = − 0.10 , X 6 ( 0 ) = 1.06

and the target point X ˉ = 2 , multi-agent system (2) was simulated using MATLAB. Figure 3 shows the state trajectories, which clarify that a consensus has been reached about X ˉ = 2 .

OPEN IN VIEWER

Figure 3.

State trajectories

5. Conclusion

We have studied the stochastic consensus problem of continuous time multi-agent systems with unknown nonlinear couplings and Markov switching coupling topologies. A sufficient condition for stochastic consensus has been presented. A simulation example was provided to illustrate our results.