Visual servoing control of omnidirectional mobile robots based on disturbance observer

Abstract

This study studies a critical issue that linear velocity and steering angular velocity of the wheels of omni-directional mobile robot are easily affected by the unknown external disturbances such as wind direction and road flatness during the visual servoing stabilization task. To overcome such challenging problems, this study proposes a disturbance observer based visual servoing control strategy. In configuration, the controller is divided into two parts: the quasi-min-max model predictive controller is designed to accommodate the state constraint and input constraint of the system, and complete the visual stabilization task without disturbance. A nonlinear disturbance observer is designed to estimate the external disturbance and feedback compensate the effect of the actual disturbance on the wheel speed. The simulation results and real experiments show that the proposed control strategy can stably accomplish the visual servoing stabilization task even with a large external disturbance.

Keywords

Omnidirectional mobile robot visual servoing quasi-min-max model predictive control disturbance observer

Introduction

Visual servoing has always been a hot research field in mobile robot technology,^1,2 which mainly controls the position and pose of mobile robots through visual sensor feedback information at the current instance, so that mobile robots can achieve the functions of stabilization, trajectory tracking, and navigation. In recent years, there have been many researches on visual servoing stabilization methods for mobile robots. Liang et al.³ proposed an adaptive image-based trajectory tracking control scheme for wheeled mobile robots with uncalibrated fixed cameras. Yan et al.⁴ developed a hybrid visual trajectory strategy for wheeled mobile robots equipped with vehicle-mounted vision systems, in which 2.5D visual servoing framework was used to enhance trajectory tracking behavior and helped preserve visual objects within the camera’s visual range. In He et al.,⁵ a model predictive controller design method based on quasi-min-max strategy was proposed for the limited mobile robot visual servoing system to achieve asymptotic stability of the closed-loop vision system of mobile robots.

The above research approaches are based on the mobile robot visual servoing task without considering the disturbance of the external environment. However, in practical situations and applications, the structure of mobile robot is complex, and mobile robot itself is easily affected by the changes of system parameters, coupled with the external environment disturbance, such as external force disturbance, wheel skid, and other factors, which increase the difficulty of mobile robot system control. How to reduce the external disturbance of the system and improve the control of mobile robot have been critically difficult problems in robot control analysis and design. To effectively suppress the influence of disturbance on the system, adaptive control method, disturbance observer method, and the adaptive method combined with the two methods can be adopted.^6–8 In recent years, active disturbance rejection control (ADRC) technology has achieved good results in robot control by establishing an observer to estimate disturbances because it does not need an accurate model of the object. Ma et al.⁹ estimated the system disturbance through the extended state observer, and introduced the disturbance compensation term into the active disturbance rejection controller, so as to reduce the uncertainty of system parameters and the impact of external disturbances on the system. Wang et al.¹⁰ from Hebei University, China designed an active disturbance rejection controller based on linear extended observer for the unknown wheel sliding and skidding phenomenon of mobile robots, which enabled mobile robots to achieve better trajectory tracking effect. In addition, there is a kind of model-based control technology, such as sliding mode control, which also has good anti-interference effect. In Li et al.,¹¹ in order to overcome the torque disturbance of wheeled mobile robots, a sliding mode speed tracking controller with disturbance observer is proposed in literature, and finally the mobile robot can track the desired path. In Huang et al.,¹² a sliding mode control method based on high-order disturbance observer is designed for the general underactuated system to improve the disturbance estimation ability of the underactuated system. However, when the synovial track enters the dead zone, the sliding model controller will be an unpredictable chattering interval. Therefore, Qiu et al.¹³ proposed an uncalibrated visual servoing adaptive model predictive control method based on disturbance observer, which solved the visual servoing tasks of mobile robots under the conditions of unknown internal and external parameters of the camera, external interference and system constraints, and improved the anti-interference performance of the system. Yu et al.¹⁴ took the two-wheel differential mobile robot model as the object and designed a predictive control estimation method based on nonlinear disturbance observer to compensate the slow-varying disturbance of the system, and got a good path tracking effect. Sun et al.¹⁵ proposed a model predictive control method for disturbance suppression and designed two disturbance observers, one to estimate the unknown disturbance and the other to compensate for the disturbance of the known harmonic frequency, finally realizing the non-offset tracking of the trajectory. To compensate the external disturbance of the multi-input multi-output robot system, Liu and Liu¹⁶ constructed a nonlinear observer to estimate the disturbance of the external system, and adopted the backstepping method to suppress the disturbance and improve the performance of position tracking. In Chen et al.,¹⁷ disturbance estimation technology is comprehensively summarized, including disturbance observer control (DOBC), active Disturbance Rejection control (ADRC), disturbance regulation control (DAC), etc., and the future development direction of disturbance compensation technology is also discussed.

Inspired by the quasi-min-max predictive control method,⁵ this paper studies the visual servoing stabilization control problem of omnidirectional mobile robot in the presence of external environmental disturbances. Different from nonholonomic two-wheel differential mobile robot, omnidirectional mobile robot can move 360° in all directions, so it is more suitable for application in the scene with limited range of activities and has greater practical significance. The major contribution of the study can be briefly described below.

This paper establishes completely the error model and kinematics model of omnidirectional mobile robot with three off-centered direction wheels. The full model integrates the omnidirectional wheeled mobile robot and visual system.

In order to improve the effectiveness and robustness of the control, we designed the quasi-min-max MPC based on a non-linear disturbance observer that provided robust performance regardless of the modeling uncertainties and perturbation.

The simulation results show that the proposed strategy has the advantage of anti-disturbance performance.

The rest of this paper is organized as follows. Section “Problem formulation” presents the visual servoing error model and omnidirectional mobile robot kinematics model. Section “Design of control strategy based on disturbance observer” presents the detailed design process based on disturbance observer control strategy. Section “Simulation” shows off the simulation results to demonstrate the performance of the control strategy. Finally, Section “Conclusions” concludes the study.

Problem formulation

Nonholonomic OWMR description

As shown in Figure 1, an omnidirectional wheeled mobile robot (OWMR) is assumed to move in a plane, which is independently driven by three off-centered direction wheels¹⁸ and equipped with a monocular camera above the OWMR. O_w-X_wY_wZ_w is the world coordinate frame attached to the ground, O_r-X_rY_rZ_r is the robot coordinate frame fixed to the OWMR, the origin O_r is located at the mass center of the OWMR, O_c-X_cY_cZ_c is the camera coordinate frame attached to the camera, the origin O_c is located at the Z_r axis of the robot coordinate frame, and the X_c and Y_c axes are in the same direction as the X_r and Y_r axes. Assuming that the OWMR moves on the plane under the nonholonomic constraints of pure rolling and non-slipping, then the speed and the pose of the omnidirectional mobile robot satisfy the following relationship:

[\begin{matrix} \overset{\cdot}{x} (t) \\ \overset{\cdot}{y} (t) \\ \overset{\cdot}{θ} (t) \end{matrix}] = [\begin{matrix} \cos θ & - \sin θ & 0 \\ \sin θ & \cos θ & 0 \\ 0 & 0 & 1 \end{matrix}] [\begin{matrix} v_{x} (t) \\ v_{y} (t) \\ ω (t) \end{matrix}]

(1)

Figure 1.

Omnidirectional wheeled mobile robot visual servoing model.

where (x, y) is the coordinates of the robot’s mass center in the world coordinate system, θ is the angle between the world coordinate system and the robot coordinate system, and counterclockwise is positive. q = [x yθ]^T is the posture of the mobile robot, v_r = [v_x v_yω]^T is the speed of the omnidirectional mobile robot, that is, the control input, v_x and v_y are the longitudinal and lateral linear velocities of the omnidirectional mobile robot, and ω is the angular velocity of the omnidirectional mobile robot.

Define the coordinates of feature point P in the world coordinate system as (X, Y, Z), in the robot coordinate system as (x_r, y_r), and in the camera coordinate system as (x_c, y_c, z_c). The equation can be obtained according to the geometric relationship in Figure 1:

{\begin{matrix} X = x_{r} \cos θ - y_{r} \sin θ + x \\ Y = x_{r} \sin θ - y_{r} \cos θ + y \\ x_{r} = x_{c} \\ y_{r} = y_{c} \end{matrix}

(2)

Since the feature point P is stationary, X and Y are constants. Taking the derivative of equation (2), and then combining with equation (1) we can obtain the change of robot motion position of feature points in the camera coordinate system, that is, the robot-camera-target model is as follows:

{\begin{matrix} {\overset{\cdot}{x}}_{c} = y_{c} ω - v_{x} \\ {\overset{\cdot}{y}}_{c} = - x_{c} ω - v_{y} \end{matrix}

(3)

There are various disturbances encountered frequently in the actual environment in the visual servoing task of omnidirectional mobile robot, such as lateral wind, ground slip, and so on, which will affect the linear velocity and angular velocity of mobile robot. When external disturbances are considered, model (3) of the omnidirectional mobile robot system can be written in the following general form:

\overset{\cdot}{x} = g_{1} (x) v_{r} + g_{2} (x) d

(4)

Where $x = {[\begin{matrix} x_{c} & y_{c} \end{matrix}]}^{T}$ is state vector, $v_{r} = {[\begin{matrix} v_{x} & v_{y} & ω \end{matrix}]}^{T}$ , $g_{1} (x) = [\begin{matrix} - 1 & 0 & y_{c} \\ 0 & - 1 & - x_{c} \end{matrix}]$ , $g_{2} (x) = [\begin{matrix} - 1 & 0 & y_{c} \\ 0 & - 1 & - x_{c} \end{matrix}]$ , $d = {[\begin{matrix} d_{1} & d_{2} & d_{3} \end{matrix}]}^{T}$ are the outside disturbance.

Visual servoing error model of OWMR

As shown in Figure 1, P is the feature point the world coordinate frame. We assume that (x_p, y_p) is the image coordinate of the projection point p of feature point P in the image coordinate system, (u_p, v_p) is the corresponding image pixel coordinate, (u₀, v₀) is the center of the image plane, f is the focal length of the camera. According to the image projection model,^19,20 the relationship between the camera coordinate system and the image coordinate system can be derived by

u_{p} = f_{x} x_{c} / y_{c} + u_{0}, v_{p} = f_{y} z_{c} / y_{c} + v_{0}

(5)

where $f_{x} = f / Δ x, f_{y} = f / Δ y$ , f is the focal length of the camera, and Δx and Δy are actual size per unit pixel on the axes X_p and Y_p, respectively.

By normalizing the image coordinates, we have that

{[\begin{matrix} {x'}_{p} & 1 & {y'}_{p} \end{matrix}]}^{T} = {[\begin{matrix} (u_{p} - u_{0}) / f_{x} & 1 & (v_{p} - v_{0}) / f_{y} \end{matrix}]}^{T}

(6)

with the normalized image coordinates (x_p′, y_p′). Taking the derivatives of (6) and combining (5), it is obtained that

[\begin{matrix} {\overset{\cdot}{x}'}_{p} \\ {\overset{\cdot}{y}'}_{p} \end{matrix}] = [\begin{matrix} ({\overset{\cdot}{x}}_{c} - {x'}_{p} {\overset{\cdot}{y}}_{c}) / y_{c} \\ ({\overset{\cdot}{z}}_{c} - {y'}_{p} {\overset{\cdot}{y}}_{c}) / y_{c} \end{matrix}] .

(7)

Let υ_c = [v_x, v_y, v_z, ω_x, ω_y, ω_z]^T be the velocity of the camera on the robot. Since the robot moves in plane, that is, v_z = ω_x = ω_y = 0 and ω_z = ω, the motion velocity of p is obtained by

[\begin{matrix} {\overset{\cdot}{x}'}_{p} \\ {\overset{\cdot}{y}'}_{p} \end{matrix}] = L (x'_{p}, y'_{p}) [\begin{matrix} v_{x} \\ v_{y} \\ ω \end{matrix}]

(8)

with $L (x'_{p}, y'_{p}) = [\begin{matrix} - 1 / y_{c} & {x'}_{p} / y_{c} & 1 + {x'}_{p}^{2} \\ 0 & {y'}_{p} / y_{c} & {x'}_{p} {y'}_{p} \end{matrix}]$ .

Define the posture variables (ρ₁, ρ₂) of the robot with its desired posture (ρ₁^*, ρ₂^*) as

{\begin{matrix} ρ_{1} = \frac{{x'}_{p}}{{y'}_{p}} = \frac{u_{p} - u_{0}}{v_{p} - v_{0}} \frac{f_{y}}{f_{x}} \\ ρ_{2} = - \frac{1}{{y'}_{p}} = \frac{- f_{y}}{v_{p} - v_{0}} \end{matrix}, {\begin{matrix} ρ_{1}^{*} = \frac{u_{p}^{*} - u_{0}}{v_{p}^{*} - v_{0}} \frac{f_{y}}{f_{x}} \\ ρ_{2}^{*} = \frac{- f_{y}}{v_{p}^{*} - v_{0}} \end{matrix}

(9)

where (u_p^*, v_p^*) are the desired coordinates of the point p in the image pixel coordinate frame. Then we have that

{\overset{\cdot}{ρ}}_{1} = - v_{x} / z_{c} - ρ_{2} ω_{c}, {\overset{\cdot}{ρ}}_{2} = v_{y} / z_{c} + ρ_{1} ω_{c}

(10)

The error signals of the robot can be defined as

{\begin{matrix} [\begin{matrix} e_{1} \\ e_{2} \end{matrix}] = [\begin{matrix} ρ_{1} \\ ρ_{2} \end{matrix}] - [\begin{matrix} \cos θ & - \sin θ \\ \sin θ & \cos θ \end{matrix}] [\begin{matrix} ρ_{1}^{*} \\ ρ_{2}^{*} \end{matrix}] \\ e_{3} = θ - θ^{*} \end{matrix}

(11)

where θ^* is the desired orientation of the robot and θ can be the estimated using the motion-estimated technique.

Taking the derivative of (11) and using (10), the visual servoing error system of the robot is described by

[\begin{matrix} {\overset{\cdot}{e}}_{1} \\ {\overset{\cdot}{e}}_{2} \\ {\overset{\cdot}{e}}_{3} \end{matrix}] = [\begin{matrix} - 1 / z_{c} & 0 & - e_{2} \\ 0 & 1 / z_{c} & e_{1} \\ 0 & 0 & 1 \end{matrix}] [\begin{matrix} v_{x} \\ v_{y} \\ ω \end{matrix}]

(12)

Kinematic model

This paper considers an omnidirectional WMR consisting of three eccentric steering wheels, whose top view is shown as Figure 2, where M is the mass center of the WMR, and A, B, and C are the fixed points of the three wheels relative to the robot body. These fixed points are located on the central axis of the vertical steering axis of each wheel when moving. The three wheels of the omnidirectional WMR are assumed to be exactly the same and the angles between MA, MB, and MC are identical to be 2π/3. In Figure 2, l is the distance from the center of the vertical axis of the wheel to M, α is the constant angle from the X_r axis of the robot coordinate system to MA, d, r, β, and φ are the eccentric distance, radius, turning angle, and rolling angle of the wheel, respectively. Note that the angles are positive when rotating counter-clockwise and the turning angle β is from MA to the axis of the wheel.

Figure 2.

Top view of omnidirectional wheeled mobile robot.

The conversion relation between the rotation velocity $\overset{\cdot}{φ}$ and deflection velocity $\overset{\cdot}{β}$ of the three wheels and the velocity v_r of OWMR can be obtained as follows²¹

[\begin{matrix} {\overset{\cdot}{φ}}_{i} \\ {\overset{\cdot}{β}}_{i} \end{matrix}] = J (β_{i}) v_{r}, i = 1, 2, 3

(13)

where ${\overset{\cdot}{φ}}_{i} = {[\begin{matrix} {\overset{\cdot}{φ}}_{1} & {\overset{\cdot}{φ}}_{2} & {\overset{\cdot}{φ}}_{3} \end{matrix}]}^{T}$ are the rotation velocity of the three wheels respectively, ${\overset{\cdot}{β}}_{i} = {[\begin{matrix} {\overset{\cdot}{β}}_{1} & {\overset{\cdot}{β}}_{2} & {\overset{\cdot}{β}}_{3} \end{matrix}]}^{T}$ are the deflection velocity of the three wheels respectively, and

\begin{matrix} J (β_{i}) = [\begin{matrix} \frac{\sin (α_{i} + β_{i})}{r} & \frac{- \cos (α_{i} + β_{i})}{r} & \frac{- l \cos β_{i}}{r} \\ \frac{- \cos (α_{i} + β_{i})}{d} & \frac{- \sin (α_{i} + β_{i})}{d} & \frac{- d - l \sin β_{i}}{d} \end{matrix}], \\ i = 1, 2, 3 \end{matrix}

Combining (12), the visual servoing error system of the mobile robot with respect to the rotation velocity and deflection velocity of the wheel can be obtained, that is,

\overset{\cdot}{e} (t) = \bar{B} (e (t)) u (t), t \geq 0

(14)

with the pseudoinverse matrix of J(β)

\bar{B} (e (t)) = [\begin{matrix} - 1 / z_{c} & 0 & - e_{2} \\ 0 & 1 / z_{c} & e_{1} \\ 0 & 0 & 1 \end{matrix}] J^{- 1} (β_{i}) i = 1, 2, 3

with J(β) = [J₁^T(β₁) J₂^T(β₂) J₃^T(β₃)]^T. For implementation purpose, the model is discretized using Euler’s method with sampling time T_s > 0 as:

e (k + 1) = e (k) + \bar{B} (e (k)) T_{s} u (k), k = 0, 1, . . .

(15)

To ensure safety and effective execution of visual servoing systems, the robot should satisfy the velocity constraints and the visibility constraints for each sampling time k, that is,

e_{1 min} \leq e_{1} (k) \leq e_{1 max}, e_{2 min} \leq e_{2} (k) \leq e_{2 max}

(16a)

{\overset{\cdot}{φ}}_{i min} \leq {\overset{\cdot}{φ}}_{i} (k) \leq {\overset{\cdot}{φ}}_{i max}, {\overset{\cdot}{β}}_{i min} \leq {\overset{\cdot}{β}}_{i} (k) \leq {\overset{\cdot}{β}}_{i max}

(16b)

where i = 1, 2, 3 and the subscript “min” and “max” indicate the lower and upper bounds of variables, respectively.

The objective of this paper is that the OWMR is driven to reach the feature point while meeting the constraints (16), and in the design process, the error e(k) and the velocity u(k) eventually approach zero.

Design of control strategy based on disturbance observer

In order to ensure the successful completion of the visual servoing task of omnidirectional mobile robot, the state constraints and output constraints of the visual system need to be considered. Therefore, a stability control law u(k) is designed for system (15) using the quasi-min-max MPC strategy. But in the actual system, the disturbance will disturb the rotation velocity and deflection velocity of the robot. By designing a nonlinear disturbance observer to estimate the external disturbance, and feedback compensation control input, the stabilization control of OWMR is finally realized. The visual servoing control block diagram of omnidirectional mobile robot based on disturbance observer is shown in Figure 3.

Figure 3.

Visual servoing control block diagram of OWMR based on disturbance observer.

In the framework shown in the figure, the visual servoing task is divided into two sub-problems:

Without considering the existence of external interference, model mismatch, and parameter uncertainty in the model, the quasi-min-max MPC controller⁵ is designed to obtain the control quantity based on the error state quantity at the current moment, which ensures the stabilization control of omnidirectional mobile robot, and satisfies the state constraint and input constraint control in the process of motion;

When there is lateral wind, complex ground environment and other conditions, that is, there is external disturbance, the disturbance is estimated and obtained by designing disturbance observer. The disturbance feedback is compensated into the control input channel to weaken or even eliminate the influence of the disturbance on the system.

Integrating the quasi-min-max MPC controller and the disturbance observer, the actual control quantity of the OWMR is $u^{*} = u - {\hat{d}}_{u} + d_{u}$ , with ${\hat{d}}_{u} = J (β_{i}) \hat{d}, i = 1, 2, 3$ , $d_{u} = J (β_{i}) d, i = 1, 2, 3$ . It can be seen that if the designed disturbance observer can converge asymptotically, the influence of the actual disturbance can be compensated by the disturbance observer, so as to ensure the asymptotic convergence and feasibility of the global control algorithm.

Design of controller based on quasi-min-max MPC

Consider the visual servoing error system (15) and let g(u) = J⁻¹(β)u. Then the system (15) can be rewritten as:

e (k + 1) = e (k) + B (e (k)) g (u (k))

(17)

where the matrix

B (e) = [\begin{matrix} - T_{s} / z_{c} & 0 & - e_{2} T_{s} \\ 0 & T_{s} / z_{c} & e_{1} T_{s} \\ 0 & 0 & T_{s} \end{matrix}]

Notice that g(u) is a static nonlinear function. Hence, the system (17) is a multi-variable time-varying Hammerstein system which typically consists of a static nonlinear input followed by a linear dynamic model.²² Let ū = g(u) and we have the linear time-varying dynamic model of (17)

e (k + 1) = e (k) + B (e (k)) \bar{u} (k)

(18)

It is further noted that B(e) is linear on the states e₁ and e₂. Then due to the constraints on those states, it is obtained from Baranyi²³ that B(e) can be covered by the convex polytope Ω, that is,

B (e) \in Ω = C_{0} {B_{1}, B_{2}, B_{3}, B_{4}}

(19)

where C₀ denotes convex hull and the matrices B_i = B(p_i) for i = 1, …, 4 are the vertices with p₁ = [e_1min, e_2min]^T, p₂ = [e_1min, e_2max]^T, p₃ = [e_1max, e_2min]^T, and p₄ = [e_1max, e_2max]^T, respectively. Therefore, combining (19), the linear time-varying dynamic model (18) can be modeled as a polyhedral LPV system

e (k + 1) = e (k) + \sum_{i = 1}^{4} λ_{i} B_{i} \bar{u} (k)

(20)

with the unknown parameters λ_i ≥ 0 satisfy that λ₁ + λ₂ + λ₃ + λ₄ = 1.

Consider the visual servoing system (20) and define the infinite horizon cost function

J_{0}^{\infty} (k) = \sum_{i = 0}^{\infty} e {(i | k)}^{T} Qe (i | k) + \bar{u} {(i | k)}^{T} R \bar{u} (i | k)

(21)

where •(i|k) is the value at k + i predicted at the current time k and weighted matrices Q > 0 and R > 0. In order to minimize the function (21), we consider the structure of (18) and use the system (20) to be the predicted model for (21). Notice that at each current time k, the matrix B(e(k)) can be determined via the measure of e(k) whilst B(e(k + i)) is not known but is covered by the convex polytope (19). Here the quasi-min-max MPC^24–26 is adopted to achieve the control goal. Online solve the optimization problem to obtain the solution ū^*(k).

Then from the system (17) or (18), it is known that the corresponding optimal control u(k) = u^mpc(k) is uniquely determined by

u^{mpc} (k) = J (β (k)) {\bar{u}}^{*} (k), k = 0, 1, \dots

(22)

which yields the closed-loop visual servoing system

e (k + 1) = e (k) + B (e (k)) g (u^{mpc} (k)), k = 0, 1, \dots

(23)

Design of disturbance observer

Considering that it is very difficult to establish an accurate interference model or accurately measure the external interference through the sensor, a disturbance observer is used to estimate the disturbance.

According to the OWMR system model (4), the disturbance observer is designed as follows:

{\begin{matrix} \overset{\cdot}{z} = - l (x) g_{2} (x) z - l (x) [g_{2} (x) p (x) + f (x) + g_{1} (x) v_{r}] \\ \hat{d} = z + p (x) \end{matrix}

(24)

where z is the internal state of the nonlinear disturbance observer, p(x) is the function to be designed, L(x) is the gain of the observer, and

l (x) = \frac{\partial p (x)}{\partial x}

(25)

Theorem 1: If l(x) in disturbance observer (24) can satisfy that $- l (x) g_{2} (x)$ is Hurwitz, then disturbance observer (24) can have tracking disturbance d with error, and the maximum tracking error $‖ e_{d} ‖_{\infty}$ is proportional to $δ$ ; For a constant bounded disturbance, the disturbance observer (24) can track disturbance d asymptotically.

Proof: Define estimation error

e_{d} = d - \hat{d}

(26)

where $\hat{d}$ is the estimated error. It is assumed that the disturbance changes slowly compared with the dynamic characteristics of the disturbance observer, therefore we can assume $‖ \overset{\cdot}{d} ‖ < δ$ , $δ$ is a given constant.

When the disturbance d is a slow-varying disturbance, the dynamic estimation error can be written as

\begin{matrix} {\overset{\cdot}{e}}_{d} = \overset{\cdot}{d} - \overset{\cdot}{\hat{d}} \\ = \overset{\cdot}{d} - \overset{\cdot}{z} - \overset{\cdot}{p} (x) \\ = \overset{\cdot}{d} - l (x) g_{2} (x) e_{d} \end{matrix}

(27)

Differential equation solution of (27) can be obtained:

e_{d} (t) = e^{- l (x) g_{2} (x) t} e_{d} (0) - \int_{0}^{t} e^{- l (x) g_{2} (x) (t - ς)} \overset{\cdot}{d} d ς

(28)

According to norm property, the following inequality can be obtained by (28)

‖ e_{d} (t) ‖ \leq ‖ e^{- l (x) g_{2} (x) t} e_{d} (0) ‖ + δ \int_{0}^{t} ‖ e^{- l (x) g_{2} (x) (t - ς)} ‖ d ς

(29)

If l(x) satisfies that $- l (x) g_{2} (x)$ is Hurwitz, then there exists $K \geq 1, α < 0$ such that

‖ e^{- l (x) g_{2} (x) t} ‖ \leq K e^{α t}

(30)

Therefore, combining (29) with (30), it can be obtained

\begin{matrix} ‖ e_{d} (t) ‖ \leq K e^{α t} ‖ e_{d} (0) ‖ + δ \int_{0}^{t} K e^{α (t - ς)} d ς \\ = K (‖ e_{d} (0) ‖ + \frac{δ}{α}) e^{α t} - \frac{δ K}{α} \end{matrix}

(31)

Therefore, by analyzing (31), it can be known that when $t \to \infty$ and $α < 0, δ, K$ is a constant greater than 0, then

‖ e_{d} (t) ‖ \to - \frac{δ K}{α}

(32)

that is, $‖ e_{d} (t) ‖$ is bounded.

When the disturbance d is a constant bounded disturbance, the dynamic (27) of the estimation error can be written as

{\overset{\cdot}{e}}_{d} = - l (x) g_{2} (x) e_{d}

(33)

Since l(x) satisfies that $- l (x) g_{2} (x)$ is Hurwitz, the disturbance observer (24) can track disturbance d asymptotically.

Theorem 2: Select

l (x) = [\begin{matrix} - 1 & 0 \\ 0 & - 1 \\ 0 & 0 \end{matrix}]

Through l(x), the linear function p(x) to be designed can be obtained according to (25). Then the disturbance observer (24) obtained is asymptotically stable.

Proof: Since $l (x) = [\begin{matrix} - 1 & 0 \\ 0 & - 1 \\ 0 & 0 \end{matrix}]$ , the characteristic roots of $- l (x) g_{2} (x) = - [\begin{matrix} 1 & 0 & - y_{c} \\ 0 & 1 & x_{c} \\ 0 & 0 & 0 \end{matrix}]$ are located in the left half plane of the imaginary axis. Therefore, the dynamic system (27) corresponding to the estimation error is asymptotically stable to the input state.

Simulation

In this section, simulation experiments are implemented by using Matlab 2019a to verify the effectiveness of the method proposed in this paper, and LMI toolbox in the Matlab 2019a was used to solve the optimization problem. In this study, the physical parameters of the robot are selected as (l, d, r) = (0.8, 0.2, 0.1) m and (α₁, α₂, α₃) = (π/3, π, −π/3) rad. The focal length of the camera mounted is f = 6 mm, the image resolution is 680 × 480 pixels, and the height z_c from the origin of the camera coordinate system to the feature point P is 0.4 m. Moreover, the visual servoing system of the robot satisfies that −4.25 ≤ e₁(k) ≤ 0, −12.5 ≤ e₂(k) ≤ 0, and u_max = −u_min = [50 50 30 30 30]^T. The sampling time T_s = 0.2 s, the diagonal matrices Q = diag{15, 1, 1}, R = diag{0.01, 0.01, 0.01, 0.5, 0.5, 0.5}. Let the initial steering angles of the three wheels of the robot be β₁ = −3π/2, β₂ = 3π/2, and β₃ = 2π/3, respectively. Pick an initial pose (1.6, −5, 0) and the expected pose as (0, 0, 0).

Simulation 1: A disturbance observation experiment is conducted to verify the effectiveness of the disturbance observer’s ability to estimate the external environment interference for the OWMR visual servoing task. Select external disturbance $d = {[\begin{matrix} d_{1} & d_{2} & 0 \end{matrix}]}^{T}$ , where

\begin{matrix} d_{1} = {\begin{matrix} 0.3 (1 - e^{- 0.5 (t - 2)}), t \geq 2 \\ 0, t < 2 \end{matrix} \\ d_{2} = {\begin{matrix} 0.4 (1 - e^{- 0.4 (t - 2)}), t \geq 2 \\ 0, t < 2 \end{matrix} \end{matrix}

In other words, disturbances are added to the lateral and longitudinal velocities of the omnidirectional mobile robot to simulate external disturbances. The initial state of the disturbance observer (27) is $z = {[\begin{matrix} - 6.4 & 64 & 0 \end{matrix}]}^{T}$ , the nonlinear function $p (x) = {[\begin{matrix} - 8 x_{c} & - 8 y_{c} & 0 \end{matrix}]}^{T}$ . As shown in Figures 4 and 5, the solid line is the actual disturbance quantity, and the red dotted line is the estimated disturbance quantity. It can be seen from Figures 4 and 5 that the nonlinear disturbance observer designed in this paper can better estimate the external disturbance.

Simulation 2: To verify the effectiveness of the OWMR visual servoing control strategy in this paper based on disturbance observer, external disturbances $d = {[\begin{matrix} d_{1} & d_{2} & 0 \end{matrix}]}^{T}$ are added to the robot system, the Figure 6 shows the error of OWMR state simulation diagram, Figures 7 and 8 shows rotation velocity and deflection velocity of wheels of OWMR. Figure 9 shows the trajectory of the OWMR in the world coordinate system Xw-Yw plane, and Figure 10 shows the trajectory of the feature points in the camera plane during the process of motion.

Figure 4.

The disturbance d₁.

Figure 5.

The disturbance d₂.

Figure 6.

Error state.

Figure 7.

Wheel rotation velocity.

Figure 8.

Wheel deflection velocity.

Figure 9.

Trajectories of feature points in the image.

Figure 10.

Trajectory of the OWMR on the X-Y plane.

As shown in Figure 6, the red solid line represents DOBC, and the dotted line represents quasi-min-max MPC. When the disturbance observer is not added, the dotted line error e₁ has static error, so the omnidirectional mobile robot cannot reach the desired target point under large external interference and cannot complete the corresponding visual servoing task. When the disturbance observer is combined with the disturbance observer, the disturbance observer can compensate the external disturbance well, and then eliminate the influence of the external disturbance. The mobile robot can reach the target point, and complete the visual servoing task better finally.

As shown in Figures 7 and 8, the red solid line represents DOBC, and the dotted line represents quasi-min-max MPC. When no disturbance observer is added, the rotation velocity and deflection velocity of the wheels of OWMR are greatly affected by external disturbance, and the fluctuation changes are obvious. And the velocity of rotation is not stabilized to zero. In the control strategy based on disturbance observer, the rotation velocity and deflection velocity of the robot are less affected by external disturbance, and the final velocity tends to zero, that is, the OWMR reaches the target position.

It can be seen from Figures 9 and 10 that under the control strategy presented in this paper, the OWMR in the world coordinate plane and the feature points in the camera coordinate plane finally reach the desired positions. Figure 9 shows the trajectory of the robot from the initial posture to the desired posture, where the red triangle denotes the desired posture. The feasibility of the designed control strategy is proved. According to Figure 10, the feature points are always located within the visual field of the camera, which ensures the visual visibility of the mobile robot.

Conclusions

This study has proposed a disturbance observer based visual servoing stabilization predictive control procedure. Firstly, it studies the visual servoing error system to model of the omnidirectional mobile robot dynamics with the rotation velocity and deflection velocity of the eccentric direction wheel. On this basis, the asymptotic stability of the closed-loop visual servoing system without disturbance is guaranteed by combining the quasi-min-max MPC strategy. A nonlinear disturbance observer is designed to compensate the impact of external disturbances on the system. Simulation and comparison experiments verify the feasibility and effectiveness of the proposed strategy. The procedure has been tested by simulation and real experiments. In addition, new solutions to problems such as camera occlusion should be explored and evaluated in practical applications. For such control system design, a potential approach worthwhile being investigated is the model free sliding mode control.²⁷

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by State Key Laboratory of Robotics and Systems (HIT) (SKLRS-2021-KF-11).

ORCID iDs

Kexin Xing

Defeng He

References

Said

Marie

Stephant

, et al. Skeleton-based visual servoing in unknown environments. IEEE/ASME Trans Mechatron 2018; 23: 2750–2761.

Qiu

Shi

, et al. Visual servo tracking of wheeled mobile robots with unknown extrinsic parameters. IEEE Trans Ind Electron 2019; 66: 8600–8609.

Liang

Wang

Chen

, et al. Adaptive image-based trajectory tracking control of wheeled mobile robots with an uncalibrated fixed camera. IEEE Trans Control Syst Technol 2015; 23: 2266–2282.

Yan

Shi

, et al. Hybrid visual servo trajectory tracking of wheeled mobile robots. IEEE Access 2018; 6: 24291–24298.

. Quasi-min-max predictive control for visual servoing stabilization of mobile robots. Control Decis 2020; 35: 1391–1396.

Jeong

Chwa

. Sliding-mode-disturbance-observer-based robust tracking control for omnidirectional mobile robots with kinematic and dynamic uncertainties. IEEE/ASME Trans Mechatron 2021; 26: 741–752.

Sun

Liu

Feng

, et al. Visual servoing of flying robot based on fuzzy adaptive linear active disturbance rejection control. IEEE Trans Circuits Syst II Express Briefs 2021; 68: 2558–2562.

Chen

Langari

, et al. Driver-behavior-based adaptive steering robust nonlinear control of unmanned driving robotic vehicle with modeling uncertainties and disturbance observer. IEEE Trans Vehicular Technol 2019; 68: 8183–8190.

Zhao

Ren

. Passivity-based active disturbance rejection control of an omni-directional mobile robot. Control Decis 2018; 33: 116–121.

10.

Wang

Dong

, et al. Active disturbance rejection tracking control of wheeled mobile robots under sliding and slipping conditions. Control and Decision 2020; 37: 206–213.

11.

Yang

. Path-following control for multi-axle car-like wheeled mobile robot with nonholonomic constraint. In: IEEE/ASME international conference on advanced intelligent mechatronics, Wollongong, NSW, Australia, 2013, pp. 268–273.

12.

Huang

Fukuda

, et al. A disturbance observer based sliding mode control for a class of underactuated robotic system with mismatched uncertainties. IEEE Trans Automat Contr 2019; 64: 2480–2487.

13.

Qiu

Liang

. Disturbance observer based adaptive model predictive control for uncalibrated visual servoing in constrained environments. ISA Trans 2020; 106: 40–50.

14.

Yang

Meng

, et al. MPC for path following problems of wheeled mobile robots. In: 6th IFAC conference on nonlinear model predictive control, Wisconsin, USA, 2018, pp. 281–286.

15.

Sun

Xia

Dai

, et al. Disturbance rejection MPC for tracking of wheeled mobile robot. IEEE/ASME Trans Mechatron 2017; 22: 2576–2587.

16.

Liu

. Composite control of nonlinear robotic system with exogenous disturbance. IEEE Access 2019; 7: 19564–19571.

17.

Chen

Yang

Guo

, et al. Disturbance-observer-based control and related methods: an overview. IEEE Trans Ind Electron 2016; 63: 1083–1095.

18.

Kanjanawanishkul

. Omnidirectional wheeled mobile robots: wheel types and practical applications. Int J Adv Mechatronic Syst 2015; 6: 289–302.

19.

Yang

. Robust tube-based predictive control for visual servoing of constrained differential-drive mobile robots. IEEE Trans Ind Electron 2018; 65: 3437–3446.

20.

Chaumette

Hutchinson

. Visual servo control, Part I: Basic approaches. IEEE Robot Autom Mag 2006; 13: 82–90.

21.

Campion

Bastin

Dandrea-Novel

. Structural properties and classification of kinematic and dynamic models of wheeled mobile robots. IEEE Trans Robot Autom 1996; 12: 47–62.

22.

Ding

Ping

. Dynamic output feedback model predictive control for nonlinear systems represented by Hammerstein–Wiener model. J Process Control 2012; 22: 1773–1784.

23.

Baranyi

. TP model transformation as a way to LMI-based controller design. IEEE Trans Ind Electron 2004; 51: 387–400.

24.

Arkun

. Quasi-min-max MPC algorithms for LPV systems. Automatica 2000; 36: 527–540.

25.

Arkun

. A scheduling quasi–min-max model predictive control algorithm for nonlinear systems. J Process Control 2002; 12: 589–604.

26.

Huang

Chen

. Quasi-min-max MPC for constrained nonlinear systems with guaranteed input-to-state stability. J Franklin Inst 2014; 351: 3405–3423.

27.

Zhu

. Complete model-free siding mode control (CMFSMC). Sci Rep 2021; 11: 22565.