Fuzzy vs Nonfuzzy in 2D Visual Servoing for Robot Manipulators

Abstract

In this paper we consider the tracking control problem of planar robots manipulators via visual servoing in the presence of parametric uncertainties associated with the robot dynamics and the camera parameters. An image–based visual servo control is developed using a Takagi–Sugeno (T–S) fuzzy model. The design includes an image coordinate velocity observer. To the best knowledge of the authors such a proposal remains hitherto unpublished. The new scheme is compared experimentally with two different non–fuzzy algorithms showing the good performance and advantages of the complete system.

Keywords

Fuzzy Tracking Control Fuzzy Observer Design Stability Analysis Visual Servoing

1. Introduction

Among the model–based fuzzy control approaches, the Takagi–Sugeno (T–S) is one of the most effective strategies. They proposed their model in 1985 Takagi and Sugeno (1985) and it has emerged as one of the most active areas of research. Basically, a complex dynamic model can be composed in a set of local linear subsystems via fuzzy inference. Hence, it is not surprising that practical applications for control started to appear very quickly after the method had been introduced in different works. On the other hand, in the design of fuzzy controllers a good option is to use parallel distributed compensation (PDC), which offers a procedure to compute a control law from the T–S model. In Kim et al. (2001) a solution to target identification and grasping using a real platform in combination with a vision system is proposed. The task for the robot end–effector is to approach a randomly placed spherical object of known size. The control algorithm is approximated by using an approach based on fuzzy rules that are built in a supervised way, after studying the system behavior. In Wai and Yang (2008) a visual servoing control scheme is proposed for positioning a robot manipulator, divided into two steps: first a fuzzy logic–based visual servoing controller is used to guide the gripper into the neighborhood of the object, and then a local neural network is applied to set the gripper in a desired position. In Fares et al. (2007) a fuzzy logic system and a genetic algorithm are integrated for adaptive visual servoing. Genetic algorithms are employed to optimize the internal parameters of membership functions. In Goncalves et al. (2008) a new uncalibrated eye–to–hand visual servoing based on inverse fuzzy modeling to obtain an inverse relationship of the mapping between image feature variations and joint velocities is proposed. This approach is independent of the robot's kinematic model and camera calibration and avoids the necessity of inverting the manipulator Jacobian online. Different methods can be employed to estimate and identify the internal states of a system when only information from input and output are available, e. g. observer design Tanaka et al. (1998). Fuzzy logic began to be used for the design of observers not long ago, and it is a strategy that is currently under research Zhang and Fei (2006). In Meza (2003) one of the few works on this subject is presented and different designs applied to several nonlinear fuzzy observers are shown with excellent results. In this paper, a fuzzy visual servoing approach for trajectory tracking of planar robot manipulators in the absence of camera parameters, robot dynamics and velocity measurements is introduced. However, on the contrary to the work in Abdelmalek et al. (2007), where a Takagi–Sugeno model based on joint coordinates is employed, in this work we present the model in image coordinates. This strategy is not yet published according to the literature review. Furthermore, an experimental comparison with other well–known algorithms is carried out.

The paper is organized as follows: Section 2 presents the mathematical preliminaries. Section 3 gives the basic concepts about parallel distributed compensation for the Takagi–Sugeno Fuzzy Model and observer design. The control–bserver approach is given in Section 4, while the experimental results are presented in Section 5. Finally, some conclusions are given in Section 6.

2. Preliminaries

The dynamics of a rigid robot arm with revolute joints can adequately be described by using the Euler–Lagrange equations of motion Sciavicco and Siciliano (2000), resulting in

H (q) \ddot{q} + C (q, \dot{q}) + D \dot{q} + g (q) = τ - τ_{p^{'}}

(1)

where q ∊ ℝⁿ is the vector of generalized joint coordinates, H(q) ∊ ℝ^n×n is the symmetric positive definite inertia matrix, C(q,q̇)q̇∊ℝⁿ is the vector of Coriolis and centrifugal torques, g(q) ∊ ℝⁿ is the vector of gravitational torques, D ∊ ℝ^n×n is the positive semi definite diagonal matrix accounting for joint viscous friction coefficients, τ ∊ E ℝⁿ is the vector of torques acting at the joints, and τ_p ∊ ℝⁿ represents any bounded external perturbation or friction force.

Throughout this paper, we will assume that the robot is a 2DOF planar manipulator, i. e. it is n = 2 in (1). The direct kinematics is a differentiable map f_k(q):ℝ²→ℝ² relating the joint positions q ∊ ℝ² to the Cartesian position x_R ∊ ℝ² of the centroid of a target attached at the arm end–effector as x_R ∊ f_k (q). In this work we used the camera fixed configuration.

The output of the system is the position y ∊ ℝ² of the image feature, i. e. the position of the target in the computer screen. As shown in Figure 1 the robot workspace is in the x–y plane. The camera is not assumed to be completely parallel to this plane, so that there are two angles of rotation φ,Ψ ∊ ℝ associated to the map from image to workspace coordinates. In order to write y in terms of x_R, the image feature y can be computed through transformation and a perspective projections. As shown in the figure, the following coordinates systems are defined: ∊_o ∊_c·, ∊_c·, ∊_c, ∊_c ∊_c ∊_i and ∊_y. The position of the origin of the camera frame ∊_C with respect to the robot frame is given by o_C = [o_c1o_c2o_c3]^T. Then, similar to the procedure explained in Kelly and Reyes (2000), by assuming cosψ ≈ 1, sinψ ≈ Ψ, tan Ψ ≈ Ψ, Ψ² ≈ 0 one gets after some manipulation

Figure 1.

Planar robot system with not parallel camera.

y = α_{λ} R φ {x_{R} - [\begin{matrix} 0_{c 1} \\ 0_{c 2} \end{matrix}]} + [\begin{matrix} u_{0} + α_{λ} \cos (φ) 0_{c 3} ψ \\ v_{0} + α_{λ} \sin (φ) 0_{c 3} ψ \end{matrix}],

where λ is the focal length, α is a conversion factor from meters to pixels, [u_ov_o]^T is the center offset and

R_{φ} = [\begin{matrix} \cos (φ) & \sin (φ) \\ \sin (φ) & - \cos (φ) \end{matrix}] ​ ​

(2)

3. Parallel distributed compensation for Takagi–Sugeno Fuzzy Model

The parallel distributed compensation (PDC) approach provides a procedure to design a fuzzy controller from a given T–S fuzzy model. The construction of the continuous T–S fuzzy model is based on rules of the form Takagi and Sugeno (1985)

\begin{array}{l} R u l e i: I f z_{1} {is M}_{i1} and ... {and z}_{g} {is M}_{ig} \\ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ Then x = A_{i} x + B_{i} u ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ \\ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ y = C_{i} x, i =, 1,2, ...,r . ​ \end{array}

(3)

Here M_ij(j = 1,2,…,g) are fuzzy sets, r is the number of model rules, x(t) ∊ ℝⁿ is the system state vector, u(t) ∊ ℝ^m is the input, y(t) ∊ ℝ^q is the output vector, A_i ∊ ℝ^n×n, B_i ∊ ℝ^n×m,C_i ∊ ℝ^q×n,z(t) = [z₁(t),…,z_g(t)] are known premise variables that may be functions of the state variables, external disturbances, and/or time. Given a pair of (x(t),u(t)), a standard fuzzy inference method is used, i. e. a singleton fuzzifier, product fuzzy inference and weighted average defuzzifier. The final state of the fuzzy system is inferred as

\dot{x} = \frac{\sum_{i = 1}^{r} w_{i} (z (t)) (A_{i} x (t))}{\sum_{i = 1}^{r} w_{i} (z (t))} = \sum_{i = 1}^{r} h_{i} (z (t)) (A_{i} x (t) + B_{i} u (t))

(4)

y = \frac{\sum_{i = 1}^{r} w_{i} (z) C_{i} x}{\sum_{i = 1}^{r} w ​_{i} (z)} = \sum_{i = 1}^{r} h_{i} (z) C_{i} x

(5)

where

w_{i} ((t)) = \prod_{j = 1}^{g} M_{i j} (z_{j} (t))

(6)

h_{i} (z (t)) = \frac{w_{i} (z (t))}{\sum_{i = 1}^{r} w_{i} (z (t))} ​

(7)

M_ij(z(t)) is the grade of membership of z_j(t) in M_ij. It is assumed that

w_{i} (z (t)) \geq 0, i = 1, 2, ..., r ​

(8)

and

\sum_{i = 1}^{r} w_{i} (z (t)) > 0, ​

(9)

for all t. Therefore

h_{i} (z (t)) \geq 0, i = 1, 2, ..., r

(10)

and

\sum_{i = 1}^{r} h_{i} (z (t)) = 1.

(11)

For the convenience of notation h_i(z(t)) = h_i and w_i (z(t)) = w_i. Then the dynamics of the final state of the fuzzy system can be represented as

\dot{x} (t) = \sum_{i = 1}^{r} h_{i} A_{i} x (t) + \sum_{i = 1}^{r} h_{i} B_{i} u (t) . ​

(12)

The main goal is to determine the local feedback gain F_i in the consequent parts. With PDC we have a simple and natural procedure to handle nonlinear control systems. Other nonlinear control techniques require special and rather involved knowledge. The overall output is given by

u = - \sum_{i = 1}^{r} h_{i} F_{i} x .

(13)

The PDC scheme that stabilizes the T–S fuzzy model was proposed by Wang et al. Liu and Lam (2006) as a design framework comprising a control algorithm and a stability test using optimization involving LMI constraints.

3.1 Quadratic Stability Conditions

Consider the following definition.

Definition 3.1. Abdelmalek et al. (2007) The system x(t) = f(x(t),u(t)) is said quadratically stable if there exists a quadratic function V(x(t)) = x^T(t)Px(t),V(0) = 0, called Lyapunov function, satisfying the following conditions

V (x (t)) > 0, \forall x (t) \neq 0 \Leftrightarrow P > 0,

(14)

V (x (t)) < 0, \forall x (t) \neq 0

(15)

By substituting (13) in (4), we obtain the Takagi–Sugeno closed loop fuzzy systems as follows Abdelmalek et al. (2007)

\dot{x} = \sum_{i = 1}^{r} \sum_{j = 1}^{r} h_{i} (z) h_{j} (z) (A_{i} - B_{i} F_{j}) x,

(16)

which can be rewritten as

\dot{x} = \sum_{i = 1}^{r} h_{i} (z) h_{i} (z) G_{i i} x + 2 \sum_{i = 1}^{r} \sum_{i < j}^{r} h_{i} (z) h_{j} (z) (\frac{G_{i j} + G_{j i}}{2}) x,

(17)

where G_ij = A_i – B_iF_j and G_ii = A_i – B_iF_i. The stabilization of a feedback system containing a state feedback fuzzy controller has been extensively considered. The stability conditions corresponding to a quadratic Lyapunov function were derived by Tanaka and Sugeno in Tanaka and Sugeno (1992). The approach requires to find a common positive definite matrix for the r subsystems, which makes it very conservative.

3.2 Stabilization Approach

The stabilization approach used in this paper was introduced by Tanaka et al. in Tanaka et al. (2003), where a fuzzy Lyapunov function is proposed. A candidate Lyapunov function is of the form

V (x (t)) = \sum_{i = 1}^{r} h_{i} (z (t)) x^{T} (t) P_{i} x (t), ​

(18)

where P_i is a positive definite matrix and h_i (z(t))≥0, ∊^r_i=1h_i(z(t)) = 1. An interesting advantage is that this form of Lyapunov function takes into account the speed variation of the decision variables, what allows more relaxed stability conditions since this type of functions reduces the global stability of the nonlinear system to the analysis of the local stability of each local linear model (sub–model) separately. The candidate Lyapunov function (18) satisfies the following conditions:

Equation (18) is said to be a fuzzy Lyapunov function for the T–S fuzzy system if the time derivative of V(x(t)) is always negative at x(t) ≠ 0, where P_i is a positive definite matrix. Based on the fuzzy Lyapunov function, we use an approach that gives less conservative stability conditions Abdelmalek et al. (2007), the key assumptions are:

Assumption 3.1. The time derivative of the premise membership function is upper bounded and such that h_i (z(t))≤ φ_i, where φ_i are given positive constants with i = 1, …, r.

Assumption 3.2. The local quadratic Lyapunov functions x^T(t)P_ix(t), i = 1, …, r are proportionally related such that P_j=α_ijP_i for i,j = 1, …,r, where α_ij ≠ 1 and α_ij > 0 for i ≠ j, and α_ij =1 for i = j.

Now consider now the following theorem.

Theorem 3.1. Under Assumptions 3.1 and 3.2, the fuzzy system (16) is stable by using the PDC fuzzy controller (13) if there exist φ_o, α_ij for i,j,ρ = 1, … r, positive definite matrices P₁,P₂, …, P_r and matrices F₁,F₂, …, F_r such that for all i,j,k ∊ {1, …,r}

P_{i} > 0 ​

(19)

\sum_{ρ = 1}^{r} φ_{ρ} P_{ρ} + (G_{j j}^{T} P_{i} + P_{i} G_{j j}) < 0

(20)

{(\frac{G_{j k} + G_{k j}}{2})}^{T} P_{i} + P_{i} (\frac{G_{j k} + G_{k j}}{2}) < 0 (f o r j < k),

(21)

where G_jk = A_j – B_jF_k and G_jj = A_j – B_jF_j.

Proof:

The derivative of V(x(t)) in (18) along (17) can be computed after a rather direct but long manipulation as

\dot{V} (x) = x^{T} [\sum_{i = 1}^{r} \sum_{j = 1}^{r} h_{i} (z) h_{j} (z) (G_{j j}^{T} P_{i} + P_{i} G_{j j}) + \sum_{ρ = 1}^{r} {\dot{h}}_{ρ} (z) P_{ρ} ​ + \sum_{i = 1}^{r} \sum_{j = 1}^{r} \sum_{j < k} h_{i} (z) h_{j} (z) h_{k} (z) ({(\frac{G_{j k} + G_{k j}}{2})}^{T} P_{i} + P_{i} (\frac{G_{j k} + G_{k j}}{2}))] x

(22)

\dot{V} (x) \leq x^{T} [\sum_{i = 1}^{r} \sum_{j = 1}^{r} h_{i} h_{j} (G_{j j}^{T} P_{i} + P_{i} G_{j j}) + \sum_{ρ - 1}^{r} φ_{ρ} P_{ρ} + \sum_{i = 1}^{r} \sum_{j = 1}^{r} \sum_{j < k}^{​} h_{i} h_{j} h_{k} ({(\frac{G j k + G k j}{2})}^{T} P i + P i (\frac{G j k + G k j}{2}))] x

(23)

If Equations (19), (20) and (21) hold, the time derivative of the fuzzy Lyapunov function is negative, i. e. · V(x) ≤ 0, and the closed loop fuzzy system (16) is stable.

Theorem 3.1 is proven by taking into account Assumption 3.1. The constraint imposed on the time derivative of the premise membership functions and hence on the derivative of the premise variables, i. e. the speed of the state variables for the case of z(t) = x(t), is transformed into LMIs of Theorem 3.2 that are solved simultaneously with those of Theorem 3.1 to stabilize the T–S fuzzy systems. The new LMIs that support Assumption 3.1 allow to increase the performance by limiting the displacement rate in the polytope, implying a facility to find the Lyapunov functions and thus a faster stabilization.

Theorem 3.2. Assumption 3.1 holds if there exist positive definite matrices P₁,P₂, …, P_r and matrices F₁,F₂, …, F_r satisfying

\begin{array}{l} [\begin{matrix} 1 & x^{T} (0) \\ x (0) & P_{i}^{- 1} \end{matrix}] ​ ​ ​, ​ ​ ​ [\begin{matrix} φ_{ρ} P_{i} & W_{i j ρ l}^{T} \\ W_{i j ρ l} & φ_{ρ} I \end{matrix}] ​ ​ \geq O, \end{array}

(24)

∀i,j,ρ ∊ 1,2, …,r and ∀l, where W_ijρl = ξ_ρl = (A_i – B_iF_j).

Proof:

For|̇h_p(z)|≤ø_pand (z = x) one has

| {\dot{h}}_{ρ} (z (t)) | = | \frac{\partial h_{ρ} (z (t))}{\partial x (t)} \dot{x} (t) | \leq φ_{ρ^{'}}

(25)

where

\frac{\partial h_{ρ} (z (t))}{\partial x (t)} = \sum_{l = 1}^{s} v_{ρ l} (z (t)) ξ_{ρ l}, ​

(26)

with ν_ρl ≥ 0 and ∊^s_l=1 ν ρ_l(z(t)) = 1. In (26) ν_ρl (z(t)) depends on the number of membership functions. This term represents the maximum value that each input can have when evaluated in the different functions. ξ_ρl depends on the maximum and minimum values that can take a membership function evaluated at the extreme values of the universe of discourse. This term represents the interval that covers each function. Using (26) we obtain LMIs that satisfy (25), from which we can derive

{(\frac{\partial h_{ρ} (z (t))}{\partial x (t)} \dot{x} (t))}^{T} (\frac{\partial h_{ρ} (z (t))}{\partial x (t)} \dot{x} (t)) \leq φ_{ρ}^{2} .

(27)

By replacing (16) in (27) it is

\frac{1}{φ_{ρ}^{2}} x^{T} [\begin{array}{l} {(\sum_{l = 1}^{s} υ_{ρ l} ξ_{ρ l} (\sum_{i = 1}^{r} \sum_{j = 1}^{r} h_{i} h_{j} [A_{i} - B_{i} F_{j}]))}^{T} \\ (\sum_{l = 1}^{s} υ_{ρ l} ξ_{ρ l} (\sum_{i = 1}^{r} \sum_{j = 1}^{r} h_{i} h_{j} [A_{i} - B_{i} F_{j}])) \end{array}] x \leq 1.

(28)

If the fuzzy Lyapunov function (18) satisfies

V (x (t)) \leq V (x (0)) \leq 1, t \geq O,

(29)

then

\begin{array}{l} ​ ​ \sum_{i = 1}^{r} h_{i} (z (t)) x^{T} (t) P_{i} x (t) \leq \\ \sum_{i = 1}^{r} h_{i} (z (0)) x^{T} (0) P_{i} x (0) \leq 1, \end{array}

(30)

so that

1 - \sum_{i = 1}^{r} h_{i} (z (0)) x^{T} (0) P_{i} x (0) \geq O

(31)

and

1 - x^{T} (0) (\sum_{i = 1}^{r} h_{i} (z (0)) P_{i}) x (0) \geq O . ​

(32)

This last equation can be expressed via LMIs by using the Schur complement as follows:

[\begin{matrix} 1 & x^{T} (0) \\ x (0) & (\sum_{i = 1}^{r} h_{i} (z (0)) P_{i})^{- 1} \end{matrix}] \geq O, ​

(33)

because

[\begin{matrix} 1 & x^{T} (0) \\ x (0) & P_{i}^{- 1} \end{matrix}] \geq O, f o r ​ ​ i = 1, ..., r .

(34)

This leads to the LMI condition (24). On the other hand, by considering inequalities (28) and (30), inequality (25) holds if

\begin{array}{l} \frac{1}{φ_{ρ}^{2}} [(\sum_{l = 1}^{s} υ_{ρ l} (z) ξ_{ρ l} (\sum_{i = 1}^{r} \sum_{j = 1}^{r} h_{i} h_{j} A B F_{i}^{T})) . \\ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ (\sum_{l = 1}^{s} υ_{ρ l} (z) ξ_{ρ l} (\sum_{i = 1}^{r} \sum_{j = 1}^{r} h_{i} h_{j} A B F_{i}))] \\ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ - \sum_{i = 1}^{r} h_{i} P_{i} \leq O, \end{array}

where $A B F_{i} \overset{Δ}{=} A_{i} - F_{i} F_{j} .$ . Equivalently one has

[\begin{matrix} φ_{ρ} \sum_{i = 1}^{r} h_{i} P_{i} & \sum_{l = 1}^{s} υ_{ρ l} ξ_{ρ l} Q^{T} \\ \sum_{l = 1}^{s} υ_{ρ l} ξ_{ρ l} Q & φ_{ρ} I \end{matrix}] \geq O, ​

(35)

where $Q = \sum_{i = 1}^{r} \sum_{j = 1}^{r} h_{i} h_{j} [A_{i} - B_{i} F_{j}] .$ . This leads to the LMI condition (24).

Remark 3.1. ξ_ρl can be obtained from h_i(z(t)) by using the procedure given in Tanaka and Wang (2001). However, note that the conditions of Theorem 3.2 depend on the initial system state, so that they need to be known.

3.3 Observer Design

In practice, all states may not be fully measurable and it may be necessary to design an observer to implement (13). If the pairs (A_i,C_i) are observable, the fuzzy system (3) is called locally observable. In this work we will assume that this is the case. The design is based on fuzzy implications, with fuzzy sets in antecedents, and a Luenberger observer in the consequents. Each rule is responsible for estimating the states of a locally linear subsystem. The final outputs are obtained by fuzzy blending the local observers outputs. Based on the PDC structure, the local observers are designed as

\begin{array}{l} R u l e i : i f z_{1} (t) i s M_{i 1} a n d ... a n d z_{g} (t) i s M_{i g} \\ ​ ​ ​ ​ ​ ​ T h e n \dot{\overset{⌢}{x}} = A_{i} \overset{⌢}{x} + B_{i} u + L_{i} (y - \overset{⌢}{y}) \\ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ {\overset{⌢}{y}}_{i} = C_{i} \overset{⌢}{x}, i = 1,2, ...,r \end{array}

(36)

where L_i with i = 1,2, …, r is the observer gain for the i-th rule. The corresponding PDC fuzzy controller takes form

u = - \sum_{i = 1}^{r} h_{i} F_{i} \hat{x},

(37)

instead of (13). Note that the same weight w_i(z) is used. The dependence of the premise variables on the state makes necessary to consider two cases for fuzzy observer design:

z₁, …, z_g do not depend on the estimated state

z₁,…, z_g depend on the estimated state

In the second case in general h_i(z) ≠ h_i(ẑ) because z ≠ ẑ a priori. Then the premise variables can be the outputs or a combination of the measured states. In this work, we make the following assumption.

Assumption 3.3. z(t) is independent of x.

After Assumption 3.3, it is proposed

\dot{\overset{⌢}{x}} = \sum_{i = 1}^{r} h_{i} [A_{i} \overset{⌢}{x} + B_{i} u + L (y - \overset{⌢}{y})],

(38)

with final output

\overset{⌢}{y} (t) = \sum_{i = 1}^{r} h_{i} (z (t)) C_{i} \overset{⌢}{x} (t) .

(39)

The design consists in finding the gains L_i in the consequent part, for which it is necessary to compute the observer error dynamics. Substitute (5), (37) and (39) in (38) to obtain

\dot{\overset{⌢}{x}} = \sum_{i = 1}^{r} h_{i} A B F_{i} \overset{⌢}{x} + \sum_{i = 1}^{r} \sum_{j = 1}^{r} h_{i} h_{j} L_{i} C_{i} (x - \overset{⌢}{x}) .

(40)

On the other hand, by substituting (37) in system (12) one has

\dot{x} (t) = \sum_{i = 1}^{r} \sum_{j = 1}^{r} h_{i} h_{j} (A_{i} x - B_{i} F_{j} \overset{⌢}{x}) . ​

(41)

Define the observer error as

\tilde{x} = x - \overset{⌢}{x} .

(42)

Then, by subtracting (40) from (41) one gets the observer error dynamics as

\dot{\tilde{x}} = \sum_{i = 1}^{r} \sum_{j = 1}^{r} h_{i} h_{j} (A_{i} - L_{i} C_{j}) \overset{⌢}{x} . ​

(43)

Now consider the following theorem.

Theorem 3.3. The observer error dynamics (43) is asymptotically stable if there exist positive definite matrices Po₁,P₀₂, …, P_or and matrices L₁,L₂, …, L_r such that the following is satisfied:

P_{o i} > 0 ​ ​

(44)

\begin{array}{l} \sum_{ρ = 1}^{r} φ_{ρ} P_{0 ρ} + (G_{j j}^{T} P_{0 i} + P_{0 i} G_{j j}) < O ​ ​ \\ {(\frac{G_{j k} + G_{k j}}{2})}^{T} P_{0 i} + P_{0 i} (\frac{G_{j k} + G_{k j}}{2}) ​ ​ ​ ​ ​ ​ ​ ​ ​ < O ​ for ​ j < k \end{array}

(45)

\begin{matrix} [\begin{matrix} 1 & x^{T} (0) \\ x (0) & P_{0 i}^{- 1} \end{matrix}], [\begin{matrix} φ_{ρ} P_{0 i} & W_{i j ρ l}^{T} \\ W_{i j ρ l} & φ_{ρ} I \end{matrix}] \geq O \\ \forall i, j, k, p \in {1, ..., r} a n d \forall l, \end{matrix}

(46)

where

G_{j k} = A_{j} - L_{j} C_{k,} G_{j j} = A_{j} - L_{j} C_{j} a n d W_{i j p l} = ξ_{p l} (A_{i} - L_{i} C_{j})

Remark 3.2. By augmenting the states of the system with the state estimation error, we obtain the following 2n dimensional state equations for the observer/controller closed–loop system

[\begin{matrix} \dot{x} \\ \dot{\tilde{x}} \end{matrix}] = [\begin{matrix} {\bar{A}}_{11} & {\bar{A}}_{12} \\ 0 & {\bar{A}}_{22} \end{matrix}] [\begin{matrix} x \\ \tilde{x} \end{matrix}] ​

(47)

y = [\begin{matrix} \sum_{i = 1}^{r} h_{i} C_{i} & 0 \end{matrix}] [\begin{matrix} x \\ \tilde{x} \end{matrix}], ​

(48)

With

A_{11} = \sum_{i = 1}^{r} \sum_{j = 1}^{r} h_{1} h_{1} (A B F_{j}), A_{12} = \sum_{i = 1}^{r} \sum_{j = 1}^{r} h_{i} h_{j} (A_{i} - L_{i} C_{j}) .

Finally, as shown in Abdelmalek (2009), the separation principle holds for system (4)–(5), so that the error dynamics (47) is asymptotically stable.

4. Visual Servoing Fuzzy Controller Design

Consider a two–link robot arm (see Figure 2). In this section we are interested in a T–S fuzzy model by using PDC laws. The fuzzy controller design is to determine the local feedback gains F_i for the closed loop T–S fuzzy system (16).

Figure 2.

Experimental test bed.

We define X_i = P_i⁻¹, F_i =M_iX_i⁻¹, X_i = α_ijX_j for i,j = 1, …, r, where α_ij ≠ 1 and α_ij > 0 for i ≠ j, and α_ij = 1 for i = j. By giving φ_ρ > 0 and α_ij for i,j,ρ = 1, …, r, we obtain the following LMIs conditions that constitute a stable fuzzy controller design problem:

​ X_{i} > 0

(49)

​ ​ \sum_{ρ = 1}^{r} φ_{ρ} X_{ρ} + X_{i} A_{j}^{T} - α_{i j} M_{j}^{T} B_{j}^{T} + A_{j} X_{i} - α_{i j} B_{j} M_{j} < O

(50)

\begin{array}{l} X_{i} A_{j}^{T} - α_{i k} M_{k}^{T} B_{j}^{T} + X_{i} A_{k}^{T} ​_{​} - α_{i j} M_{j}^{T} B_{k}^{T} + \\ + A_{j} X_{i} - α_{i j} B_{j} M_{k} + A_{j} X_{i} - α_{i j} B_{j} M_{j} < O, \end{array}

(51)

for each set of i,j,k ∊ {1,…,r} such as j < k

[\begin{matrix} 1 & x^{T} (0) \\ x (0) & X_{i} \end{matrix}] \geq O ​ ​ ​ ​ ​ f o r ​ ​ ​ ​ ​ i = 1, ..., r,

(52)

[\begin{matrix} φ_{ρ} X_{i} & W_{i j ρ l}^{T} \\ W_{i j ρ l} & φ_{ρ} I \end{matrix}] \geq O,

(53)

where W_ijρl = ξ_ρl (A_iX_i – α_ijB_iM_j). Note that from X_i = α_ijX_j we have X_j = (1/α_ij)X_i = α_ijX_i, so that α_ij = 1/ α_ij ∀_i, j ∊ {1, …, r}, and hence, for given i and j, the relation α_ijα_ji = 1 is used. Note that X_i and F_i are computed by solving (13), while M_j can be obtained from these two matrices.

The coefficients α_ij and φ_ρ for i,j,ρ = 1, …, r and i ≠ j, can be chosen heuristically according to the application. In particular, the φ_ρ's are chosen to obtain a fast switching among IF-THEN rules in order to keep the speed of response for a closed–loop system Tanaka and Wang (2001). α_ij and β_ij must be different from 1 (for i = j, α_ij = β_ij = 1) and selection of ξ_ρl is obtained from h_iz(t). Subsequently, we will consider that the premise variables do not depend on the estimated states ○(t). The objective of the tracking control is to make the system outputs follow some predefined trajectories. The inputs are given by the vision system and correspond to the image features, and the output is the torque applied in each joint. The fuzzy observer–based control is summarized as follows

Select the fuzzy rules and membership functions for nonlinear system (3).

Calculate the matrices A_i, B_i and C_i by using model (3).

Establish the reference model or path to follow.

Compute L_i.

Solve the LMI to obtain F_i.

Solve the LMI to obtain P_i.

Solve the LMI to obtain P_0i.

Construct the fuzzy observer (36).

Construct the fuzzy controller (37).

The system state is given by

\begin{array}{l} x (t) = {[\begin{matrix} x_{1} & x_{2} & x_{3} & x_{4} \end{matrix}]}^{T} \\ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ = {[\begin{matrix} y_{1} & y_{2} & y_{3} & y_{4} \end{matrix}]}^{T}, \end{array}

(54)

where (Δy₁, Δy₂) represents the robot end–effector position in image coordinates and (Δẏ₁, Δẏ₂) are the corresponding velocities. Note that it is possible to establish the following limits for the state due to physical restrictions.

\begin{array}{l} x_{1} (t) \in [x_{1_{\min}}, x_{1_{\max}}] = [0, ​ 1000] [p i x e l s] \\ x_{2} (t) \in [x_{2_{\min}}, x_{2_{\max}}] = [- 50, ​ 50] [p i x e l s / s] \\ x_{3} (t) \in [x_{3_{\min}}, x_{3_{\max}}] = [0, ​ 1000] [p i x e l s] \\ x_{4} (t) \in [x_{4_{\min}}, x_{4_{\max}}] = [- 50, ​ 50] [p i x e l s / s] \end{array}

To minimize the design effort and complexity, we try to use as few rules as possible. The maximum and minimum values for state variables are defined based on images acquired. x₁ and x₃ are measurable through the camera. Based on the dynamic model equation which is a function of image coordinates are calculated matrices A_i, B_i and C_i. The T–S fuzzy model is given by the following rules whose membership functions are of triangular form. The premise variables are the tracking error in image coordinates, which is obtained by

[\begin{matrix} Δ y_{1} \\ Δ y_{2} \end{matrix}] = [\begin{matrix} y_{1} - y_{d 1} \\ y_{2} - y_{d 2} \end{matrix}], ​

(55)

where (y₁,y₂) are obtained through the camera image on the computer screen, and (y_d1,y_d2) represent the desired trajectories.

Rule 1: If Δy₁ is about–10 and Δy₂ is about 10 Then Ẋ(t) = A₁x(t) + B₁u(t),y = C₁x(t),

Rule 2: If Δy₁ is about – 10 and Δy₂ is about 0

Then Ẋ(t) = A₂x(t) ₊ B₂u(t),y = C₂x(t),

Rule 3: If Ay₁isabout – 10 and Δy₂ is about – 10 Then Ẋ(t) = A₃x(t) + B₃u(t),y = C₃x(t),

Rule 4: If Δy₁ is about 0 and Δy₂ is about – 10

Then Ẋ(t) = A₄x(t) + B₄u(t),y = C₄x(t),

Rule 5: If Δy₁ is about 0 and Δy₂ is about 0

Then Ẋ(t) = A₅x(t) + B₅u(t),y = C₅x(t),

Rule 6: If Δy₁ is about 0 and Δy₂ is about 10

Then ẋ(t) = A₆x(t)₊B₆u(t),y = C₆x(t),

Rule 7: If Δy₁ is about 10 and Δy₂ is about – 10 Then ẋ (t) = A₇x(t) + B₇u(t),y = C₇x(t),

Rule 8: If Δy₁ is about 10 and Δy₂ is about 0

Then ẋ(t) = A₈x(t)₊B₈u(t),y = C₈x(t),

Rule 9: If Δy₁ is about 10 and Δy₂ is about 10

Then ẋ(t) = A₉x(t)₊B₉u(t),y = C₉x(t).

In order to get the matrices (A_i, B_i,C_i), a model of the form (57) is necessary. In Appendix A a sketch of the robot model in image coordinates, as well as and example of a set (A_i,B_i,C_i), with i = 1, are given. The amount of 9 sets was gotten by trial and error. The image coordinates and their corresponding velocities were considered as the system states, which will be stabilized to zero by the T – S fuzzy controller and observer. The experimental results show that tracking errors tend to zero. The tracking errors are considered in the premise of fuzzy rules; the time derivatives are not considered because the trajectories are small and this does not affect the design of the controller. It is noted that the T-S fuzzy model for the controller and observer design is only an approximation of the robot dynamics given in (57). Therefore, asymptotical stabilization of the fuzzy model does not mean that the original robot dynamics can be asymptotically stabilized, unless the T-S fuzzy model is an exact representation of the original robot dynamics. The number of fuzzy rules allow having a robust system that operates correctly in the operation zone defined. Finally, by employing the MATLAB LMI Toolbox, the corresponding gains F_i and L_i are computed. The results are also shown in the appendix. Note that the MATLAB LMI Toolbox easily allows to satisfy the conditions given in Theorems 3.1, 3.2 and 3.3 to guarantee asymptotical stability of tracking and observation errors.

5. Experimental Results

In order to test the theory given in Section 4, the robot A465 of CRS Robotics shown in Figure 2 has been used. It has six degrees of freedom, but only joints 2 and 3 (renumbered 1 and 2, respectively) are employed. The visual system consists of a CCD monochrome camera (Pike F–505B). A circular object is attached to the robot end–effector to recognize it. In order to have a point of comparison, two algorithms have also been programmed. Although they are not explicitly designed to cope with two angles of rotation, we consider that the comparison is fair because both of them are robust schemes.

The Comparative Algorithm 1 is an adaptive second order sliding mode global tracking visual feedback controller developed in Parra-Vega et al. (2003). It has been chosen because it is also meant for planar manipulators in a modified image–based approach. Furthermore, the basic assumption is that the robot model parameters are unknown. The Comparative Algorithm 2 is the one given in Zergeroglu et al. (2001). It is an adaptive calibration controller that compensates for uncertain camera parameters and ensures global asymptotic position tracking. The interested reader can see the reference to check out these schemes, omitted here for lack of room.

5.1 Experimental Outcomes

One experiment has been carried out for all control schemes which is considered representative enough to check out tracking performance. It consists in following a circle in the y₁ – ?y₂ plane described by

y_{d} (t) = [\begin{matrix} 60 \times \sin (0.2 t) + 650 \\ - 60 \times \cos (0.2 t) + 450 \end{matrix}] ​ ​ ​ ​ ​ ​ [p i x e l] . ​

(56)

With raw eye it was set ø ≈ 45° and Ψ ≈ 25°. Note that the comparative algorithms are only designed for the nominal case Ψ ≈ 0°, thus they rely on their robustness properties to get good results.

In Figure 3, the desired and actual trajectories in the image plane are shown for all approaches. The initial position has been chosen the same in each case and inside the circle. As can be appreciated, the performance appears to be similar in all cases. Note that since the origin (0,0) is in the upper left corner, a minus sign has been added to have the y₂ in the right direction even though it is actually a positive value always. To have a better insight, we have computed the Root Mean Square Error (RMSE) as $\sqrt{\frac{1}{n} \sum_{i = 1}^{n} e_{i} ^{2}}$ where i is the current sample number, e_i is the error associated to i and n is the total number of samples. As can be appreciated in Table 1, the performance of the proposed approach is better. Another advantage of fuzzy logic is that, on the contrary to the adaptive approaches, it does not need any regressor matrix and thus its implementation may be more direct.

Table 1.

Root mean square errors [PIXELS].

Error	Proposed Algorithm	C.Algorithm 1	C.Algorithm 2
Δy₁	4.3123	5.7234	5.9960
Δy₂	10.7642	15.1442	14.1335

Figure 3.

Trajectory in the image plane (y₁,y₂). Desired trajectory (…). Proposed Algorithm (—). Comp. Algorithm 1 (– –). Comp. Algorithm 2 (- – -).

Finally, the observation errors for the proposed algorithm are shown in Figure 4. While the error s˜₂ becomes zero, s˜₁ has a value of 1[pixel] during more than half the experiment to become zero in the end.

Figure 4.

Observer errors for the proposed algorithm. a) s˜₁. b) s˜₂.

6. Conclusions

This paper presents a new fuzzy tracking control scheme for visual servoing applied in a robot manipulator. Based on the T-S model, a fuzzy observer and a fuzzy controller are developed to reduce the tracking errors arbitrarily. Furthermore, the stability of the closed-loop nonlinear system is discussed. The advantage of the proposed tracking control design is that only a simple fuzzy controller is used without feedback linearization or an adaptive scheme. A state space model for a robot manipulator with two degrees of freedom in image coordinates has been suggested which can be employed with other controllers. Experimental results are shown by comparing the proposed scheme with well-known adaptive control laws. The outcomes clearly show the good performance of the algorithm introduced in this work.

Footnotes

Acknowledgment

This work was supported by the CONACYT under grant 58112 and by the DGAPA-UNAM under grant IN109611.

Appendix A

In order to implement the schemes for the experiments shown in Section 5, the dynamic model of robot A465 is necessary for implementation. By considering (1) as two degrees of freedom manipulator and computing the derivative of (2) one gets the differential perceptual kinematic model given by ẏ = α_λR_φ ẋ _R, with ẋ_R =J(q)q̇ J(q) = ∂f_k(q)/∂q is the so–called geometrical Jacobian matrix of the robot Sciavicco and Siciliano (2000). As long as the robot is not in a singularity, one gets $\dot{q} = \frac{1}{α_{λ}} J^{- 1} (q) R_{φ} \dot{y}$ . This relationship defines the visual mapping that relates the velocity in Cartesian space with the velocity in image space. To use a visual control algorithm is necessary to rewrite the model of the robot manipulator in terms of image coordinates y. This is a rather standard procedure leading to

(57)

\bar{H} (q) \ddot{y} + \bar{C} (q, \dot{q}) \dot{y} + \bar{g} (q) = \bar{τ} - {\bar{τ}}_{p},

where

\begin{array}{l} ​ ​ ​ ​ ​ \bar{H} (q) = \frac{1}{α_{λ}} R_{φ} J^{- T} (q) H (q) J^{- 1} (q) R_{φ} \\ \bar{C} (q, \dot{q}) = \frac{1}{α_{λ}} R_{φ} J^{- T} (q) [H (q) J^{- 1} (q) | R_{φ} + C (q, \dot{q}) J^{- 1} (q) R_{φ} + D (q) J^{- 1} (q) R_{φ}] \\ ​ ​ ​ ​ \bar{g} (q) = R_{φ} J^{- T} (q) g (q) \\ ​ ​ ​ ​ \bar{τ} (q) = R_{φ} J^{- T} (q) \end{array}

The parameters of model (57) can be found in Pérez et al. (2009) and the references cited there. Note that it is exclusively used to generate the state space representation (A_i,B_i,C_i) for the Takagi–Sugeno model. Because of lack of room, we write down only the matrices for the operation point p₁. In this case one has

\begin{array}{l} A_{1} = [\begin{matrix} 0 & 1 & 0 & 0 \\ 3.9 & - 0.001 & - 0.32 & - 8.4 \cdot 10^{- 6} \\ 0 & 0 & 0 & 1 \\ - 6.9 & 0.002 & 3.2 & 6.2 \cdot 10^{- 6} \end{matrix}] ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ \\ ​ ​ B_{1} = {[\begin{array}{l} \begin{matrix} 0 & ​ ​ ​ ​ 1 & 0 & - 1 \end{matrix} \\ \begin{matrix} 0 & - 1 & 0 & 2 \end{matrix} \end{array}]}^{T} ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ \\ ​ ​ C_{1} = [\begin{array}{l} \begin{matrix} 1 & 0 & 0 & 0 \end{matrix} \\ \begin{matrix} 0 & 0 & 1 & 0 \end{matrix} \end{array}] \end{array}

Achieving the conditions given in Theorem 3.1 can be readily done by using the MATLAB LMI Toolbox, so that the control–observer gains are found to be

\begin{array}{l} L_{1} = [\begin{matrix} 3.2458 \times 10^{2} & - 5.5324 \times 10^{2} \\ 1.2084 \times 10^{2} & - 2.7651 \times 10^{2} \\ - 5.7012 \times 10^{2} & 1.0698 \times 10^{2} \\ - 7.1989 \times 10^{2} & 7.1763 \times 10^{2} \end{matrix}] ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ \\ ​ F_{1} = [\begin{array}{l} \begin{matrix} - 263.49 & - 100.42 & - 143.45 & - 54.11 \end{matrix} \\ \begin{matrix} - 164.11 & - 63.37 & - 138.30 & - 53.05 \end{matrix} \end{array}] \end{array}

\begin{array}{l} P_{1} = [\begin{matrix} 0.0034 & 0.0005 & - 0.0004 & 0.000 \\ 0.0005 & 0.0005 & 0.0000 & 0.0000 \\ - 0.0004 & 0.0000 & 0.0190 & 0.0011 \\ 0.0000 & 0.0000 & 0.0011 & 0.0004 \end{matrix}] ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ \\ P_{01} = [\begin{matrix} 0.6571 & 0.0010 & 0.0013 & 0.0161 \\ 0.0010 & 0.6615 & 0.0044 & 0.0764 \\ 0.0013 & 0.0044 & 0.0054 & 0.0565 \\ 0.0161 & 0.0764 & 0.0565 & 0.8634 \end{matrix}] \end{array}

References

Abdelmalek

(2009). Non linear systems control: LMI fuzzy approach, Master's thesis, Université Hadj Lakhdar de Batna.

Abdelmalek

Golea

and Hadjili

M. L.

(2007). A new fuzzy Lyapunov approach to non quadratic stabilization of Takagi-Sugeno fuzzy models, International Journal of Applied Mathematics and Computer Science, 17(1): 39–51.

Fares

E. A.

Elbardiny

and Sharaf

M.M.

(2007). Adaptive fuzzy logic controller of visual servoing robot system by membership optimization using genetic algorithms, Computer Engineering and Systems, 2007. ICCES.

Goncalves

Mendonca

Sousa

and Pinto

(2008). Uncalibrated eye-to-hand visual servoing using inverse fuzzy models, IEEE Transactions On Fuzzy Systems, 16(2): 341–353.

Kelly

and Reyes

(2000). On vision systems identification with application to fixed camera robotic systems, International Journal of Imaging Systems and Technology, 11(3): 170–180.

Kim

Seol

W. H.

Han

S. H.

and Khatib

(2001). Fuzzy logic control of a robot manipulator based on visual servoing, ISIE 2001. IEEE International Symposium on Industrial Electronics. Pusan, Korea.

Liu

and Lam

(2006). Uncalibrated visual servoing of robots using a depth-independent interaction matrix, IEEE Transactions On Robotics Systems, 22(4): 804–817.

Meza

A. G.

(2003). Observadores difusos y control adaptable difuso basado en observadores, Master's thesis, Cinvestav.

Parra-Vega

Fierro-Rojas

and Espinosa-Romero

(2003). Adaptive sliding mode uncalibrated visual servoing for finite-time tracking of 2D robot, International Conference on Robotics and Automation, 3(3): 3048–3054.

10.

Pérez

Arteaga-Pérez

Kelly

and Espinosa

(2009). On output regulation of direct visual servoing via velocity fields, International Journal of Control, 82(4).

11.

Sciavicco

and Siciliano

(2000). Modeling and Control of Robot Manipulators, London, Great Britain, Springer-Verlag.

12.

Takagi

and Sugeno

(1985). Fuzzy identification of systems and its applications to modeling and control, IEEE Transactions on Systems, Man and Cybernetics, 6(1): 116–132.

13.

Tanaka

Hori

and Wang

H. O.

(2003). A multiple Lyapunov function approach to stabilization of fuzzy control systems, IEEE Transactions On Fuzzy Systems, 11(4): 582–589.

14.

Tanaka

Ikeda

and Wang

H. O.

(1998). Fuzzy regulators and fuzzy observers: Relaxed stability conditions and LMI-based designs, IEEE Transactions on Fuzzy Systems, 6(2): 250–265.

15.

Tanaka

and Sugeno

(1992). Stability analysis and design of fuzzy control systems, Fuzzy sets and systems, 45(2): 135–156.

16.

Tanaka

and Wang

(2001). Fuzzy Control Systems Design and Analysis. A Linear Matrix Inequality Approach, JohnWiley and Sons.

17.

Wai

R. J.

and Yang

Z.W.

(2008). Adaptive fuzzy neural network control design via a TS fuzzy model for a robot manipulator including actuator dynamics, IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics., 38(5): 1326–1346.

18.

Zergeroglu

Dawson

de Queiroz

and Behal

(2001). Visionu based nonlinear tracking controllers with uncertain robotu camera parameters, IEEE/ASME Transactions on Mechatronics, 6(3): 332–337.

19.

Zhang

and Fei

(2006). Analysis and design of robust fuzzy controllers and robust fuzzy observers of nonlinear systems, 6th World Congress on Intelligent Control and Automation. Dalian, China, Vol. 1, pp. 3767–3771.