Line Tracking Control of a Two-Wheeled Mobile Robot Using Visual Feedback

Abstract

This article presents the development and control of a two-wheeled mobile robot as the base of a human carrier for an amusement/transportation vehicle. The robot has a combined structure of two systems: a line tracking mobile robot and an inverted pendulum system that maintains balance while following a line on the floor. The mobile robot is purposely designed to carry a human operator or humanoid arms. The robot has the capability to follow the line on the floor using visual feedback, as well as maintaining its balance on two wheels. A visual servoing technique allows the robot to follow the line on the floor captured by a camera as the desired trajectory. Controllers are designed to have good line tracking and balancing performance using sensor fusion techniques. Experimental studies involving the robot following a line demonstrate the feasibility of it being an amusement vehicle.

Keywords

Two-Wheeled Mobile Robot Visual Servoing Control Line Tracking Amusement Vehicle

1. Introduction

A balancing mechanism is one of the most important characteristics of dynamic systems that require a stable balancing posture to perform given tasks. The human body is a good example of a structure that uses balancing mechanisms in everyday life to stand, walk and run. Humans learn balancing skills over many years, from crawling as a baby to standing as a child.

Recently, human balancing skills have been endowed to dynamical systems. A robot that mimics a human being's balancing mechanism is called a humanoid robot [1 –4]. If the balancing mechanism is broken in the dynamical system, the system becomes unstable. Therefore, robust balancing mechanism is quite an important skill in the dynamical system and requires a sophisticated control mechanism based on accurate sensing techniques.

One typical well-known system for balancing is the inverted pendulum system that consists of a cart and an inverted pendulum. The cart moves on the fixed rail track to balance the inverted pendulum. The goal of the inverted pendulum system is to control both the balancing angle and the position as desired. Control of the inverted pendulum system is quite challenging since a single input force has to control both angle and position, as in a single-input multiple-output configuration. More challengingly, combining two inverted pendulum systems together in the x and y-axes yields a two degrees-of-freedom (DOF) inverted pendulum system that moves on a plane [5]. Other variations of inverted pendulum systems have been presented in the literature [6 –8].

Replacing the fixed track of the inverted pendulum system with two wheels adds another degrees-of-freedom (DOF) to allow navigation on the plane and this forms the mobile inverted pendulum system whose structure is a combination of two systems, a mobile robot system and an inverted pendulum system [9 –19]. Since wheels can move on the plane, control of the mobile inverted pendulum system becomes more difficult to consider an additional heading angle to be controlled. Thus, for successful control of the mobile inverted pendulum system, three variables, namely a balancing angle, an orientation angle and a position should be controlled properly by input torques to drive the wheels.

Recently, the concept of a mobile inverted pendulum system has been applied to commercial service robots such as a personal transportation vehicle called Segway [9]. The reasons of using mobile inverted pendulum systems as a transportation vehicle are better manoeuvring and turning control in narrow spaces [10].

Successful commercial market of Segway has attracted many researchers' attention to develop and control mobile inverted pendulum systems as in the category of service robots. Movement of Segway is controlled based on the change of the centre of gravity by shifting the body and by manipulating handles. Thus, driving control of the Segway is not easy for beginners wishing to stabilize the vehicle.

The small-sized JOE has also demonstrated successful balancing control performance [11]. Control of the mobile inverted pendulum system based on system modelling has been presented [12 –14] and a disturbance observer has been designed to control the mobile inverted pendulum system [13]. Interaction control of a mobile inverted pendulum system with other objects has also been presented [14, 15]. Compensation for the drift problem of a gyro sensor has been addressed to obtain an accurate angle [16]. Several balancing robots called BalBots have been developed and controlled with various control algorithms [17 –19]. BalBot I and II from the BalBot series, have a simple structure of a slender rod for the pendulum with a two-wheeled balancing mechanism. Control of BalBot I and II has been achieved with linear controllers with the help of a neural network controller. A multi-layered perceptron network is used and implemented on a DSP 6000 series for BalBot I [17] and the radial basis function network is used and implemented on DSP 2812 for BalBot II [18, 19]. The design of BalBot III is quite different from previous ones: it has two arms instead of a single rod on the balancing platform so that it can play boxing games and is called BoxingBot [20].

Borrowing the concept of balancing control from two-wheeled mobile robots, this paper presents a new model for the BalBot series called AmuseTransBot, as shown in Fig. 1, which is purposely designed for carrying human beings for entertainment and service in an amusement park. The base of the AmuseTransBot is designed and developed to carry a human being as a personal transportation vehicle or to carry a humanoid robot arm [21]. A mobile inverted pendulum, BalBot IV is presented [22].

Figure 1.

Concept of Human-carrying BalBot, AmuseTransBot

The goal of this paper is to summarize [22] how the bottom part of the human-carrying BalBot is built and tested to have a robust balancing performance while following a line on the floor captured by a CCD camera. The visual servoing control technique of the human-carrying BalBot is employed to follow the line while maintaining balance. A linear controller is designed to achieve stable tracking performances when balancing with sensor fusion techniques. Line tracking experiments are demonstrated to show the control performance of the human-carrying BalBot.

2. Design of Human-carrying BalBot

The size of the human-carrying BalBot is designed to be larger than the previous BalBot series. The purpose of the human-carrying BalBot is to carry human beings as a transportation vehicle. The bottom part of the whole system is designed using a CAD program, as shown in Fig. 2. Fig.2 (a) shows the overall shape of the human-carrying BalBot and Fig. 2 (b) shows the inside of the BalBot. The BalBot has two wheels and two supporters to prevent it from falling down. The inside of the housing structure of the DC motors and batteries is shown in Fig.2 (b).

Figure 2.

CAD Design of the human-carrying BalBot

3. Kinematics

The BalBot is a two-wheeled drive mobile robot with an inverted pendulum structure. The kinematics of the BalBot is similar to that of a wheeled drive mobile robot. Fig. 3 shows the coordinates of the BalBot. Since the BalBot is a non-holonomic system, position x, y and the orientation (heading) angle φ should be realized by two wheels.

Figure 3.

Coordinates of BalBot

The Cartesian velocities have a relationship with a linear velocity v and an angular velocity φ of the robot:

[\begin{matrix} \dot{x} \\ \dot{y} \\ \dot{φ} \end{matrix}] = [\begin{matrix} \frac{r}{2} \cos φ - \frac{r d}{L} \sin φ & \frac{r}{2} \cos φ + \frac{r d}{L} \sin φ \\ \frac{r}{2} \sin φ + \frac{r d}{L} \cos φ & \frac{r}{2} \sin φ - \frac{r d}{L} \cos φ \\ \frac{r}{L} & - \frac{r}{L} \end{matrix}] [\begin{matrix} w_{R} \\ w_{L} \end{matrix}],

(1)

where φ is the heading angle, d is the distance between the centre of the actuating axis and the centre of gravity, r is the radius of a wheel, L is the distance between two wheels. w_R = θ_R is the angular velocity of the right wheel and w_L = θ_L is the angular velocity of the left wheel.

In our system the wheels are designed and assumed to be located in the centre of the body, such that d = 0. Therefore, the relationship becomes simplified.

[\begin{matrix} \dot{x} \\ \dot{y} \\ \dot{φ} \end{matrix}] = [\begin{matrix} \frac{r}{2} \cos φ & \frac{r}{2} \cos φ \\ \frac{r}{2} \sin φ & \frac{r}{2} \sin φ \\ \frac{r}{L} & - \frac{r}{L} \end{matrix}] [\begin{matrix} w_{R} \\ w_{L} \end{matrix}] .

(2)

4. Control Schemes

The BalBot is kinematically constrained and has the non-holonomic characteristic dynamic equation:

M (q) \overset{‥}{q} + C (q, \dot{q}) \dot{q} + G (q) = A^{T} λ + P τ

(3)

where M(q) is the 6×6 inertia matrix, C(q,q) is the 6×1 vector of Coriolis and centrifugal torques, G(q) is the 6x1 gravitational torques, λ is the Lagrangian multiplier, P is the 6×2 input transition matrix and τ is the 2×1 vector of the actuator joint torques when the joint vector is given by q = [x,y,φ,θ,θ_R,θ_L]^T where θ is the balancing angle. Since the system is kinematically constrained in the lateral direction, we have the constraint matrix, A(q):

A (q) = [\begin{matrix} \sin φ & - \cos φ & 0 & 0 & 0 & 0 \\ \cos φ & \sin φ & \frac{L}{2} & 0 & - r & 0 \\ \cos φ & \sin φ & - \frac{L}{2} & 0 & 0 & - r \end{matrix}]

(4)

To eliminate the Lagrangian multiplier, we select the matrix S to satisfy the relationship of S^TA^T(q) = 0.

S (q) = [\begin{matrix} \cos φ & 0 & 0 \\ \sin φ & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \frac{1}{r} & \frac{L}{2 r} & 0 \\ \frac{1}{r} & - \frac{L}{2 r} & 0 \end{matrix}]

(5)

Since we have a Jacobian relationship of q = S(q)ν, q = Sν ₊ Sν, multiplying S^T to (3) yields the Cartesian dynamic equation:

D \dot{υ} + H = S^{T} P τ

(6)

whereD = S^TMS, H = S^TMSν + S^T(C ₊ G).

Controlled variables include a balancing angle θ, an orientation angle φ and a position p. A PD controller is used for controlling the balance angle and PID controllers are used for position and heading angle control. A combined control method for each wheel is given as:

\begin{array}{l} τ_{R} = K_{A p} (θ_{d} - θ) + K_{A d} ({\dot{θ}}_{d} - \dot{θ}) \\ + K_{H p} (φ_{d} - φ) + K_{H d} (ω_{d} - ω) + K_{H i} \int_{0}^{t} (φ_{d} - φ) d t \\ + K_{P p} (p_{d} - p) + K_{P d} (v_{d} - v) + K_{P i} \int_{0}^{t} (p_{d} - p) d t \\ τ_{L} = K_{A p} (θ_{d} - θ) + K_{A d} ({\dot{θ}}_{d} - \dot{θ}) \\ - K_{H p} (φ_{d} - φ) - K_{H d} (ω_{d} - ω) - K_{H i} \int_{0}^{t} (φ_{d} - φ) d t \\ + K_{P p} (p_{d} - p) + K_{P d} (v_{d} - v) + K_{P i} \int_{0}^{t} (p_{d} - p) d t \end{array}

(7)

where K_ii is the controller gain, ω = φ, v = p and φ_d,θ_d,p_d are the desired orientation angles, a desired balancing angle and a desired position, respectively. The desired balancing angle is set to zero and the desired orientation angle is calculated using a camera. The corresponding control block diagram is shown in Fig. 4.

Figure 4.

Control block diagram of human carrying BalBot

5. Experimental Studies

5.1. Experimental Setup

The overall hardware system structure is shown in Fig. 5. The experimental setup consists of the BalBot with a CCD camera, a PC and the control hardware. The line on the floor can be detected by the CCD camera and the captured images are processed in real time on a PC to generate the desired trajectory (p_d and φ_d). Each joint torque command can be calculated from a control algorithm that forms errors between the line trajectory captured from a CCD camera through the kinematics of an actual trajectory. The robot position p and the orientation angle φ are calculated by encoder signals. The balancing angle θ is measured from a fused algorithm by a tilt and a gyro sensor.

Figure 5.

Overall hardware system structure

Based on the kinematic equation given in (2), state variables for the controller design are defined and obtained from sensors as shown in Fig. 6. Encoder measurements are used to calculate the position and orientation of the robot through kinematics and tilt and gyro sensors are used for detecting the balancing angle, as shown in Fig. 6. FPGA chips are used for collecting encoder data and converting analogue signals to digital.

Figure 6.

Control variables of BalBot

Encoder counters embedded in FPGA count the number of rotations of the motors to provide position p. Then velocity information v is estimated through the finite difference method. Captured tilt and gyro sensor data are digitized by an AD converter and used in the complementary filter to generate the balancing angle θ and the angular velocity θ. The mechanical layout of the BalBot is shown in Fig. 7. The control hardware has DSPs and a FPGA to control the wheel motors, as shown in Fig. 8.

Figure 7.

Experimental setup of BalBot

Figure 8.

Control hardware: DSP and FPGA

Fig. 9 shows the GUI on the screen that describes the laser sensor data and captured image data. From the image data, the line trajectories are calculated for the robot so that it follows the desired trajectories.

Figure 9.

Line capturing monitor

The PD and PID controller gains are listed in Table 1. Gains are selected based on experimental studies. Several experiments were conducted to decide which controller gains are suitable.

Table 1.

Controller gains

	Gains	Values
Balancing angle control	K_Ap	25
Balancing angle control	K_Ad	1.5
Heading angle control	K_Hp	10
	K_Hd	1
	K_Hi	0.1
Position control	K_Pp	−4
	K_Pd	−5
	K_Pi	−0.01

5.2. Experimental Results

5.2.1. Balancing control test

Firstly, a balancing control task was tested. The BalBot was initially standing and then external disturbance was applied to the system. Fig. 10, 11 and 12 show the response results of the balancing angle, the position and the heading angle, respectively. Fig. 10 shows an interesting result that indicates that the robot is balanced not at zero but within a 0.1 radian. This behaviour occurs when the actual centre of gravity (COG) of the system is not located exactly in the centre of the system body, but deviates slightly from the COG point. At around 2 seconds, there is an intentional disturbance from an external hit, but the BalBot maintains balance well. We see from Fig. 11 that a small offset in position occurs after the external hit. The corresponding heading angle is shown in Fig. 12.

Figure 10.

Balancing angle error of BalBot

Figure 11.

Balancing position error of BalBot

Figure 12.

Heading angle error of BalBot

5.2.2. Line tracking control task

Next, the line tracking control performance is tested. The red line is drawn on the green floor, as shown in Fig. 13. The robot is required to follow that red line. Fig. 13 shows the actual movements of the BalBot when following the line. We see the successful line-tracking performances from the starting point to the ending point as desired.

Figure 13.

Line tracking performances of BalBot

The corresponding results are shown in Fig. 14 –18. Fig. 14 shows the successful balancing angle result. The position tracking error between the desired trajectory captured by the camera and the actual trajectory is shown in Fig. 15. The corresponding heading angle is shown in Fig. 16. We see the actual trajectory of the BalBot on the x-y plane in Fig. 17. The corresponding heading angle is also shown in Fig. 18.

Figure 14.

Balancing angle error of BalBot

Figure 15.

Position error of BalBot

Figure 16.

Heading angle error of BalBot

Figure 17.

XY Position of BalBot

Figure 18.

Heading angle of BalBot

6. Conclusion

A base of the balancing robot was designed and implemented to carry human beings for amusement and service applications. The robot is required to follow a line on the floor while balancing. Image processing of captured images from a CCD camera on a PC provides the desired trajectories for the robot to follow. Necessary control hardware such as a DSP and an FPGA chip is implemented to generate control torque for each wheel. Multi-sensors such as tilt and gyro are fused with a complementary filter to detect the balancing angle accurately. Relying on linear controllers, the BalBot achieves successful line tracking performances. In the future, the BalBot will be equipped with a seat for a human operator.

Footnotes

7. Acknowledgements

This work was partially supported by the Korea Research Foundation under a grant from the Basic Research Program in 2008 and 2010.

References

“ASIMO”, http://world.honda.com/ASIMO/

“HUBO”, http://ohzlab.kaist.ac.kr

Sekiguchi

Kameta

Yuichi

and Nenchev

, “Biped walk based on vertical pivot motion of linear inverted pendulum”, IEEE Conf. on Advanced Motion Control, 2007, 1–6.

Hirabayashi

Ugurlu

and Kawamura

, “Yaw moment compensation of biped fast walking using 3D inverted pendulum”, IEEE Conf. on Advanced Motion Control, 2008, 296–300.

Jung

Cho

H.T.

Hsia

T. C.

, “Neural network control for position tracking of a two-axis inverted pendulum system: Experimental studies”, IEEE Transaction on Neural Networks, 18, (4), 2007, 1042–1048.

Spong

M. W.

Corke

and Lozano

, “Nonlinear control of the inertia wheel pendulum”, Automatica, 37, 2001, 1845–1851.

Cheng

Zhong

and Xu

, “Fuzzy control of a double inverted pendulum”, Fuzzy sets and systems, 79, 1996, 315–321.

White

and Fales

, “Control of double inverted pendulum with hydraulic actuation: a case study”, Proc. Of the American Control Conference, pp.495–499, 1999.

“Segway.”, http://www.segway.com

10.

Ambrose

R. O.

Savely

R. T.

Goza

S.M.

Strawser

Diftler

M. A.

Spain

and Radford

, “Mobile manipulation using NASA's robonaut”, IEEE ICRA, 2004, 2104–2109.

11.

Grasser

D'Arrigo

Colmbi

Rufer

Alfred C.

, “JOE:A Mobile Inverted Pendulum”, IEEE Trans. On Industrial Electronics, 49 (1), 2002, 107–114.

12.

Pathak

Franch

and Agrawal

, “Velocity and position control of a wheeled inverted pendulum by partial feedback linearization”, IEEE Trans. on Robotics, 21, 2005, 505–513.

13.

Jeong

S. H.

and Takayuki

, “Wheeled Inverted Pendulum Type Assistant Robot: Design Concept and Mobile Control”, IEEE IROS, 2007, 1932–1937.

14.

Sasaki

and Murakami

, “Pushing operation by two-wheel inverted mobile manipulator”, IEEE Workshop on Advanced Motion Control, 2008, 33–37.

15.

Abeygunawardhana

P. K.

and Toshiyuki

, “Environmental interaction of two-wheeled mobile manipulator by using reaction torque observer”, IEEE Workshop on Advanced Motion Control, 2008, 348–353.

16.

Imamura

Takei

and Yuta

, “Sensor drift compensation and control of a wheeled inverted pendulum mobile robot”, IEEE Workshop on Advanced Motion Control, 2008, 137–142.

17.

Kim

S. S.

and Jung

, “Control experiment of a wheel-driven mobile inverted pendulum using neural network”, IEEE Trans. On Control Systems Technology, 16 (2), 2008, 297–303.

18.

Noh

J. S.

Lee

G. H.

and Jung

, “Position control of a mobile inverted pendulum system using radial basis function network”,IEEE WCCI, 2008, 371–377.

19.

Noh

J. S.

Lee

G. H.

Choi

H. J.

and Jung

, “Robust control of a mobile inverted pendulum system using RBF neural network controller”, IEEE ROBIO, 2008, 1932–1937.

20.

Lee

H. J.

Choi

H. J.

Park

J. H.

Lee

J. H.

and Jung

, “Center of Gravity Based Control of a Humanoid Balancing Robot for Boxing Games: BalBot V”, ICCAS, 2009, 124–128.

21.

Song

D. H.

Lee

W. K.

and Jung

, “Kinematic analysis and implementation of a motion-following task for a humanoid slave robot controlled by an exoskeleton master robot”, International Journal of Control, Automation and Systems, 5 (6), 2007, 681–690.

22.

Lee

G. H.

Lee

S. J.

and Jung

, “Line tracking control using visual feedback of a mobile inverted pendulum: BalBOT IV”, IASTED Conference on Robotics and Applications, 2009, 188–193