Tuning a PD Controller Based on an SVR for the Control of a Biped Robot Subject to External Forces and Slope Variation

Abstract

Real-time balance control of an eight-link biped robot using a zero moment point (ZMP) dynamic model is difficult to achieve due to the processing time of the corresponding equations. To overcome this limitation an intelligent computing technique based on Support Vector Regression (SVR) is developed and presented in this paper. To implement a PD controller the SVR uses the ZMP error relative to a reference and its variation as inputs, and the output is the correction of the angle of the robot's torso, necessary for its sagittal balance. The SVR was trained based on simulation data generated using a PD controller. The initial values of the parameters of the PD controller were obtained by the second Ziegler-Nichols method. In order to evaluate the balance performance of the biped robot, three performance indexes are used.

The ZMP is calculated by reading four force sensors placed under each of the robot's feet. The gait implemented in this biped is similar to a human gait, which is acquired and adapted to the robot's size.

The main contribution of this paper is the fine-tuning of the ZMP controller based on the SVR. To implement and test this, the biped robot was subjected to external forces and slope variation. Some experiments are presented and the results show that the implemented gait combined with the correct tuning of the SVR controller is appropriate for use with this biped robot. The SVR controller runs at 0.2 ms, which is about 50 times faster than a corresponding first-order TSK neural-fuzzy network.

Keywords

Tuning PD SVR Biped Robot Balance ZMP

1. Introduction

A biped robot has a leg structure similar to human anatomy. To be able to maintain its stability in dynamic situations such a robotic system requires a good mechanical design, force sensors to acquire the Zero Moment Point (ZMP) and the design of appropriate real-time controllers. Many such biped humanoid robots have been developed, including ASIMO by Honda, WABIAN 2R by Waseda University, HUBO KHR-3 by KAIST and QRIO by Sony. Vukobratović et al. have developed a mathematical model for a biped robot and its method of control [1]. Many researchers [2 –4] have investigated the gait of biped robots based on human kinematic data; a particularly good study of the kinematics of a human body was done by Winter [5]. Because a biped robot is easily knocked down, to assure its dynamic stability Hirai et al. proposed a standard method for gait synthesis based on the ZMP [6]. Basically, this method consists of designing a desired ZMP trajectory, and afterwards, during the robot's motion, making on-line control corrections to the movement of the torso and pendulum to materialize the defined ZMP trajectory, based on the measurements of the force sensors on the feet.

For humanoid robotics, static walking is when the projection of the centre of mass (CoM) on the floor is always within the support polygon during the walking motion. The supporting polygon corresponds to the support foot in the single support phase, if flat contact with the ground is verified. In the double support phase the support polygon is the convex polygon inscribing the two parts of the feet that are touching the ground. In static walking the robot is always in static equilibrium, so it can stop its motion at any moment and does not fall down. Note that fast motion is not possible, since the dynamic couplings of the body parts could affect the static equilibrium. In stable dynamic walking the projection of the CoM on the floor is outside the supporting polygon during some phases of the gait. The ZMP, however, is always inside the support polygon. The equilibrium of the robot depends on the dynamics, and in general the motions performed are faster and smoother than with static walking [23].

Intelligent computing techniques have found wide application in the area of advanced control of biped robots, due to their strong learning and cognitive abilities and good tolerance to uncertainty and imprecision. To solve the biped robot's balance problem many researchers have been developing controllers using intelligent computing methods like fuzzy neural networks or neuro-fuzzy networks [12 –14] and SVR [7, 17]. A survey of these techniques was undertaken by Katić et al. [15]. The control of a biped robot using the ZMP with an eight-link model is more accurate than methods based on a two-link model with mass concentrations, which is normally used for real-time balance control. In the two-link model, the active joint can either be the ankle [8 –10] or the hip [11] to determine and apply the necessary torque for the robot's balance.

Sagittal balance control using an eight-link model is difficult to apply in real time due to the excessive computational effort. To overcome this problem a computational intelligence technique, the Support Vector Regression (SVR) technique, is used in this paper. The SVR is trained with the simulation data from an eight-link robot model and data generated by empirical rules based on the Ziegler-Nichols method [21]. As the ZMP control is nonlinear, an SVR is appropriate because it calculates the optimal hyper plane for the training data and is faster than a neural network. The SVR technique was initially developed by Vapnik [16]. Using the eight-link biped model together with one computational intelligence technique allows the real-time control of the biped robot with greater precision than using the biped robot's simplified two-link model.

In [26] the authors compared the SVR with a first-order Takagi-Sugeno-Kang (TSK) [25] neuro-fuzzy network controller using real experiments, and concluded that the SVR controller presents a slightly better (between 1% and 5%) stability than the neuro-fuzzy network. Also, the SVR controller runs at 0.2 ms, which is about 50 times faster.

The present work has the objective of improving the performance of the SVR controller. Three performance indexes are used to evaluate the performance of a biped robot's balance control method [7]. The main contribution of this paper is to use these performance indexes to fine-tune the initial proportional and derivative controller parameters obtained with the Ziegler-Nichols method, in order to achieve a better performance. This fine-tuning consists of correcting scale factors in the SVR inputs instead of changing the initial PD parameters in the simulator controller, with subsequent retraining of the SVR. The Ziegler and Nichols method [21] uses a set of empirical rules for tuning PID controllers based on experimental results of the system to be controlled.

The gait implemented in this biped robot is similar to a human gait, which was acquired and adapted to the robot's size [4, 22].

The experiments were performed with a biped robot, shown in Figure 1, that was designed and built at the Institute of Systems and Robotics, University of Coimbra, Portugal [7].

Figure 1.

Implemented robot

2. Training data for the SVR

The method used to obtain the equilibrium of the robot in the sagittal plane consists of correcting the angle of the hips (torso) using the SVR [18 –20] real-time output. Balance in the lateral plane is achieved by positioning the pendulum (θlateral) at its extreme lateral positions during the single phase. This way the lateral coordinate of the ZMP is neglected.

The SVR was trained with 239 uniformly distributed and normalized data points, and tested with another 68 data points [7], generated by simulation using a set of empirical rules proposed by Ziegler and Nichols [21].

The second Ziegler and Nichols method, the stability-limit method, sets the controller parameters based on an evaluation of the system at the limit of stability. Its first step is to determine experimentally the value of the critical proportional gain (Kc), defined as the smallest value of the controller gain that results in sustained oscillations when a pure proportional controller is used. The period of these oscillations is called the critical period of oscillation (T_c). The proportional parameter of the PD controller is K_p = 0.6·K_c and the derivative parameter (T_d) is calculated from T_c, using the relationship T_d= T_c/8.

The second Ziegler and Nichols method was applied for the biped robot system. In the experiment to determine K_c and T_c the robot was maintained with only one foot on the ground and the proportional controller gain was increased until the robot presented sustained oscillations, as shown in Figure 2.

Figure 2.

X_ZMP and θ_torso obtained at the limit of stability with the proportional controller active and K_c=10.3. The robot has only one foot on the ground.

The value of K_c obtained for the limit of stability was 10.3. Thus, K_p is 6.2, because K_p = 0.6K_c.

The critical frequency of oscillation (ω_c) is equal to 2.7 rad·s⁻¹, resulting in K_d equal to 1.8, because K_d = K_pT_d. Using this constant derivative, the training data and the testing data for the SVR were determined. The integral parameter was ignored to prevent oscillations of the torso.

The training data consisted initially of 34 pairs of points obtained by simulation [7] of the biped robot model with steps of four seconds (seven points for each of five step lengths, excluding the pair (EX_ZMP, Δθ_torso) = (0, 0)). For each of the previous 34 pairs, eight new pairs of points were generated with DX_ZMP $D X_{Z M P_{k}} = E X_{Z M P_{k}} - E X_{Z M P_{k - 1}}$ varying uniformly between–0.002 m and 0.002 m, which is the maximum expected range for DX_ZMP. This range was determined by multiplying the maximum velocity of the X_ZMP occurring in the experiment above (Figure 2), which is 0.043 m s⁻¹ for the sampling time (Δt), which is 0.046 s. The red lines in Figure 2 represent the edge of the foot. The value of Δθ_torso for each of these new points (Δθ_{torso Nk})was obtained by

Δ θ_{t o r s o}_{N_{k}} = Δ θ_{t o r s o_{k}} + K_{d} \cdot D X_{Z M P_{k}} / Δ t .

(1)

The first term of this equation is Δθ_{torso k}, obtained by simulation of steps taking four seconds [7].

The following values were obtained: 307 (34×9+1), 239 (34×7+1) of those used for training and 68 (34×2) for testing the SVR.

3. Real-Time Control Strategy

The control strategy is one of the most important issues in controlling a biped robot. Many control strategies are available and may be based on fuzzy systems, neural networks, classic control, support vector machines, and hybrid systems.

The main blocks of our biped robot control are presented in Figure 3. The control system block is implemented by an SVR controller.

Figure 3.

Balance control strategy of the biped robot

For real-time control, the actual value of the ZMP is needed. When the ZMP is within the stable region, the ZMP is equal to the centre of pressure (CoP) [24]. To determine the CoP, four force sensors are implanted under each foot of the robot. The CoP is calculated by

C o P = \frac{\sum_{i = 1}^{8} F_{i} \cdot {\bar{r}}_{i}}{\sum_{i = 1}^{8} F_{i}}

(2)

where F_i is the measured force in sensor i, and r̂_i is the position vector.

The force sensors' values are acquired by an analogue to digital converter (ADC) with 10-bit resolution and a maximum 30 Hz sampling rate. The force measurements are noisy because the force sensors are sensitive to vibrations during motion, so a second-order Butterworth low-pass filter is used to remove the high-frequency noise from the force sensor signals. A cut-off frequency of 3 Hz was set.

4. Experimental Results of Tuning

The choice of the proportional and derivative parameters of the controller was based on the Ziegler-Nichols method, but these parameters needed to be refined in order to optimize system performance. To refine the parameters the entries of the SVR (EX_ZMP and DX_ZMP) are adapted by the gain factors F_P and F_D, which indirectly influence the proportional and derivative terms, respectively.

The factors used in the experiments and the results of these experiments are presented in Tables 1 and 2. In the experiments the robot was walking (0.07 m) on a flat horizontal surface, using the trajectories of the human gait, dragging a mass of 1.5 kg (providing an effective pulling force about 5 N), as Figure 4 shows. Figure 5 shows the behaviour of the main variables of the biped robot during four steps. The values presented in this figure were normalized such that the unit values correspond to 25 degrees for θ_torso, 10 degrees for θ_ankle, 55 degrees for the pendulum lateral angle (θ_lateral) and 0.047 m for X_ZMP. In the figure it can be noticed that the θ_torso is deviated forward relative to the θ_{torso D} in order to keep the sagittal balance and X_ZMP near zero (X_ZMPref =0).

Figure 4.

Snapshots of one step walked on a horizontal flat surface pulling a mass with SVR control active

Figure 5.

X_ZMP, X_ZMPref, designed torso (θ_torsoD), torso and lateral angles when the robot walks on a horizontal flat surface pulling a mass with SVR control active

Table 1.

Performance indexes - derivative case, experiments with a mass

F_D	NX_RMS	MNSM	MSM
0.5	0.197	0.834	0.45
0.75	0.196	0.833	0.41
1	0.193	0.831	0.53
1.125	0.186	0.837	0.58
1.25	0.181	0.842	0.60
1.375	0.194	0.832	0.57
1.5	0.218	0.828	0.29

Table 2.

Performance indexes – proportional case, experiments with a mass

F_P	NX_RMS	MNSM	MSM
0.5	0.375	0.756	0.20
0.75	0.297	0.761	0.18
0.875	0.237	0.823	0.52
1	0.181	0.842	0.60
1.125	0.207	0.827	0.58
1.25	0.279	0.751	0.55
1.5	0.281	0.753	0.42

The time of the swing phase of the step is about two seconds, which represents a step time of about four seconds. The implemented step time is about five seconds (three seconds for the double phase), due to the need to perform the lateral control (the pendulum must move from 50 to −50 degrees or vice versa).

To determine which are the best parameters for the PD controller, and because the result plots are inconclusive, three performance indexes are proposed. The first is the normalized root mean square of X_ZMP - X_{ZMP_ref} (NX_RMS); the second is the mean of the normalized stability margin (MNSM) and the third is the minimum of stability margin (MSM). These indexes were calculated for four walking steps and are described by

N X_{R M S} = \frac{\frac{1}{k} \sum_{i = 1}^{k} \sqrt{\frac{1}{n} \sum_{j = 1}^{n} {(X_{Z M P} (i, j) - X_{Z M P_r e f} (j))}^{2}}}{X_{S}}

(3)

M N S M = \frac{\frac{1}{k} \sum_{i = 1}^{k} \frac{1}{n} \sum_{j = 1}^{n} (X_{S} - | X_{Z M P} (i, j) |)}{X_{S}}

(4)

M S M = \min_{\begin{array}{l} i = 1, …, k \\ j = 1, …, n \end{array}} (\frac{X_{S} - | X_{Z M P} (i, j) |}{X_{S}})

(5)

where k is the number of steps, n is the number of the force sensor samples and X_s is the X absolute coordinate of the force sensor locations, which corresponds to the maximum possible value of X_ZMP (in our robot, this is 0.047 m). The optimal value for NX_RMS is zero and for both MNSM and MSM it is one.

Tables 1 and 2 present the three performance indexes' values for the experiments. The best values are highlighted in bold. The results in Table 2 were obtained with F_D = 1.25.

The experiments show that the derivative controller parameter should be altered by the factor 1.25, and the proportional by 1. Since the proportional factor exhibits its best performance when F_P = 1, there is no need for more iterations to find another F_D.

Experiments with the robot without external disturbances were also performed to verify the correctness of the factors. Tables 3 and 4 give the results of these experiments, confirming the previous factors, although the MSM index indicates F_P=1.125 as the best.

Table 3.

Performance indexes – derivative case

F_D	NX_RMS	MNSM	MSM
1	0.203	0.829	0.45
1.125	0.179	0.849	0.60
1.25	0.178	0.851	0.62
1.375	0.211	0.821	0.58

Table 4.

Performance indexes – proportional case

F_P	NX_RMS	MNSM	MSM
0.875	0.204	0.825	0.55
1	0.178	0.851	0.62
1.125	0.199	0.832	0.68
1.25	0.241	0.823	0.52

Experiments with variation of the inclination slope were performed, using the initial and the improved SVR controller. In the following experiments the robot's right foot was placed in the air in an ascending (see Figure 8) and descending (see Figure 11) slope. The ascending slope was varied continuously during 10 s, from 0 to 10 degrees and again to 0 degrees using the initial (see Figure 6) and the improved (see Figure 7) SVR controller. The descending slope was varied continuously during 8.5 s, from 0 to −10 degrees and again to 0 degrees using the initial (Figure 9) and the improved (Figure 10) SVR controller.

Figure 6.

X_ZMP and θ_torso when the robot is standing with one leg in the air; the slope varies from 0 to 10 degrees and 10 to 0 degrees with the initial SVR controller

Figure 7.

X_ZMP and θ_torso when the robot is standing with one leg in the air and the slope varies from 0 to 10 degrees and 10 to 0 degrees, with the improved SVR controller

Figure 8.

Snapshots of the behaviour of the robot when it is standing with one foot in the air and the slope varies from 0 to 10 degrees and 10 to 0 degrees, with the improved SVR controller

Figure 9.

X_ZMP and θ_torso when the robot is standing with one leg in the air, and the slope varies from 0 to −10 degrees and −10 to 0 degrees, with the initial SVR controller

Figure 10.

X_ZMP and θ_torso when the robot is standing with one leg in the air and the slope varies from 0 to −10 degrees and −10 to 0 degrees, with the improved SVR controller

Figure 11.

Snapshots of the behaviour of the robot when it is standing with one foot in the air and the slope varies from 0 to −10 degrees and −10 to 0 degrees, with the improved SVR controller

Again, the values shown in figures 6, 7, 9 and 10 are normalized with the constants used previously. The inclination of the slope is normalized by dividing by 10. The value of the slope of the ramp was obtained using the images from a digital video camera.

In the slope experiments it can be seen that the initial SVR controller keeps the X_ZMP between −0.6 and 0.6. In the improved SVR controller the values of X_ZMP are lower (between −0.4 and 0.4), increasing the stability of the robot.

5. Conclusions

The real-time control of a biped robot using the dynamic model of the ZMP is difficult to achieve because of the time required to process the corresponding equations.

An SVR balance controller allows the real-time control of the robot using an eight-link biped model. The controller uses the real ZMP, acquired by force sensors placed under the robot's feet. The control method was tested and satisfactory results were obtained.

The biped robot did not fall in any of the experiments with the balance controller active, and it kept a good stability margin, thereby demonstrating that the SVR controller is a good solution for biped robot balance control.

Three performance indexes were used to fine-tune the PD parameters using gain factors. It was shown that the gain factors obtained improve the performance of the robot.

In future work, we intend to use other recent computational intelligence control methods, like the extreme learning machine, and compare the results obtained with the SVR and with other classic methods.

6. Acknowledgments

The authors would like to thank the Portuguese Fundação para a Ciência e a Tecnologia for financial support.

References

Vukobratović

Borovac

Surla

Stokic

, “Biped locomotion: Dynamics, Stability, Control and Application”. Berlin: Springer-Verlag 1990, pp. 50–60.

Nakamura

Mori

Nishii

, “Trajectory planning for a leg swing during human walking”, 2004 IEEE International Conference on Systems, Man and Cybernetics, pp. 784–790.

Yoo

J.-H.

Nixon

M. S.

Harris

C. J.

, “Extracting Human Gait Signatures by Body Segment Properties”, Fifth IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI.02), pp. 35–39.

Ferreira

J. P.

Crisóstomo

M. M.

Coimbra

A. P.

Carnide

Marto

, “a Human Gait Analyzer”, 2007 IEEE International Symposium on Intelligent Signal Processing-WISP'2007, Madrid, Spain, 3–5 October 2007, pp. 71–75.

Winter

D. A.

, “The Biomechanics and Motor Control of Human Movement”, 2nd Ed., John Wiley & Sons, 1990, pp.73–96.

Hirai

Hirose

Haikawa

Takenaka

, “The Development of Honda Humanoid Robot”, Proc. Int. Conf. Robotics and Automation, pp. 1321–1326, 1998.

Ferreira

J. P.

Crisóstomo

Manuel

Coimbra

A. P.

Ribeiro

, “SVR Controller for a Biped Robot with a Human-like Gait Subjected to External Sagittal Forces”, Biped Robots, InTech Publisher, ISBN: 978–953–307–216–6, 2011, pp. 77–98.

Park

I.W.

Kim

J. Y

Lee

J. H.

, “Online free walking trajectory generation for biped humanoid robot KHR-3(HUBO)”, Proceedings of the 2006 IEEE International Conference on Robotics and Automation, Orlando, Florida, May 2006, pp. 1231–1236.

Kim

J. Y.

Lee

J. H.

, “Experimental realization of dynamic walking for a human-riding biped robot, HUBO FX-1”, Advanced Robotics, vol. 21, no. 3–4, pp. 461–484, 2007.

10.

Prahlad

Dip

Hwee

C. M.

, “Disturbance rejection by online ZMP compensation”, Robotica, pp. 1–9, 2007.

11.

Low

K. H.

Liu

Goh

C. H.

, “Locomotive Control of a Wearable Lower Exoskeleton for Walking Enhancement”, Journal of Vibration and Control, 2006, pp. 1311–1336.

12.

Ferreira

J. P.

Amaral

T. G.

Pires

V. F.

Crisóstomo

M. M.

Coimbra

A. P.

, “A Neural-Fuzzy Walking Control of An Autonomous Biped Robot”, Proceedings of the 10th International Symposium on Robotics with Applications, IEEE CNF, Sevilha, 21–23 June 2004, pp. 253–258.

13.

Sim

Seo

Park

G. T.

, “Zero Moment Point Trajectory modeling of a Biped Walking Robot using an adaptative neuro-fuzzy system”, IEE Proc. Control Theory Appl., vol. 152, no. 4, pp. 411–426, July 2005.

14.

Behnke

, “Online trajectory Generation for Omnidirectional Biped Walking”, Proc. of the 2006 IEEE International Conference on Robotics and Automation, Orlando, Florida – May 2006, pp. 1597–1603.

15.

Katić

Vukobratović

, “Survey of Intelligent Control Algorithms For Humanoid Robots”, Proceedings of the 16th IFAC World Congress, Prague, Czech Republic, July 2005, ISBN: 978–0–08–045108–4, Elsevier Science, 27 June 2006, pp. 117–141.

16.

Vapnik

, “The Nature of Statistical Learning Theory”, New York: Springer, 1998.

17.

Ferreira

J. P.

Crisóstomo

M. M.

Coimbra

A. P.

Ribeiro

, “Simulation control of a biped robot with Support Vector Regression”, 2007 IEEE International Symposium on Intelligent Signal Processing-WISP'2007, Madrid, Spain, 3–5 October 2007, pp. 41–46.

18.

Mohamed

R. M.

Farag

A. A.

, “Classification of Multispectral Data Using Support Vector Machines Approach for Density Estimation”, IEEE Seventh International Conference on Intelligent Engineering Systems, INES03, Assiut, Egypt, March 2003, pp. 51–57.

19.

Vapnik

Golowich

Smola

, “;Support Vector Method for Multivariate Density Estimation, Advances in Neural Information Processing Systems”, vol. 12, pp. 659–665, April 1999.

20.

Chang

C.-C.

Lin

C.-J.

, “LIBSVM: a Library for Support Vector Machines”, January 2 2007.

21.

Ziegler

J. G.

Nichols

N. B.

, “Optimum settings for automatic controllers,” Trans. ASME, vol. 64, pp. 759–768, Nov. 1942.

22.

João Ferreira

Crisóstomo

Manuel

Coimbra

A. Paulo

, “Human Gait Acquisition and Characterization”, IEEE Transaction on Instrument and Measurement Vol. 58, Issue 9, pp. 2979–2988, September 2009.

23.

Vukobratović

Borovac

Potkonjak

, “Towards a unified understanding of basic notions and terms in humanoid biped robotics”, Robotica, vol. 25, pp. 87–101, 2006.

24.

Vukobratovic

Borovac

, “Zero-Moment Point – thirty five years of its life”, International Journal of Humanoid Robotics, vol.1, no.1, pp. 157–173, 2004.

25.

Takagi

Sugeno

, “Fuzzy identification of systems and its applications to modelling and control”, IEEE Trans. Syst. Man Cybern., vol. 15, pp. 116–132, 1985.

26.

Ferreira

J. P.

Crisóstomo

Coimbra

A. P.

, “SVR vs. Neural-Fuzzy Network controllers for the sagittal balance of a biped robot”, IEEE Transactions on Neural Networks, vol. 20, no. 12, pp. 1885–1897, December 2009.