Abstract
Autonomous landing is a challenging but important task for Unmanned Aerial Vehicles (UAV) to achieve high level of autonomy. The fundamental requirement for landing is the knowledge of the height above the ground, and a properly designed controller to govern the process. This paper presents our research results in the study of landing an autonomous helicopter. The above-the-ground height sensing is based on a 3D vision system. We have designed a simple plane-fitting method for estimating the height over the ground. The method enables vibration free measurement with the camera rigidly attached on the helicopter without using complicated gimbal or active vision mechanism. The estimated height is used by the landing control loop. Considering the ground effect during landing, we have proposed a two-stage landing procedure. Two controllers are designed for the two landing stages respectively. The sensing approach and control strategy has been verified in field flight test and has demonstrated satisfactory performance.
Introduction
Unmanned Aerial Vehicles (UAV) are ideal platforms for missions that are dangerous, expensive, or impossible for human to carry out. With the capabilities of hovering and Vertical Take-off and Landing (VTOL), autonomous unmanned small scale helicopters can be applied in such applications as surveillance, infrastructure inspection, mine detection, search-and-rescue and so on. The unique hovering and VTOL flying characteristics make such autonomous task scenario as “go-search-find-take-return” possible.
Recent study on autonomous helicopters ranges from system modeling and identification (Mettler, B., Tischler, M. B., & Kanade, T., 2002), controller design (Lai, G., Fregene, K. & Wang, D., 2000; Enns, R. & Si, J., 2003; Shin, J., Nonami, K., Fujiwara, D. & Hazawa, K., 2005) to sensor integration and hardware implementation. The hovering capability has been most studied. However, the VTOL capability is less addressed, which is another important factor towards fully autonomous mission. In order to land a helicopter autonomously, we need to know the relative height between helicopter and the ground. Also we need a well-performed controller to smooth the landing process.
In literature, different landing strategies have been reported. The sensing schemes for landing can be mainly summarized into two groups: (1), vision only, (2), combination of multiple sensors. The group one measures height and vertical velocity for landing directly by using vision only. While group two makes use of multiple sensors for the measurements, and typically the sensors may include GPS, sonar, laser range meter or vision system. The sensing scheme in the work (Shakernia, O., Ma, Y., Koo J. & Sastry, S.S., 1999) belongs to the first group. In the paper, a single camera is used to track features in a landing pad and estimate the ego-motion. The pose of the helicopter with respect the landing pad is derived and used for landing control. The performance of this approach is presented with vision-in-the-loop simulation. The work (Kanade, T., Amidi, O. & Ke, Q., 2004) shows another vision based motion estimation for Micro Air Vehicles (MAV) control. The paper (Saripalli, S., Montgomery, J.F. & Sukhatme, G. S., 2003) and (Nakamura, S., Kataoka, K. & Sugeno, M., 2001) present the study of landing control with multiple sensors, which falls into the group 2 sensing scheme. The first paper uses GPS and sonar sensor for height measurement, and a camera is used for searching landing pad. The vision system is not used directly for height measurement but instead for target recognition and localization. The second paper presents a similar approach where an active vision system is used for finding the landing pad. The height measurement is by using differential GPS.
Our sensing strategy belongs to the group one. The difference between our approach and above mentioned approach lies in the fact that:
we use binocular 3D vision system, which measure range without relying on any predefined landmark. the vibration free height measurement is derived from the range map by the proposed plane-fitting algorithm. vertical velocity is estimated by fusing the height measurement with the acceleration measurement from Inertial Measurement Unit (IMU).
With the sensing strategy, we proposed a two-stage landing control procedure to address the dynamics variation and performance requirement difference in high altitude and low altitude. The 3D vision based landing control approach has been verified in our field experimental study.
The paper is organized as follows. Section 2 describes the helicopter tested and system architecture. Section 3 shows the application of 3D vision to determine the height over a flat ground. Section 4 describes the two-stage controller design for landing. Section 5 presents our field experimental results, and Section 6 gives our conclusion and future work.
In the research we use a SF-40 model RC helicopter as our flying platform. It is powered by a two-stroke gasoline engine with a maximum payload of 8.5kg. A picture of the helicopter is shown in Fig.1 and the detailed specifications are shown in Table 1.
Helicopter platform: SF40 model helicopter
Specification of SF40 model helicopter
The system consists of a ground processing part and an onboard sensing part. The ground part is responsible for executing control law and sending control command to the helicopter. The ground part also provides user interface to operator. The onboard part is responsible for sensing the helicopter states necessary for control purpose. The sensors are interfaced via an onboard SBC (Single Board Computer). Sensor readings are forwarded to the ground computer for calculating the actuating command according to the control law. Then the command is sent to the helicopter to control the movement of servomotors. Fig. 2 shows an architectural view of the system.
System architecture
The onboard sensors we used are all commercial-off-the-shelf (COTS) products. The sensors equipped are a Bumblebee binocular 3D vision camera and an Attitude Heading Reference System (AHRS). An onboard Pentium III class SBC is used to run vision processing algorithm, and interface between sensors and host computer. The Bumblebee camera and the AHRS are used to measure the vertical position and velocity during the landing control. The camera is mounted on the helicopter with its lens facing directly down to the ground. In Table 2 and Table 3, we show the pictures and primary parameters of Bumblebee camera and AHRS respectively.
Vision system specification
AHRS specification
Vision system has become popular in the field of robot control for its passive nature and the ability to percept the environments. Application of vision system for such purposes generally requires multidimensional signal processing technique to extract useful information from the redundant vision data. In addition, a small-scale helicopter manifests unstable dynamics and requires constant attention to keep it stable. The control interaction together with the engine vibration make it necessary to handle the vibration induced noise when applying vision system in a small scale helicopter.
In our application, the 3D vision camera is fixed on the helicopter with lens looking downward. The camera vibrates with the helicopter. The vibration induced measurement error will get bigger as the helicopter flies higher. We can minimize the vibration effect by using a gimbal mechanism to keep the camera from vibrating with the helicopter or by compensating the measurement with the attitude information from an attitude sensor. But either way will complicate the system design and burden the limited payload/space constraint. We solve the problem in a soft way by developing algorithm to get vibration free height estimation without adding any hardware support.
The height estimation is to decide the height of the helicopter above the ground. The height measurement comes from the Bumblebee 3D camera. The vision system performs Sum of Absolute Differences (SAD) correlation between the left and right images to derive the depth information and outputs the depth image as result. The size of the depth image is configurable. In our case, we configure the dimension of the depth image to be 120 pixels (height) by 160 pixels (width). A typical depth image generated by the vision system is shown in Fig.3.
Depth image from Bumblebee camera
At each sample instance, we receive a depth image from the Bumblebee camera which contains all measurements in the area that the camera can see. This kind of high dimensional data cannot be directly used by the controller. In this study, we propose to use plane-fitting method to map the high dimensional data into one dimensional measurement.
In developing the algorithm, we assume that the ground surface is relative flat. Thus the plane approximation will not cause significant error. The method is stated as follows. Firstly we apply Least Mean Square (LMS) technique to fit the depth image data with a plane. Then we compute the perpendicular distance from the center of gravity of the helicopter to the plane. The value is used as the height estimation. We perform the calculation in the camera's frame. Since the Euclidean distance is invariant to the rotation. The estimation is immune to the attitude change of helicopter.
The definition of the camera's body coordinate is shown in Fig. 4. The z axis is towards the ground. The y axis is perpendicular to the z axis and towards the front. The x axis points inside the paper which forms the right hand rule body coordinate frame.
The camera mounting configuration and the camera body frame definition
In the camera coordinate system we can express the plane by Eq.(1),
where n = [a b c d] T is the plane parameter vector, p = [x y z 1] T is the homogeneous coordinate of a point on the plane.
The plane parameter n can be solved by minimizing Eq.(2).
where
The solution of n is the associated eigenvector of PT P with the least eigenvalue. Once the plane equation (1) is determined, we can calculate the perpendicular distance from the C.G. (Center of Gravity) point of the helicopter to the plane.
where (xc yc zc) is the coordinate of helicopter C.G. point.
The accuracy of the measurement is evaluated by comparing with a Novatel RTK-2 (Real Time Kinematic) GPS which has a resolution of 1cm. The comparison result is shown in Fig. 5.
Height measurement comparison : Vision and GPS
The data is gathered in real flight with GPS collocated with 3D vision system. Excepting the time from about 88 second, the measurement accuracy of vision system is comparable to that of the GPS. The mismatch is because GPS lost differential compensation and degraded to standalone working mode. During the time, we can see vision provide the right height measurement.
The vertical velocity is estimated by using a steady state Kalman filter. The filter design is based on the kinematic relationship between acceleration and position, which is shown in Eq. (4).
where z(t) represents the height and az(t) is the vertical acceleration. The acceleration can be measured by AHRS sensor.
Landing is to descend from some arbitrary height till contacting the ground. This process can be divided into two phases: descending phase and landing phase. We define the “descending phase” as the process before the helicopter's altitude gets lower than a specified height. The “landing phase” refers to the process that the helicopter descends from the specified height till touching the ground. The definition is illustrated in Fig. 6. In this study, the specified height is chosen to be 2 meters. At this height the helicopter is just free from the ground effect.
Definition of two landing phases
The reason for identifying the landing process as two phases lies in the fact that the working conditions and control requirements of the two phases are different. In the descending phase, the helicopter flies at a relative higher altitude where the helicopter is away from the ground effect but may experience stronger wind. For the controller design, we expect that the controlled system can respond fast and can tolerate wind disturbance robustly. In the landing phase, the helicopter flies near the ground where the ground effect is significant. The dynamics in this phase may differ from that of the descending phase. Undershoot, overshoot or rebounding are undesirable. For safe landing we also expect some management support in the touch down phase, for example, we want the engine to be stopped or be kept in idle state as soon as the helicopter touches the ground. This kind of function is not required in the descending phase. The two phase treatment gives us flexibility in designing controller to satisfy the performance requirement in different phase without compromise. We have designed two controllers who are responsible for the two phases respectively.
Our landing control is planned in a 3 layered framework as shown in Fig.7. The top layer is “Landing planner”. The middle layer is “Landing coordinator”. The low layer is the two stage controller which controls the landing dynamics directly. The landing planner makes landing decision and instructs the landing coordinator to execute landing task. Currently this function layer is not implemented yet. We are working to enable the planner functionality by using vision data to find a safe landing site. The landing coordinator coordinates the two low layer landing controllers by sending reference command and transferring control authority from stage-1 controller to stage-2 controller at proper time. Current implementation of the coordinator is based on the response time of the stage-1 controller and the difference between actual system response and the commanded reference. To be specifically, if the time passed since landing start is longer than the response time of stage-1 controller, and the error between the actual states and commanded target have been less than some threshold for a specified time, the control authority is transferred to stage-2 controller. The two stage landing controllers directly handle the landing dynamics. The dynamic model and controller design are described in the next two subsections.
Three layer control architecture
When ignoring the cross-couplings, the vertical motion of helicopter can be simplified as a rigid body with gravitational force and thrust acting on it. The thrust T generated by main rotor can be expressed by Eq. (5).
where the symbol meaning is listed in Table 4.
Parameter definitions of Eq. (5)
In experiments, the rotor speed Ω is kept as constant by a governor. The thrust generated by the main rotor is thus proportional to the sum of collective pitch angle and inflow angle. The inflow angle is not controllable. By ignoring this term, we arrive a simple linear relationship between thrust and the collective pitch angle. The effect of the ignored term φ t is treated as disturbance and is to be compensated by the controller.
Linearizing around the hovering state and considering the transportation delay, we get the transfer function from collective pitch to vertical displacement. (Hazawa, K., Shin, J., Fujiwara, D., Igarashi, K., Fernando, D. & Nonami, K., 2003)
where K is a constant which is identified from experimental data, and Td is the transportation delay.
The controllers are designed based on the linearized model in Eq. (6). However, different K is used to design the stage-1 and stage-2 controllers. We apply Linear Quadratic optimal control theory for the design. Two LQI (Linear Quadratic with Integral) controllers are designed for the landing phase 1 and phase 2 respectively. The integral action is for rejecting constant disturbance and achieving zero steady tracking error.
Linear Quadratic control offers a systematic procedure to find the feedback gain by minimizing the integral of weighted plant states and plant input.
where Q>0, R≥0 are weighting matrix, x(t) is plant state and u(t) is plant input.
A key step is to select the state weighting matrix Q and controller output weighting matrix R. The performance specification is encoded in the weighting matrix.
In our design, we initially choose the weighting matrix based on the maximum allowed variation range of states and controller output. Then the controller is refined through experiments.
The experimental results of the landing control are shown in Fig. 8 to Fig. 10. Fig. 8 shows the height trajectory during landing. Fig. 10 shows vertical velocity and Fig. 10 shows the controller output.
Height trajectory during landing
Vertical velocity during landing
Controller output during landing
From Fig. 8 and Fig. 9 we can see the smooth landing process. The stage-1 controller is commanded by the landing coordinator to descend till 2 meters high above the ground at time 0. At the time about 66 second, the control authority is transferred to the stage-2 controller. At the time of 76 second, the helicopter touched ground. Around that time, we can see the vision measurement is quite noisy. This is because of the limit nearest measurable range of the 3D vision system. In our case, the 3D vision system cannot measure distance nearer than 30cm. The noisy height data cannot be used for control. Instead we integrate the vertical acceleration twice for height measurement when the vision system cannot measure.
Fig. 10 shows the controller output which is the pulse command sent to the servomotor responsible for changing the collective pitch. The ramp like control data starting from 76 second is to decrease the collective pitch and makes the helicopter away from neutral balance.
In this study, we present a 3D vision based approach for landing control of unmanned helicopter. We have designed a plane fitting method for height estimation. The method is insensitive to the attitude change of the helicopter. For a smooth and stable landing, we proposed a two stage landing strategy, which constitutes the low control layer in our 3 layered landing control framework. The two stage landing separates the landing process into descending phase and landing phase. Two controllers are deployed to address the difference requirement in the landing phases. The effectiveness of the proposed 3D vision based over-ground height estimation method and the two stage landing strategy have been verified in field flight test. We have successfully landing the helicopter in experiment.
Our future work will focus on enabling the helicopter to land in an unknown environment autonomously. We will develop safe area detection algorithm to find safe landing site. This self landing site locating capability will become the core of the landing planner in the 3 layered landing control framework.