Human-vehicle classification scheme using doppler spectrum distribution based on 2D range-doppler FMCW radar

Abstract

In this paper, we proposed a human-vehicle classification scheme using a Doppler spectrum distribution based on 2D Range-Doppler FMCW (Frequency Modulated Continuous Wave). Typically, because humans have non-rigid motion, multiple reflection points can appear on the Doppler spectrum. However, in the actual field, the Doppler spectrum distribution of a walking human is highly variable over time. Thus method using only this characteristic of the extended Doppler spectrum is limited with regard to human-vehicle classification. In order to improve the target classification performance, we designed two feature. The first is the Doppler spectrum extension features, which is expressed as the number of Doppler reflection points with magnitudes exceeding reference threshold. Next, we defined the Doppler spectrum variance feature, which is extracted as the difference the reflection points between two successive frames. We can determine how the Doppler spectrum expands with the first feature, and how the Doppler spectra change based on the second feature. To verify the proposed target classification scheme, we measured real data using a 24 GHz FMCW transceiver on an actual road with various scenarios of walking humans and moving vehicles. From an analysis of the results, we confirmed that the thresholds effectively classify humans and vehicles based on the two proposed features. Finally, we verified that the results of the proposed classification scheme using the two features were much better than those using the first feature alone.

Keywords

Automotive radar feature extraction pedestrian classification radar recognition FMCW radar

1 Introduction

Currently, commercial automotive radar supports the target detection to estimate the range, angle position, and radial velocity. Because radar sensors are very robust against external environment conditions compared to a camera device, radar sensros are becoming very important for ADAS (Advanced Driver Assistance Systems) [1 –3].

According to a report by a Daimler AG vendor [1], next generation automotive radar systems will provide the following functions: dense point-cloud generation, radar grids, co-representation of static and dynamic environments, ego-motion estimation, radar-based localization, motion prediction, object classification, and sensor fusion.

Among these requirements, target classification is the most significant issue for pedestrian safety [1].

Typically, the micro-Doppler signature is a popular means of distinguishing pedestrians and vehicles [4, 5]. That is, additional patterns exist in the Doppler frequency spectrum, appearing as side-bands around the Doppler frequency of a bulk motion such as a walking human [6]. However, because the micro-Doppler based solution has a high computational load and real time target classification may not be possible, this method is not appropriate for automotive radar applications.

Another solution for real-time problem is to use different features of the range and velocity profiles [7]. That is, as geometric and kinematic properties of a human, the measured range profile may be point-shaped and the Doppler profile may has multiple reflection points. In contrast, because a moving vehicle is an object with several range reflection points but with a single radial velocity, an extended range profile and a point-shaped Doppler spectrum occur. When using this method, because only one frame range and the Doppler profile are used, the processing complexity is very low.

However, in order for the method to be very effective, the range resolution should be very high so as to distinguish between the width of a human and that of a vehicle. Moreover, in the field, the Doppler spectrum distribution of an echo signal reflected from a walking human is highly variable over the measurement time [8]. Thus, the method is limited in terms of how much it can increase the target recognition performance.

In previous work [8], we presented this problem simply. In addition, we also reported that other features were needed when using the Doppler spectrum.

Thus, in this paper, we proposed the new human-vehicle classification scheme using the distribution characteristic of Doppler spectrum.

First, we assume that the Doppler spectrum received from a human will mostly be wide. At times, the Doppler spectrum of the current frame is not wide enough but there will be much variation compared to that of the previous frame.

Second, for a vehicle, the shape of the Doppler spectrum will scarcely change through all frames.

These two assumptions from the key idea of this paper. Thus, in this paper, when monitoring the Doppler spectra distributions of the current and last frames, we extract two features: the number of Doppler reflection points in the current frame and the difference in the Doppler reflection points between two successive frames. Based on information pertaining to these two features, we undertake human-vehicle classification.

Because the proposed method uses only the Doppler spectra of two successive frames, the computation complexity is very low compared to that of the typical micro-Doppler based method. Moreover, because the range-profile is not used and only two Doppler spectra are used, we can solve the problems associated with the typical method using the range-velocity profile.

In Section 2, we propose the human-vehicle classification scheme based on Doppler spectrum distribution. In Section 3, we present the measurement results tested using a 24 GHz FMCW radar transceiver and a real-time data logging system. The conclusions from our study are presented in Section 4.

2 Proposed Pedestrian Classification Scheme

For automotive radar systems, FMCW (Frequency Modulated Continuous Wave) radar is typically used because the signal processing complexity can be dramatically reduced compared to that of pulse radar. Specifically, fast-ramp based 2D range-Doppler FMCW radar is very effective when used measure the range and velocity of a target simultaneously [9 –11].

The basic concept of fast-ramp based FMCW radar is illustrated using the example of a single moving human, as shown in Fig. 1.

Fig.1

Basic concept of fast-ramp based 2D range-Doppler FMCW radar. Here, T is the modulation period, B is bandwidth, and L is the number of ramps.

A ramp train with a saw-tooth modulation shape is transmitted and the beat signal is received at every ramp. When using this method, because the transmission time is considerably shorter than the movement time of the target, it can be assumed that the range and radial velocity remain unchangeable for all ramps.

Generally, we can express the received beat-signal reflected from a single target as S (t) = A · e^j2πf_rt · e^j2πf_DlT, where A is the signal amplitude, f_r is the range frequency, f_D is the Doppler frequency, and l = 1∼L is the number of ramps. As shown in S(t) equation, Thus, we can extract one range frequency and one Doppler frequency through 2D FFT processing.

However, because a moving human has non-rigid body motion, the received signal has multiple reflection points echoed from various components of the body. Because all components can be detected in one range-bin, the received signal S(t) reflected from a moving human can be expressed as Equation (1). Here, n_c = 1∼N_c is the number of components and f_{D,n_c} is the corresponding Doppler freqeuency. Typically, among the components of the body, echoes of the torso, leg, and arm mainly appear on the Doppler spectrum. $S (t) = e^{j 2 π f_{r} t} \cdot \sum_{n_{c} = 1}^{I} (A_{i} \cdot e^{j 2 π (l - 1) {Tf}_{D, n_{c}}})$ (1)

Based on the Doppler spectrum characteristics of a moving human, we propose total signal processing scheme consisting of a detection part, a tracking part, and a classification part, as presented in Fig. 2.

Fig.2

Proposed radar signal processing scheme.

The detection part is based on a typical structure as follow:

We apply beat signal S(t) to range-processing and Doppler-processing including windowing and the FFT (Fast Fourier Transform). Thus, a 2D range-Doppler map {X (n, m) , n = 1∼ N, m = 1 ∼ M } is generated. Here, N is range-bin size and M is Doppler-bin size.

With regard to human detection, a human with weak signal power and a low Doppler frequency may be masked by strong clutter. In order to overcome this problem, we insert a clutter suppression technique between the range-processing and Doppler-processing steps, as proposed in previous work [11].

Finally, thresholding is carried out to determine whether or not each cell of the 2D map X(n,m) is a target. In this paper, for adaptive thresholding, a conventional CA-CFAR (Cell Averaging Constant False Alarm Rate) detector is with the Doppler dimension [12].

Finally, in the detection part, we can extract Y (k) ={ r_k, v_k }, where r_k and v_k are the range and velocity of the detected target and k = 1∼K is the number of detected targets.

Next, we also employ a tracking technique based on previous work [13]. In order to monitor the Doppler spectra of the same target in all frames, we identify the same track from the detections. This step is very important to support the proposed classification concept. However, in this paper, because a single moving target is initially considered, it is implemented very simply without a tracking filter. The tracking part outputs in Fig. 2 are briefly described below.

$Y (k^{'}) = {r_{k}^{'}, v_{k}^{'}}$ denotes the track estimations and n_k is the range-bin index of the kth track.

Using n_k, we can select the Doppler spectrum Z_k ={ X (n_k, 1) , …, X (n_k, M) } over range r_{n
_k} from 2D range-Doppler X(n,m). Here, k = 1∼K is used as the track ID to identify the same target.

We also present the proposed human-vehicle classification scheme using the Doppler spectra Z_k, as shown in Fig. 3. In this paper, for a simple description, we consider the Doppler spectrum (Z₁) of a single target.

Fig.3

Proposed human-vehicle classification scheme.

For the proposed classification scheme, we normalize Z_k with the strongest magnitude for every frame, as the same thresholds are used regardless of the received power.

Based on the normalized Doppler spectrum, the proposed classification scheme consists of three functions: the buffer, the feature extraction, and the classifier. First, the Doppler spectrum Z₁ of the current frame is saved in one buffer and the spectrum Z’ ₁ of the last frame is also stored in another buffer. In order to hold the Doppler spectrum of the last frame for every frame, we insert a frame delay element between the two buffers.

Next, for the human-vehicle classification, we define two features extracted from Doppler spectrum distribution, as follows:

① Doppler spectrum extension feature

From the normalized Doppler spectrum Z₁ in the current frame, we determine the reflection points echoed from the an actual target in order to express the Doppler spectrum extension feature.

To do this, we define N_ext as the number of reflection points. N_ext is calculated while counting the reflection points to meet the condition |Z₁| > T_mag, where T_mag is the magnitude threshold.

In an earlier study [14], RCS (Radar Cross Section) difference between the torso and the limbs (legs and arms) of the body was determined to be approximately 10 dB. Thus, based on these results, we determine T_mag through an actual measurement in Section 3.

② Doppler spectrum variation feature

First, from the Doppler spectrum of the last frame, we count the number of reflection points (N’ _ext ) to meet $| Z_{1}^{'} | > T_{mag}$ .

Next, by subtracting N_ext from N’ _ext , we can determine the second feature parameter N_var. That is, N_var refers to the Doppler spectrum variation between two successive frames.

If the target is a vehicle, N_var will reach convergence with a low value. On the other hand, in the case of a human, the value will be much larger.

N_var is very important feature for the proposed target classification in this paper.

Finally, the proposed classifier determines whether the detected target is a human or a vehicle using two features N_ext and N_var.

In this paper, we define the extension threshold as T_diff and the variation threshold as T_sct.

If the number of target Doppler reflection points (N_ext) exceeds extension threshold (T_sct), this target can be determined as a human because this Doppler spectrum is sufficiently extended.

Moreover, if the condition of |N_var|> T_sct is met, we can regard this target as a human. This step involves findng whether the target Doppler spectra are variable.

To summarize the proposed classification scheme, we express the decision procedure using as Eq. (2). Here, two thresholds T_diff and T_sct will be determined through actual measurements.

$\begin{matrix} human, if N_{ext} > T_{sct} or | N_{var} | > T_{dff} \\ vehicle, if N_{ext} \leq T_{sct} and | N_{var} | \leq T_{dff} \end{matrix}$ (2)

3 Measurement results

In order to verify the effectiveness of the designed Doppler spectrum distribution based human-vehicle classification scheme, we assembled the measurement set-up shown in Fig. 4. The measurement set-up consists of a 24 GHz transceiver module with antennas and a real-time data logging board developed at the DGIST ART Lab [15]. The beat signals are captured at every modulation period and transferred to a PC through Ethernet. All classification algorithms were implemented in the PC using Matlab.

Fig.4

Measurement set-up using a 24 GHz FMCW transceiver and a real time data logging system.

In this paper, we designed the waveform shown in Table 1. Here, the wave transmitting time is 102.4 ms and the time for one frame is 200 ms due to delay time for data saving process.

Table 1

Parameters of the radar system used in this paper

Specification	Symbol	Value
Center frequency (GHz)	fs	24
Bandwidth (MHz)	B	1,000
Number of antenna	–	1
Modulation period (us)	T	400
Number of ramps	L	256
Range-FFT point	N	2,048
Doppler-FFT point	M	256
One frame time (ms)	–	200

In this paper, we considered the eight scenarios shown in Fig. 5. Details are given in Table 2. Moreover, Fig. 6 shows photos of the first and the seventh scenarios for a walking human and a moving vehicle.

Fig.5

Configuration of the eight measurement scenarios in an outdoor environment.

Fig.6

Photos of the first and seventh scenarios.

Table 2

Measurement scenario description

Scenario	Description
1	The human is moving along the middle line of the radar sensor within approximately 10 m.
2	The human is moving 30 degree away from the center line within approximately 10 m.
3 and 4	The human is crossing the road diagonally from 10 m or 5 m away from the radar system.
5 and 6	The human is laterally moving 5 m or 10 m away from the radar system.
7	The vehicle is moving along the middle line of the radar sensor within approximately 20 m
8	The vehicle is laterally moving 15 m away from the radar system.

In this paper, the total number of detections for actual humans is 1,082, consisting of 163, 166, 187, 182, 169 and 215 detections in scenarios #1 ∼ #6. The number of detections of an actual vehicle for scenarios #7 and #8 are 462 in total (= 255 + 207).

Figure 7 presents the detected results (Y₀ shown in Fig. 2) of the human or the vehicle in scenarios #1 and #7 over the frame time (sec). It is apparent here, that both targets have point-shaped ranges (m).

Fig.7

Detected range and velocity over the frame time for the first scenario (a) and for the seventh scenario (b).

However, while the detected velocity (m/s) of the vehicle is distributed with a smooth curve, as shown in Fig. 7(b), the detections of the human fluctuate as presented Fig. 7(a).

Figure 8 shows the Doppler spectrograms, which are a collection of Doppler spectrums (Z_o referred in Fig. 2) extracted from the same target over frame time. Here, the color denotes the normalized magnitude: red is strongest and blue is weakest.

Fig.8

Doppler spectrograms of the detected targets: (a) scenario #1, (b) scenario #6, (c) scenario #7, and (d) scenario #8.

Figure 8(a) and 8 (b) present spectrograms of a walking human for scenarios #1 and #6, and Figure 8(c) and (d) shows the results for scenarios #7 and #8.

Compared to the spectrogram of the moving vehicle, we can see that the walking human has micro-Doppler characteristics despite the fact that the walking direction of human is lateral with respect to the radar sensor due to the non-rigid motion

In order to analyze the Doppler spectrum in detail, we select the Doppler spectra over three successive frames, as shown in Figs. 9 and 10.

Fig.9

Doppler spectra in three consecutive frames for the detected human (scenario #1): (a) the non-normalized linear scale and (b) the normalized dB scale.

Fig.10

Doppler spectra in three consecutive frames for the detected vehicle (scenario #7): (a) the non-normalized linear scale linear scale and (b) the normalized dB scale.

In Fig. 9(a), the non-normalized Doppler spectrum of the walking human has various Doppler reflection points. However, we also note that the echo power and spectrum extension change over time.

On the other hand, in Fig. 10(a), we find that the shapes of the Doppler spectra of the vehicle are similar regardless of time.

Next, Fig. 9(b) shows three Doppler spectra of the walking human in one graphic on the dB scale. Here, we find the differences between the strongest returns and the smallest echo within the extended Doppler spectra to approximately 10 dB.

In contrary, for the vehicle, compared to dominant reflection point, nearly all points were distributed below 15 dB.

Therefore, in this paper, we determine the magnitude threshold T_mag as – 10 dB to count the reflection points of an actual target from the normalized Doppler spectrum Z₀.

Figure 11 shows the normalized histogram of the proposed Doppler extension feature (N_ext). That is, when T_mag = – 10 dB, we calculated the statistics of the normalized N_ext values for each scenario.

Fig.11

Normalized histogram about the number of Doppler spectra reflection points for 8 scenarios. Here, T_mag is 0.1.

In the results, the corresponding plots of the human (scenarios #1 ∼ #6) and those of the vehicle (scenarios # 7 and #8) have completely different shapes.

In the case of the vehicle, the N_ext values were mostly placed within about 15, but there was an opposite distribution in the case of the human. Thus, in this paper, we determined the extension threshold T_diff to be 15.

However, because a few N_ext values for the human classification step are less than 15, the method using only the spectrum extension feature can lead to classification decision errors.

Figure 12 presents the two-dimensional distributions of the Doppler extension feature (N_ext) and the Doppler variation feature (N_var). Here, the blue circle plots are the results of the vehicle and the black dotted plots indicate those of the human.

Fig.12

Two-dimension distributions of two features for humans and vehicles when T_mag is 0.1.

We find that the circle plots are placed within areas to meet the condition of N_ext ≤15 and N_var ≤10. Moreover, although the Doppler spectrum extension is low, in this case N_ext ≤15 for the human, we find that Doppler spectrum variations exist between two successive frames. Thus, in this paper, the determined variation threshold T_diff is 10.

To summarize results analyzed so far, we designed parameters of the proposed classification scheme such as Table 3.

Table 3

Designed decision parameters of the proposed classification scheme

Specification	Symbol	Value
Magnitude threshold	T_mag	– 10 dB
Extension threshold	T_sct	15
Variation threshold	T_dff	10

Using the designed classification parameters, we calculate the classification decision rate given in Tables 4 and 5. In these matrixes, the horizontal axis indicates the actual type of target, and the vertical axis indicates the determined class of the target.

Table 4

Classification decision rate when using only the spectrum extension feature when T_mag is 0.1

Actual class\Determined class	Human	Vehicle
Human	90.48%	9.52%
Vehicle	6.71%	93.29%

First, Table 4 shows the results when using only the spectrum extension feature (N_ext) together with the corresponding threshold (T_sct).

In these results, we find that the average classification decision rates of the human and the vehicle are 90.48% and 93.29%, respectively. In addition, the decision error rates of the human and the vehicle are correspondingly estimated to be 9.52% and 6.71%.

Next, in Table 5, human classification is conducted effectively together with the spectrum variation feature (N_var) and the corresponding threshold (N_dff).

Table 5

Classification decision rate of the proposed method based on both the spectrum extension feature and the spectrum variation feature when T_mag = 0.1

Actual class\Determined class	Human	Vehicle
Human	97.60%	2.40%
Vehicle	7.36%	92.64%

In the results, it was found that the average classification decision rates and error rates of the human and vehicle are 97.60% and 92.64%, and 2.40% and 7.36%, respectively.

Despite the fact that the vehicle decision error rate is increased by 0.65%, the classification probability of the human is increased by 7.12%.

In these results of vehicle classification, we find that the slight rise in the decision error rate occurs. It means that a Doppler spectrum extension or some variation are occasionally found even when the target is a vehicle. We surmise that the error is due to ghost target detection caused by clutter. Thus, in order to compensate for these errors, we will apply a full tracking algorithm in the future.

4 Conclusions

Given that a walking human has non-rigid motion, various Doppler components are reflected from the human body. Based on this characteristic, as a typical classification method, there are micro-Doppler signature techniques and a range-/Doppler- profile shape based solution. However, because the former has high computational complexity, it is not appropriate for automotive radar applications. Moreover, as the Doppler spectrum distribution of an echo signal reflected from a walking human is highly variable over the measurement time, the second typical method is also limited in how much it can increase the target classification performance.

Thus, in this paper, we proposed a human-vehicle classification scheme based on the Doppler spectrum distribution using 2D range-Doppler FMCW radar. First, to measure the extent of Doppler spectrum expansion, we counted the Doppler reflection points with power exceeding reference threshold. Second, in order to measure the degree of change of the Doppler spectra, we calculate the difference in Doppler reflection points between the successive two frames.

To verify the performance of the proposed algorithm, eight scenarios were considered, including a walking human and a moving vehicle. In these cases, we used a real-time data logging system and a 24 GHz FMCW radar transceiver developed by DGIST. That is, using raw radar data obtained in the real field, we verified the designed human-vehicle classification algorithm. In an analysis of the measured data, we also determined two thresholds which effectively classified humans and vehicles from the two proposed features.

The results of the proposed classification scheme with the two features show that the class decision rates of human and vehicle are 97.60% and 92.64%, respectively. Compared to the classification method using only the first feature, the class decision rate of human is increased by 7.12%.

In the future, we will apply the proposed algorithm to various target types, such as a bicycle or a cart. We will also consider various environments, such as a multi-target situation. Moreover, in order to enhance the detection performance, we will fully implement the tracking algorithm while taking into account track maintenance and the track filter. Finally, in order to use the angle position of the target, we will also employ transceiver with multiple receive antennas previously developed by DGIST [16].

Footnotes

Acknowledgments

This research was supported by the DGIST R&D Program funded by the Ministry of Science and ICT, Korea. (18-IT-01 an 18-FA-07).

References

Dickmann , Klappstein

, Hahn

, Appenrodt

, Bloecher

H.-L.

, Werber

and Sailer

, Automotive radar the key technology for autonomous driving: From detection and ranging to environmental understanding, 2016 IEEE Radar Conference, 2016.

Hyun

, Jin

Y.-S.

and Lee

J.-H.

, Design and development of automotive blind spot detection radar system based on ROI preprocessing, IJAT18(1) (2017), 165–177.

Geronimo

, Lopez

A.M.

, Sappa

A.D.

and Graf

, Survey of pedestrian detection for advanced driver assistance systems, IEEE Transactions on Pattern Analysis and Machine Intelligence32(7) (2010), 1239–1258.

Rytel-Andrianik

, Samczynski

, Gromek

, ., Micro-range, micro-Doppler joint analysis of pedestrian radar echo, IEEE Signal Processing Symp. (SPSympo), Debe, Poland, 2015, pp. 1–4.

Villeval

, Bilik

, Gurbuz

S.Z.

, Application of a 24GHz FMCW automotive radar for urban target classification, IEEE Radar Conf, Cincinnati, USA, 2014, pp. 1237–1240.

Kim

and Ling

, Human activity classification based on micro-doppler signatures using a support vector machine, IEEE Transactions on Geoscience and Remote Sensing47(5) (2009), 1328–1337.

Heuel

and Rohling

, Pedestrian Recognition in Automotive Radar Sensor, IEEE IRS (2013), 2013.

Hyun

and Jin

Y.-S.

, Feature extraction scheme for moving human detection and classification of automotive surveillance radar, International Conference on Green and Human Information Technology2018 (2018), 168–172.

Hyun

, Kim

S.-D.

, Choi

J.-H.

, Yeom

D.-J.

and Lee

J.-H.

, Parallel and pipelined hardware implementation of radar signal processing for an FMCW multi-channel radar, Elecktronika Ir Elecktrotechnika21(2) (2015), 65–71.

10.

Hyun

and Lee

J.-H.

, Waveform design with dual ramp-sequence for high-resolution range-velocity FMCW radar, Elecktronika Ir Elecktrotechnika22(4) (2016), 46–51.

11.

Hyun

, Jin

Y.-S.

and Lee

J.-H.

, A pedestrian detection scheme using a coherent phase difference method based on 2D range-doppler FMCW radar, MDPI Sensors 201616(124) (2016).

12.

Winkler

, Range Doppler detection for automotive FMCW Radar, European Microwave Conference, 2007, pp. 166–169.

13.

Hyun

and Lee

J.-H.

, Multi-target tracking scheme using a track management table for automotive radar systems, IEEE 17^th International Radar Symposium, 2016.

14.

Vahidpour

and Sarabandi

, Millimeter-wave doppler spectrum and polarimetric response of walk bodies, IEEE Transactions on Geoscience and Remote Sensing50(7) (2012), 2866–2879.

15.

Hyun

, Jin

Y.-S.

and Lee

J.-H.

, Moving and stationary target detection scheme using coherent integration and subtraction for automotive FMCW radar systems, IEEE International Radar Conference, 2017.

16.

Hyun

, Jin

Y.-S.

and Lee

J.-H.

, Design and implementation of 24 GHz multichannel FMCW surveillance radar with a software-reconfigurable baseband, Hindawi Journal of Sensors2017 (2017).