An automatic in-home fall detection system using Doppler radar signatures

Abstract

One in three elders over the age of 65 falls each year in the United States. This paper describes a non-invasive fall detection system based on a Doppler radar sensor. The developed system has been tested in two environments: laboratory and real senior living apartments. While some laboratory results appeared in our previous papers, the main novelty of this paper consists in the deployment of our fall detection system in six apartments from TigerPlace (a senior living facility in Columbia, Missouri). The fall detection results obtained in our laboratory were excellent, with the radar placed on the ceiling performing better than on the floor. The fall detection system was then evaluated using radar data collected over two weeks in six TigerPlace apartments. The fall detection system successfully detected all six natural senior falls in an apartment for the examined one week.

Keywords

Doppler radar eldercare fall detection MFCC

1. Introduction

Falls are the dominant reason for accidental death in population above age 65 [25]. Moreover, the death rate caused by falling is increasing in the past decade as more Americans live longer and more independent lives [34]. While prevention is desirable, falls still occur due to the limited success of prediction algorithms and intervention strategies [31]. After an injurious fall occurs, it is important that medical intervention follows as soon as possible in order to reduce long term consequences [15,24]. This paper describes a Doppler radar automatic fall detection system that could be used to alert the nursing personnel as soon as a fall happened.

Recently, many fall monitoring devices have been described in the literature. They can be divided into two categories: wearable and non-wearable. Wearable devices, such as accelerometers, gyroscopes [13] or “push-buttons”, are suitable for active elders but are less appropriate for less active or cognitively impaired ones [10]. Moreover, wearable devices are not suitable for night time or shower use [26]. Conversely, non-wearable devices such as floor vibration sensors [4,5,39], video cameras [6], passive infrared sensor [33,39], microphone arrays [18,27,28], and sensor networks [9,36] are more appropriate for less active elders and night-time use. In assistive living and ambient intelligence applications [7,8,29], each non-wearable device has its own advantages and disadvantages. Aside from privacy preserving, Doppler radars have advantages, such as independence on lighting conditions, inexpensive, non-invasive, sensing through occlusions, and deployable in bathrooms.

Based on the idea that different human body parts produce specific radar signatures, different types of radar sensors could recognize various human activities, such as continuous-wave (CW) radar for gait analysis [12], two-element receiver array for running, walking, and standing [17], M-sequence-based ultra-wideband sensor network for vitality monitoring [32]. We note that none of the above radars are deployed in a real life setting collecting realistic data. There are two main categories of Doppler signature representation methods. The methods in the first category are based on the pattern extracted from the spectrogram. Kim et al. [17] extracts six features from a filtered spectrogram: torso Doppler frequency, total bandwidth (BW), offset of the total Doppler, BW without micro-Dopplers, the normalized standard deviation (STD) of the signal strength, and the period of the limb motion. Wu et al. [37] applies image segmentation and morphological operations using the spectrogram as a binary image. The maximum/minimum frequency and time values generate three features: extreme frequency magnitude, extreme frequency ratio, and the time span of the event. In the second category, the features are derived from sound processing techniques due to the fact that sound waves and radar waves share similar characteristics in frequency changes. The sound wave frequency changes when the listener moves toward or away from the source [14]. An example of this kind of feature is represented by the mel-frequency cepstral coefficients (MFCCs) [16,20–22,38]. This approach is supported by the fact that human operators can differentiate certain targets by listening to the audio output of the surveillance Doppler radar [16].

In a related project, we used floor radars to compute gait velocity and stride duration to estimate the fall risk [38]. Inspired by this system, we proposed the idea of an automatic fall detection system based on Doppler radar signatures and presented preliminary laboratory results in [22]. In subsequent work [20,21], we proposed a two radar sensor system together with a decision-level fuzzy fusion method for integrating multiple information sources [21]. We examined the system performance in our laboratory for various sensor positions and fall directions because only the radial velocity (i.e. collinear to the beam) component of the moving subject can be detected by a Doppler radar [20]. We concluded that for sensors positioned on the floor, the detection of a fall lateral to the radar is not as good as a fall towards or away from it. At the same time, sensors installed on the ceiling are less sensitive to falling direction, since there will always be a downward (away from the radar) component of the radar signal.

In this paper we introduce two new refinements to our radar processing algorithm, one related to feature alignment and another needed for online fall detection. The developed algorithms were tuned first in our laboratory and then were evaluated in TigerPlace, a senior living facility in Columbia, MO [31]. Unlike other fall detection systems that are only evaluated in artificial environments [11], the main novelty of this paper is that it evaluates a Doppler radar fall detection system in a real living environment. A real living environment poses many challenges for fall detection such as antenna configuration, room size, furniture occlusions and movement, building materials, pets and visitors.

The context of our research is a parent study for addressing falls in elderly with a two-fold aim: to monitor gait for fall risk [10] and to develop methods for fall detection using Doppler radar. Radar sensors were first deployed on the floor for optimal gait detection. Although we previously showed that a ceiling placement is better than a floor one for fall detection [20], TigerPlace fall experiments presented here were performed with both of the floor radars due to our emphasis on fall prevention using gait evaluation, and ceiling radar for better performance in detecting natural falls.

This paper is organized as follows. Section 2 introduces the system overview. Section 3 describes the methods and algorithms. Section 4 provides the data collection and evaluation. Section 5 presents the experimental results. More details about the results are discussed in Section 6. We draw conclusions in Section 7.

Fig. 1.

Radar sensor hardware setup. (a) Schematic view. (b) Floor sensor 0.42 m × 0.32 m × 0.23 m deployment box.

2. System overview

The fall detection system is composed of a Doppler radar sensor, and the related signal processing and pattern recognition algorithms.

Fig. 2.

Block diagram for the processing procedure to detect a fall.

We used a commercial GE range controlled Doppler radar (RCR) as in [20–22] with a center frequency of 5.8 GHz and a target velocity between 0.15 and 1.5 m/s. The range of the radar is set to 6 meters with a 90° viewing angle. Figure 1(a) shows the diagram of the hardware setup. We use a wireless router to transmit and store the sensor data from RCR to the apartment computer using a DI-710 DataQ (www.dataq.com) data logger. The data are then transferred from each apartment computer to a central server for further processing.

We used a 0.42 m × 0.32 m × 0.23 m wood box to deploy the RCR and its auxiliary equipment such as power supply, data acquisition board, and wireless router (see Fig. 1(b)). The RCR was mounted on a bracket inside the box at a height of 10 cm from the bottom. The data logger was mounted on the top of the RCR and the wireless router was placed on top of the DataQ. The signal and power supply wires are placed outside of the RCR range. The floor RCR surface is wrapped by foil with a slot to exclude potential false alarm sources, such as possible electronic fields formed by electric motors or high voltage equipment, and moving or vibrating objects, such as fans, pulleys and conveyor belts.

For ceiling RCR, we did not use the slotted foil in order to allow for a wider view range. In the laboratory tests, we placed the RCR on the ceiling of the laboratory at the room center (about 3.5 m high). In the apartment, we fixed the RCR in a wooden bracket in the attic, above the ceiling dry wall, at the living room center facing down to the floor. All the RCRs were carefully calibrated to obtain a clear signal before installation.

In order to assess the timing of the fall and to investigate the types of false alarms detected in each apartment by the fall detection system, we used a depth camera (Kinect). The Kinect camera was installed in the living room, above the entrance door.

The block diagram of the fall detection algorithm is displayed in Fig. 2. After the temporal position of a potential fall is identified, features are extracted in a two second window around it. Then, mel-frequency cepstral coefficients (MFCC) features are computed and aligned with similar feature vectors from the training set. The feature alignment procedure makes the fall detection algorithm more resilient to the fall location in the temporal window. Finally, a classifier is used to determine whether or not the feature vector represents a fall. A detailed description of each block in Fig. 2 is given in the following Section 3.

3. Methodology and algorithms

3.1. Potential fall segmentation

A typical time domain radar signal waveform segment in time domain with two falls is shown in Fig. 3. Two second windows around each fall are marked by dotted lines.

The fall location procedure is based on finding the energy burst peaks in the spectrogram. To compute the spectrogram of the radar signal $x (n)$ we first compute the short time Fourier transform (STFT) as: $\begin{array}{rcl} (1) & STFT (m, ω) = \sum_{n = - \infty}^{\infty} x (n) w (n - m) e^{- j ω n}, \end{array}$ where $w (n)$ is the Hamming window with N-width ( $N = 512$ ) defined by $\begin{matrix} (2) & w (n) = 0.54 - 0.46 cos (\frac{2 π n}{N - 1}) . \end{matrix}$

The spectrogram is defined as the magnitude of squared of STFT as: $\begin{matrix} (3) & spectrogram {x (n)} = | STFT (m, ω) |^{2} . \end{matrix}$

The spectrogram of the time domain radar signal from Fig. 3 is shown in Fig. 4.

Fig. 3.

A typical time domain radar signal waveform segment including two falls (the signal waveform measured along time). Each fall is labeled with a 2-s green dotted window.

To identify falls using the spectrogram shown in Fig. 4, we compute the spectral energy burst (EB) by summing the energy in the [A, B] Hz frequency range as: $\begin{matrix} (4) & EB (m) = \sum_{ω = B / (2 π)}^{ω = A / (2 π)} | STFT (m, ω) |^{2} . \end{matrix}$ A peak in the energy burst curve (see Fig. 5) is associated with the occurrence of a high velocity motion, such as a fall.

Fig. 4.

Spectrogram of the signal waveform from Fig. 3. The non-negative frequencies are only displayed because the negative frequency STFT values are complex-conjugates of the positive frequency values. So the magnitudes-squared are identical to their positive counterparts.

Fig. 5.

Energy burst curve for Fig. 4.

Fig. 6.

Influence of the frequency range on detecting potential falls (marked by red stars) on energy burst curve: a) original radar signal including 4 falls (green lines) and walking (red box); b) [10, 50] Hz range; c) [25, 50] Hz range; d) [30, 200] Hz range.

One question is determining the best choice for the frequency range [A, B]. The Doppler shift (hence the signal waveform in Fig. 3) $Δ f$ is proportional to the speed variation $Δ v$ along radar axis, $Δ f \sim Δ v$ , for every point of the falling body. In the falling process, some points of the body are going toward while others are going away from the radar (imagine a twisting motion during falling), hence generating the frequency patterns in Fig. 4. To choose the useful frequency range in the spectrogram, we analyzed a 20 min radar signal segment by identifying the energy bursts over a given threshold and correlated them to the ground truth (see Fig. 6).

The frequencies below 25 Hz are typically associated with low speed activities such as walking (see extra detections in Fig. 6(b) vs 6(c)). As we increase the lower frequency threshold, A, the number of false alarms produced by walking decreases. Similarly, increasing the higher threshold, B, seems to slightly decrease the false alarms (see Fig. 6(c) vs 6(d)). However, less useful information seems to be present past 200 Hz (see Fig. 4). In this paper, for fall detection, we choose the [A, B] frequency range as [25, 50] Hz.

After the EB curve is computed, we then smooth it using a moving average with $K = 5$ (K was selected empirically by trial and error) as: $\begin{matrix} (5) & \overline{EB} (m) = \sum_{i = 1}^{K - 1} EB (m - i) . \end{matrix}$

On the energy burst curve, a peak means the occurrence of an extreme motion, such as a fall. A peak finder with a threshold is used to locate this potential fall location. By adjusting the threshold, the number of potential fall activities varies. Figure 5 shows an energy burst curve with located fall peaks.

3.2. Feature extraction

To describe the Doppler signal we used features employed in speech recognition such as Mel-frequency cepstral coefficients (MFCC) and 1-D local binary patterns [40]. The reason for this choice is that human radar operators were able to identify certain targets by listening to the Doppler audio output [16]. In our laboratory experiments MFCCs outperformed other features, such as zero crossing rate and energy frequency bands [27] and LBP (see Fig. 12) due to, we believe, a superior false alarm representation.

We extracted MFCCs features from a 2 s window located around the potential fall. The window size is selected from the exhaustive searching of window size in [0.5 s, 6 s] with 0.5 s interval on floor radar dataset. Each 2 s window is considered as an input signal frame, which needs to be divided into sub-frames to extract features. If the signal sampling frequency is 960 Hz, an input signal frame has $2 \times 960$ samples. The sub-frame in each frame has a size of 256 samples with an overlap of 11.72% to each other. So the total number of sub-frames in an input signal frame is computed with $L = ⌊(960 \times 2 - 256) / (256 \times 11.72 %)⌋ = 55$ . For each sub-frame, we choose (see Section 5.1) to compute $N = 7$ cepstral coefficients. Therefore, the feature matrix for the i-th signal window is given by: $\begin{matrix} (6) & C_{i} = [\begin{matrix} C_{1, 1} & \dots & C_{1, 55} \\ ⋮ & ⋱ & ⋮ \\ C_{N, 1} & \dots & C_{N, 55} \end{matrix}] . \end{matrix}$

After ignoring the first coefficient (DC term) in each sub-frame, we obtain a feature vector with the dimension of $55 \times 6 = 330$ by concatenating the columns of the above matrix, $C_{i}$ . Typical fall and non-fall feature vectors for a 2 s signal window are shown in Figs 7(a) and (b), respectively. The fall feature vector contains larger values within a blob shape at the vector center to represent the fall event in Fig. 7(a), while the non-fall feature vector shows a random variation pattern in Fig. 7(b).

Fig. 7.

Examples of fall (a) and nonfall (b) feature vectors for a 2-second signal window.

3.3. Feature alignment algorithm

One of the most challenging parts of developing our fall recognition system was developing a radar signature library. Although the ground truth has been observed by the depth camera monitoring system, there may be some uncertainty in the fall timing due to the fall type, direction and the clock difference between the PC and DataQ logger. In fact, it is hard to tell just by looking at a radar signature if it comes from a fall or not (see Fig. 3). To alleviate the effect of the fall timing in the 2 s window, we propose to apply a feature alignment procedure (somewhat similar to the ones used in bioinformatics [19]) for the entire training dataset. This will ensure the presence of fewer outliers in the training data and more compact fall and non-fall clusters in the feature space.

The feature vectors were aligned such that their correlation is maximized. For two feature vectors $v_{1}$ and $v_{2}$ , we compute their correlation as: $\begin{array}{l} {corr}_{v_{1}, v_{2}} (d) \\ (7) & = \frac{\sum_{i = 1}^{P} (v_{1} (i) - \overline{v_{1}}) (v_{2} (i - d) - \overline{v_{2}})}{\sqrt{\sum_{i = 1}^{P} {(v_{1} (i) - \overline{v_{1}})}^{2} \sum_{i = 1}^{P} {(v_{2} (i - d) - \overline{v_{2}})}^{2}}}, \end{array}$ where $P = 330$ is the dimension, and $\overline{v_{k}} (k = 1, 2)$ are the means of the feature vectors. The above ${corr}_{v_{1}, v_{2}}$ is computed at all shifts $d = 1, \dots, P$ . The total amount of shift or lag τ required to align $v_{2}$ to $v_{1}$ is the one that maximizes the correlation between $v_{1}$ and $v_{2}$ . It is defined as: $\begin{matrix} (8) & τ = \underset{d}{argmax} ({corr}_{v_{1}, v_{2}} (d)) . \end{matrix}$

We tried two alignment strategies: a simple one based on zero padding and a more complicated one based on search in a larger feature window. The zero-padding strategy consisted of shifting $v_{2}$ by τ index and padding the remaining places in the 2 s window with zeros. For example, if we shift $v_{2}$ to the right by τ locations, we set $v_{2} (1) = 0, \dots, v_{2} (τ) = 0$ and discard $v_{2} (P - τ + 1) \dots v_{2} (P)$ . A bigger window, 4 s in our case, allowed us to avoid zero-padding and feature discarding at the expense of more computation time. All feature vectors $v_{2}$ in the library were aligned to a single fall vector $v_{i}$ chosen such that the resulting classifier had the best overall performance. The reference fall feature vector was found by exhaustive search. For a dataset with $M_{f}$ falls and $M_{nf}$ non-falls, the i-th vector from $M_{f}$ falls is the reference to alignment the rest $M_{f} + M_{nf} - 1$ feature vectors. We calculated an area under the receiver operating characteristic curve (AUC, defined in Section 4.2 and Eq. (9)) for each aligned dataset. The reference vector is the one that generates the largest AUC. The feature shifting algorithm was summarized in Algorithm 1.

Algorithm 1

Feature shifting algorithm

3.4. Classifiers

Support vector machines (SVM) are a popular classification method for two class problems. We employed LibSVM implementation for both training and testing steps [1,2]. We used a linear kernel SVM for computational efficiency.

The classifiers were first trained and tested on signature libraries (ceiling and floor) obtained in our laboratory using a leave-one-out cross validation approach. Then, we used the classifiers trained on the laboratory signature to classify the continuous data obtained in the TigerPlace apartments. We also used a set of classifiers trained on the stunt actor falls obtained in TigerPlace to test the influence of the environment in the training process. This aspect is important for the real life application since we would like to avoid training the system for each apartment and person, if possible (see Fig. 11). The fall detection algorithm used for experiments on a signature library is summarized in Algorithm 2 below.

The online classification procedure used for continuous data was similar to Algorithm 2 except for three details. First, a sliding 2 s window with a 0.5 overlap rate was employed in step 1 above. Second, a 4 s buffer was available to perform the feature alignment procedure. Third, to reduce the number of false alarms, we used an energy threshold e to decide if steps 1–7 from Algorithm 2 will be performed or not. If the energy of the signal in window i, $E_{i} = \sum_{n = 0}^{N - 1} x_{i} {(n)}^{2}$ , is smaller than e we label the window as a non-fall (i.e. fall $confidenceValue = 0$ ). The classifier used for the online classification procedure was trained on the laboratory floor signature library (all the radars in TigerPlace were placed on the floor at the time of this data collection).

Algorithm 2

Fall detection algorithm

4. Data collection and criteria of performance evaluation

4.1. Data collection

The fall/non-fall activities were performed by stunt actors in both our laboratory and the senior apartments. Our nursing collaborators trained stunt actors using 21 fall types at the University of Missouri to fall in ways similar to older adults [30]. In our laboratory, aside from elderly falls, the stunt actors performed other possible activities of daily living in an independent living facility, such as walking, picking objects from the floor, sitting and standing [3] which might generate false alarms. While stunt actor falls were still used in TigerPlace, we obtained a large amount of data (mostly normal daily activities) from the TigerPlace residents themselves. While a variety of activities were recorded in the apartment, no real falls were observed during the two weeks period of the data collection. Note that our project had University of Missouri Institutional Review Board approval and written consents from all senior apartment residents involved in data collection. In this paper, we used three datasets denoted as DATA_Lab, DATA_Apt1 and DATA_Apt2, respectively.

Table 1
Stunt actor profile in DATA_lab

Subject# Gender Age Height Weight (Lbs)

1 female 32 5’3” 135

2 female 46 5’4” 117

3 male 30 5’8” 170

Subject#	Gender	Age	Height	Weight (Lbs)
1	female	32	5’3”	135
2	female	46	5’4”	117
3	male	30	5’8”	170

Fig. 8.

Lab environment. RCR 1 is on floor and RCR 2 is on ceiling. (a) Front camera view. (b) Right side camera view.

Fig. 9.

Senior apartment in TP. RCR 1 is on the floor and RCR 2 is in the attic. (a) Left side camera view. (b) Front camera view.

DATA_Lab was generated in a 9 m × 8 m × 3.5 m room situated in our laboratory (see Fig. 8). One radar sensor (RCR) was placed on the floor and another on the ceiling (denoted by triangles). For the data collection, the floor mat was placed in the center of room and directly under the ceiling RCR. The stunt actors performed 21 types of falls in the following five categories: a) lose balance – forward, backwards, left, right; b) lose consciousness – forward, backwards, left, right, crumple; c) trip & fall – forwards, sideways, forwards, sideways, backwards; d) reach & fall on chair – forwards, left, right, forwards, backwards; e) fall from couch – upper body first, hips first. Each fall type is repeated 5 times to generate 105 falls. We extracted 8 types of non-fall activities from 5 daily scenarios including 383 non-fall activities: walk, turn around, sit down, stand up, body sway, bend over, stretch arms, and step over. We had three different stunt actors during our data collection in order to capture some variance in body size as shown in Table 1. Each event was captured by both of the ceiling and the floor RCR.

DATA_Apt1 was collected in six senior apartments similar to the one shown in Fig. 9 using radars placed on the floor. The size of the living room was 5.8 m × 3.6 m × 2.7 m. The height of the laboratory room (3.5 m) is about 30% higher than the typical living room in a senior apartment, which resulted in a different coverage range for the ceiling RCR between the two settings. As we explained later, that impacted the performance of the ceiling radar in the senior apartments. DATA_Apt1 had three parts: DATA_Apt1_s and DATA_Apt1_s_b collected from stunt actors, and DATA_Apt1_r collected from residents. DATA_Apt1_s consisted of 20 falls and 16 non-falls from the stunt actors in the living room. DATA_Apt1_s_b contained 72 falls and 98 non-falls from the stunt actors in the bathroom during five months. We were limited to a small number of falls in order to minimize the test time in the resident’s apartment. The fall/non-fall types were similar to the 21 types of falls performed in the laboratory. DATA_Apt1_r was collected from six residents during two weeks. No natural falls by residents occurred during that time. The mix of stunt actor and resident data was necessary for algorithm evaluation since we expected a low number of natural falls.

DATA_Apt2 was collected in an apartment of a frequent faller. During one week, the resident fell 6 times. The radar placement was in the attic above the ceiling. This dataset is important for assessing if we can detect non-actor falls and will answer the question if our algorithm training is adequate for detecting falls of persons not included in the training data.

4.2. Performance evaluation

We used a receiver operating characteristic (ROC) curve to evaluate the performance of the classifier in the fall detection algorithm. The ROC is obtained by thresholding the fall confidence (a number between 0 and 1) generated by the classifier with a series of n thresholds, $T_{i} \in [0, 1]$ , $i = 1, \dots, n$ . For each threshold $T_{i}$ we obtain a true positive rate ( ${TPR}_{i}$ ), and false positive rate ( ${FPR}_{i}$ ) pair. Then, the area under the ROC curve (AUC) can be calculated using: $\begin{array}{l} (9) & AUC = \sum_{i = 2}^{n} \frac{({TPR}_{i} + {TPR}_{i - 1}) ({FPR}_{i} - {FPR}_{i - 1})}{2} . \end{array}$

We mention that in the case of continuous radar data (DATA_Apt1_r and DATA_Apt2) the number of events that can lead to false alarms is unknown. In this case we report the number of detected false positives (false alarms) per unit time (hour). Since this is not a traditional ROC, we will call it a “pseudo ROC”. In our opinion, this pseudo ROC is more suitable for evaluating online algorithms that run in real settings. Reporting the number of false alarms per hour instead of the false alarm rate, allows the clinical personnel to more easily evaluate and understand the fall detection system.

5. Experimental results

In this section we present the results of several experiments designed to answer the following questions: (1) Is feature alignment useful? (2) What is the best position of the radar sensor (floor/ceiling)? (3) What is the performance of a radar fall detection system in real settings? (4) Is this performance different from the one observed in the laboratory? Why?

5.1. Influence of feature alignment on classification

This paper used a window based approach for feature alignment. The size of the window is chosen and compared to other known windows using features. Although a segmentation based approach might be better, it requires a good segmentation process and a dynamic time warping method for alignment with a variable signal window. One of the greatest problems for the segmentation based approach would be that it is not easy to produce the ground truth for Doppler signal segmentation. So the window based approach is chosen to investigate the effect of feature alignment. We used the data collected in our laboratory, DATA_Lab (only the floor RCR), and ran multiple classification experiments with different numbers of MFCC coefficients for three cases: unaligned features, aligned using zero padding, and aligned using a 4 s window (see Fig. 10). We used a classifier and AUC to quantify the algorithm performance for each case. As expected, both feature alignment procedures resulted in better classification performances than the unaligned case: zero-padding by about 3% and 4 s window by about 6%. The increase in performance was independent of the number of MFCC coefficients employed. However, the best classification performance was observed in all three cases for $N = 7$ cepstral coefficients number, which is used throughout all our experiments. For later application, we use SVM to avoid the possible over-fitting due to the small training data size from real senior apartment.

Fig. 10.

Fall detection performance for three feature alignment options: no alignment, zero-padding, and 4 s window, with the variable number of MFCC features, respectively.

5.2. Influence of the radar position on detection performance

To determine which radar position leads to better fall detection performance, we first used our laboratory data, DATA_Lab, and then the stunt actor part of the TigerPlace data, DATA_Apt1_s.

5.2.1. Fall detection in the laboratory setting

The ROC curves obtained for two radar positions using a SVM classifier with DATA_Lab are given in Fig. 11. From Fig. 11, we see that the ceiling RCR generates a larger AUC value. In addition, the ceiling RCR detects all the falls with only 20% false positive rate, which is better than the floor RCR. Clearly, placing the radar on the ceiling produced better results than placing it on the floor.

Fig. 11.

Fall detection in lab environment with DATA_Lab.

5.2.2. Fall detection in apartment setting

We conducted similar experiments using stunt actor data collected in TigerPlace, DATA_Apt1_s and DATA_Apt1_s_b.

The features were validated on DATA_Apt1_s_b, which has a clean signal background and full coverage to the room because of the limited space and furniture in the bathroom. For comparison, the performance of the LBP features [40] (AUC = 0.79) is about 20% worse than MFCC (AUC = 0.98) on the ceiling data collected in TigerPlace bathroom in Fig. 12.

Fig. 12.

Performance comparison between different features: LBP and MFCC features on the ceiling data collected in TigerPlace bathroom.

To test if a classifier trained in the laboratory can be used in real settings, we performed two sets of experiments, as shown in Fig. 13: we tested DATA_Apt1_s with classifiers trained on DATA_Apt1_s (circles), and with classifiers trained on DATA_Lab (triangles). As we can see from Fig. 13, the loss in performance is only of about 2–3% between the two classifiers. This finding has important implications for the development of an online fall detection system. Immediately after deployment time, we can use a classifier trained in the laboratory that will later be improved using some online adaptive training strategy.

Fig. 13.

Fall detection in senior apartment environment with DATA_Apt1_s by using different training datasets, DATA_Lab and DATA_Apt1_s, respectively.

From Fig. 13 we also see that the performance of both RCRs decreased in the real setting due to various factors such as lower ceilings, apartment clutter and motion interference from other people present in the apartment. However, the results from Fig. 13 confirm that the best placement of the radar is on the ceiling. We note that the actual RCR position in the TigerPlace apartment was above the ceiling (in the attic) which on one hand was clean (no apartment modification required) but may not be accessible for some homes (such as apartment buildings).

5.3. Results obtained on DATA_Apt1_r

We used the fall detection algorithms described above, trained on DATA_Lab and DATA_Apt, to detect falls in the two weeks of data collected in 6 TigerPlace apartments using floor RCRs. We centered our data collection on the day that the stunt actor performed 12 falls in each apartment near the floor radar, to make sure that our data included enough falls. Details about the detection algorithm used in this experiment are given in Algorithm 2 and its performance is shown in Fig. 14. For two apartments (1 and 4) we obtained 100% detection with about 2 false alarms per week (0.015 FA/hour). This proves that a Doppler radar can be used for fall detection. However, in two other apartments (5 and 6) the number of false alarms was greater than 1 FA/hour at 100% detection, which is clearly inacceptable.

Fig. 14.

Fall detection and false alarm of floor RCR with two weeks senior apartment data in 6 different TigerPlace apartments.

5.4. Results obtained on DATA_Apt2

In Fig. 15, we show the results on DATA_Apt2. A 75 years old resident lives alone in this apartment and she is a frequent faller. The data contains 6 natural falls during the one week long data collection.

Fig. 15.

Result obtained for DATA_Apt2.

As we see from Fig. 15, the Doppler radar system is able to detect all the real senior falls but with a significant number of false alarms. So this answers the question if we can identify falls from persons that were not included in the training dataset. To further reduce the false alarms, we can use a fusion approach with other sensors such as motion detectors. Using the depth cameras installed in the apartment (i.e. Microsoft Kinect) we were able to determine which activities produced false alarms. Among the most frequent false alarm sources we mention a big cat (around 4 kg) jumping on/off the chairs/couch and visits by the nursing staff and other visitors that come to help the resident multiple times every day in his apartment. All those activities contribute to a high false alarm rate during this one week data collection. However, in the meantime, our motion detectors are pet-insensitive and we should be able to tell if the activity is performed by the resident or by the cat [23]. Moreover, our Kinect based gait detection system is able to identify the resident in the room based on his gait [35]. Using the radar in conjunction with other sensors (such as motion sensors [23] or Kinect [35]) can further reduce the false alarm rate.

Table 2

False alarms from two weeks data in each apartment

Apt	False alarm						Total

	bend over/pick up	drop	sit/stand	run/fast walk	walk in/out of door	other
1	0	0	0	0	5	0	5
2	2	2	1	0	6	4	15
3	6	0	1	0	0	1	8
4	0	1	1	1	0	6	9
5	2	1	5	19	0	3	30
6	5	1	2	8	2	5	23

Fig. 16.

Motions for different daily activities: (a) sit down on floor with butt on floor first and keep both leg straight in front; (b) cross leg and sit down on floor; (c) lay down on floor; (d) sit down on a chair; (e) kneel down on floor; (f) drop a book on floor, bend over and squat down to pick it; (g) walking; (h) fall. The y-axis units are voltage (V), frequency (Hz) and squared-voltage (v²) for the raw signal, spectrogram and energy burst, respectively. The x-axis unit is time in second.

6. Discussions

We showed in Section 5.2 that the ceiling is the best position for RCR to detect fall. However, since our work was part of a parent study that comprises both fall risk assessment and fall detection, the team decided to focus on prevention using gait assessment, which requires radars on the floor for a higher accuracy. Since the parent study has ended, we are in the process of moving all our floor radar on the ceiling to collect more data for fall detection.

The use of the floor RCR for gait detection had another implication on fall detection. To optimize gait detection, a thin aluminum foil shielding with a slit was placed on the detector in order to prevent secondary reflections to reach the antenna [10]. The size and placement of the shielding were not identical among the six apartments, being a possible source of the variability observed in Fig. 14.

In Table 2 we show an analysis of the false alarms detected by our algorithm in all six apartments. We chose the false alarms with fall confidence greater than 0.5 for apartments 1–4, and 0.95 for apartments 5 and 6. The false alarms were identified based on the Kinect depth images and classified into six categories: bend over or pick up an object, drop things on floor/chair, sitting or standing, running or fast walking, leave/enter the room and other. Figure 16 uses the raw signal waveform, spectrogram and energy burst curve to present the patterns for eight daily activities. Those activities in Figs 6(a)–(g) represent the most common false positives in a daily living environment. Figure 6(h) shows the occurrence of a fall.

Another possible explanation for the variability observed in Fig. 14 is the number of people and visitors in the apartments. For example, apartment 5 has a couple living in it, rather than just one person. The residents like to spend a lot of time in the living room and have many visitors. The fast runs observed in apartment 5 are due to kids running and sitting on the floor. In apartment 6, the female resident has severe osteoarthritis, and she uses an aluminum walker during walking. It is possible that the reflections caused by the walker resulted in the extra false alarms. Moreover, a physical therapist and nursing staff visit her apartment frequently leading to more false alarms. Pets are also a problem for our current system. The “other” false alarms (see Table 2) observed in apartment 4 are due to a dog jumping up/down from the couch.

The radar position on the floor is another source of variability. The floor radars are by the front door in the first two apartments being prone to detect walks in/out of the apartment. In apartment 4, the floor radar is in the bedroom and pointing toward the living room. This could lead to various false alarms (“other” in Table 2) caused by sudden turns or entering the bedroom. In the apartments 5 and 6, the radar is put beside the counter which is close to the center of the livingroom. This placement captures more visitor activity than the bedroom placement.

7. Conclusions

In this paper, we describe a fall detection system based on a Doppler radar sensor in senior homes. This system can differentiate a fall from daily non-fall activities based on MFCC features extracted from the radar signal using a SVM classifier. We first deployed the system in a simulated home-like laboratory environment and investigated classification performance for various variables such as the number of MFCC coefficients, feature alignment and radar position. We found that using 7 MFCC coefficients, feature alignment and ceiling positioning leads to the best performance.

Then, we deployed the system in six apartments in TigerPlace, an aging in place facility in Columbia, MO. The results obtained in TigerPlace varied from about 2 false alarms/week to about 4 false alarms/hour at 100% detection rate. After analyzing the false alarms, we concluded that the variability of the results was due to factors such as radar construction and positioning, number of people in the room (resident or visitors), pets and walkers. Moreover, the proposed system effectively detected all six natural falls in a continuous one week data from an apartment where a senior faller is living there.

In future work, we will try to address the variability of results by developing algorithms more robust to visitors, pets and interferences. We are currently deploying Doppler radars on the ceiling of ten TigerPlace apartments in order to further confirm the results presented in this paper.

Footnotes

Acknowledgement

This paper was funded by the AHRQ grant 5R01HS018477.

References

http://neural.cs.nthu.edu.tw/jang/books/audioSignalProcessing/speechFeatureMfcc.asp?title=12-2MFCC.

http://www.csie.ntu.edu.tw/~cjlin/libsvm/.

http://www.americareusa.net/independent_living/columbia_mo/zip_65201/americare/1335.html.

Addlesee,

Jones,

Livesey and

Samaria, The ORL active floor, IEEE Personal Communications4 (1997), 35–41. doi:10.1109/98.626980.

Alwan,

P.J.

Rajendran,

Kell,

Mack,

Dalal,

Wolfe and

Felder, A smart and passive floor-vibration based fall detector for elderly, in: 2nd IEEE Int. Conf. on Inf. & Comm. Tech., Damascus, Syria, Apr. 24–28, Vol. 1, 2006, pp. 1003–1007.

Anderson,

R.H.

Luke,

Keller,

Skubic,

Rantz and

Aud, Linguistic summarization of activities from video for fall detection using voxel person and fuzzy logic, Computer Vision and Image Understanding113(1) (Jan. 2009), 80–89. doi:10.1016/j.cviu.2008.07.006.

J.C.

Augusto,

Nakashima and

Aghajan, Ambient intelligence and smart environments: A state of the art, in: The Handbook of Ambient Intelligence and Smart Environments, 2010, pp. 3–31. doi:10.1007/978-0-387-93808-0_1.

A.J.

Carlos and

Paul, Ambient intelligence: Concepts and applications, Computer Science and Information Systems4 (2007), 1–26. doi:10.2298/CSIS0701001A.

Castanedo,

Lopez-de-Ipina,

Aghajan and

Kleihorst, Learning routines over long-term sensor data using topic models, Expert Systems Journal31(4) (Sep. 2014), 365–377. doi:10.1111/exsy.12033.

10.

P.E.

Cuddihy,

Yardibi,

Z.J.

Legenzoff,

Liu,

C.E.

Phillips,

Abbott,

Galambos,

Keller,

Popescu,

Back,

Skubic and

Rantz, Radar walking speed measurements of seniors in their apartments: Technology for fall prevention, in: Proc. of 34th EMBS, San Diego, CA, Aug. 28–Sep. 1, 2012, pp. 260–263.

11.

Garripoli,

Mercuri,

Karsmakers,

P.J.

Soh,

Crupi,

Vandenbosch,

Pace,

Leroux and

Schreurs, Embedded DSP-based telehealth radar system for remote in-door fall detection, IEEE Journal of Biomedical and Health Informatics19(1) (Jan. 2015), 92–101. doi:10.1109/JBHI.2014.2361252.

12.

J.L.

Geisheimer,

W.S.

Marshall and

Greneker, A continuous-wave (CW) radar for gait analysis, in: The 35th Asilomar Conf. on Signals, Systems and Computers, Vol. 1, Nov. 2001, pp. 834–838.

13.

Giansanti,

Maccioni and

Macellari, The development and test of a device for the reconstruction of 3D position and orientation by means of a kinematic sensor assembly with rate gyroscopes and accelerometers, IEEE Trans. Biomed. Eng.52(7) (2005), 1271–1277. doi:10.1109/TBME.2005.847404.

14.

Gupta,

Morris,

Patel and

Tan, Soundwave: Using the Doppler effect to sense gestures, in: Proc. of the SIGCHI Conference on Human Factors in Computing Systems, 2012, pp. 1911–1914.

15.

R.J.

Gurley,

Lum,

Sande,

Lo and

M.H.

Katz, Persons found in their homes helpless or dead, N. Engl. J. Med.334(26) (Jun. 27, 1996), 1710–1716. doi:10.1056/NEJM199606273342606.

16.

E.J.

Hughes and

Lewis, The application of speech recognition techniques to radar target Doppler recognition: A case study, in: The Institution of Engineering and Technology Seminar on High Resolution Imaging and Target Classification, Nov. 21–21, 2006, pp. 145–152. doi:10.1049/ic:20060082.

17.

Kim and

Ling, Human activity classification based on micro-Doppler signatures using a Support Vector Machine, IEEE Transactions on Geoscience and Remote Sensing47(5) (May 2009), 1328–1337. doi:10.1109/TGRS.2009.2012849.

18.

Li,

K.C.

Ho and

Popescu, A microphone array system for automatic fall detection, IEEE Transactions on Biomedical Engineering59(2) (May 2012), 1291–1301.

19.

D.J.

Lipman,

S.F.

Altschul and

J.D.

Kececioglu, A tool for multiple sequence alignment, Proc. Natl. Acad. Sci. USA86(12) (1989), 4412–4415. doi:10.1073/pnas.86.12.4412.

20.

Liu,

Popescu,

Rantz and

Skubic, Doppler radar sensor positioning in a fall detection system, in: Proc. of 34th EMBS, San Diego, Aug. 28–Sep. 1, 2012, pp. 256–259.

21.

Liu,

Popescu,

Rantz and

Skubic, Fall detection using Doppler radar and classifier fusion, in: Proc. of EMBS on BHI, Shenzhen, Jan. 2–7, 2012, pp. 180–183.

22.

Liu,

Popescu,

Rantz,

Skubic,

Yardibi and

Cuddihy, Automatic fall detection based on Doppler radar motion, in: Proc. of 5th International Conference on Pervasive Computing Technologies for Healthcare, Dublin, Ireland, May 23–26, 2011, pp. 222–225.

23.

Liu,

Popescu,

Skubic and

Rantz, An automatic fall detection framework using data fusion of Doppler radar and motion sensor network, in: Proc. of 36th EMBS, Chicago, Aug. 26–30, 2014, pp. 5940–5943.

24.

C.G.

Moran,

R.T.

Wenn,

Sikand and

A.M.

Taylor, Early mortality after hip fracture: Is delay before surgery important, J. of Bone and Joint Surgery87(3) (Mar. 2005), 483–489. doi:10.2106/JBJS.D.01796.

25.

S.L.

Murhy, Deaths: Final data for 1998, National Vital Statistics Reports48(11) (July 24, 2000), 1–105.

26.

Noury,

Fleury,

Rumeau,

A.K.

Bourke,

G.O.

Laighin,

Rialle and

J.E.

Lundy, Fall detection-principles and methods, in: Proc. of the 29th IEEE EMBS, Lyon, France, Aug. 23–26, 2007, pp. 1663–1666.

27.

Popescu and

Coupland, A fuzzy logic system for fall detection, in: AAAI Fall Symposium, Washington, DC, Nov. 7–9, 2008, pp. 78–83.

28.

Popescu,

Li,

Skubic and

Rantz, An acoustic fall detector system that uses sound height information to reduce the false alarm rate, in: 30th Int. IEEE EMBS Conf, Vancouver, BC, Aug. 20–24, 2008, pp. 4628–4631.

29.

Ramos,

J.C.

Augusto and

Shapiro, Ambient intelligence – The next step for artificial intelligence, IEEE Intelligent Systems23 (2008), 15–18. doi:10.1109/MIS.2008.19.

30.

Rantz,

Aud,

Alexander,

Wakefield,

Skubic,

R.H.

Luke,

Anderson and

Keller, Falls, technology, and stunt actors: New approaches to fall detection and fall risk assessment, Journal of Nursing Care Quality23(3) (2008), 195–201. doi:10.1097/01.NCQ.0000324581.59557.42.

31.

Rantz,

Marek,

Aud,

Johnson,

Otto and

Porter, TigerPlace: A new future for older adults, Journal of Nursing Care Quality20(1) (Jan.–Mar. 2005), 1–4. doi:10.1097/00001786-200501000-00001.

32.

Sachs and

Hermann, M-sequence-based ultra-wideband sensor network for vitality monitoring of elders at home, IET Radar, Sonar & Navigation9(2) (Feb. 2015), 125–137. doi:10.1049/iet-rsn.2014.0214.

33.

Sixsmith,

Johnson and

Whatmore, Pyrolytic IR sensor arrays for fall detection in the older population, J. Phys. IV France128 (2005), 153–160. doi:10.1051/jp4:2005128024.

34.

J.A.

Stevens, Fatalities and injuries from falls among older adults – United States, 1993–2003 and 2001–2005, MMWR55(45) (Nov. 17, 2006), 1221–1224.

35.

Stone and

Skubic, Unobtrusive, continuous, in-home gait measurement using the Microsoft kinect, IEEE Transactions on Biomedical Engineering60(10) (2013), 2925–2932. doi:10.1109/TBME.2013.2266341.

36.

Tessens,

Morbee,

Aghajan and

Philips, Camera selection for tracking in distributed smart camera networks, ACM Transactions on Sensor Networks10(2) (Jan. 2014), 23:1–33.

37.

Wu,

Y.D.

Zhang,

Tao and

M.G.

Amin, Radar-based fall detection based on Doppler time-frequency signatures for assisted living, IET Radar, Sonar & Navigation9(2) (Feb. 2015), 164–172. doi:10.1049/iet-rsn.2014.0250.

38.

Yardibi,

Cuddihy,

Genc,

Bufi,

Skubic,

Rantz,

Liu and

Phillips, Gait characterization via pulse-Doppler radar, in: IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops), Mar. 21–25, 2011, 662–667.

39.

Yazar,

Keskin,

B.U.

Toreyin and

A.E.

Cetin, Fall detection using single-tree complex wavelet transform, Pattern Recognition Letters34(15) (Nov. 2013), 1945–1952. doi:10.1016/j.patrec.2012.12.010.

40.

Zhu,

Chatlani and

J.J.

Soraghan, 1-D local binary patterns based VAD used INHMM-based improved speech recognition, in: Proc. of the 20th European Signal Processing Conference (EUSIPCO), Aug. 27–31, 2012, pp. 1633–1637.

An automatic in-home fall detection system using Doppler radar signatures

Abstract

Keywords

1. Introduction

3.1. Potential fall segmentation

4.1. Data collection

Table 1 Stunt actor profile in DATA_lab Subject# Gender Age Height Weight (Lbs) 1 female 32 5’3” 135 2 female 46 5’4” 117 3 male 30 5’8” 170

5. Experimental results

5.1. Influence of feature alignment on classification

5.2.1. Fall detection in the laboratory setting

7. Conclusions

Footnotes

Acknowledgement

References

Table 1
Stunt actor profile in DATA_lab

Subject# Gender Age Height Weight (Lbs)

1 female 32 5’3” 135

2 female 46 5’4” 117

3 male 30 5’8” 170