Abstract
Extensive field tests were carried out to assess the performance of adaptive thresholds algorithm for footstep and vehicle detection using seismic sensors. Each seismic sensor unit is equipped with wireless sensor node to communicate critical data to sensor gateway. Results from 92 different test configurations were analyzed in terms of detection and classification. Hit and false alarm rates of classification algorithm were formed, and detection ranges were determined based on these results. Amplification values of low-intensity seismic data were also taken into account in the analysis. Algorithm-dependent constants such as adaptive thresholds sample sizes were examined for performance. Detection and classification of seismic signals due to footstep, rain, or vehicle were successfully performed.
1. Introduction
Seismic sensors are invaluable parts of security systems that focus on perimeter or compound security. Footstep detection is the foremost application of these sensors [1–7]. Fusion of different sensors has also been considered for the same task [8, 9]. Many detection algorithms have been proposed in the past, but some of them place too much burden on computational resources and some of them are simply too complex to be implemented on a wireless sensor network, and yet some of them lack field tests. Field tests compromise of footsteps and vehicles at different ranges of the sensors.
Another critical aspect of footstep detection is the amount of analog signal gain that is applied to seismic data. Low-intensity seismic data require large amounts of amplification before being digitized. On one hand, high amplification is desired for increased range at the expense of increased noise level. On the other hand, low amplification is ideal for suppressing and identifying noise but has limited application range [10, 11]. Any signal processing algorithm must pay attention to amplification level as well as algorithm-dependent variables. In this study, we also study the impact of signal amplification on detection performance by analyzing data with different amplification values. Seismic data after being processed at the node, has been transferred to other wireless nodes or directly to the gateway to signal alarm conditions. Wireless sensor network nodes, illustrated in Figure 1, were specifically developed for this purpose, utilizing Texas Instruments transceiver and microcontroller family.
Pictures of test setup.
2. Test Setup
Field tests were performed at Yeditepe University, and test conditions are summarized in Table 1. Test site is shown in Figure 2. Two students with different body weights were chosen for footstep detection. Footstep, vehicle, and combined footstep-vehicle tests were used to test the detection algorithm under various signal amplification conditions.
Field test conditions.
The flowchart of the detection and alarm algorithm.
3. Detection Probability and Classification Performance
The detection algorithm was explained previously in [10]. Flowchart of the algorithm is given in Figure 2. The algorithm utilizes slow adaptive threshold (SAT) to identify ambient dynamic noise and quick adaptive threshold (QAT) to detect any disturbance in sensor readings. For field tests, first, noise-only data were collected with different gain values. Signal processing algorithm is expected not to produce any alarm to these noise-only data. Then, a large number of footstep tests were executed to assess the performance of the algorithm. Afterwards, vehicle tests were evaluated, and lastly, vehicle and footstep combined condition were assessed.
First, we define “True Detection”, “False Detection”, “Missed Detection”, and “Correct Rejection” according to Table 2. “true detection” means that the signal type is detected correctly. “Missed detection” refers to the condition that the present signal is not detected. “False detection” refers to incorrect detection; that is, if a vehicle signal is detected in footstep test, this detection is called “false detection.” “Correct rejection” condition occurs only if there is no signal and no detection.
Chart for detection performance.
Since detection is different than alarm, a similar table is formed for alarm conditions in Table 3.
Chart for classification performance.
“True Classification” means intruder type is classified correctly. “Missed classification” means intruder type is not classified. “False classification” means signal type is classified incorrectly; for example, if a vehicle signal was classified in footstep test, this classification is called “false classification”. “Correct Rejection in Classification” condition occurs only if there is neither intruder nor any alarm.
Specifying hits and false alarms is sufficient as the others (miss and correct rejection) can be easily extracted; that is, probability (Hit) equals to probability (Miss).
3.1. Noise-Only Tests
In noise-only tests, ambient environment is recorded and processed with 20 different gain values in the absence of any movement. Detection and classification results are presented in Table 4. A large range of gain values were tested. Gain values greater than 7.5 K were not tested as noise levels reach full ADC swing.
Noise-only data (no disturbance).
As it is seen in the Table 4, the signal processing algorithm did not produce any alarm or any vibration detection in noise-only condition at all amplification values.
3.2. Footstep Tests
Nine circles concentric to the seismic sensor were traversed by two different persons. The test setup is illustrated in Figure 3. In each test, 60 footsteps were taken (for larger radius circles, only some portion of the circle is traversed.). A total of 72 tests were formed according to Table 5 where combinations of different persons with different amplification values on test circles were evaluated.
Footstep test configurations.
Footstep test setup.
The classification result was “true” when only footstep alarm was generated, “false” if rain or vehicle alarm were triggered, and “miss” if no alarm occurred. The probability of each classification results according to different conditions was evaluated by using data from both persons. The results are presented in Figures 4 and 5 and Tables 6 and 7.
Footstep classification with 5 K gain.
Footstep classification with 7.5 K gain.
True classification rates of footsteps with different gain and range values.
False classification rates of footsteps with different gain and range values.
Classification performance can vary depending on gain and range. For >90% true classification rate and less than 10% false classification, 7.5 K amplification at 14 m provides best results. However, 7.5 K is still too large for vehicle tests, and if target detection range is lowered to less than 10 m, 5 K gain is as good as 7.5 K gain.
3.3. Vehicle Tests
Test setup for vehicle tests is shown in Figure 6. Vehicle tests were executed on a straight line with varying distances from the geophone sensor, 10 meters, 20 meters, and 30 meters. All lines are passed with two different vehicle speeds, 20 km/hr and 40 km/hr. Test IDs were formed as GXXXXSYYDZZ, where XXXX represents gain amount, YY represents speed, and ZZ represents distance to sensor. Tests were performed with 1 K and 5 K gains. Although 7.5 K gain produced very good results, we did not use that in vehicle tests due to increased noise levels, which in turn degraded the sensitivity of the system. Particular tests, with corresponding test IDs were given in Table 8, were performed.
Vehicle tests configurations.
Vehicle test paths.
Summary of the tests are given in Table 9. There were false vibration type detections. However, these false detections were filtered in the classification processes. Only 1 false classification was observed in 12 tests.
Vehicle classification with different test parameters.
3.4. Vehicle and Footstep Combined Tests
The test setup is illustrated in Figure 7. Vehicle with footstep tests were executed on two different lines at the same time. Walking path distance to geophone sensor was 5 meters, and vehicle path distance to sensor is 10 meters. On vehicle path, two different vehicle speeds (20 km/hr and 40 km/hr) were used; all tests were repeated twice.
Vehicle and footstep combined test configuration.
As before, test ID's were assigned with GXXXXSYYTZ where XXXX, YY, and Z represent gain, vehicle speed, and test number, respectively. The tests, all configurations were given in Tables 10 and 8, were performed.
Vehicle and footstep combined tests.
Here, false Alarm was defined as “Rain” alarm, and missed classification was defined as missing any one of the two alarms, either vehicle or footstep. Test results were summarized in Table 11.
Vehicle and footstep classification results.
There were false vibration type detections, but these false detections were filtered in the classification. Only one false classification was observed in eight tests.
4. Algorithm-Dependent Parameters
Detection algorithm heavily relies on QAT and SAT memory sizes, that is, number of samples where moving average is calculated. Thus, we studied detection performance by varying these critical memory sizes. Amplification was set to 5 K for all tests. True, false, and missed detection for various QAT memory sizes as a function of range are shown in Figures 8, 9, and 10. Same analysis was performed for SAT memory sizes and the results are presented in Figures 11, 12, and 13.
True footstep detection rate with different QAT memory sizes.
False footstep detection rate with different QAT memory sizes.
Missed footstep detection rate with different QAT memory sizes.
True footstep detection rate with different SAT memory sizes.
False footstep detection rate with different SAT memory sizes.
Missed footstep detection rate with different SAT memory sizes.
Best performance for true footstep detection was achieved when QAT memory size was 30 samples and SAT memory size was 20000 samples. Lower or higher than 30 samples QAT memory size prevents detection of footsteps due to oscillations in the footstep vibrations. Larger SAT memory sizes provide stability at the intruder movement moments. However, due to memory limitations of WSN, SAT memory size cannot be increased indefinitely.
5. Conclusion
Extensive field tests were performed to assess the performance of adaptive thresholds detection algorithm. Over 92 different test scenarios were performed, and results were evaluated in terms of detection performance. Performance tests also included the amplification amount of seismic sensor signals operating in a wireless sensor network. As higher amplification values lead to better detection range, they also increase noise level which, in turn, increases false alarm classification. For detection range under 10 m, 100% footstep classification with less than 5% false alarm rate was observed with 5 K gain. For detection range less than 15 m, 7.5 K gain produced about 95% hit rate and less than 10% false classification rate. Although the number of experiments with vehicle tests were limited, 5 K gain was also successful (nearly 68% true detection and no false detection). Algorithm-dependent constants such as QAT and SAT sample sizes were analyzed for best performance, and these values were used in detection and classification.