An effective method for the abnormal monitoring of stage performance based on visual sensor network

Abstract

Abnormal monitoring of stage performance plays a vital role in the stage performance. For the real-time stage performance, detection efficiency and accuracy are particularly important. As the traditional monitoring method based on sparse description model to realize abnormal behavior of stage performance did not realize the manifold structure during the performance, the behavior characteristics are sparse, and the decomposition has higher volatility, the recognition accuracy of abnormal behavior is low. Therefore, an abnormal monitoring method of stage performance based on visual sensor network is proposed, the overall structure of the abnormal monitoring system of stage performance based on the vision sensor network is analyzed, the hardware structure and software composition of the system are designed, and the method of monitoring the abnormal behavior of the system is analyzed emphatically. Through the background subtraction, the weighted threshold-based segmentation of the target image from the background image, the chaotic search particle swarm optimization algorithm based on image target detection and tracking algorithm for target tracking by mean shift, the abnormal behavior of local linear embedding and detection method based on sparse representation, a comprehensive analysis of the local manifold structure of sample is set. Enhance the stage performance of abnormal behavior detection efficiency and accuracy. The experimental results show that the proposed method has higher detection efficiency and accuracy and has higher robustness.

Keywords

Visual sensor network stage performing behavior abnormality monitoring method

Introduction

Abnormal monitoring of stage performance plays a vital role in the stage performance. Through monitoring and recognizing abnormal stage performance, the stage performers and the stage directors can be able to prevent the performers from leading up to the errors and ensure that the process of performance is working correctly. This article provides a clear and feasible solution different from the traditional monitoring method based on the visual sensor network.

This solution put into use the visual sensor technology in stage performance, which is based on computer vision image manipulation and pattern recognition and consists of checking the target and behavior in the stage performance and achieving intelligent monitoring of the target state, is of great significance to improve the performance of the actors. The traditional monitoring method based on sparse description model to realize abnormal behavior of stage performance did not realize the manifold structure during the performance, the behavior characteristics are sparse, the decomposition has higher volatility, and the recognition accuracy of abnormal behavior is low.^1–7

In order to solve the problem, this article proposes an effective method for the abnormal monitoring of stage performance based on visual sensor network. In this article, we proposed abnormal behavior recognition in the visual sensor network platform based on local linear embedding (LLE) and sparse description to realize the high efficiency and high accuracy of abnormal monitoring of stage performance. Figure 1 shows stage abnormal performance monitoring based on the visual sensor network technology.

Figure 1.

Stage abnormal performance monitoring based on the visual sensor network technology.

System architecture

As shown in Figure 2, this system is intended to construct the system architecture from three aspects: hardware architecture design, software module design, and algorithm application modeling.

Figure 2.

System implementation methods.

System design strategies

When image that does not match the predefined system appears in the process of the video monitoring of the stage performances, the system will automatically alarm and call the attention of the monitoring personnel. Then, monitoring personnel can immediately collect sample of the predefined exception behavior and take action correspondingly. Based on visual sensor network, design strategy of the abnormal monitoring system of stage performance is shown in Figure 3.

Figure 3.

Design strategy of the research on abnormal monitoring of stage performance based on visual sensor network.

It can be seen from Figure 3 that the system consists of image acquisition module, image manipulation module, and result operation module. The image acquisition module consists of charge coupled device (CCD) camera and video editor and is used for collecting stage performance images. The image manipulation module consists of network transmission and abnormal monitoring algorithm of stage performances, in which network transmission transmits the collected stage performance image data to the video central processor for processing and analyzing.⁸ The result operation module consists of result show and alarm system and outputs abnormal behavior of stage performance.

This article adopts H.264 as the video editor to transmit the stage performance image data by sensor network.

Composition of the system hardware

The composition of the system hardware is shown in Figure 4.

Figure 4.

Composition of the system hardware.

The system hardware mainly consists of PC, CCD camera, motor, and other control modules. After receiving order from the PC server, CCD camera begins to collect stage performance images⁹ and feed the collected image information to computer memory through the transmission equipment. Then, image data are presented by the image manipulation software and are used for judging whether there is an abnormal stage performance. Finally, the alarm system takes measures on the stage performance according to the test results.

Design of the system software

The key of the proposed system is the video analysis algorithm, which consists of image preprocessing, motion detection, target tracking, and abnormal behavior recognition.¹⁰ Operable control interface should be provided via system software to realize the analysis, acquisition, and manipulation of the stage performance.

Functional modules of the system software

This article uses the Windows Visual Studio 2008 to design the system software. The structure of the functional modules of the system software is shown in Figure 5. The modules are designed according to the system function to realize the image acquisition, image manipulation, and the result operation. Each module interacts with others through the image information flow. The video data receiving/transmitting module collects the information of the stage performance behavior and transmits the information through multicast mode. The image manipulation module is the key component of the system which provides pre-operation and abnormal behavior recognition of images on the basis that the exception behavior samples are given. The result operation module is used for storing the output results.

Figure 5.

Structure of the functional modules of the system software.

Process of the abnormal monitoring of stage performance

To monitor the abnormal stage performance with high efficiency and accuracy, detection and identification of the stage performance are of great significance. The flow process diagram of the abnormal monitoring of stage performance is shown in Figure 6.

Figure 6.

Flow process diagram of the abnormal monitoring of stage performance.

Target monitoring of stage performance is to identify the moving target from the background image given by fixed angle monitoring and denotes the original stage performance image series. After target monitoring, image preprocessing and noise reduction will be done to the moving target image to add clarity of the moving target and reduce the noise of the image. Then, the image will be segmented and the feature of stage performance will be extracted from the segmented target, which complete the process of the abnormal monitoring.

The abnormal behavior monitoring method mentioned is applied to the center of the city’s national song and dance in the stage performance, using PC, CCD camera, motors, and other control module acquisition stage image and H.264 digital video encoder shown in Figure 4, through the sensor network transfer stage image data, sending them to the computer memory. After the video image input, the function module of the system software is used to preprocess the image and reduce noise processing, segmentation image target, and extracting image features. According to the image movement target characteristics, the abnormal behavior monitoring and recognition of stage performance is realized.

Concrete proposal

The monitoring stage scene has variability. If the background is not changed in time, it will have an impact on the target monitoring of the stage performance. So it is necessary to update the background in real time. This article adopts the method combined background subtraction with the weighted threshold value factor,¹¹ which is able to filter the particle swarm of chaos searching to find the parameters of the weighting factor.

Segmenting target based on the weighted threshold value background subtraction method

Set the pixel of the stage perform digital image to be M × N. The background image sequence is set to be $f' (x, y) = {f'_{i} (x, y), i = 1, 2, \dots, n}$ , and the performance image sequence is $f (x, y) = {f_{i} (x, y), i = 1, 2, \dots, n}$ . Then, the computation of the threshold value method is done in two steps:

The average $\bar{f' (x, y)}$ of the background image sequence is given by the overall planning method. The background image sample of the initial stage is

\bar{f' (x, y)} = \frac{1}{n} \sum_{i = 1}^{n} {f'}_{i} (x, y)

(1)

The background image of the next frame is $f'_{i + 1} (x, y)$ and denote by $D (x, y)$ the update. On the basis of the background image $\bar{f' (x, y)}$ of the initial stage, define the threshold value $σ$ . The updated background image is

D (x, y) = | {f'}_{i + 1} (x, y) - \bar{f' (x, y)} |

(2)

If $D (x, y) > σ$ , which means the background image changes frequently, then the background should be updated in time. If $0 \leq D (x, y) \leq σ$ , then the noise may affect the clarity of the background image. The problem that the noise may influence the result can be solved using the weighting factor.¹⁰

The value of $σ$ directly influences whether the background should be updated. If the value of $σ$ is too small and there is too much noise, the noise will be mistaken for the update image and make the background image blur. However, if the value of $σ$ is too large and the pixel of the update image is stable, part of the update image will be lost. The threshold value method can determine whether the background image should be updated. However, the update data may contain the noise, which means that the threshold value is influenced by other factors.¹² This article uses the weighted factor and update the background image through the control of the weighted factor.¹³

Denote by $λ$ the weighting factor and by $g (x, y)$ the sum pixel of updated background image m × n of frame i + 1. Then, the threshold value $σ$ can be represented as

σ = λ * \frac{g (x, y)}{\bar{f' (x, y)}}, 0 < λ < 1

(3)

Compare the pixel of the current background image with the pixel of the initial background image and multiply by the weighted harmonic to get the threshold value. Then, the problem that the pixels change with the noise is solved and the clarity of the background image of the sample is retained to an extreme.

Moving target monitoring using particle swarm optimization algorithm based on chaos searching

When target detection of stage performance behavior is in progress, the chaos searching algorithm is integrated into the particle swarm optimization algorithm, that is, if the locally optimal solution problem appears in the particle swarm optimization algorithm,¹⁴ the chaos searching algorithm would be adopted to obtain the global optimal location. The locally optimal solution will be filtered out and the locally optimal solution problem is assured to not appear, which enhances the convergence efficiency of the algorithm. The process is shown in Figure 7:

The initialization of all the parameters: denote by $c_{1}$ and $c_{2}$ the learning factors, by T_max the maximum number of the iteration, by U the control parameter, and by T the number of initial iteration of the chaos searching algorithm to initialize the particle swarm, including random positions and velocities.

Evaluate the adaptation of all the particles to the targets. Lock the optimal location P_j that the jth particle has found and the optimal location P_g of the entire particle swarm up to now.

Combine equations (1) and (2) to imply iteration for particles in the particle swarm. If the current optimal individual is in the scope of the convergence condition or the maximum number of iterations has been reached, run process (5).

If the optimal location P_g of the entire particle swarm has not changed or only slightly changed after T times iteration, denote $X'_{j} = P_{g}$ and the optimal value $X'_{j}$ is available using the chaos searching algorithm, then run the process (3) for the follow-up particle swarm algorithm. Otherwise, run process (5).

The evolutionary process is finished and the global optimal solution is returned. The fitness function of the uniformity measure optimization¹⁵ is introduced to get the optimal threshold value for image segmentation. Assume that the image is divided into two parts with threshold value t, where the two parts are represented by $R_{I}$ , $I = 1, 2$ . Then, the uniformity measure is

UM (t) = 1 - \frac{σ_{I}^{2} + σ_{2}^{2}}{C_{0 l}}

(4)

where $σ_{I}^{2} = \sum_{(x, y) \in R_{I}} {(f_{i} (x, y) - U_{I})}^{2}$ . Denote by $f_{i} (x, y)$ the gray value of the image region $(x, y)$ . Denote

μ_{I} = \frac{\sum_{(x, y) \in R_{I}} f_{i} (x, y)}{A_{I}}

Denote by $A_{I}$ the number of pixel value in the region $R_{I}$ , and by $C_{0 l}$ the normalized constant of $UM (t)$ . Then, the optimal parameter $λ$ satisfies $λ * = \arg max UM (t * (λ))$ .

Figure 7.

Particle swarm optimization algorithm for moving target monitoring flowchart.

Target tracking of the stage performance based on the mean shift algorithm

This article uses the mean shift algorithm to realize the target tracking of the stage performance, which is able to deal with the occlusion problem of moving targets and the problem of information missing. The edge feature of the moving target is collected with LOG operator detection method and is tracked using the mean shift algorithm. Then, the subsequent position prediction is done basing on Kalman filtering. The mean shift algorithm provides non-parametric prediction using density gradient. It provides efficient iteration via optimization of the probability density function of the pixel feature point when tracks the moving target frame and can obtain the local maximum of the density function,¹⁶ complete high-efficiency orientation of the moving target, and adapt to the complex motions of the stage performance target such as deformation and rotation.

Set $d$ as there are m sample points $x_{J}$ , $J = 1, 2, \dots, m$ in space $R^{d}$ . Then, the mean shift vector on point x is

M_{h} (x) = \frac{1}{k} \sum_{x_{J} \in S_{h}} (x_{J} - x)

(5)

where $(x_{J} - x)$ is the offset vector from $x$ to $x_{J}$ , $S_{h}$ is a high-dimensional globular region with radius $h$ , and the vector $M_{h} (x)$ is the average vector of the offset vectors from $x$ to the $k$ sample points in the region $S_{h}$ . Denote by $y$ the set of the sample points in $S_{h}$ , which is given by

S_{h} = {y : {(y - x)}^{T} (y - x) \leq h^{2}}

(6)

Described by $k$ that there is $k$ sample points in $S_{h}$ among the m sample points $x_{J}$ . If the sample points $x_{J}$ are obtained from the probability density function $f (x)$ , most sample points in $S_{h}$ distribute along the probability density gradient due to the average algorithm and the fact that the non-zero probability density gradient is in accordance with the direction that the probability density increases greatest. Thus, the offset vector $M_{h} (x)$ based on the mathematical analysis average is in accordance with the trend that the sample points increases, which is also the gradient direction of the probability density function. Here, the big circle represents the region $S_{h}$ , the small circles represent the ample point $x_{J} \in S_{h}$ , the datum point X of the mean shift algorithm is described by the center dark spot and the offset vectors of the sample points from the datum point X are described by the arrows.

The mean shift algorithm draws the colored histograms of the target model and the alternative model in the first place and then realizes the target tracking after analyzing the similarity of the two models. The flow is shown in Figure 8.

Figure 8.

The stage performance target tracking based on mean shift algorithm.

The modeling of the target

Use stage performance detection to divide the moving targets into several regions. The regions are selected according to the gray or colored histograms of the moving targets. If the center of the object is $x_{o}$ , denoted by $x_{J}$ each pixel points. The eigenvalues are $μ = 1, 2, \dots, m'$ . Denote by ${\hat{q}}_{μ}$ the probability of the eigenvalue in corresponding region

\begin{matrix} {\hat{q}}_{u} = C \sum_{J = 1}^{m} k (‖ \frac{x_{J}^{s} - x_{o}}{h} ‖) δ [b (x_{J}^{s}) - u] \\ C = \frac{1}{\sum_{J = 1}^{m} k ({‖ \frac{x_{o} - x_{J}}{h} ‖}^{2})} \\ {\hat{p}}_{u} (y) = C_{h} \sum_{J = 1}^{m_{h}} k ({‖ \frac{y - x_{J}}{h} ‖}^{2}) δ [b (x_{J}) - u] \end{matrix}

(7)

The chosen of the candidate target

After the moving object detection, the target region that moves more intensely will be regarded as the candidate target. The center coordinates of the target range kernel function is y and the sum of the pixel points in the target region is $m_{h}$ . Then, the probability density function of the uth eigenvalue of the candidate model is

{\overset{⌢}{p}}_{u} (y) = C_{h} \sum_{J = 1}^{m_{h}} k ({‖ \frac{y - x_{J}}{h} ‖}^{2}) δ [b (x_{J}) - u]

(8)

in which

C_{h} = \frac{1}{\sum_{J = 1}^{m_{h}} k ({‖ \frac{y - x_{J}}{h} ‖}^{2})}

is a normalization constant. Then, the target can be regarded as the process to get a optimal y and guarantee the maximum similarity of ${\overset{⌢}{p}}_{u} (y)$ and ${\overset{⌢}{q}}_{u}$ .

The likelihood function

Compute ${\overset{⌢}{q}}_{u}$ of the target model and ${\overset{⌢}{p}}_{u} (y)$ of the candidate model and analyze the distribution of the target model via the Bhattacharyya coefficient ${\overset{⌢}{q}}_{y}$ of the likelihood function. Then

{\overset{⌢}{ρ}}_{y} \equiv ρ [p (y), q] = \sum_{u = 1}^{m^{'}} \sqrt{p_{u} (y) {\overset{⌢}{q}}_{u}}

(9)

The value range of ${\overset{⌢}{q}}_{y}$ is [0, 1]. The greater the value is, the higher the similarity between the ${\overset{⌢}{q}}_{u}$ model and the ${\overset{⌢}{p}}_{u} (y)$ model is. The Taylor expansion of equation (9) at point ${\overset{⌢}{p}}_{u} ({\overset{⌢}{y}}_{0})$ implies

ρ [p (y), q] \approx \frac{1}{2} \sum_{u = 1}^{m'} \sqrt{p (y_{0}) q_{u}} + \frac{1}{2} \sum_{u = 1}^{m'} p_{u} (y) \sqrt{\frac{q_{u}}{p_{u} (y_{0})}}

(10)

Putting equation (9) into equation (10) implies

ρ [p (y), q] \approx \frac{1}{2} \sum_{u = 1}^{m'} \sqrt{p (y_{0}) q_{u}} + \frac{C_{h}}{2} \sum_{J = 1}^{m} w_{J} k ({‖ \frac{y - x_{J}}{h} ‖}^{2})

(11)

w_{i} = \sum_{u = 1}^{m'} δ [b (x_{J}) - u] \sqrt{\frac{q_{u}}{p_{u} (y_{0})}}

(12)

The computation of the offset vector

The optimal mean shift vector is computed using the Bhattacharyya coefficient. The vector is the change of position of the moving target in stage performance. The optimal position $y_{1}$ of the target in the follow-up frames is given by iteration

y_{1} = \frac{\sum_{J = 1}^{m} x_{J} w_{J} g ({‖ \frac{y - x_{J}}{h} ‖}^{2})}{\sum_{J = 1}^{m} w_{J} g ({‖ \frac{y - x_{J}}{h} ‖}^{2})}

(13)

The abnormal behavior detection method based on LLE and sparse description

Method and procedure

The detection of the abnormal behavior in stage performance can be seen as the classification process of the time-varying data, which compares the input sequence with the set of reference behavior sequence. The abnormal behavior in stage performance is a complex abnormal behavior, which needs the auxiliary model to describe the behavior of stage performance as comparison to complete the accurate recognition of abnormal behavior in stage performance. Since the behavior characteristic of stage performance has three properties: high-dimensional, large amount of data and dynamic change of the local manifold structure, the characteristic recognition of the abnormal behavior is completed with the abnormal behavior detection method based on LLE and sparse description. This kind of method gathers the characteristics of the movement via optimized spectroscopic flow algorithm, which divides each frame image into several blocks, collects the low-dimensional behavior characteristics, classifies the behavior characteristics via LLE and sparse description,¹⁷ and deals with the dynamic change of the local manifold structure through adding the LLE regular term to the sparse classification model. The identification function of the samples is greatly enhanced since the regular term of the LLE retains the local manifold decoupling of the test samples to the maximum. The procedure of the algorithm is shown in Figure 9.

Figure 9.

Procedure of the abnormal behavior detection method based on local linear embedding and sparse description.

Sparse description of the stage performance behavior

Sparse description algorithm based on $l_{1} - norm$

Set M classes with marked difference. Assume that there is $N_{i}$ training samples in the ith class, then the training samples are $D^{i} = [α_{1}^{i}, \dots, α_{N_{i}}^{i}] \in R^{d \times N_{i}}$ . The same class of features approximate in a low-dimensional subspace. Assume that a test sample y is m-dimensional and is in the ith class, then y can be described with the linear combination of the training samples of $D^{i}$ , that is

y = d_{a}^{i} a_{1}^{i} + \dots + d_{N_{i}}^{i} α_{N_{i}}^{i} = D^{i} α^{i}

(14)

in which $α^{i}$ is the simplified description of y in $D^{i}$ , and y has a sparse description with the dictionary constructed by $D = [D^{1}, \dots, D^{M}] = [d_{1}, \dots, d_{N}] \in R^{d \times N}$ . The sparse description of y is

a = \min_{a} {‖ y - D a ‖}^{2} + λ^{'} {‖ a ‖}_{1}

(15)

in which $a$ is the sparse description coefficient of y in the dictionary D and $λ'$ is the restraint sparse coefficient.

Sparse description based on LLE

LLE is a kind of manifold learning algorithm. The algorithm assumes that the samples are obtained from smooth manifold. A sample in low-dimensional area can be described with the linear combination of several neighboring points, and the algorithm ensures that the local linearity of the high-dimensional samples is steady. If the data set of the stage performance is $Y = [y_{1}, y_{2}, \dots, y_{n}]$ , basing on LLE, assume that $y_{i}$ can be described with the linear combination of the nearest neighbor in the test samples and the sparse description coefficient of $y_{i}$ can be linear description performed by the same weight of the neighbor. Then, the quadratic limit of LLE is

\sum_{i = 1}^{n} {‖ a_{i} - \sum_{j \in N (y_{i})} v_{ij} α_{j} ‖}^{2}

(16)

in which $v_{ij}$ is the refactoring weight of $α_{j}$ , $N (y_{i})$ is the nearest neighbor of $y_{i}$ and

v_{ij} = \arg min_{v_{j}} ‖ y_{i} - \sum_{j \in N (y_{i})} v_{ij} y_{j} ‖^{2} s . t . \sum_{j \in N (y_{i})} v_{ij} = 1

(17)

Then, formula (17) can be rewritten as

\sum_{i = 1}^{n} {‖ a_{i} - \sum_{j \in N (y_{i})} v_{i j} α_{j} ‖}^{2} = {‖ A - AV ‖}^{2} = {‖ {(A - A V)}^{T} ‖}^{2} = T r (A (I - V) {(I - V)}^{T} A^{T}) = Tr ({AMA}^{T})

in which I is the unit matrix and $M = (I - V) {(I - V)}^{T}$ is according LLE matrix. Then

V_{ji} = {\begin{matrix} v_{ji}, & If y_{j} is in the k - nearest samples of y_{i} \\ 0, & Otherwise \end{matrix}

Plug equation (18) into the sparse description implies the sparse description based on LLE

{min}_{A} ‖ Y - DA ‖^{2} + λ_{0} ‖ A ‖_{1} + λ_{1} Tr ({AMA}^{T})

(19)

in which $λ_{0}$ and $λ_{1}$ are regularization parameters.

The sparse description coefficient

The $α_{i}$ in equation (19) can be described with

{min}_{a_{1}, a_{2}, \dots, a_{n}} \sum_{i = 1}^{n} {‖ y_{i} - D a_{i} ‖}^{2} + λ_{1} \sum_{i, j = 1}^{n} M_{i, j} a_{i}^{T} a_{j} + \sum_{i = 1}^{n} λ_{0} {‖ a_{i} ‖}_{1}

(20)

If there exists stable vector ${a_{j}}_{j \neq i}$ , the procedure of solving the problem can be described with

{min}_{a_{i}} g (α_{i}) = ‖ y_{i} - D a_{i} ‖^{2} + λ_{1} M_{ii} α_{i}^{T} α_{j} + α_{i}^{T} h_{i} + λ ‖ a_{i} ‖_{1}

(21)

In which $h_{i} = 2 λ_{1} (\sum_{j \neq i} M_{ij} a_{j})$ and $a_{i}^{(j)}$ is the jth component of $a_{i}$ . This article uses feature-sign searching algorithm to solve equation (21) and defines

f (α_{i}) = ‖ y_{i} - D a_{i} ‖^{2} + λ_{1} M_{ii} α_{i}^{T} α_{i} + α_{i}^{T} h_{i}

(22)

Then, $g (α_{i}) = f (α_{i}) + λ ‖ α_{i} ‖_{1}$ .

The detection of the abnormal behavior in stage performance

The category of the test samples can be gotten basing on the computed sparse description coefficient $a^{i}$ . Define the ith reconstruction error, which is the error of the test samples caused by the ith reconstruction

r^{i} (y) = ‖ y - D^{i} a^{i} ‖, i = 1, \dots, M

(23)

in which the non-zero coefficient in $a^{i}$ describes the atom in corresponding category. The minimum residual error algorithm to get the category of y is

class (y) = \arg {min}_{i = 1, - M} r^{i} (y)

(24)

Due to the high-dimension disturbance of the performance characteristics of the stage performance, the marking of the performance training samples in the real detection process is complicated. In this situation, equation (23) is over-determined. Raising the dimension of D and using the $d \times d$ unit matrix imply the underdetermined function

y = {min}_{α} ‖ y - B α ‖^{2} + λ ‖ a ‖_{1}

(25)

in which

B = [\begin{matrix} D & I \end{matrix}] \in R^{d \times (N + d)}, a = [\begin{matrix} α^{T} & η^{T} \end{matrix}]^{T} \in R^{(N + d)}

and $η \in R^{(N + d)}$ is the error vector. Equation (25) ensures that equation (23) has been turned into underdetermined equation and errors can exist in equation (23). Then, the objective function with the error vector is

R^{i} (y) = ‖ y - D^{i} α^{i} - {\overset{⌢}{η}}^{i} ‖, i = 1, \dots, M

(26)

The minimum residual error algorithm to get the category of y is

class (y) = \arg {min}_{i = 1, - M} R^{i} (y)

(27)

The above sparse description classification algorithm cannot give accurate description of the local manifold structure of the sample of the stage performance behavior.¹⁸ In order to enhance the classification accuracy of the algorithm, this article adopts the sparse description method which is based on LLE to increase the accuracy of the detection of the stage abnormal behavior. The specific procedure is as follows.

Input: the training sample matrix

D = [D_{1}, D_{2}, \dots, D_{i}] \in I R^{m \times n}

with M categories in total. The test sample

y \in [y_{1}, y_{2}, \dots, y_{n}]

For $1 \leq i \leq m$

Step 1. Take normalization of the training samples and the test samples.

Step 2. Solve the sparse description problem basing on LLE and get the sparse description coefficient $a^{i}$ .

Step 3. Compute the error $R^{i} (y)$ .

end for

Output: $c l a s s (y) = {argmin}_{i = 1, - M} R^{i} (y)$ .

Experimental analysis

Analysis of feature recognition rate of abnormal behavior of stage performance

Experiment 1

As shown in Figure 10, the data are the stage performance video in a city’s national art center taken by Sony HVR-V1C camera. The size of each frame of the video is 360 × 240. There are a large number of abnormal stage performance behaviors such as falling done, pauses, stomping, wrong directions, and not enough height of jumping. To test the effectiveness of the proposed method, the method is employed after giving the video sequences of all the abnormal stage performance behaviors in the same background scene. Collect the 6 performance behaviors in Table 1 from 10 videos as samples. Take 65% of the feature set as training set and 35% as test set and obtain the training dictionary from the training of sparse model. After obtaining the sparse description dictionary and detecting the characteristics of the stage performance behavior from the video, classify the behavior characteristics through LLE and sparse description. The experiment uses 300 frames of the video as training sample and 150 frames of each kind of behavior as test sample.

Figure 10.

The extracting result of the stage performance characteristics.

Table 1.

Training and test samples of the video sequence.

No.	Name	Training (frames)	Test (frames)
1	Normal	300	150
2	Falling done	300	150
3	Pauses	300	150
4	Stomping	300	150
5	Wrong directions	300	150
6	Not enough height of jumping	300	150
Total		1800	900

The experiment uses the Windows 7 laptop, with the configuration is Intel^® Core™ Duo CPU T6500 3.2 GHz, 4 GB memory. The experimental software platform is MATLAB 7.2.1.

The experiment gives the comparison analysis of the monitoring method in this article, and the traditional method based on sparse description algorithm. The recognition rates of six behavior characteristics are shown in Table 2. It can be seen from Table 2 that the recognition rates of the proposed method in this article are higher than the traditional method.

Table 2.

Recognition rates of the traditional method and the method in this article.

Type	Traditional method	The method in this article
Normal	77.1	83.7
Falling done	79.8	85.6
Pauses	82.3	88.7
Stomping	85.6	92.5
Wrong directions	83.7	93.7
Not enough height of jumping	80.5	95.8

The data in Table 1 include solo performance and multiplayer interactive performance. Solo performances of 25 persons are taken together with one or two multiplayer interactive performances for each two persons. The training results of the stage performance behavior with traditional method are shown in Figure 11. Desired error is achieved if the training times is 140. For comparison, Figure 12 gives the training results of the proposed method in this article, which achieves the desired error when the training times is 60. It is clear that the proposed method in this article is more efficiency.

Figure 11.

Training results of the traditional method (horizontal line: the desired error).

Figure 12.

Training results of the method in this article (horizontal line: the desired error).

Experiment 2

Take the D-Harris interest points as the characteristics of the stage performance. Apply the proposed method in this article to some YouTube data bank of stage performance behavior to monitor abnormal behavior. The recognition rates of different abnormal characteristics using the proposed method in this article is given in Table 3, in which HNF abnormal stage performance characteristics is the combination of histogram of oriented gradient (HOG) and histogram of optical flow (HOF) characteristics. It can be seen from Table 3 that the HNF characteristics are with the highest recognition rate of 83.36%. The recognition rate of HOG characteristics is 77.91% and the recognition rates of motion boundary histogram (MBH), HOF, and IMHCD characteristics are lower than 75%.

Table 3.

Recognition rates of different abnormal characteristics with the proposed method in this article.

Recognition rates	HNF	MBH	HOG	HOF	IMHCD
10	79.5	71.8	76.6	76.8	78.9
9	89.7	70.5	82.5	69.8	79.8
8	82.6	65.8	82.6	73.4	72.5
7	82.8	68.4	73.8	70.8	72.4
6	79.6	69.2	71.9	73.8	74.2
5	80.2	71.3	79.6	78.5	78.9
4	85.7	72.6	81.5	75.6	74.2
3	82.6	68.7	76.4	70.8	69.5
2	81.4	68.5	77.8	76.2	73.8
1	89.5	69.2	76.4	75.8	76.8
Average	83.36	69.6	77.91	74.15	75.1

MBH: motion boundary histogram; HOG: histogram of oriented gradient; HOF: histogram of optical flow.

To test the influence of the quantity of samples to the recognition rates, Table 4 gives the recognition rates of the HNF abnormal characteristics using the proposed method in this article with different quantities of samples.

Table 4.

Recognition rates (%) of the HNF abnormal characteristics with different quantities of samples.

Recognition rates	20	100	150
10	84.6	93.5	94.5
9	83.6	88.8	95.6
8	85.6	92.8	96.7
7	86.4	91.2	95.7
6	86.7	89.6	98.4
5	85.2	90.8	95.7
4	88.7	91.8	97.4
3	86.7	96.4	95.7
2	84.5	93.5	96.5
1	85.7	95.2	96.8
Average	85.77	92.36	96.3

It can be seen from Table 4 that the quantity of samples has influence on the recognition rate. The recognition rate increases with the increase in the quantity of samples. The average recognition rate is 96.3% when the quantity of samples is 150 and the average recognition rate is 92.36% when the quantity of samples is 100, which shows the efficiency and robustness of the proposed method.

Conclusion

This article proposes the abnormal monitoring method of stage performance based on visual sensor network. The analysis is based on the structure of the proposed method and gives the design of the hardware structure and software system.^16–20 The local manifold structure of the sample is analyzed using the abnormal behavior detection method, which is based on LLE and sparse description. That improves the efficiency and accuracy. From the comparison of the results of design experiments (Table 2 –4, Figures 11 and 12), it can be seen that this method can improve the detection efficiency and accuracy of stage performance abnormality to a greater extent than the traditional method. In the future, it is proposed to introduce a deep learning algorithm to further improve the detection efficiency and accuracy of abnormal stage behavior.

Footnotes

Handling Editor: Jeng-Shyang Pan

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Research Program Foundation of Minjiang University under grant no. MYK17021, the Major Project of Sichuan Province Key Laboratory of Digital Media Art under grant no. 17DMAKL01, and also the Fujian Province Guiding Project under grant no. 2018H0028. This work was supported by the National Natural Science Foundation of China (61772254), Key Project of College Youth Natural Science Foundation of Fujian Province (JZ160467), Fujian Provincial Leading Project (2017H0030), Fuzhou Science and Technology Planning Project (2016S-1116), Program for New Century Excellent Talents in Fujian Province University (NCETFJ), and Program for Young Scholars in Minjiang University (Mjqn201601).

References

Wang

Zhao

Chang

. Obstacle detection based on visual sensor. Comput Eng Appl J 2015; 51(4): 180–183.

Kuang

. Monitoring method based on IoT garden vision research and simulation. Comput Simul 2016; 33(4): 447–450.

Chang

F-C

Huang

H-C

. A survey on intelligent sensor network and its applications. J Netw Intell 2016; 1(1): 1–15.

Nguyen

T-T

Chu

S-C

Dao

T-K

et al . An optimal deployment wireless sensor network based on compact differential evolution. J Netw Intell 2017; 2(3): 263–274.

Myroniv

C-W

Ren

et al . Analyzing user emotions via physiology signals. Data Sci Pattern Recog 2017; 1(2): 11–25.

Hsieh

C-H

Hung

M-H

Weng

S-K

. Visual object tracking based on color and implicit shape features. J Inform Hid Multim Signal Process 2018; 9(1): 198–210.

Zhao

Wang

. Re-expression of face images with help of samples from other face datasets. J Inform Hid Multim Signal Process 2017; 8(4): 818–822.

Zhang

. Research and design of wireless sensor network energy model based on Semi-Markov chain. Electr Design Eng 2016; 24(11): 95–98.

Tong

Chen

et al . Dual-band stereo vision based on heterogeneous sensor networks. Signal Process 2016; 126: 87–95.

10.

Lin

. Sensorless non-contact mechanical arm collision estimate based on the disturbance of thinking. Bull Sci Technol 2016; 32(6): 104–108.

11.

Frontoni

Mancini

Zingaretti

. Embedded vision sensor network for planogram maintenance in retail environments. Sensors 2015; 15(9): 21114–21133.

12.

Wang

. Design of target tracking system based on visual sensing network. Mod Electr Tech 2016; 39(8): 88–91.

13.

Chen

Yang

. Research on multi-dimensional database activity monitor. J Univ Electr Sci Technol China 2015; 44(2): 266–271.

14.

Feng

Fukuda

Feng

et al . Nontarget vision sensor for remote measurement of bridge dynamic response. J Bridge Eng 2015; 20(12): 04015023.

15.

Dai

Wang

Tan

et al . Self-adaptive calibration of hybrid visual sensor networks for pedestrian detection. Chin J Sci Instrum 2016; 37(3): 683–689.

16.

Wang

Shen

et al . Research of multi-sensor data fusion based on binocular vision sensor and laser range sensor. Key Eng Mat 2016; 693: 1397–1404.

17.

Zhang

. Wireless sensor network coverage of the network topology design method. Sci Technol Eng 2016; 16(25): 126–130.

18.

Luo

Tang

Zhao

et al . Abnormal behavior detection for elderly based on 3D structure light sensor. Optic Tech 2016; 42(2): 146–151.

19.

. Research and design of quality control system based on wireless sensor network. Chin J Power Source 2014; 38(2): 373–374.

20.

Wang

Liu

et al . Abnormal human body behavior recognition using pose estimation. Chin J Sci Instrum 2016; 37(10): 2366–2372.

An effective method for the abnormal monitoring of stage performance based on visual sensor network

Abstract

Keywords

Introduction

System architecture

System design strategies

Composition of the system hardware

Design of the system software

Functional modules of the system software

Process of the abnormal monitoring of stage performance

Concrete proposal

Segmenting target based on the weighted threshold value background subtraction method

Moving target monitoring using particle swarm optimization algorithm based on chaos searching

Target tracking of the stage performance based on the mean shift algorithm

The modeling of the target

The chosen of the candidate target

The likelihood function

The computation of the offset vector

The abnormal behavior detection method based on LLE and sparse description

Method and procedure

Sparse description of the stage performance behavior

Sparse description algorithm based on l 1 - norm

Sparse description based on LLE

The sparse description coefficient

The detection of the abnormal behavior in stage performance

Experimental analysis

Analysis of feature recognition rate of abnormal behavior of stage performance

Experiment 1

Experiment 2

Conclusion

Footnotes

Declaration of conflicting interests

Funding

References

Sparse description algorithm based on $l_{1} - norm$