Smart urban management identification by integrating optical flow characteristics and HSV space

Abstract

n smart cities, sanitation workers are the key to urban construction. Accurate targeting of sanitation workers can help managers better monitor and manage them. Pedestrian detection is the core and key component of object detection technology. The difficulty of pedestrian detection in the actual feature recognition is still how to quickly and accurately identify the identity in the complex video scene. To realize the effective detection of sanitation workers, the study designs an identity identification scheme conducive to the friendly management of smart urban management. Since the optical flow feature extraction method and HSV color space extraction method can not meet the actual detection efficiency, this study innovatively integrates the two methods to improve the detection accuracy of the mode. Meanwhile, the study also introduces PCA algorithm to identify the specific identity of sanitation workers. In the actual detection of sanitation workers, the identification rate of two sanitation workers is high, and the similarity is 98.61%. This technology greatly reduces the false detection rate of actual detection and improves the detection accuracy.

Keywords

Smart urban management identification optical flow characteristics HSV space technology integration PCA

1. Introduction

The process of urban intelligence is accelerating, among which sanitation workers is essential in urban construction. Target identification of sanitation workers is helpful for managers to better supervise and manage them. In the actual construction of smart cities, the corresponding equipment and systems should be updated, and the detection and management of sanitation workers should be optimized, which can also better serve modern life. Therefore, it is very necessary to use modern intelligent means to effectively identify the identity of sanitation workers. Based on the progress of science and technology, intelligent transportation system plays an important role in modern life [1]. Among them, pedestrian detection and face recognition are the core technologies of modern intelligent transportation systems, which have important research and application value [2]. Traditional intelligent traffic target detection methods rely on manual detection, which is inefficient and unfavorable to the efficient operation of the system. The early intelligent algorithm detection methods are greatly affected by the detection environment, which are prone to high false detection rate and missing detection rate. Therefore, it is an urgent problem to further optimize pedestrian detection technology to improve the operating efficiency of intelligent transportation system [3]. Currently, many researchers have proposed some classical pedestrian detection methods, which have greatly promoted the development of pedestrian detection technology. However, most methods still have great shortcomings in terms of robustness, real-time performance and stability, which need to be improved [4, 5]. Therefore, to better detect sanitation workers, this study constructs a detection technology based on optical flow characteristics and HSV spatial fusion. The system can accurately detect and identify sanitation workers, which has important social and economic value for the benign development of modern transportation system. The innovation of this study lies in the integration of optical flow characteristics and HSV spatial detection algorithm to build a sanitation worker detection model. Meanwhile, PCA method is proposed to extract the identity characteristics of sanitation workers. This method combines the advantages of optical flow feature that can remove background noise and the related advantages of HSV color space conversion, whose core idea is to convert the obtained optical flow threshold segmentation map to HSV color space to complete HSV threshold segmentation and fusion at the same time.

The study consists of five main components. First, the literature review part discusses the research status of the technical means. Second, a pedestrian target detection model with optical flow method and HSV is constructed. Thirdly, an identity recognition model based on PCA is proposed. Fourthly, the effectiveness of the suggested method is verified. Thirdly, the paper summarizes the content of the full text and looks forward to the future research direction.

2. Related work

The basic idea of the optical flow feature detection method is to use the information of the current image and its adjacent image, so that the image of each pixel is oriented towards the velocity vector, so as to obtain a two-dimensional image. That is, in the imaging state, the moving brightness of the object is expressed by optical flow. The optical flow method has a good detection effect when the detected object and the camera move me or there is a blocking phenomenon. In recent years, this method has been favored by scholars and widely discussed. Caldelli and other researchers proposed a new video forensics method. The method was based on CNN and used optical flow field to distinguish possible motion differences in the time structure of video sequences. The proposed method could effectively distinguish between fake and original video sequences, and it also showed excellent robustness in cross-forgery operation scenarios, and improved the global effectiveness of the detection scheme [6]. From the perspective of anomaly video monitoring, Dong et al. suggested a semi-supervised method of generating adversarial network structure with double discriminator. This method integrated more information of target motion, that was, the generator predicted future frames of normal events, and the discriminator generated more realistic continuous frames. The discriminator in the model could determine the true or false optical flow [7]. Paredes-Valles et al. constructed an unsupervised learning model for layered peak neural networks for light flow estimation. The model could capture temporal and spatial information meanwhile to distinguish abnormal behavior in surveillance video. In this model, the time sequence capture module extracted the key information of human body by simulating the generation of optical flow, so as to realize the recognition of human body. This model had good generalization ability and could be used to measure human movement behavior [8]. Ullah et al. proposed an activity recognition model based on time-optical flow convolutional features and multi-layer LSTM. A new method combining optical flow field detection with chroma saturation was proposed. The algorithm proposed in this scheme could effectively detect abnormal behaviors in abandoned surveillance videos based on numerical color space [9]. Almatrafi and other researchers constructed a new optical flow detection method for neuromorphic cameras. The method used a dynamic vision sensor to report the logarithmic intensity change of any pixel above the reservation threshold, while introducing the concept of "distance surface" to be used as an input to the intensity based optical flow method. The method could estimate the Angle and velocity of each abnormal event with high accuracy [10].

HSV (Hue, Saturation, Value) color space is proposed based on the intuitiveness of color, which is mainly evolved from RGB color space. Where “H” represents the hue; “S” stands for saturation; “V” stands for brightness. Currently, many researchers have discussed HSV technology. Ito and other researchers built a system, which could assist visually impaired people to walk independently. The system depended on the dynamic threshold method in HSV color space to enhance the robustness of image processing under changing lighting, environment and color conditions. The system could warn users of sidewalks and collision hazards, and it could achieve a high tactile pavement detection rate under various conditions, which had strong practical significance [11]. In response to the problems existing in the production process of the textile industry, Guan proposed a method for detecting textile defects based on visual salience. The algorithm had the ability to automatically detect the defects of unpatterned and patterned fabrics, and used HSV to detect the defects of fabrics. Experiments showed that the proposed method was more accurate and effective than other existing methods [12]. Hassan and Gutub proposed two high-payload embedding methods using a hue saturation value (HSV) color model to reduce image distortion. The proposed method was based on tonal embedding (HBE), which used the H-plane to hide one or two bits in non-grayscale pixels. The method scored highest among six advanced techniques in red-green-blue, HSV, hue saturation intensity, and YCbCr space, five of which had the highest average peak signal-to-noise ratio. [13]. The model proposed by Yan and Woç„ iak adopted the non-uniform quantization HSV space method and introduced one-dimensional eigenvector to reduce the influence of lens transformation. The accuracy and recall rate were higher than 99% [14]. Saif et al. proposed an innovative skin color modeling method based on HSV color model, which is mainly implemented by convolutional neural network technology. In this method, the face can be effectively separated from the complex background environment accurately by setting the background mask, and then the specific posture of the face can be further determined. This new modeling method has high practical value and application prospect. Compared with some existing skin color modeling algorithms, the proposed method shows better performance in both visual quality and quantitative measurement [15].

In summary, optical flow feature detection and HSV space have received more attention and have more applications in various scientific fields. However, there are few researches that combine the two methods and apply them to pedestrian target detection. Therefore, an algorithm combining optical flow feature detection and HSV space detection is suggested in this paper, which is used to the target recognition of sanitation workers. This study aims to use the intelligent model constructed to achieve accurate detection of sanitation workers, to assist managers in managing and supervising them, and to promote the construction of smart cities.

3. Construction of sanitation worker detection model integrating optical flow characteristics and HSV space

3.1 Research on detection algorithm based on optical flow characteristics

The Detection Algorithm(DA) based on optical flow characteristics first needs to eliminate the noise of the acquired image. After re-processing through binarization, it is used for subsequent specific analysis [16, 17]. The first step of the detection technology based on the fusion of optical flow characteristics and HSV space in this study is to select optical flow characteristics for specific detection of dynamic sanitation workers. Among them, the specific process of the DA based on optical flow characteristics mainly includes the pre-processing of the captured video image to eliminate the noise to the maximum extent. Then, the optical flow diagram needs to be obtained by calculation, and the region segmentation and threshold segmentation are performed on the obtained optical flow diagram. The main function of segmentation is to distinguish or separate the background region from the target region. Next, the threshold segmentation graph needs to be analyzed and studied again, and finally, the background noise is removed by mathematical morphology [18, 19]. The data analysis of optical flow characteristics can eliminate the interference information outside the target area in the actual detection and recognition, and can improve the detection accuracy. Thus, the optical flow feature DA can directly obtain the dynamic features of the target object. Therefore, it can be integrated with the HSV color space in the subsequent process, so as to better carry out pedestrian detection and identification. Figure 1 shows the specific flow of pedestrian detection based on optical flow characteristics.

Figure 1.

Optical flow feature DA flow and threshold processing diagram.

In Fig. 1, the optical flow field region segmentation in the optical flow feature DA mainly consists of preliminary segmentation, main segmentation and subsequent processing. Among them, the key point of the preliminary segmentation is to merge the pixels with similar optical flow. That is, two adjacent regions are selected in the image first, the average value of optical flow is calculated, and then the similarity of the value is judged. If similar, merge. In the main segmentation stage, local fitting is mainly carried out, that is, the merging of regions is determined according to the sum of the squares of fitting residuals. The fitting model generally uses quadratic polynomials. Assume that regions $R_{i}$ and $R_{j}$ are two adjacent regions. If the fitting support region containing $n$ is selected as $R_{h}$ , then any point $p$ in $R_{h}$ satisfies the conditions shown in Eq. (1):

$\displaystyle\left\{\begin{array}[]{lll}\forall p^{\prime}\in R_{i}-R_{h},&D% \left({p^{\prime},R_{j}}\right)\geqslant D\left({p,R_{j}}\right),&\textit{if}% \ p\in R_{i}\\ \forall p^{\prime}\in R_{j}-R_{h},&D\left({p^{\prime},R_{i}}\right)\geqslant D% \left({p,R_{i}}\right),&\textit{if}\ p\in R_{j}\\ \end{array}\right.$ (1)

In Eq. (1), $d\left(,\right)$ represents the distance equation between two points; $D\left({p,R}\right)$ denotes the equation representing the distance from the point to the region. So, the $n$ points in $R_{h}$ are the ones most likely to cluster together in the $R_{i}$ and $R_{j}$ regions. After that, the hypothesis test is carried out to determine the optimal fitting model, and the region $R_{h}$ is fitted. Primary segmentation stops until no two adjacent regions are merged. The last step is the follow-up processing, which mainly includes threshold segmentation and mathematical morphology region processing. Assume that the initial threshold is the median value of the image gray scale, denoted as Th; the target area is $C_{1}$ ; if the background region is $C_{2}$ , the gray mean value of the two backgrounds is calculated as follows:

$\displaystyle\mu_{i}=\frac{1}{N_{C_{i}}}\sum\limits_{\left({x,y}\right)\in C_{% i}}{f\left({x,y}\right)}\left({i=1,2}\right)$ (2)

In Eq. (2), $N_{C_{i}}$ represents the number of pixels in class $i$ . Subsequently, the functional expression of the new threshold is:

$\displaystyle\textit{Th}_{\textit{new}}=\left({\mu_{1}+\mu_{2}}\right)/2$ (3)

Repeat the previous steps until the new threshold and the old threshold are less than a specific value to print the result. Usually, the background information of the image contains noise and other interference information. The mathematical morphology can eliminate the interference information of the image and improve the detection accuracy. It is a commonly used means to deal with image performance, including corrosion and expansion. Corrosion refers to the process of eliminating boundary points and making the boundary shrink, and its calculation equation is shown as Eq. (4):

$\displaystyle A\Theta B=\left\{{x\left|{\left(B\right)_{x}\subseteq A}\right.}\right\}$ (4)

In Eq. (4), after the structural element $x$ is translated, a set is formed so that $B$ is included in $A$ , which is the result of $A$ being corroded by $B$ . Expansion value refers to the process of combining an object with all the points it touches, so that the area of the object increases the number of pixels corresponding to it, and its function is expressed as:

$\displaystyle A\oplus B=\left\{{x\left|{\left(B\right)_{x}\subseteq A}\right.}\right\}$ (5)

In Eq. (5), the structural element $x$ is translated to form a set of non-empty points at the intersection of $B$ and $A$ , and the result is the result of $A$ being expanded by $B$ . Corrosion and expansion can be combined in cascade, that is, “open and closed operation,” and its function expression is shown in Eq. (3.1):

$\displaystyle\textit{Open}\ \ A\circ B=\left({A!B}\right)\oplus B$ (6) $\displaystyle\textit{Close}\ \ A\cdot B=\left({A\oplus B}\right)!B$

In Eq. (3.1), $\circ$ represents the open operator; $\cdot$ represents the closed operator. The open operation can filter the punctures smaller than the structural elements, and the closed operation can fill the gaps smaller than the structural elements. The expressions of the open property theorem and the closed property theorem are shown in Eq. (7):

$\displaystyle A\circ B=\left\{{x\in A\left|{t\in A!B,x\in\left(B\right)_{t}% \textit{or}\left(B\right)_{t}\subseteq A}\right.}\right\}$ (7) $\displaystyle A\cdot B=\left\{x\left|X\in\left(\hat{B}\right)\Rightarrow\left(% \hat{B}\right)\cap A\neq\emptyset\right.\right\}$

In Eq. (7), to open $A$ with $B$ is to select some points matching $B$ in $A$ ; The result condition that closes $A$ with $B$ contains that the intersection of $A$ with mapped and shifted structural elements is not zero.

Although the background noise has been relatively removed in the target image after optical flow field region segmentation, threshold segmentation and morphological region processing, some noise remains in the image obtained due to the defects of the algorithm itself. In view of this, in addition to the need for optical flow field region segmentation, the technology fusion of optical flow feature extraction DA and HSV space is carried out to further improve the detection accuracy.

3.2 Research on detection algorithm based on HSV space

The DA based on HSV space is mainly composed of three non-interfering and independent components of HSV [20, 21, 22]. The DA based on HSV space first needs to pre-process the target image, that is, remove the image noise and determine the target image region. The H component, S component and V component were extracted respectively, and the maximum and minimum values of H component, S component and V component are determined for threshold segmentation. Finally, the obtained images are processed by morphology. In general, color space conversion is mainly divided into two types: from HSV to RGB and from RGB to HSV. In Fig. 2, the conversion mode from RGB to HSV is adopted in this study.

Figure 2.

RGB-HSV conversion.

Transfer the color image to the expression equation of HSV space function, as shown in Eq. (8):

$\displaystyle\left\{\begin{array}[]{l}V=\max(R,G,B)S=\frac{V-\min(R,G,B)}{V}\\ H=\left\{\begin{array}[]{l}\frac{(G-B)\times 60}{S},V=R\\ \frac{(G-B)\times 60}{S}+120,V=G\\ \frac{(R-G)\times 60}{S}+240,V=B\\ \end{array}\right.\\ \end{array}\right.$ (8)

In general, the improved form iteration method is used for component adaptive segmentation of the target image region. That is, the H component, S component and V component of the image and the corresponding biggest and smallest values are obtained respectively. After the adaptive threshold segmentation of H component, the initial threshold $T_{0}=\left({\textit{Hmax}+\textit{Hmin}}\right)/2$ is set; then the obtained image is segmented into target image and background image by using the initial threshold. Meanwhile, the respective mean H0 and H1 are calculated, and the new threshold is calculated. When the new threshold is stable and unchanged, it is the final threshold, and finally the segmentation result of the H threshold is obtained [23, 24]. Similarly, the threshold segmentation results of S component and V component are obtained. The HSV DA flow chart and processing diagram are shown in Fig. 3.

Figure 3.

Flow chart and processing diagram of HSV DA.

Generally, the video image captured by the device has multiple scenes. When HSV color space is used alone, only the target object in the static scene needs to be detected. As can be seen from Fig. 3, the segmented image has high detection accuracy. However, for the target object in the dynamic scene, the detection effect of this algorithm is not ideal.

3.3 Sanitation worker detection algorithm based on the fusion of optical flow characteristics and HSV space

In view of the need for real-time detection of sanitation worker targets in dynamic scenarios, this study proposes that the dynamic targets extracted by optical flow method and HSV threshold segmentation are combined to extract specific targets to complete the final detection [19]. The core idea is to convert the obtained optical flow threshold segmentation graph to HSV color space, and perform fusion while completing HSV threshold segmentation. The algorithm flow and image processing diagram are shown in Fig. 4.

Figure 4.

Optical flow characteristics and HSV spatial fusion ring DA flow and image processing diagram.

The optical flow feature and HSV color space fusion algorithm first applies the optical flow feature to get the optical flow threshold segmentation map, and then converts the dynamic image to the static image [25]. After that, processing such as HSV threshold segmentation is carried out to remove noise information in the background area by taking the common part. Finally, the results are obtained by morphological processing. In view of the low detection accuracy of the optical flow method for sanitation workers with specific characteristics, and in view of the HSV color space can only detect static sanitation workers. To improve the detection rate, this study proposes a spatial fusion technology based on optical flow characteristics and HSV.

4. Research on the identification method of sanitation workers based on PCA algorithm

4.1 Principal component analysis method based on PCA algorithm

The Principal Component Analysis (PCA) method of identity identification based on PCA algorithm is essential in solving the problem of high dimensionality of data space. Computing principal components for specific identification is achieved by projecting high-dimensional data into a lower-dimensional space [26, 27]. Given m observations of n variables, form a data matrix with a specific $n\times m$ , where n is usually large. That is, it is difficult to accurately understand something described by multiple variables. Therefore, it is required to find the main variables for analysis, but in general, it is difficult to directly characterize the key variables. In view of this, PCA algorithm uses the linear combination of the original variables to represent the main aspects of things. The function thought expression is as follows:

$\displaystyle\left\{{\begin{array}[]{l}\eta_{j}=\frac{\lambda_{j}}{\lambda_{1}% +\lambda_{2}+\ldots+\lambda_{\mu}}\ \ E(\chi)=0\\ y=\sum\limits_{k=1}^{m}{w_{k}x_{k}=w^{T}x\ \ \left\|{\left.w\right\|}\right.}=% (w^{T}w)^{\frac{1}{2}}=1\\ E(y^{2})=E((w^{T}w)^{2})=w^{T}E(xx^{T})w=wC_{x}w\\ C_{x}w_{j}=\lambda_{j}w_{j}\ \ j=1,2,3,\ldots,m\\ \end{array}}\right.$ (9)

Principal component refers to the variable obtained after linear combination of m variables of the original data. In Eq. (9), $\lambda_{j}$ represents the degree of dispersion of this point in the direction of the principal component; $\eta_{j}$ represents the contribution rate of principal component $\lambda_{j}$ . Let $\chi$ represent a random vector of the environment m; let $w$ represent the M-dimensional unit vector and $x$ be the projection on it; where the inner product of vectors $x$ and $w$ satisfies certain constraints. PCA is to determine that the weight vector w maximizes the value of the expression, that is, the maximum $w$ is the eigenvector corresponding to the eigenvalue of the matrix.

4.2 Identification feature extraction of sanitation workers based on PCA

Face feature extraction based on PCA algorithm takes the global divergence matrix of training sample set as the production matrix of K-L transformation, and can also be the interclass dispersion matrix of training sample set. Set of N face images about $H\times W$ {F1,F2, $\ldots$ FN}. According to the specific imaging environment of the image, it is generally necessary to pre-process the image first, and then convert the original image matrix into $n=$ HW dimension column vector or row vector. The function expression is as follows:

$\displaystyle\left\{{\begin{array}[]{l}\{F_{1},F_{2},\ldots,F_{N}\}\Rightarrow% \{x_{1},x_{2},\ldots,x_{N}\}\\ X=\{x_{1}-\mu,x_{2}-\mu,\ldots,x_{N}-\mu\}\\ \mu=\frac{1}{N}\sum\limits_{k=1}^{N}{x_{k}S_{r}=\sum\limits_{k=1}^{N}{(x_{k}-% \mu)(x_{k}-\mu)^{T}}}\\ S_{T}\Phi=\Phi D_{\lambda}\mbox{ }S_{T}\Phi_{i}=\Phi{}_{i}D_{i}\\ \end{array}}\right.$ (10)

In Eq. (10), it is necessary to subtract its mean vector. In the equation, $X$ is the sample matrix; $\mu$ denotes the mean vector of $X$ . Therefore, the matrix $R$ in the $K-L$ transformation is the covariance matrix $S_{T}$ , which is the population spread matrix. It is also the matrix of the $K-L$ transformation. To solve the generated matrix and implement the transformation is to solve the eigenvalues and eigenvectors of the generated matrix.

5. Result analysis

5.1 Analysis of face detection results

To test the effectiveness of the suggested model, a total of 8,000 photos of 100 sanitation workers are collected from 4 different locations in a city. Sanitation workers wear yellow overalls, and the collection period includes early morning, noon, afternoon, evening and night. The experimental platform consists of software platform and hardware platform. The software platform is MATLAB 2016a; Hardware platform is InterCorei5-6300HQCPU processor, 4 G memory, 64 bit operating system.

The threshold value of light flow of sanitation workers determined by the light flow method is shown in Fig. 5. In the figure, when detecting sanitation workers, the optical flow method is applied to extract optical flow characteristics and segment the optical flow threshold of the image, non-target objects such as pedestrians on the road will also be separated. Therefore, for some special cases of sanitation workers, the simple use of optical flow method, its identification accuracy is not high.

Figure 5.

Result of optical flow threshold.

According to the color characteristics of sanitation workers’ clothes, two different component threshold segmentation methods of H and S are adopted. Through the analysis of image light saturation, the V component subthreshold is obtained. After the region segmentation, morphological processing is carried out to obtain an ideal region segmentation map. Figure 6(a) shows the results after the simultaneous threshold segmentation of the three components of HSV. Figure 6(b) shows the image results of sanitation workers obtained after morphological processing. It is not difficult to find from the processing result chart that the image of sanitation workers with more comprehensive information can be obtained after segmentation, thus improving the final detection accuracy.

Figure 6.

Results of threshold segmentation and morphological processing.

The difference of grayscale distribution in face images is an obvious feature. By projecting grayscale into horizontal and vertical directions within the face area, the approximate regional range of the facial organs can be obtained by the position of the minimal points. To complete face detection, it is necessary to place the built skin color model into a specific color space to achieve a specific description and complete feature detection. The gray feature projection results and HSV color space histogram are shown in Fig. 7.

Figure 7.

Gray feature projection results and HSV color space histogram.

In Fig. 7, the gray feature is affected by uneven illumination. In view of this, linear light fitting and histogram equalization can make up for uneven lighting. In addition, nonlinear transformation can also reduce the influence of uneven illumination. Based on the above advantages, skin color features have obvious advantages in the face detection of color images. In addition, this study gives the identification of sanitation workers based on PCA and average map. Through the extraction and comparison of the facial features of the sanitation workers, their identities are finally identified. Among them, the convergence curve of the sanitation worker identification algorithm based on PCA and the average graph is shown in Fig. 8.

Figure 8.

Convergence curve of PCA identification.

Figure 9.

TP and FN curves corresponding to the detection results of the four algorithms.

In Fig. 8, due to the instability and sparseness of the high-dimensional space, dimensionality reduction of the training set and test set can reduce the number of variables, and the reduction of the number of variables can more easily extract the essential information of face features. In the training stage, the selection of feature vector is very important, which has a great influence on the accuracy of the algorithm recognition rate. Generally, the covariance matrix has many eigenvectors corresponding to non-zero eigenvalues. But in practice, the eigenvectors with less information can be discarded to improve the operation speed. In short, based on the convergence curve of PCA identification, it can be seen that the recognition performance of this algorithm is very good.

5.2 Comparison of different fusion technologies

To verify the superiority of the fusion features designed in this study, another 4 groups of comparison experiments are set up in this study to perform comparative real analysis with the HOG-LBP features, HPG features and LBP features separately. Meanwhile, to verify the superior performance of the fusion algorithm proposed in this study, TP and FN are taken as measurement indicators, TP is pedestrian detection, FN is pedestrian detection. The larger the TP or the smaller the FN, the lower the false detection rate and missing rate of the algorithm, the better the performance of the method. Among them, the TP and FN curves corresponding to the detection results of the four algorithms are shown in Fig. 9.

In Fig. 9, the feature fusion algorithm proposed in this study has a good recognition rate, and its recognition rate is significantly higher than that of the other experimental groups. Therefore, the proposed method fusion features can effectively carry out identification detection. The method used in this study has the highest TP and the lowest FN, which verifies that the algorithm proposed in this study has a good performance in extracting image texture features.

5.3 Analysis of identification results of sanitation workers

Under normal circumstances, the background of the video image obtained in practical application is extremely complex, and the target objects in the background image are in a specific dynamic condition. Since the optical flow method will recognize the non-target image as the specific target image when obtaining the optical flow pattern, the detection rate will be affected. In addition, the HSV color space detection method can only detect static objects, but has the advantage of extracting specific features. In view of this, this study proposes a DA based on the fusion of optical flow characteristics and HSV, and finally realizes the easy extraction and detection of sanitation workers with specific characteristics in various complex dynamic scenarios. Among them, the results of motion scene detection and identification are shown in Fig. 10. Due to the actual detection condition in the moving scene, the optical flow feature detection alone mistakes the noise information of the non-target image as the target image for marking detection, and thus the detection accuracy can be generated. In view of the fact that HSV color space can extract specific features but cannot be applied to dynamic scenes, a fusion algorithm based on optical flow feature detection and HSV color space is selected in this study. That is, the obtained optical flow threshold segmentation map is segmentated by HSV threshold to remove the interference information of the target image, and then the accuracy of the detection results is comprehensively improved.

Figure 10.

Results of motion scene detection and identification.

After using Matlab program, the identity of sanitation workers is identified. Figure 11 illustrates the identification results of the two sanitation workers. The identification algorithm based on PCA and average map has a high identification rate of sanitation workers, and its similarity can reach 98.61%.

Figure 11.

Identification results of 2 sanitation workers.

Figure 12 shows the parameter information of feature value, feature vector and feature face in the experimental training stage. In Fig. 12, the first few eigenvalues account for the largest amount of energy, but the overall trend is decreasing. The proportion of energy numbers corresponding to eigenvalues after the 10th is almost zero. When the eigenvalue is 80, the energy number accounts for almost 100%. Therefore, it can be said that the fusion algorithm based on light flow feature detection and HSV color space has very good recognition efficiency.

Figure 12.

Energy ratio of eigenvalues.

6. Conclusion

Based on the progress of science and technology and the development of society, intelligent transportation system plays an important role in modern life. This study takes sanitation workers as the research object, and designs an identity identification scheme conducive to the friendly management of smart urban management. Since the optical flow feature extraction method and HSV color space extraction method alone is difficult to meet the actual detection efficiency, this study proposes a detection technology combining optical flow feature and HSV space, and applies PCA algorithm for specific identification. Experiments are carried out to test the overall performance of the model. The experimental results show that the image of sanitation workers with more comprehensive information can be obtained after segmentation, thus improving the final detection accuracy. In this method, the obtained optical flow threshold segmentation map is segmented by HSV threshold to remove the interference information of the target image, and then the accuracy of the detection results is improved comprehensively. From the convergence curve of identity recognition based on PCA, it can be seen that the recognition performance of this algorithm is very good. Finally, the identification rate of the two sanitation workers is higher, and their similarity can reach 98.61%. Therefore, the system can accurately detect and identify sanitation workers, which has important social and economic value for the benign development of modern transportation system. The shortcoming of this study is that the recognition accuracy of more complex scenes is relatively limited, which can be further optimized in the future.

Footnotes

Funding

“Research on Teaching Innovation Mode Construction and Practice Reform of Digital Signal Processing under the background of New Engineering” in “Industry-university Cooperative Education Program of the Ministry of Education” Program number: 231105522094551.

References

Ayzel

Heistermann

Winterrath

. Optical flow models as an open benchmark for radar-based precipitation nowcasting (rainymotion v0.1). Geosci Model Dev. 2019; 12(4): 1387-1402.

Alper

Goudreau

Daniel

. Pose and Optical Flow Fusion (POFF) for accurate tremor detection and quantification. Biocybern Biomed Eng. 2020; 40(1): 468-481.

Bodduna

Weickert

. Removing multi-frame Gaussian noise by combining patch-based filters with optical flow. J Electron Imag. 2021; 30(3): 033031.

Tehrani

AKZ

Rivaz

. Displacement estimation in ultrasound elastography using pyramidal convolutional neural network. IEEE T Ultrason Ferr. 2020; 67(12): 2629-2639.

Yoon

Shin

Spencer

. Structural displacement measurement using an unmanned aerial system. Comput-Aided Civ Inf. 2018; 33(3): 183-192.

Caldelli

Galteri

Amerini

Del Bimbo

. Optical flow based CNN for detection of unlearnt deepfake manipulations. Pattern Recogn Lett. 2021; 146: 31-37.

Dong

Zhang

Nie

. Dual discriminator generative adversarial network for video anomaly detection. IEEE Access. 2020; 8: 88170-88176.

Paredes-Vallés

Scheper

KYW

De Croon

GCHE

. Unsupervised learning of a hierarchical spiking neural network for optical flow estimation: From events to global motion perception. IEEE T Pattern Anal. 2020; 42(8): 2051-2064.

Ullah

Muhammad

Del Ser

Baik

de Albuquerque

VHC

. Activity recognition using temporal optical flow convolutional features and multilayer LSTM. IEEE T Ind Electron. 2019; 66(12): 9692-9702.

10.

Almatrafi

Baldwin

Aizawa

Hirakawa

. Distance surface for event-based optical flow. IEEE T Pattern Anal. 2020; 42(7): 1547-1556.

11.

Ito

Premachandra

Sumathipala

Premachandra

HWH

Sudantha

. Tactile paving detection by dynamic thresholding based on HSV space analysis for developing a walking support system. IEEE Access. 2021; 9: 20358-20367.

12.

Guan

. Fabric defect delaminating detection based on visual saliency in HSV color space. J Text I. 2018; 109(12): 1560-1573.

13.

Hassan

Gutub

. Improving data hiding within colour images using hue component of HSV colour space. CAAI T Intell Techno. 2022; 7(1): 56-68.

14.

Yan

Woźniak

. Accurate key frame extraction algorithm of video action for aerobics online teaching. Mobile Netw Appl. 2022; 27(3): 1252-1261.

15.

Saif

Esmail

Ragheb

Alshawi

Alshebeili

. Machine learning techniques for optical performance monitoring and modulation format identification: A survey. IEEE Commun Surv Tut. 2020; 22(4): 2839-2882.

16.

Zhang

Wang

. Vehicle recognition algorithm based on Haar-like features and improved Adaboost classifier. J Amb Intel Hum Comp. 2021; 14: 807-815.

17.

Zhao

. A low-cost and fast vehicle detection algorithm with a monocular camera for adaptive driving beam systems. IEEE Access. 2021; 9: 26147-26155.

18.

Kim

Lee

Ahn

Jung

Ahn

. Broken stitch detection system for industrial sewing machines using HSV color space and image processing techniques. J Comput Des Eng. 2023; 10(4): 1602-1614.

19.

Das

Piciucco

Maiorana

Campisi

. Convolutional neural network for finger-vein-based biometric identification. IEEE T Inf Foren Sec. 2019; 14(2): 360-373.

20.

Palanisamy

Shankar

Ponnusamy

Gpoi

. A hybrid feature preservation technique based on luminosity and edge based contrast enhancement in color fundus images. Biocybern Biomed Eng. 2020; 40(2): 752-763.

21.

Wang

Yao

. Color image multiplicative noise and blur removal by saturation-value total variation. Appl Math Model. 2021; 90: 240-264.

22.

Salman

Siddiqui

Shafait

Mian

Shortis

Khurshid

Ulges

Schwanecke

. Automatic fish detection in underwater videos by a deep neural network-based hybrid motion learning system. ICES J Mar Sci. 2020; 77(4): 1295-1307.

23.

Bao

Zhang

Chen

Gao

. KalmanFlow 20:. Efficient video optical flow estimation via context-aware kalman filtering. IEEE T Image Process. 2019; 28(9): 4233-4246.

24.

Gallego

Delbrück

Orchard

Bartolozzi

Taba

Censi

Leutenegger

Davison

Conradt

Daniilidis

Scaramuzza

. Event-based vision: A survey. IEEE T Pattern Anal. 2022; 44(1): 154-180.

25.

Huang

Wang

. A novel key-frames selection framework for comprehensive video summarization. IEEE T Circ Syst Vid. 2020; 30(2): 577-589.

26.

Gehrig

Rebecq

Gallego

Scaramuzza

. EKLT: Asynchronous photometric feature tracking using events and frames. Int J Comput Vision. 2020; 128(3): 601-618.

27.

Luo

Liu

Lian

Tang

Duan

Peng

Gao

. Video anomaly detection with sparse coding inspired deep neural networks. IEEE T Pattern Anal. 2021; 43(3): 1070-1084.