Urban management image classification approach based on deep learning

Abstract

Based on the case images in the smart city management system, the advantage of deep learning is used to learn image features on its own, an improved deep convolutional neural network algorithm is proposed in this paper, and the algorithm is used to improve the smart city management system (hereinafter referred to as “Smart City Management”). These case images are quickly and accurately classified, the automatic classification of cases is completed in the city management system. ZCA (Zero-phase Component Analysis)-whitening is used to reduce the correlation between image data features, an eight-layer convolutional neural network model is built to classify the whitened images, and rectified linear unit (ReLU) is used in the convolutional layer to accelerate the training process, the dropout technology is used in the pooling layer, the algorithm is prevented from overfitting. Back Propagation (BP) algorithm is used for optimization in the network fine-tuning stage, the robustness of the algorithm is improved. Based on the above method, the two types of case images of road traffic and city appearance environment were subjected to two classification experiments. The accuracy has reached 97.5%, and the F1-Score has reached 0.98. The performance exceeded LSVM (Langrangian Support Vector Machine), SAE (Sparse autoencoder), and traditional CNN (Convolution Neural Network). At the same time, this method conducts four-classification experiments on four types of cases: electric vehicles, littering, illegal parking of motor vehicles, and mess around garbage bins. The accuracy is 90.5%, and the F1-Score is 0.91. The performance still exceeds LSVM, SAE and traditional CNN and other methods.

Keywords

Urban management image classification Convolution Neural Network (CNN)Zero-phase Component Analysis (ZCA)-whitening dropout Rectified Linear Unit (ReLU)

1. Introduction

With the rapid development of Internet technology, urban management is gradually moving towards digitization, networking, and intelligence [29], and urban management technology based on Internet technology is becoming more and more mature [30], such as in the smart city management system [32], ordinary citizens can use the smart phone APP (Application) to report problems in urban management (such as random parking of vehicles, city appearance and environmental problems, etc.) with one click. At the same time, when reporting this kind of urban management case, the user needs to upload at least one on-site picture, on-site location and other information corresponding to the case. The urban management staff can immediately follow the picture information and location information of the reported case after seeing the problem case reported by the citizen, they go to the location of the incident for corresponding processing, this greatly improves the efficiency of urban management [18]. However, as the number of users continues to increase, the number of cases reported every day is also increasing. It is time-consuming, laborious and inefficient to select cases of different types by manual only. Therefore, there is an urgent need for a method that can quickly classify urban management cases to assist urban management and make urban management more intelligent. Ambient intelligence (AmI) is intrinsically and thoroughly connected with artificial intelligence (AI). Multi-agent systems and the Semantic Web, ambient assisted living and e-healthcare, AmI for assisting medical diagnosis, ambient intelligence for e-learning and ambient intelligence for smart cities are researched from information-society laws, superintelligence [9]. The increasing deployment and usage of ‘smart’ technologies determining a wide range of everyday life activities. From taking different perspectives on common issues, commonalities and relationships are provided between them, and anchor points for important challenges are provided in the field of ambient intelligence [24].

Based on the information in the case image, the case can be automatically classified by the method of image classification. At present, there are many methods for image classification, such as the Support Vector Machines (SVM) classification method proposed by Vapnik et al., and the Lagrangian support vector machine (LSVM) method used by Lu et al. [17]. However, it is generally necessary to extract texture features such as HOG (Histogram of Oriented Gradient) features of the image before using these methods, and the quality of image feature extraction will have a direct impact on the final classification result. More commonly used image feature extraction methods include Gabor wavelet image texture feature extraction [28], Gauss Markov Random Field (GMRF) remote sensing image feature extraction [20], SIFT (Scale-invariant feature transform) image feature extraction [16]. Deep learning was proposed by Hinton in 2006. Its remarkable feature is that it can learn image features by itself, and form a more abstract high-level feature representation by combining low-level features of data [10], it can greatly reduce various overheads for manual feature extraction. Subsequently, some deep learning methods developed rapidly, such as Denoising Autoencoder (DAE) [12], Sparse Autoencoder (SAE) [5], Convolutional Neural Networks (CNN) [14], etc. Convolutional neural network (CNN) is used to recognize the minist handwriting data set, the error rate has been lower than that of human manual recognition [7]. Yang et al. used SAE to classify images [31]. In 2012, Krizhevsky et al. performed image classification on the ImageNet database, and the test error rate of top-5 was 15.3% [14]. Internet of things (IoT) applications for smart cities have become a primary target for advanced persistent threats (APT) of botnets. A botnet detection system is proposed based on a two-level deep learning framework for semantically discriminating botnets and legitimate behaviors at the application layer of the domain name system (DNS) services [27]. In the first level of the framework, the similarity measures of DNS queries are estimated using siamese networks based on a predefined threshold for selecting the most frequent DNS information across Ethernet connections. In the second level of the framework, a domain generation algorithm (DGA) based on deep learning architectures is suggested for categorizing normal and abnormal domain names. The framework is highly scalable on a commodity hardware server due to its potential design of analyzing DNS data. The explosive growth of Internet and the recent increasing trends in automation using intelligent applications have provided a veritable playground for malicious software (malware) attackers. With a variety of devices connected seamlessly via the Internet and large amounts of data collected, the escalating malware attacks and security risks are a big concern. While a number of malware detection methods are available, new methods are required to match with the scale and complexity of such a data-intensive environment. A novel and unified hybrid deep learning and visualization approach is proposed for an effective detection of malware [26]. The application of hybrid image-based approaches with deep learning architectures are proposed and investigated for an effective malware classification. Diseases in plants are a great threat to the yield of the crops thereby causing famines and economy slow down. Machine learning model is used for classifying tomato disease image dataset [8], necessary steps are proactively taken to combat such agricultural crisis. In this work, the dataset is collected from publicly available plant–village dataset. The significant features are extracted from the dataset using the hybrid-principal component analysis–Whale optimization algorithm. Further the extracted data are fed into a deep neural network for classification of tomato diseases. The grid denotes the electric grid which consists of communication lines, control stations, transformers, and distributors that aids in supplying power from the electrical plant to the consumers. Presently, the electric grid constitutes humongous power production units which generates millions of megawatts of power distributed across several demographic regions. There is a dire need to efficiently manage this power supplied to the various consumer domains such as industries, smart cities, household and organizations. In this regard, a smart grid with intelligent systems is being deployed to cater the dynamic power requirements. A smart grid system follows the Cyber-Physical Systems (CPS) model, in which Information Technology (IT) infrastructure is integrated with physical systems. In the scenario of the smart grid embedded with CPS, the Machine Learning (ML) module is the IT aspect and the power dissipation units are the physical entities. In this research, a novel Multidirectional Long Short-Term Memory (MLSTM) technique is being proposed to predict the stability of the smart grid network [3]. The obtained results are evaluated against other popular Deep Learning approaches such as Gated Recurrent Units (GRU), traditional LSTM and Recurrent Neural Networks (RNN).

Internet of Things (IoT) empowered Heating, Ventilation, and Air Conditioning (HVAC) buildings are considered to monitor and control the regulation of thermostats, sensors, actuators, and control devices smartly. A novel model named PersonalisedComfort is proposed to predict the thermal sensation votes of individuals living in a building [21]. Conventional machine learning algorithms and deep learning algorithms are used to predict the thermal sensation vote. The Internet of Things (IoT) provides smart solutions for future urban communities to address key benefits with the least human intercession. A smart home offers the necessary capabilities to promote efficiency and sustainability to a resident with their healthcare-related, social, and emotional needs. In particular, it provides an opportunity to assess the functional health ability of the elderly or individuals with cognitive impairment in performing daily life activities. This work proposes an approach named Cognitive Assessment of Smart Home Resident (CA-SHR) to measure the ability of smart home residents in executing simple to complex activities of daily living using pre-defined scores assigned by a neuropsychologist. CA-SHR also measures the quality of tasks performed by the participants using supervised classification. Furthermore, CA-SHR provides a temporal feature analysis to estimate if the temporal features help to detect impaired individuals effectively. The goal of this study is to detect cognitively impaired individuals in their early stages. CA-SHR assess the health condition of individuals through significant features and improving the representation of dementia patients. For the classification of individuals into healthy, Mild Cognitive Impaired (MCI), and dementia categories, ensemble AdaBoost is used [13]. This results in improving the reliability of the CA-SHR through the correct assignment of labels to the smart home resident compared with existing techniques. A collaborative health care plan using a multi-agent system assists adult individuals to live an independent healthy life by analyzing their routine life activities. Robust recognition of activities provides services such as health monitoring and fitness assessment. A novel Collaborative Health Care Plan system is proposed [22], the independent living of an individual is improved by using a smartphone sensor, machine learning algorithm, multiple agents i.e. doctor, gym trainer, guardian, and intelligent ranker agent. The novelty of the devised approach is that it shares the daily life assessment of activities among care provides which in return provides a care plan or recommendation to ensure good health of the individual. A machine learning algorithm is used to recognize the adult individual’s daily life physical activities.

The images in urban management cases are all captured by various ordinary mobile phones, and the background information is relatively complicated and the image quality is low. Therefore, the above-mentioned method of classifying such images cannot achieve ideal results. Aiming at this problem, the images are analyzed in the city management case, the convolutional neural network is used to automatically extract the characteristics of the image features, and an improved convolutional neural network algorithm (ZCNN) is designed. First, unified ZCA (Zero-phase Component Analysis)-whitening processing is performed on the acquired case images [1], the correlation is effectively reduced between image features, and then an 8-layer convolutional neural network is built according to the image features, and the appropriate convolution kernel size is set. After the convolutional layer, a down-sampling layer is connected to the pooling layer, and the Rectified Linear Unit (ReLU) is used to accelerate the training process [11,19]. The dropout technology is used in the pooling layer to randomly disconnect the network nodes, the algorithm is prevented from overfitting. Finally, when the model is fine-tuned, the stochastic gradient descent method is used to calculate the parameters of the model layer by layer. In addition, in order to improve the accuracy of the algorithm, the BP (Back Propagation) algorithm is used to optimize the network parameters [6]. In this article, two types of case pictures (1300 in each class, 2600 in total) and four types of case pictures (1300 in each class, 5200 in total) experiments are used to verify the effectiveness of the algorithm.

The smart city management platform is based on the digital city management system, with the goal of solving the pain points and difficult problems that need to be solved urgently in city management, better serving the public, and forming a typical application of “smart city”. The smart urban management platform not only focuses on the comprehensive business management of the Urban Management Comprehensive Administrative Law Enforcement Bureau itself, but also provides city-wide public services to functional departments at all levels and the public from the perspective of large urban management. The research content of this article is part of smart city management, and the application of artificial intelligence helps the automation and intelligence of smart city management.

2. Materials and methods

2.1. Data preprocessing-ZCA whitening

Image data preprocessing is a crucial step in the task of image classification, especially for natural images. Preprocessing affects the final classification effect because of the strong correlation between adjacent pixels of the image. Image neighboring pixel correlation is Fig. 1. Therefore, how to effectively remove the correlation of image data and it is especially necessary to reduce its redundancy [2]. Whitening is the process of transforming the covariance matrix of the data into the identity matrix. Its purpose is to reduce the redundancy of the input data and make the whitened data closer to the original data. In terms of value, it is mainly to make the data have a uniform covariance, and each feature has the same variance

Fig. 1.

Image neighboring pixel correlation.

Specific steps are as follows:

First, the brightness and contrast of the data are normalized. For the pixel value $p^{(i)}$ of each pixel in the image, subtract the average gray value of the image, and then divide by the standard deviation, so that the denominator is not 0, usually a small constant ε is added to the denominator. The calculation equation (1) is as follows:

\begin{array}{l} (1) & p^{(i)} = \frac{{\tilde{p}}^{(i)} - mean ({\tilde{p}}^{(i)})}{\sqrt{var} ({\tilde{p}}^{(i)}) + ε} \end{array}

Where ${\tilde{p}}^{(i)}$ is the original input image data, $mean ({\tilde{p}}^{(i)})$ and $var ({\tilde{p}}^{(i)})$ respectively represent the mean and variance of the image ${\tilde{p}}^{(i)}$ . When the image gray value is $[0, 255]$ , let the constant $ε = 10$ .

The covariance matrix of the training sample is calculated in equation (2):

\begin{array}{l} (2) & Σ = \frac{1}{n} \sum_{i = 1}^{n} (p^{(i)}) {(p^{(i)})}^{T} \end{array}

Since the data are correlated, the covariance matrix calculated at this time is a non-diagonal matrix.

Reduce the correlation between the data and transform the covariance matrix into a diagonal matrix. Equation (3) is as follows:

\begin{array}{l} (3) & [V, D] = eig (Σ) \end{array}

Where $V$ is the eigenvector matrix. $D$ is a diagonal matrix composed of the eigenvalues of the covariance matrix Σ.

ZCA whitening is used, the equation is as follows:

\begin{array}{l} (4) & \hat{p} = V {(D + δ I)}^{- 1 / 2} V^{T} p \end{array}

Wherein, δ is a very small constant, set to 0.01, and $I$ is the identity matrix.

After the image data is whitened by ZCA (Zero Components Analysis), the correlation between the pixels and the redundancy of the data can be reduced, so that the variance of each dimension of the feature vector is equal, and the data is effectively unified.

In order to make the whitened data as close to the original data as possible, the processed daIta can be transformed back to the original space, which is ZCA whitening. The full name of ZCA Whitening is Zero-phase Component Analysis Whitening. The understanding of [zero phase] is that, relative to the original space (coordinate system), the whitened data does not rotate (coordinate transformation). Interrelationships between features sometimes appear in training data. For image data, the issue of interrelationship is even more serious. Although the convolutional layer can solve these local correlations through learning, it is always not straightforward to obtain through learning. If you directly manipulate the input data to solve some data correlation problems, it will definitely make training easier.

2.2. Convolutional neural network

Convolutional Neural Networks (CNN) is proposed by Huber and Wiesel [15]. Its unique network structure can effectively reduce the complexity of feedback neural networks. The basic structure mainly includes two layers: one layer is convolution Layer, an image is convolved with the set convolution kernel, the convolution value is weighted and biased, and then it is obtained through an activation function; the other layer is the down-sampling layer, it is also called the pooling layer, the principle of image local correlation is used to sub-sampling the image, the amount of data processing is reduced while retaining useful information.

In order to understand the process of convolution and pooling more clearly, as shown in Fig. 2, the size of the input image is $h \times d$ , $k_{x}$ is a convolution filter (convolution kernel) of size $r \times c$ , and the step size is set to 1. Then the size of the feature map of the convolutional layer is $(h - r + 1) \times (d - c + 1)$ . The number of feature planes of the pooling layer $C_{x}$ is equal to the number of feature planes of the convolutional layer, and each neuron is connected to the receptive area of the convolutional layer, and the maximum or average value is calculated. The output of the neuron on the j-th feature plane of the l-th convolutional layer through the activation function is equation (5). $\begin{array}{l} (5) & a_{j}^{l} = S (\sum_{i \in M_{j}^{l}} a_{i}^{l - 1} * k_{i j}^{l} + b_{j}^{l}) \end{array}$

Fig. 2.

Convolution and pooling structure. Note: $k_{x}$ is a digital filter, $b_{x}$ is a bias, the convolution feature map $C_{x}$ of the pooling layer is obtained after processing by the unsaturated nonlinear function ReLUs; $W_{x + 1}$ represents the weight of the pooling layer, and $b_{x + 1}$ is the corresponding bias, the pooling layer $S_{x + 1}$ is obtained after non-linear processing.

Wherein, $a_{j}^{l}$ is the neuron output on the j-th feature plane of the l-th convolutional layer, $b_{j}^{l}$ is the bias vector, * represents the two-dimensional convolution operation, and $k_{i j}^{l}$ is the convolution kernel. $S (\cdot)$ is the nonlinear activation function. Non-linear activation functions usually include hyperbolic function (tanh), sigmoid function and modified linear unit (ReLU). The first two are saturated nonlinear functions, and the latter is unsaturated nonlinear functions. The first two are used in network model training. The speed of the species is significantly lower than the latter, especially in the deep neural network, this time difference is more obvious. Therefore, all activation functions of the convolutional layer in this article use the ReLU function, that is, $S (z) = max (0, z)$ .

2.3. ZCNN model

The data studied in this article are images of urban management cases. This type of images are all captured by ordinary mobile phones, and the photographers are ordinary citizens. The shooting locations are all corners of the city. Therefore, this type of image has a complex background, low picture pixels, and the characteristics of different image sizes, and there is strong correlation between image pixels. Considering that traditional image recognition and classification methods require very complex feature extraction, this article is based on the convolutional neural network (CNN) algorithm in deep learning. CNN can learn the advantages of picture features by itself and design network models. Considering that this model should be applied to smart city management case data, the network structure should not be too large while ensuring the classification accuracy, so as to minimize training time and reduce operating costs. In order to remove the correlation between the pixels of the image and make the image size uniform, image normalization processing and ZCA whitening processing are used in the model.

In this algorithm, firstly, normalization processing and ZCA whitening processing are applied to the input image. Secondly, the size of the convolution kernel is designed by this method, it is $13 \times 13$ , and the ReLU activation function is used in the convolution layer to speed up network training. After the pooling layer, the dropout technology is used to randomly disconnect 10% of the network nodes to prevent the algorithm from overfitting. Finally, the BP algorithm is used to fine-tune the network parameters, and in the fully connected layer, the softmax classifier is used for classification. The structure of the model is in turn: Input layer-Convolutional layer C₁-Downsampling layer S₂-Convolutional layer C₃-Downsampling layer S₄-Convolutional layer C₅-Downsampling layer S₆-Full Connected layer. The specific process is as follows:

The image data after ZCA whitening preprocessing is uniformly processed into $100 \times 100$ size as the input layer of the model.

Set the size of the convolution kernel to $13 \times 13$ , use 6 filters to convolve the input image, and then add the offset to obtain the convolution layer C₁, which contains 6 feature maps with a size of $88 \times 88$ .

Each feature map in the C₁ layer is obtained through average pooling, that is, the average value of every 4 pixel values in the feature map is calculated, then the weighted value is added, the bias is added, and ReLU is used as the activation function. At the same time, in the pooling layer, dropout of this method is used to randomly disconnect 10% of the nodes, and finally 6 feature maps of the S₂ layer with a size of $44 \times 44$ are output.

The obtained S₂ layer is used as the input layer of the next layer, the size of the convolution kernel is $13 \times 13$ , the number of output feature maps is 12, and the C₃ layer is obtained after convolution, and the size of the feature map is $32 \times 32$ .

After the feature map of layer C₃ is processed in the same step (3), the S₄ layer is obtained, and the size of the feature map is $16 \times 16$ .

In the same step (4), the S₄ layer is used as the input layer, the size of the convolution kernel is still $13 \times 13$ , the number of output feature maps is 18, and the feature map size of C₅ obtained after convolution is $4 \times 4$ .

After the C₅ layer is processed in the same step (3), the feature map of the S₆ layer is obtained, and the size is $2 \times 2$ .

Finally, the obtained S₆ layer is fully connected, and the classification result is obtained through the softmax classifier.

The implementation convolutional neural network structure (ZCNN) is in Fig. 3, and the implementation process of convolutional neural network model is in Table 1. (https://www-sciencedirect-com-443.web.bisu.edu.cn/topics/engineering/convolutional-neural-network).

Fig. 3.

Implementation convolutional neural network structure (ZCNN).

Table 1

Implementation process of convolutional neural network model

No.	Layer	Width	Height	Kernel	Feature Maps
0	Input	100	100	–	–
1	Convolution	88	88	13	6
2	Relu	88	88	–	6
3	Average Pooling	44	44	–	6
4	Dropout	44	44	–	6
5	Convolution	32	32	13	12
6	Relu	32	32	–	12
7	Average Pooling	16	16	–	12
8	Dropout	16	16	–	12
9	Convolution	4	4	13	18
10	Relu	4	4	–	18
11	Average Pooling	2	2	–	18
12	Dropout	2	2	–	18
13	Full Connected	1	1	–	–
14	Softmax	1	1	–	–

3. Experiment, comparative analysis and discussion of experimental performance

3.1. Experiment based on urban management case data

The data of this experiment are mainly derived from the smart city management system (referred to as “Urban Guantong” system) developed in cooperation with Qingxiu District, Nanning City, China. This system includes the web page terminal and the smart phone APP terminal (supporting iOS And Android two platforms), when citizens or workers discover problems in the city, they can take pictures of the crime scene with their personal mobile phones, select the description of the case, and report to the system with one click. This article conducted two sets of experiments on different types of case data: the first group randomly selected 1300 cases are from the road traffic category and the city appearance environment category in the system, and a total of 2600 cases were used for the two-category experiment. The second group randomly selected 1300 cases are from the four types of random placement of electric vehicles, littering of garbage, illegal parking of motor vehicles, and mess around garbage bins in the system, and a total of 5200 cases were classified into four categories. Some sample images of smart city management cases are shown in Fig. 4.

Fig. 4.

Sample graph of experimental data.

3.2. Performance evaluation index

In this paper, several commonly used classification evaluation indicators are used as the evaluation criteria of the model, specifically Accuracy, Precision, Recall, F1_Score [4].

Let $TP$ denote the number of the current class that was originally positive and also predicted as positive, $FP$ denote the number of the current class that was originally negative and was predicted to be positive, and $TN$ denotes the number of the current class that was originally negative and was predicted to be negative, $FN$ represents the number of the current class that was originally a positive class and was predicted to be a negative class [33], and S represents the total number of samples of the current class. The definition calculation formula of these indicators are as follows:

Accuracy of the current class: $Accuracy = \frac{TP}{S} \times 100 %$ .

The accuracy of the current class: $Precision = \frac{TP}{TP + FP} \times 100 %$ .

The recall rate of the current category: $Recall = \frac{TP}{TP + FN} \times 100 %$ .

F₁ value of the current class: $F_{1}_Score = \frac{2 \times Precision \times Recall}{Precision + Recall}$ ,

Average accuracy: $Mean_acc = \frac{1}{n} \sum_{i = 1}^{n} {Accuracy}_{i}$ (n is the total number of categories).

Average recall rate: $Mean_recall = \frac{1}{n} \sum_{i = 1}^{n} {Recall}_{i}$ (n is the total number of categories).

Average accuracy rate: $Mean_pre = \frac{1}{n} \sum_{i = 1}^{n} {Precision}_{i}$ (n is the total number of categories).

Average F1 value: $Mean_F 1 = \frac{1}{n} \sum_{i = 1}^{n} F 1_{Score}_{i}$ (n is the total number of categories)

3.3. Two-class experiment

In this experiment, 1300 case pictures of road traffic (Road_Traffic) and city appearance environment (City_Environment) are used, for a total of 2600 pictures, of which 1,000 pictures are taken as the training set, and the remaining 300 pictures are taken as the test set. They are to calculate various Accuracy (precision), Precision (accuracy rate), Recall (recall rate) $, F 1_Score$ (F1 value) and the total average precision Mean_acc, average accuracy Mean_pre, average recall rate Mean_recall and average F1 value Mean_F1, And compared with Lagrangian SVM (LSVM), sparse self-encoding SAE and unimproved convolutional neural network CNN, the experimental results are shown in Table 2, the model ROC curve is shown in Fig. 5, and the mean square error curve is shown in Fig. 6.

Table 2
Two classification experiment results

Model Label Accuracy/% Precision/% Recall/% F1_Score

LSVM Road_Traffic 78.33 89.69 78.33 0.84

City_Environment 91.01 80.77 91.01 0.86

Mean (average) 84.67 85.23 84.67 0.85

SAE Road_Traffic 80.01 85.64 80.01 0.83

City_Environment 85.69 88.35 85.69 0.87

Mean (average) 82.85 86.99 82.85 0.85

CNN Road_Traffic 86.33 88.69 86.33 0.87

City_Environment 89.00 86.69 89.00 0.88

Mean (average) 87.67 87.69 87.67 0.88

ZCNN Road_Traffic 98.67 96.42 98.67 0.98

City_Environment 96.33 98.63 96.33 0.97

Mean (average) 97.50 97.53 97.50 0.98

Model	Label	Accuracy/%	Precision/%	Recall/%	F1_Score
LSVM	Road_Traffic	78.33	89.69	78.33	0.84
City_Environment	91.01	80.77	91.01	0.86
Mean (average)	84.67	85.23	84.67	0.85
SAE	Road_Traffic	80.01	85.64	80.01	0.83
City_Environment	85.69	88.35	85.69	0.87
Mean (average)	82.85	86.99	82.85	0.85
CNN	Road_Traffic	86.33	88.69	86.33	0.87
City_Environment	89.00	86.69	89.00	0.88
Mean (average)	87.67	87.69	87.67	0.88
ZCNN	Road_Traffic	98.67	96.42	98.67	0.98
City_Environment	96.33	98.63	96.33	0.97
Mean (average)	97.50	97.53	97.50	0.98

Fig. 5.

ROC curve.

Fig. 6.

Mean square error curve.

According to the above experimental results, it can be seen that the performance of the ZCNN model designed in this paper is better than LSVM, SAE and the traditional CNN algorithm in the two classification problems of smart urban management. And according to Fig. 5, it can be seen that this algorithm convergence can be obtained in a short time, and the error gradually decreases and tends to stabilize.

3.4. Multi-class experiment

In this experiment, four types of pictures were used: electrocar, rubbish, car parking (car), and dustbin (dustbin), 1300 pictures each. totaling 5200 sheets; among them, 1000 sheets are selected for the training set, a total of 4000 sheets are selected as the training set, and the remaining 300 sheets are selected for each, and a total of 1 200 sheets are used as the test set; the various Accuracy (precision), Precision (accuracy) and Recall (Recall rate) $, F 1_Score$ (F1 value) and the total average accuracy Mean_acc, average accuracy Mean_pre, average recall Mean_recall and average $F 1$ value Mean_F1. And compared with linear SVM (LSVM), sparse self-encoding SAE and unimproved convolutional neural network CNN, the experimental results are shown in Table 3, the model ROC curve is shown in Fig. 7, and the mean square error curve MSE is shown in Fig. 8.

Table 3
Results of four classification experiments

Model Label Accuracy/% Precision/% Recall/% F1_Score

LSVM Electrocar 54.67 41.41 54.67 0.47

Rubbish 77.01 43.58 77.01 0.56

Car 75.33 43.46 75.33 0.55

Dustbin 70.33 43.06 70.33 0.53

Mean (average) 69.33 42.88 69.33 0.53

SAE Electrocar 59.32 53.25 59.32 0.56

Rubbish 73.21 60.25 73.21 0.62

Car 76.36 62.34 76.36 0.69

Dustbin 65.87 59.98 65.87 0.63

Mean (average) 68.69 58.96 68.69 0.63

CNN Electrocar 61.67 75.51 61.67 0.68

Rubbish 80.33 77.24 80.33 0.79

Car 76.33 72.01 76.33 0.74

Dustbin 72.67 67.08 72.67 0.69

Mean (average) 72.75 72.96 72.75 0.73

ZCNN Electrocar 85.00 87.63 85.00 0.86

Rubbish 95.67 92.28 95.67 0.94

Car 87.33 85.62 87.33 0.86

Dustbin 94.00 96.58 94.00 0.95

Mean (average) 90.50 90.60 90.50 0.91

Model	Label	Accuracy/%	Precision/%	Recall/%	F1_Score
LSVM	Electrocar	54.67	41.41	54.67	0.47
Rubbish	77.01	43.58	77.01	0.56
Car	75.33	43.46	75.33	0.55
Dustbin	70.33	43.06	70.33	0.53
Mean (average)	69.33	42.88	69.33	0.53
SAE	Electrocar	59.32	53.25	59.32	0.56
Rubbish	73.21	60.25	73.21	0.62
Car	76.36	62.34	76.36	0.69
Dustbin	65.87	59.98	65.87	0.63
Mean (average)	68.69	58.96	68.69	0.63
CNN	Electrocar	61.67	75.51	61.67	0.68
Rubbish	80.33	77.24	80.33	0.79
Car	76.33	72.01	76.33	0.74
Dustbin	72.67	67.08	72.67	0.69
Mean (average)	72.75	72.96	72.75	0.73
ZCNN	Electrocar	85.00	87.63	85.00	0.86
Rubbish	95.67	92.28	95.67	0.94
Car	87.33	85.62	87.33	0.86
Dustbin	94.00	96.58	94.00	0.95
Mean (average)	90.50	90.60	90.50	0.91

Fig. 7.

ROC curve.

Fig. 8.

Mean square error curve.

3.5. Multi-classification experiment based on ImageNet data set

The ImageNet dataset is currently the world’s largest image recognition library, containing more than 14 million high-definition pictures. In the ILSVRC-2012 competition, Alex et al. designed the AlexNet model and tested the top-level image in a subset of ImageNet. The error rate of 5 is 15.3% [14]. In addition, the VGG-Net designed by Simonyan et al. has an error rate of 7.3% in the ILSVRC2014 competition based on this data set [23]. Based on this data set, the Google-Net designed by Szegedy et al. measured a top-5 error rate of only 6.67% [25]. This type of network structure is a large-scale deep neural network model based on multi-GPU parallel acceleration. Although it can obtain high accuracy in the recognition of large databases, it is much higher than the training time and cost of this article. The designed model is not suitable for the image classification of urban management cases mentioned in this article. The specific model structure is shown in Table 4.

Table 4
Model structure comparison (the total number of layers only calculates the convolutional layer and the fully connected layer)

Model name AlexNet VGG-Net Google-Net ZCNN

Total floors 8 19 22 4

Convolutional layers 5 16 21 3

Convolution kernel size 11, 5, 3 3 7, 1, 3, 5 13

Number of fully connected layers 3 3 1 1

Number of GPUs used 2 4 Multiple 0

Training time 5∼6 days 2∼3 weeks 1 week 8 hours

Model name	AlexNet	VGG-Net	Google-Net	ZCNN
Total floors	8	19	22	4
Convolutional layers	5	16	21	3
Convolution kernel size	11, 5, 3	3	7, 1, 3, 5	13
Number of fully connected layers	3	3	1	1
Number of GPUs used	2	4	Multiple	0
Training time	5∼6 days	2∼3 weeks	1 week	8 hours

It can be seen from Table 4 that the method proposed in this article does not use GPU acceleration and uses an ordinary computer configuration (Core i5, 4 GB memory, MATLAB2018a) although only part of the data in the ImageNet data set is used for experiments. It can still complete the training of the model quickly. Compared with larger network structures such as AlexNet, VGG-Net, and Google-Net, it greatly shortens the training time, reduces costs, and improves work efficiency. It is suitable for the classification of urban management cases mentioned in this article and other small and medium-sized application scenarios.

In the ImageNet database, 4 types of image data are randomly selected for classification experiments with 1300 images each, a total of 5200 images, and the experimental data is shown in Fig. 9.

Fig. 9.

ImageNet data sample graph.

In the experiment, 4 types of pictures are selected as the training set, each with 1000 pictures, a total of 4000 pictures, and the remaining 300 pictures, a total of 1200 pictures, as the test set. The experimental results are shown in Table 5. According to Table 5, it can be seen that the method proposed in this paper can still obtain a higher accuracy rate for the ImageNet data set in a shorter training time. The ROC curve is shown in Fig. 10, and the mean square error curve is shown in Fig. 11. It can be seen that convergence can be achieved in a relatively short time.

Table 5

Multi-classification experiment results of ImageNet dataset

Label	Accuracy/%	Precision/%	Recall/%	F1_Score	Train_Time/h
Cock	86.01	84.28	86.01	0.85
Carp	87.77	83.97	87.77	0.86
Gold-fish	89.23	84.19	89.23	0.82
Bird	85.15	85.25	85.15	0.85
Mean (average)	86.94	84.42	86.94	0.85	7.5

Fig. 10.

ROC curve.

Fig. 11.

Mean square error curve.

4. Conclusion

As the application of smart city management systems becomes more and more popular, the number of cases reported daily is also increasing. For a large number of urban management cases reported by citizens and staff, it is time-consuming and labor-intensive to select different types of urban management cases, which is extremely inefficient, and due to personal misunderstandings of different types of cases, a large number of cases are misclassified, it results in the city disorders in managing cases.

In this paper, based on the deep learning convolutional neural network algorithm, an improved ZCNN algorithm is designed to automatically classify the images of urban management cases, so as to achieve the effect of automatic classification of cases. First, the city management case image are obtained, the case image is preprocessed with ZCA, an 8-layer convolutional neural network model is designed, and a $13 \times 13$ convolution kernel is used to convolve the image. In order to accelerate the training of the model, a linear correction unit ReLU is used as an activation function, average-pooling is used in the pooling layer, and dropout is used after pooling to reduce the overfitting of the algorithm. The BP algorithm is used to optimize the model to improve the robustness of the model. In the fully connected layer, the softmax classifier is used for classification. Through the use of road traffic and city appearance and environment two types of case images, a total of 2600 images were used for the two-classification experiment and the use of electric vehicles for random placement, littering, illegal parking of cars, and uncleanliness around the garbage bin. A total of 5200 images were used for four-classification experiments, and compared with LSVM, SAE, and unimproved CNN algorithms to calculate various Accuracy (precision), Precision (accuracy), Recall (recall rate), F1_Score (F1 value)) and the total average accuracy Mean_acc, average accuracy Mean_pre, average recall Mean_recall and average F1 value Mean_F1, the results show that the classification results of this method are significantly better than other methods. Applying this method to the public data set ImageNet, while keeping the original number of layers unchanged and not using GPU acceleration, compared with large networks such as AlexNet, VGG-Net, Google-Net, it greatly shortens the training time, reduces the size of the model itself, saves costs, and still maintains good classification accuracy. In the future, it is necessary to further improve the automatic generation of model structure, real-time data online training and warning prediction.

Footnotes

Acknowledgements

This work was supported by the Scientific Research Project (NO. 17C0893, NO. 20B335, NO. 18C1122) of Hunan Provincial Education Department, China.

Conflict of interest

The authors have no conflict of interest to report.

References

Adam and

A.Y.

Ng, Learning feature representations with K-means, in: Neural Networks: Tricks of the Trade, Springer, Berlin Heidelberg, 2012, pp. 561–580.

Akata,

Perronnin,

Harchaoui et al., Label-embedding for image classification, IEEE Transactions on Software Engineering 38(7) (2016), 1425–1438.

Alazab,

Khan,

S.S.R.

Krishnan,

Q.V.

Pham,

M.P.K.

Reddy and

T.R.

Gadekallu, A multidirectional LSTM model for predicting the stability of a smart grid, IEEE Access 8 (2020), 85454–85463. doi:10.1109/ACCESS.2020.2991067.

Bahrampour,

N.M.

Nasrabadi,

Ray et al., Multimodal task-driven dictionary learning for image classification, IEEE ransactions on Image Processing 25(1) (2016), 24–38. doi:10.1109/TIP.2015.2496275.

L.F.

Chen,

W.J.

Su et al., A fuzzy deep neural network with sparse autoencoder for emotional intention understanding in human–robot interaction, IEEE Transactions on Fuzzy Systems: A Publication of the IEEE Neural Networks Council 28(7) (2020), 1252–1264, 2020.2966167. doi:10.1109/TFUZZ.

Cheng,

Wang,

Si et al., Maximum burning rate and fixed carbon burnout efficiency of power coal blends predicted with back-propagation neural network models, Fuel 172 (2016), 170–177. doi:10.1016/j.fuel.2016.01.035.

Dan,

Meier and

Schmidhuber, Multi-column deep neural networks for image classification, Computer Vision and Pattern Recognition 157(10) (2012), 3642–3649.

T.R.

Gadekallu,

D.S.

Rajput,

M.P.K.

Reddy,

Lakshmanna et al., A novel PCA-whale optimization-based deep neural network model for classification of tomato plant diseases using GPU, Journal of Real-Time Image Processing 6 (2020), 1–14. doi:10.1007/s11554-020-00987-8.

Gams,

I.H.

Gu,

Härmä et al., Artificial intelligence and ambient intelligence, Journal of Ambient Intelligence and Smart Environments 11(1) (2019), 71–86. doi:10.3233/AIS-180508.

10.

G.E.

Hinton and

R.R.

Salakhutdinov, Reducing the dimensionality of data with neural networks, Science 313(5786) (2006), 504–507. doi:10.1126/science.1127647.

11.

G.E.

Hintonn,

Srivastava,

Krizhevsky et al., Improving neural networks by preventing co-adaptation of feature detectors, Computer Science (2012), arXiv:1207.0580.

12.

Jain and

H.S.

Seung, Natural image denoising with convolutional networks, in: Advances in Neural Information Processing Systems 21, Proceedings of the Twenty-Second Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 8–11, 2008, pp. 769–776.

13.

A.R.

Javed,

L.G.

Fahad,

A.A.

Farhan et al., Automated cognitive health assessment in smart homes using machine learning, Sustainable Cities and Society 65(9) (2020), 102–122. doi:10.1016/j.scs.2020.102572.

14.

Krizhevsky,

Sutskever and

G.E.

Hinton, ImageNet classification with deep convolutional neural networks, in: ANIPS’12: Proceedings of the 25th International Conference on Neural Information Processing Systems, 2012, pp. 1097–1105.

15.

Lawrence,

C.L.

Giles,

A.C.

Tsoi et al., Face recognition: A convolutional neural-network approach, IEEE Transactions on Neural Networks 8(1) (1997), 98–113. doi:10.1109/72.554195.

16.

Lindeberg, Scale invariant feature transform, Scholarpedia 7(5) (2012), 2012–2021. doi:10.4249/scholarpedia.10491.

17.

S.X.

Lu and

X.Z.

Wang, A comparison among four SVM classification methods: LSVM, NLSVM, SSVM and NSVM, in: Proceedings of 2004 International Conference on Machine Learning and Cybernetics, 2004, pp. 4277–4282. doi:10.1109/ICMLC.2004.1384589.

18.

Y.W.

Lu and

G.F.

Zhai, Applications and exploration of artificial intelligence technology in urban disaster risk management, Urban Planning International 36(2) (2021), 22–31. doi:10.19830/j.upi.2021.031.

19.

A.L.

Maas,

A.Y.

Hannun and

A.Y.

Ng, Rectifier nonlinearities improve neural network acoustic models, Proc ICML (International Conference on Machine Learning) 30 (2013).

20.

D.P.

Ming,

J.C.

Luo and

Z.F.

Shen, Research on region partition in high resolution remote sensing image based on GMRF-SVM, Science of Surveying and Mapping 34(2) (2009), 33–37. doi:10.3771/j.issn.1009-2307.2009.02.010.

21.

S.U.

Rehman,

A.R.

Javed,

M.U.

Khan et al., Personalised comfort: A personalised thermal comfort model to predict thermal sensation votes for smart building residents, Enterprise Information Systems 12, 1–23. doi:10.1080/17517575.2020.1852316.

22.

M.U.

Sarwar and

A.R.

Javed, Collaborative Health Care Plan through Crowdsource Data using Ambient Application, in: 22nd International Multitopic Conference (INMIC), 2019, Islamabad, Pakistan. doi:10.1109/INMIC48123.2019.9022684.

23.

Simonyan and

Zisserman, Very deep convolutional networks for large-scale image recognition, Computer Science (2014), arXiv:1409.1556.

24.

Streitz,

Charitos,

Kaptein and

Böhlen, Grand challenges for ambient intelligence and implications for design contexts and smart societies, Journal of Ambient Intelligence and Smart Environments 11(1) (2019), 87–107. doi:10.3233/AIS-180507.

25.

Szegedy,

Liu,

Jia et al., Going deeper with convolutions, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 7–12, 2015, Boston, MA, USA, pp. 1–9. doi:10.1109/CVPR.2015.7298594.

26.

Venkatraman,

Alazab and

Vinayakumar, A hybrid deep learning image-based analysis for effective malware detection, Journal of Information Security and Applications 47 (2019), 377–389. doi:10.1016/j.jisa.2019.06.006.

27.

Vinayakumar,

Alazabm,

Srinivasan,

Q.V.

Pham,

S.K.

Padannayil and

Simran, A visualized botnet detection system based deep learning for the Internet of things networks of smart cities, IEEE Transactions on Industry Applications 56(4) (2020), 4436–4456. doi:10.1109/TIA.2020.2971952.

28.

C.M.

Wang,

Z.Y.

Wang et al., Gabor & PCA-based face image recognition method, Information Technology 5(2) (2021), 20–25. doi:10.13274/j.cnki.hdzj.2021.02.004.

29.

J.Y.

Wang,

Li,

Xiong et al., Survey of data-centric smart city, Journal of Computer Research and Development 51(2) (2014), 239–259. doi:10.7544/issn1000-1239.2014.20131586.

30.

Yang, Research on construction of digital intelligent city management system, International Journal of Hybrid Information Technology 7(5) (2014), 285–294. doi:10.14257/ijhit.2014.7.5.26.

31.

Yang,

Yu,

Gong et al., Linear spatial pyramid matching using sparse coding for image classification, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2009), 1794–1801.

32.

L.J.

Zhon,

H.W.

Wang et al., Novel perspective for urban water resource management: 5R generation, Frontiers of Environmental Science & Engineering 15(1) (2021), 1–13. doi:10.1007/s11783-020-1293-2.

33.

Zhou and

Wei, Learning hierarchical spectral spatial features for hyperspectral image classification, IEEE Transactions on Cybernetics 46(7) (2015), 1667–1678. doi:10.1109/TCYB.2015.2453359.