Improved LSTM data analysis system for IoT-based smart classroom

Abstract

To reduce the resource and energy waste of colleges and universities more accurately and efficiently, this paper has developed a smart classroom data analysis system based on the Internet of Things, which realizes a variety of sensor information (temperature, humidity, smoke). Environmental parameters such as carbon dioxide concentration and light intensity), remote collection of equipment information, data storage and data analysis functions, and intelligent control of smart classrooms. Data analysis uses an improved LSTM model to predict energy consumption. The model uses LSTM and bidirectional LSTM and uses the ELU activation function instead of the sigmoid and tanh activation functions of the LSTM. Compared with the standard LSTM model and the LSTM model without the ELU activation function, the model improves the prediction accuracy, better avoid the gradient disappearance, and reduces the over-fitting. The system implementation results show that the system can effectively reduce school energy waste.

Keywords

Internet of things smart classroom data analysis Bi-LSTM

1 Introduction

In recent years, the Internet of Things has developed rapidly, and the agricultural Internet of Things, Industrial Internet of Things, and smart cities have also developed rapidly [1, 2]. Juan Juan and others developed a water quality monitoring system for aquaculture ponds by combining with the narrowband Internet of Things [3]; Wu Hui and others realized a farmland information collection system based on the Internet of Things model [4], which includes Information collection device, cloud server, mobile phone terminal, and PC terminal; Luo Le has realized a solar power generation monitoring system based on the Internet of Things [5]; Wang Dong et al. Library Internet monitoring system for WiFi Internet of Things [6]; Li Chao et al. Implemented a classroom intelligent energy-saving system based on WSNs and power carrier waves [7], which controlled the classroom to save energy by setting thresholds on sensors [8].

Although the data collected by the above applications of the Internet of Things are all-time series, they have not been analyzed in-depth [9]. As an important place for teaching tasks in colleges and universities, classrooms are also developing in a more intelligent and information-oriented direction. Compared with middle school classrooms, college classrooms are more open, so their corresponding classroom management needs are also stronger. Most colleges and universities lack the corresponding classroom resource management control system, and there is a serious waste of energy. Because the equipment in college classrooms is generally managed manually, energy waste caused by equipment forgetting to shut down in time often occurs [10].

In response to this situation, this article uses the Internet of Things architecture to build a smart classroom and a smart classroom data analysis system. Through real-time recording of classroom environmental data, equipment usage data, and time series analysis using LSTM models, and then predicting future energy consumption, thereby reducing energy waste.

The thesis is mainly divided into the following parts. First, the overall structure of the system is introduced in related work. In the part of neural network model construction, the advantages and disadvantages of LSTM and Bi-LSTM are analyzed first, as well as the influence of different activation functions on these two models, and then the improved LSTM model is proposed by adjusting the model structure. In the LSTM experiment and analysis part, the improved LSTM model was used to predict the classroom equipment and environmental data of the School of Information Technology, Shanghai Jianqiao University. The last part is to use this model to intelligently control classroom equipment, which can achieve a certain energy-saving effect.

2 Related Work

The smart classroom data monitoring system designed in this paper includes data collection, remote transmission, storage management, data monitoring, data analysis, and intelligent control functions. It implements distributed monitoring of smart classroom environment data and intelligent management of smart classroom equipment. The system consists of a four-layer structure such as perception layer, transmission layer, platform layer and application layer.

The perception layer is composed of smart classrooms, including single-chip computers, communication modules, power modules, various sensor modules, and equipment control modules. The transport layer is an intelligent gateway, which transmits uplink and downlink data through the intelligent gateway. The platform layer uses the cloud platform to perform operations on the database, receive processing results from the application layer, and pass device control data to the perception layer through the transmission layer. The application layer uses the data analysis system to obtain the equipment control results by acquiring data from the cloud platform, performing data visualization and data analysis. The overall system structure is shown in Fig. 1.

Fig. 1

System structure diagram.

As shown in Fig 2, the Smart Classroom Data Analysis System was developed using Django, an open-source web application framework written in Python. The framework uses MTV (model, template, view) software design pattern, which can be used to develop database-driven websites easily and quickly. The system acquires data of the database from the cloud platform, and can perform real-time monitoring of classroom environment data, display classroom equipment status, and remote control. The acquired data uses a multi-layered improved LSTM model for energy prediction. Since the acquired classroom equipment data is a time series, LSTM is used here for energy consumption prediction. Finally, using data visualization, the data is displayed in the form of a line chart.

Fig. 2

Smart Classroom Data Analysis system structure.

3 Construction of neural network model

3.1 LSTM neural network model

LSTM [8 –14] is a special RNN that can learn long-term dependency information. LSTM Structure is shown in Fig. 3. There is a hidden layer state similar to the original RNN in the LSTM structure [15], which is very sensitive to short-term inputs and adds a state C to maintain long-term memory. This state, also known as the cell state, records information that is passed over time. The cell state information is updated during the transfer process by the current input, the hidden layer state at the previous time, the cell state at the previous time, and the gate structure. The gate structure uses the sigmoid function to control the degree of update by taking values between 0 and 1. There are three gate structures in the LSTM network, namely the forgetting gate, the memory gate and the output gate. These three gates control the extent to which the state of the cell C is continuously preserved, the degree to which the state of the temporary cell is applied to the state of the cell, and whether the cell state is the output of the current hidden layer state.

Fig. 3

LSTM Structure.

For the traditional form of genetic algorithm, we use, the whole computational process of the algorithm is centered on the individual [16 –18]. The advantage of an individual computing center is that it is computationally fast and can reduce the number of

As shown, it x_t is set to the current t-time input, C_t is the cell state value at time t, and h_t is the cell^′s output.

σ is the sigmoid function, as shown in Equation 1. φ is the tanh function, as shown in Equation 2. $σ (x) = \frac{1}{1 + e^{- x}}$ (1) $φ (x) = \frac{e^{x} - e^{- x}}{e^{x} + e^{- x}}$ (2)

The update of the memory unit is divided into the following steps according to the gate structure. $f_{t} = σ (W_{f} \cdot [h_{t - 1}, x_{t}] + b_{f})$ (3) $i_{t} = σ (W_{i} \cdot [h_{t - 1}, x_{t}] + b_{i})$ (4) $o_{t} = σ (W_{o} \cdot [h_{t - 1}, x_{t}] + b_{o})$ (5) ${\tilde{C}}_{t} = φ (W_{c} \cdot [h_{t - 1}, x_{t}] + b_{c})$ (6) $C_{t} = f_{t} \cdot C_{t - 1} + i_{t} \cdot {\tilde{C}}_{t}$ (7) $h_{t} = o_{t} \cdot φ (C_{t})$ (8)

(3) Forgetting gate calculation: f_t represents the forgotten gate output at time t; [h_t-1, x_t] represents the splicing of the two matrices into a matrix; W_f and b_f is the weight matrix and offset at time t-1, respectively item.

(4) Memory gate calculation: i_t represents the memory gate output at time t; W_i and b_i is the weight matrix and offset term of the input gate at time t-1, respectively.

(5) Output gate calculation: o_t is the output gate output at time t; W_o and b_o are the weight matrix and offset term of the input gate at time t-1, respectively.

(6) Temporary cell state calculation: ${\tilde{C}}_{t}$ is the temporary cell state; W_c and b_c is the weight matrix and bias term calculated by the state value, respectively. Calculate the status value generated by the current input information.

(7) Cell state calculation: C_t is the cell state; the long-term state at time t-1 is multiplied by the forgetting gate output at time t, plus the product of the input gate output and the immediate state at time t.

(8) Hidden layer state calculation: h_t is expressed as the hidden layer state output at time t; the output is determined by the output gate and the cell state.

3.2 Bi-LSTM Neural Network Model

Schuster et al. [14] proposed a bidirectional recurrent neural network model (BRNN). Compared with the RNN model, this model can enhance the processing ability of the subsequent data. The RNN model can only process data in one direction. A two-way recurrent neural network puts each training sequence into a recurrent neural network that can propagate forward and backward and connect it to the output layer. Bi-LSTM neural network model is a new model formed by combining two models of bidirectional RNN and LSTM. It replaces the cyclic unit in the classic two-way RNN model with LSTM [15, 16]. Compared with the standard LSTM model, using this method can better understand the long-term dependence of time series.

Figure 4 shows the bidirectional recursive network Bi-LSTM superposition of recursive layers in two different directions. The forward recursive layer starts from the initial moment, and the reverse recursive layer starts from the last moment. $h_{t}^{f} = ϕ (W_{h}^{f} \cdot [h_{t - 1}, x_{t}] + b_{f})$ (9) $h_{t}^{b} = ϕ (W_{h}^{b} \cdot [h_{t - 1}, x_{t}] + b_{b})$ (10) $h_{t} = h_{t}^{f} + h_{t}^{b}$ (11)

Fig. 4

Bi-LSTM Structure.

Equation 9 represents the forward propagation output, Equation 10 represents a backward propagation output, and Equation 11 represents the two-way propagation output. Among them, ϕ represents the LSTM structure, $h_{t}^{f}$ represents the forward transfer output at time t, $W_{h}^{f}$ and b_f represent the weight matrix and bias term of forward propagation respectively, and represents the reverse transfer output at time t. $W_{h}^{b}$ and b_b represent the weight matrix and bias term of the backpropagation, respectively, and h_t represents the bidirectional recursive output.

3.3 Improved energy consumption prediction method of LSTM

During the training of LSTM, gradients may disappear and overfitting is likely to occur [17]. To avoid the problem of gradient disappearance, this article will forget the activation functions of gates, memory gates, output gates, and temporary unit states, hidden layers. The state activation function is replaced by the ELU activation function. The related expressions of the ELU function are as follows: $f (x) = {\begin{matrix} x, x ⩾ 0 \\ α (e^{x} - 1), x < 0 \end{matrix}$ (12) $f' (x) = {\begin{matrix} 1, x ⩾ 0 \\ α (e^{x} - 1) + α, x < 0 \end{matrix}$ (13)

Among them, f (x) and f′ (x) are the original and derivative functions of ELU, respectively, and are the independent variables of the function [18, 19]. The ELU activation function combines the characteristics of smooth saturation on the left side of the sigmoid activation function and the linearity characteristics of the right side of the ReLU activation function [19]. The right side of the activation function is a linear part, making it have a mitigating effect on the disappearance of the gradient, and the soft saturation on the left side of the function can reduce its sensitivity to outliers and improve the robustness of the model [19]. The average value of the ELU output is close to zero, and the model can converge faster than the tanh activation function and sigmoid activation function.

Based on the improved LSTM model algorithm, the Bi-LSTM network is used to further learn and train the time series predicted by the previous LSTM network through the Dropout layer. The bidirectional LSTM model combines forward LSTM and reverse LSTM. Information, and the inverse LSTM obtains later information, which has the advantages of capturing data timing and long-range dependence [7], which can further improve the accuracy of prediction results.

The Improved LSTM energy consumption prediction algorithm flow is shown in Fig. 5 including the following steps:

Fig. 5

Improved LSTM energy consumption prediction algorithm flowchart.

Step 1 Prepare the data. Get the data you need to predict from the cloud platform. The initial effective time series is calculated based on the energy consumption of multiple classrooms at the same time interval. This data uses the total daily energy consumption during the working day. By calculating the energy consumption of all equipment in the classroom during the working day, the calculation formula is: energy consumption = rated power * usage time. The data is in five columns, the first to fifth columns are the energy consumption from Monday to Friday, and each row represents the weekly energy consumption. As shown in Table 1.

Table 1

Energy data format

Mon.	Tue.	Wed.	Thu.	Fri.
151.1352	90.6005	120.8492	105.7321	60.4096
148.8623	95.6734	135.8096	112.9123	59.3867
154.2333	105.2031	137.3524	118.9686	69.9274
155.3880	106.7123	125.0778	119.0192	66.93005

Step 2 Normalize the data. Improve the model’s convergence speed and accuracy. Use 0-1 normalization to map the data so that the result is in the interval [0,1], as shown in Equation 14. Where max is the maximum value of the sample data and min is the minimum value of the sample data. $x^{*} = \frac{x - min}{max - min}$ (14)

Step 3 Divide the sample set. Two consecutive samples (two rows of data) are taken as x, and the next third sample is taken as the label y. 30% of the samples were used as test sets and 70% were used as training sets.

Step 4 Use the improved LSTM for energy consumption prediction. Since the energy consumption changes periodically according to the season [21], the experiment uses short-term predictions. The training uses the Multiple Parallel method, that is, input is multiple sequences, and output is multiple sequences. This experiment uses two different LSTM structures. The difference between the two is that one of the models uses Bi-LSTM and the other uses standard LSTM. For these two different structures, their activation functions are respectively activated. A replacement of the ELU activation function was performed. The following steps use the Bi-LSTM layer as an example.

Set the sequence length to len_seq, and then use the first len_seq-1 data to predict the data of the len_seq.

The first layer is the LSTM layer. The input is the length and dimension of each input data, and the length and dimension of the output data are obtained through calculation.

The second layer is the Dropout layer, which uses the Dropout layer to suppress 20% of neuronal activation, preventing local optimism.

The third layer is the Bi-LSTM layer. The standard LSTM processes the sequence in chronological order, taking only historical information and ignoring future information. Therefore, using the bidirectional LSTM network to extract forward and reverse classroom energy consumption sequence information, The forward recursive layer begins with the initial classroom energy consumption time of the predicted period, and the backward recursive layer begins with the last classroom energy consumption time.

The fourth layer is the LSTM layer and the fifth layer is the Dropout layer. The last layer is a fully connected layer, without the activation function. The minimum error is the stochastic gradient descent method (SGD).

Step 5 The trained model is used to predict the test set, and the predicted classroom energy consumption value is denormalized and an error analysis is performed with the real classroom energy consumption value.

Step 6 Display the real and predicted values using a line chart to observe the effect. Save the current model.

The specific structures of the two models are shown in Figs. 6 and 7.

Fig. 6

Model structure diagram using Bi-LSTM layer.

Fig. 7

Model structure diagram using only LSTM layer.

Fig. 8

Bi-LSTM+ELU.

4 LSTM Experiment and analysis

4.1 Experimental prediction results

The data used in this article is the classroom environment and equipment data of the School of Information Technology, Shanghai Jianqiao University from March to June, and July to October 2019. The specific prediction effect is shown in the figure below. Figure 8 shows the model using the Bi-LSTM layer and the ELU activation function, Fig. 9 shows the model using the Bi-LSTM layer and the default activation function, and Fig. 10 shows the LSTM model using the ELU activation function. Figure 11 shows a model using a standard LSTM.

Fig. 9

Bi-LSTM.

Fig. 10

LSTM+ELU.

Fig. 11

LSTM.

4.2 Evaluation index

The evaluation index uses the mean square error (MSE) to measure the prediction result, as shown in Equation 15. Where y_i represents the true value and ${\hat{y}}_{i}$ represents the predicted value. $MSE = \frac{1}{m} \sum_{i = 1}^{m} (y_{i} - {\hat{y}}_{i})^{2}$ (15)

4.3 Analysis of results

After comparison, the classroom energy consumption errors for the next 15 working days are predicted to be 0.03931%, 0.04218%, 0.04651%, and 0.04926%, as shown in Table 2. The model using the Bi-LSTM layer has a smaller prediction error than the model without this layer, and the model using the ELU activation function has a smaller prediction error than the model without the ELU activation function. Since the change of classroom energy consumption itself has a certain periodicity, it is more suitable for prediction in the form of time series. The Bi-LSTM + ELU model can ensure better prediction accuracy.

Table 2
Model prediction error table

Model MSE

Bi-LSTM + ELU 0.03931%

Bi-LSTM 0.04218%

LSTM + ELU 0.04651%

LSTM 0.04926%

Model	MSE
Bi-LSTM + ELU	0.03931%
Bi-LSTM	0.04218%
LSTM + ELU	0.04651%
LSTM	0.04926%

5 Classroom equipment intelligent control experiment

5.1 Specific experimental process

As shown in Fig. 12, the intelligent classroom data analysis system reads the data uploaded by the sensor nodes of the perception layer to the database and uses the improved LSTM energy prediction method to train the model, that is, the model with the best experimental prediction performance (Bi-LSTM + ELU)), Save the trained model [26]. When using the device intelligent control function, first read the saved prediction model, obtain the state of the device at that time through the prediction model, and finally send the device state to the sensor nodes of the perception layer according to the time node.

Fig. 12

Classroom intelligent control flowchart.

5.2 Experimental results

In this experiment, the monitoring data of the School of Information Technology of Shanghai Jianqiao University in September 2019 was selected. The sensor node upload period was set to 15 s. By recording two classrooms with similar courses, the recording time was September 14–18, 2019. The data analysis system was installed in one of the classrooms and the other was not. A comparison of power consumption is shown in Fig. 13. Compared with classrooms without this system, the average daily energy consumption of a single classroom is reduced by 23.41%, and an average of 32.32 kWh is saved.

Fig. 13

Energy consumption graph.

6 Conclusion

This paper develops an IoT smart classroom data analysis system using improved LSTM. The system collects classroom data continuously and periodically through the nodes of the perception layer and uses an improved LSTM model to predict energy consumption. First use the classroom equipment to use the data to train an LSTM model, and use its output as the input of the next Bi-LSTM model, and then make further deep-level predictions. After a large number of experiments, it is verified that the algorithm is superior to the existing LSTM prediction methods in terms of accuracy, and the method is feasible and effective. After experiments, the system can reduce energy waste and has certain feasibility and practicability.

References

A Low-Overhead, Confidentiality-Assured, and Authenticated Data Acquisition Framework for IoT, Yushu Zhang; Qi He; Guo Chen; Xinpeng Zhang; Yong Xiang, IEEE Transactions on Industrial Informatics, Publication Year: 2019, pp. 1–1

, Li

, Wang

, Zhang

, Hu

and Feng

, Government Affairs Service Platform for Smart City, Future Generation Computer Systems 81 (2018), 443–451.

Juan

, Fan

, Weijian

, Hui

and Xingqiao

, Development of Water Quality Monitoring System for Aquaculture Pond Based on Narrowband Internet of Things, Transactions of the Chinese Society of Agricultural Engineering 35(08) (2019), 252–261.

Hui

, Hongyan

, Hongjun

, Gang

, Yan

, Pei

and Xuefei

, Farmland Information Collection System Based on Internet of Things Mode, Electronic Devices 42(04) (2019), 1056–1062.

, Design of Solar Power Generation Monitoring System Based on the Internet of Things, Instrumentation Technology and Sensors (11) (2018), 104–107.

Dong

, Wei

and Di

, Library Environment Monitoring System Based on WiFi Internet of Things, Computer Science 45(S2) (2018), 532–534 + 557.

Chao

, Shengli

and Yuqiang

, Design of Intelligent Energy Saving System in the Classroom Based on WSNs and Power Carrier, Measurement & Control Technology 36(07) (2017), 92–96 + 100.

Yu-Feng

, He

, Su-Ping

and Xin

, Beam Compressed System in Front of Wavefront Sensor, Acta Microscopica 28(5) (2019).

Tan

H.R.

, Global Financial Crisis and the Impact of the Food Crisis on the Great Depression Based on Agricultural Technology Iteration: A 15-year Population-Based Time-Series Study, Revista de la Facultad de Agronomia de la Universidad del Zulia 36(2), 2019.

10.

and Qian

, Text sentiment analysis based on long short- term memory. Proceedings of the 2016 IEEE First International Conference on Computer Communication and the Internet. Piscataway: IEEE (2016), 471–475.

11.

Sahoo

, Jha

and Prashar

, A novel approach for Spam Email Filtering Using Machine Learning, Journal of Cybersecurity and Information Management 2(2) (2020), 44–57.

12.

Mohamed

, A comparative study on Internet of Things (IoT): Frameworks, Tools, Applications and Future directions, Journal of Intelligent Systems and Internet of Things 1(1) (2020), 13–39.

13.

Christianto

, Boyd

R.N.

and Smarandache

, Three Possible Applications of Neutrosophic Logic in Fundamental and Applied Sciences, International Journal of Neutrosophic Science 1(2) (2020), 90–95.

14.

, Hongsu

, Sanqian

, Qunchao

and Jun

, Sentence segmentation for classical Chinese based on LSTM with radical embed-ding, The Journal of China Universities of Posts and Telecommunications 26(02) (2019), 1–8.

15.

Liu

, Dong

, Lu

and Wang

, LSTM Based Reserve Prediction for Bank Outlets, Tsinghua Science and Technology 24(01) (2019), 77–85.

16.

Gaofeng

, Xin

and Yonghong

, Using highway connection to realize deep and small LSTM-RNN speech recognition, Acta Electronica Sinica 28(01) (2019), 107–112.

17.

Cho

, van Merrienboer

, Gulcehre

, et al., Learning phrase representations using RNN en-coder-decoder for statistical machine translation. Proceedings of the 19th International Conference on Empirical Methods in Natural Language Pro-cessing. Stroudsburg: Association for Computational Linguistics (2014), 1724–1735.

18.

, Sun

, Cioffi

J.M.

and Yang

, Energy Efficient MIMO Relay Transmissions via Joint Power Allocations, IEEE Transactions on Circuits and Systems II: Express Briefs 61(7) (2014), 531–535.

19.

Luo

, Graphical Design of Deformed Koch Curve Fractal Animal Based on Wavelet Function, Revista Cientifica-Facultad de Ciencias Veterinarias 29(5) (2019).

20.

Schuster

and Paliwal

K.K.

, Bidirectional recurrent neural networks, IEEE Transactions on Signal Processing 45(11) (1997), 2673–2681.

21.

Shan

, Lai

and Liu

, Correlational Analytical Characterization of Energy Dissipation-Liberation and Acoustic Emission during Coal and Rock Fracture Inducing by Underground Coal Excavation, Energies 12(12) (2019), 2382.

22.

Zhengbin

and Yangsen

, Bidirectional LSTM word segmentation optimization method based on fixed-length sequences, Journal of Zhengzhou University (Science Edition) 50(02) (2018), 97–101.

23.

Zhaorong

, Tinglei

, Wenjuan

and Guanghuan

, Trajectory outlier detection algorithm based on Bi-LSTM model, Acta Radar 8(01) (2019), 36–43.

24.

Yongjun

and Haishan

, Research on Food Safety Network Public Opinion Early Warning Based on LSTM Model, Computer Engineering and Science 41(09) (2019), 1603–1611.

25.

Clevert

D.A.

, Unterthiner

and Hochreiter

, Fast and accurate deep network learning by exponential linear units (elus), ICLR, 2015.

26.

Cheng

, Zhao

X.G.

and Niu

W.C.

, ApplicationTechnology of New Nanomaterials Electrochemi (2019).