Intelligent analysis of classroom student state based on neural network algorithm and emotional feature recognition

Abstract

Classroom teaching in the context of artificial intelligence needs to be combined with modern intelligent recognition technology to improve classroom teaching efficiency. In order to study the auxiliary teaching system for classroom student management, this article is based on neural network technology and emotional feature recognition algorithm, and according to the actual situation of classroom teaching, an intelligent analysis system for classroom student status is constructed. The system simulates the RFID mode to tag the students. Moreover, this article sets the system function module according to the actual teaching management needs and designs the learning algorithm of the quantitative assessment model. In addition, this study uses machine learning methods to design the quantitative evaluation index system, logistic regression scoring algorithm and model training algorithm. Finally, this study uses the neural network algorithm as the comparison algorithm to verify the performance of the constructed model and analyzes the comparison results through chart comparison. The research results show that the model proposed in this paper has good performance and can be applied to practical classrooms.

Keywords

Neural network emotion recognition student status intelligent recognition

1 Introduction

With the accelerated development of information technology that takes intelligence and data as advanced representatives and characteristics, as well as the opening and promotion of artificial neural network learning frameworks, deep learning core algorithms, development languages such as Python, tensorflow and other development platforms are open and popularized, the application of artificial intelligence in various fields is showing a vigorous trend, such as news recommendation, intelligent marketing, AI art, intelligent search engine, machine translation, autonomous driving, games, smart robots, etc. It can be said that where there is accumulation of big data, there is the birth of artificial intelligence. Machine learning and artificial intelligence, like the Internet, have become an available information resource that can help people work, study and live more efficiently and freely [1]. The application of artificial intelligence in the education industry has also developed rapidly. One is the curriculum optimization aspect: Big data analysis and learning behavior monitoring can automatically identify knowledge spots and learning weaknesses in the learning process, and optimize the configuration of the course content, learning form and learning time. The second is teaching evaluation: based on the learner’s learning feedback and test results, it can automatically formulate a learning plan, teach students according to their aptitude, provide personalized learning recommendations, and improve learning efficiency. The third aspect is smart counseling: training teacher robots can provide one-to-one counseling and psychological communication and interaction with learners, and comprehensively enhance the learning experience and motivation of individual learners [2]. In addition, applying computer artificial intelligence technology to traditional classroom teaching management and efficiency improvement is also one of the current important research areas. For example, in the establishment of student portraits, through the integrated analysis of data in the history of students in various subjects, student family information, hobbies, student status data, teacher evaluation, etc., multi-dimensional information portraits and dynamic features of students are formed, which can help teachers more comprehensively understand the information of each student. Moreover, through big data prediction, teachers can observe and pay attention to students may encounter problems and difficulties in learning and life [3].

As a biological feature, human facial features are unique and difficult to be copied. Due to the limitations of computer technology and hardware in the early days, face recognition has not been well developed. As deep learning methods are widely used in artificial intelligence and even computer vision, the speed and accuracy of face recognition have been greatly improved. At present, the accuracy rate of face recognition has exceeded the limit of human eye recognition, which has laid a solid foundation for the popularization of face recognition technology [4].

As an inseparable part of students’ daily teaching activities, the causes of emotions, the process of change and development and the influencing factors are reflected in the process of interpersonal communication. Student emotions have been greatly developed, involving teacher emotions affecting teacher behavior, teacher teaching, teacher professional identity, teacher life, student behavior and learning, and awareness of educational changes. As the object of the classroom, students’ emotions should be consciously regulated and utilized.

2 Related work

A series of related technologies that collect face images or videos through camera equipment, and automatically detect and track faces in the images, and then recognize the detected faces are called face recognition [5]. The key to the success of the face recognition system is whether it has cutting-edge core algorithms and whether the recognition results have a practical recognition rate and speed [6]. Face recognition technology integrates various professional technologies such as artificial intelligence, machine recognition, machine learning, model theory, expert system, video image processing and so on. It belongs to the field of artificial intelligence machine vision research, is currently the most widely used machine vision technology, and is also one of the most important research branches of artificial intelligence [7]. Deep learning is currently the most mainstream machine learning method in the big data network environment. It uses big data correlation theory and multi-layer neural networks to implement artificial intelligence. Deep learning is a machine learning method based on artificial neural networks [8]. The machine learning principle of the multi-layer neural network structure is to use the low-level feature output value as the input variable of the next layer and as a more abstract high-level feature value, thereby discovering more complex distributed features of the data.

The convolutional neural network proposed in [9] uses spatial relative relationships to reduce the number of parameters involved in calculations, which makes it the first true multi-layer learning algorithm and effectively improves the trainability of the model.

The application of face recognition technology is also very extensive, such as security, justice, criminal investigation, electronic identity, company attendance, financial services, information security, information search and other fields. It is mainly used to identify and verify the true identity of the observed object, or to locate, search and retrieve the target object [10].

At present, in addition to the application of static face authentication, the more advanced is the recognition of facial expressions and behaviors and feature capture to analyze the performance and status of observed objects during the observation period [11]. The application of face recognition technology in the education industry has extremely critical value, just like opening the “eye of wisdom” in traditional classroom education.

The literature [12] proposed Deep Face, which uses convolutional neural networks and large-scale face datasets for training. The data set includes 4.4 million facial images from 4030 people. Moreover, this literature used the Softmax loss function. The model achieved an accuracy rate of 97.35% in the LFW test set, reaching the level of manual recognition. The literature [13] made a series of improvements to the structure of convolutional neural networks, and finally improved the accuracy of face recognition to more than 99%. FaceNet proposed in the literature [14] uses a ternary loss function instead of the traditional Softmax loss function. The ternary loss function uses the same face as a positive sample and different faces as negative samples and calculates the loss through the feature distance between the samples. The method achieves 99.63% accuracy on the LFW. The literature [15] proposed to optimize the feature layer through the Center-Loss center loss function, so that the resulting feature categories are gathered toward the center. When applied to face recognition, it can also have a good classification effect when there are few training samples. In recent years, the development of face recognition technology mainly relies on the continuous optimization of the Softmax loss function. The core is to continuously increase the extra-class spacing and reduce the intra-class distance through improvement. The L-Softmax proposed in the literature [16] removes the offset term in the last layer of the network, directly optimizes the cosine angle between features, and adds a fixed value to the angle to make the classification achieve the desired effectThe A-Softmax proposed in the literature [17], the CosFace proposed in the literature [18] and the AM-Softmax proposed in the literature [19], and the ArcFace proposed in the literature [20] been improved to varying degrees on the basis of the Softmax loss function, and have achieved relatively good results. The article [25] implementated IoT-based Smart City is achieved by exploiting IoT and BigData Analytics using Hadoop ecosystem in real time environments. The article [26] reflects on IoT and its main role in the development of human behaviors and actions. The paper also deals with the compilation of various data from different databases connected to the Internet. The literature [27] addresses the numerous issues in the field of vehicle communication with the suggestion for a mutual unified and dispersed spectrum sensing model. The introduction of a mutual cognitive paradigm minimizes conflict and multiple unknown problems. The literature [28] discusses the issue, such as large amount of bigdata, and introduces the Smart Buddy framework for creating smart and adaptive ecosystems using human behaviors and human dynamics. The article [29] talks around the development of coordinated non-cyclic chart for video coding calculations for movement estimation in parallel reconfigurable computing frameworks [30]. The partitioning algorithm moreover plays a key part in optimizing the encoding of images [31].

3 Dynamic analysis solution steps

The dynamic analysis process based on LS-DYNA mainly includes four parts: problem planning, pre-processing, solving and post-processing, as shown in Fig. 1.

Fig. 1

LS-DYNA flow chart.

LS-DYNA analyzes on the basis of discrete mathematics, which is mainly based on La-grange algorithm, and also has ALE algorithm and Euler algorithm. For the general dynamic model, the solution method is to solve the dynamic problem through the central difference method, and the relationship between displacement, velocity and acceleration is as follows.

The displacement at a certain moment is expressed as [21]: ${\dot{U}}_{i} = \frac{U_{i + 1} - U_{i - 1}}{2 Δ t}$ (1) ${\ddot{U}}_{i} = \frac{U_{i + 1} - 2 U_{i} + U_{i - 1}}{Δ t^{2}}$ (2)

In the formula, $U_{(i)}, {\dot{U}}_{(i)}, {\ddot{U}}_{(i)}$ represents the displacement, velocity and acceleration at the initial moment.

When solving dynamic problems, the equation of motion is discretized by the finite element method to obtain the differential equation of motion: $[M] \cdot {\ddot{U}} + [C] \cdot {\dot{U}} + [K] \cdot {U} = {F_{i}}$ (3)

Formulas (1) and (2) are substituted into (3) to obtain: $\begin{matrix} [M] \cdot \frac{U_{i + 1} - U_{i - 1}}{2 Δ t} + [C] \cdot \frac{U_{i + 1} - 2 U_{i} + U_{i - 1}}{Δ t^{2}} \\ + [K] \cdot U_{i} = {F_{i}} \end{matrix}$ (4)

In the formula, U represents the displacement of discrete points [22]. $U_{i} = U_{(t_{i})}, {\dot{U}}_{i} = {\dot{U}}_{(t_{i})}, {\ddot{U}}_{i} = {\ddot{U}}_{(t_{i})}, i = 0, 1, 2, \dots$ (5)

The recursive formula of the load vector solution at each discrete time point is sorted out: $\begin{matrix} [\frac{1}{Δ t^{2}} M + \frac{1}{2 Δ t} C] U_{i + 1} = F_{i} - K - \frac{2}{Δ t^{2}} {MU}_{i} \\ - (\frac{1}{Δ t} + \frac{1}{2 Δ t} C) U_{i - 1} \end{matrix}$ (6)

In the formula, $\begin{matrix} [M] = \sum_{i}^{n_{i}} {[m]}^{i} \\ [C] = \sum_{i}^{n_{i}} {[c]}^{i} \\ [K] = \sum_{i}^{n_{i}} {[k]}^{i} \end{matrix}$ (7)

M represents the mass matrix, C represents the damping matrix, and K represents the stiffness matrix. At the same time, {F_i} represents the node load vector, and Δt represents the time interval.

4 Working principle of H-S wavefront sensor

Figure 2 shows the measurement schematic diagram of the H-S wavefront sensor. The lens array divides the wavefront of the incident beam to form an array spot on the photosensitive surface of the CCD. If a wavefront with aberration is incident, the spot passing through the microlens is shifted. According to the offset information of the array spot, the slope information of each sub-aperture is obtained, and the wavefront information of the incident beam is fitted.

Fig. 2

Schematic diagram of H-S wavefront sensor measurement principle.

Based on the calibration center, the offset of the array spot is calculated. The mathematical relationship between the offset of a single sub-aperture and the slope of the wavefront is as follows [23]: ${\begin{matrix} g_{x} = \frac{1}{s} \iint_{s} \frac{\partial φ (x, y)}{\partial x} dxdy = \frac{Δ x}{y} \\ g_{y} = \frac{1}{s} \iint_{s} \frac{\partial φ (x, y)}{\partial y} dxdy = \frac{Δ y}{f} \end{matrix}$ (8)

In the formula, g_x and g_y represent the slope in the direction of x, y, s represents the area of the sub-aperture, and φ (x, y) represents the spatial distribution of the wavefront. Meanwhile, f represents the focal length, and Δx, Δy represents the offset.

The offset is obtained by formula (9): ${\begin{matrix} Δ x = \frac{\sum_{i}_{, j x_{i j} I_{i j}}}{\sum_{i}_{, j I_{i j}}} - x_{c} \\ Δ y = \frac{\sum_{i}_{, j y_{i j} I_{i j}}}{\sum_{i}_{, j I_{i j}}} - y_{c} \end{matrix}$ (9)

In the formula, I_ij represents the light intensity of the (i, j)th pixel, x_ij, y_ij represents the coordinate of the (i, j)th pixel, and x_c, y_c represents the calibrated center of mass respectively.

Wavefront slope information is obtained by formulas (8) and (9), and all wavefront information is fitted by least squares. The Legendre polynomial is used to reconstruct the wavefront to extract the azimuth information. The one-dimensional Legendre polynomial has the following expression [24]: $L_{n} (x) = \sum_{l = 0}^{[\frac{n}{2}]} \frac{2 {(- 1)}^{l} (n - 1)}{2^{n} l! (n - 1)! (n - 21)!} x^{n - 2 l}$ (10)

x ∈ [- 1, 1] is the number of terms in the Legendre polynomial, and the value of the cumulative upper limit [n/2] should satisfy the following relationship: $[\frac{n}{2}] = {\begin{matrix} \frac{n}{2}, n is even \\ \frac{n - 1}{2}, n is odd \end{matrix}$ (11)

The product of any two Legendre polynomial functions in interval x ∈ [- 1, 1] satisfies the following relationship: $\int_{- 1}^{1} L_{n} (x) L_{m} (x) dx = {\begin{matrix} 0, (m \neq n) \\ \frac{2}{2 n + 1}, (m = n) \end{matrix}$ (12)

It can be seen that the Legendre polynomial function is orthogonal in this interval, and the normalized two-dimensional Legendre polynomial has the following expression: $\begin{matrix} L_{nm} (x, y) = \sqrt{(2 m + 1) (2 n + 1)} L_{n} (x) L_{m} (x) \\ x \in [- 1, 1], y \in [- 1, 1] \end{matrix}$ (13)

After normalization, formula (13) is orthogonal in the square area of x ∈ [- 1, 1] , y ∈ [- 1, 1]. When reconstructing it, it is regarded as two independent processes in the X and Y directions. Therefore, two-dimensional Legendre polynomials independent in the X and Y directions can be used to represent the spatial distribution of the wavefront. The number of terms in its mode k is related to both n and m, and can be expressed as: $k = \frac{(m + n) (m + n + 1)}{2} + m$ (14)

Therefore, the spatial distribution of the incident beam wavefront can be further expressed as: $φ (x, y) = \sum_{k = 1}^{l} α_{k} L_{k} (x, y)$ (15)

In the formula, α_k represents the coefficient of the k-th Legendre polynomial; L_k (x, y) represents the k-th Legendre polynomial; l represents the number of Legendre polynomials used in wavefront recovery.

The wavefront slope information of the ith sub-aperture has the following mathematical relationship: ${\begin{matrix} g_{x} (i) = \sum_{k = 1}^{l} α_{k} \frac{1}{s} \iint_{s} \frac{\partial L_{k} (x, y)}{\partial x} dxdy = \frac{Δ x}{y} \\ g_{y} (i) = \sum_{k = 1}^{l} α_{k} \frac{1}{s} \iint_{s} \frac{\partial L_{k} (x, y)}{\partial y} dxdy = \frac{Δ y}{f} \end{matrix}$ (16)

It is assumed that H-S has M effective sub-apertures, when the first l term Legendre polynomial is used for wavefront reconstruction, formula (5–10) can obtain the following expression: $\begin{matrix} [\begin{matrix} g_{1 x} \\ g_{1 y} \\ g_{2 x} \\ g_{2 y} \\ ⋮ \\ g_{Mx} \\ g_{My} \end{matrix}] = [\begin{matrix} L_{11 x} & L_{12 x} & L_{13 x} & \dots & L_{1 lx} \\ L_{11 y} & L_{12 y} & L_{12 y} & \dots & L_{1 ly} \\ L_{21 x} & L_{22 x} & L_{23 x} & \dots & L_{2 lx} \\ L_{21 y} & L_{22 y} & L_{23 y} & \dots & L_{2 ly} \\ ⋮ & ⋮ & ⋮ & \dots & ⋮ \\ L_{M 1 x} & L_{M 2 x} & L_{M 3 x} & \dots & L_{Mlx} \\ L_{M 1 y} & L_{M 2 y} & L_{M 3 y} & \dots & L_{Mly} \end{matrix}] \\ \cdot [\begin{matrix} α_{1} \\ α_{2} \\ ⋮ \\ α_{l} \end{matrix}] \end{matrix}$ (17)

The expression of L_MKx in the formula: ${\begin{matrix} L_{MKx} = \frac{1}{s_{M}} \iint_{s_{M}} {[\frac{\partial L_{k} (x, y)}{\partial x}]}_{j} dxdy \\ L_{MKy} = \frac{1}{s_{M}} \iint_{s_{M}} {[\frac{\partial L_{k} (x, y)}{\partial y}]}_{j} dxdy \end{matrix}$ (18)

After finishing formula (17), it can be expressed by matrix: $G = L \cdot A$ (19)

Among them, the column vector G is the wavefront slope information of the M sub-apertures in the X and Y directions. L represents the reconstruction matrix when using the method of wavefront reconstruction; A represents the model coefficients.

The least-norm solution is obtained by the least-squares fitting method: $A = L^{+} \cdot G$ (20)L⁺ is the generalized inverse matrix of matrix L, which can be brought into Equation (15) to achieve the purpose of wavefront reconstruction.

5 System model construction

RFID middleware is the bridge between the application system and the RFID equipment system and has the purpose of intermediary agents. As shown in Fig. 3 below, its main function is to accept various types of RFID device information, integrate middleware, and provide corresponding interfaces to the application system. That is, it enables the application system to complete its own business without knowing the implementation details of the RFID device system. Using this RFID middleware method can effectively decouple the two systems. In addition, RFID middleware is a message-oriented middleware. Information is sent from one software application system to one or more software systems by means of messages. Since the information is transmitted asynchronously, the sender does not need to wait for the response of the other party but can continue to do his work. In addition, the transmission of information is not the only function of RFID middleware. It mainly includes service functions such as security, finding the path that meets the cost, data broadcasting, finding network resources, and sorting messages by rank.

Fig. 3

RHD middleware working diagram.

(1) Independent architecture. As a bridge between the RFID reader and the application layer software, the RFID middleware framework exists independently. It can also connect multiple readers and provide API interface for multiple application layer software to use at the same time. This can achieve the purpose of decoupling and reduce the complexity of the system architecture. (2) Abstract RFID data flow. Its main function is to change the object of the entity into an abstract object under the information environment, so the most important function of RFID is to process data. RFID middleware has the characteristics of data collection, integration and filtering, and transmits the correct information to the software system at the application layer. (3) Flow processing. RFID middleware adopts the functions of storage, retransmission and program logic to provide an orderly message flow, which can realize the design and management of data flow. (4) Normative standards. RFID technology can meet the needs of automatic data collection and object identification. The EPC global organization has proposed a world-wide standard, the product electronic code, which stipulates that all items in the world have only one corresponding number. The definition of EPC is to identify a specific item with a series of numbers in the entire life cycle of the system. After being collected by the RFID reader, the process in the software system that transmits the application layer is called the object naming service.

At this stage, in general, RFID middleware uses a three-layer model, namely a hardware abstraction layer (HAL), event data management layer (EDML) and application abstraction layer (AAL) as shown in Fig. 4.

Fig. 4

RFID middleware structure.

Complex event processing is a new type of information processing technology. It takes an event-driven architecture, takes analysis based on basic events, and infers complex events through prediction. It is a system-oriented architecture that further subdivides services and turns services into events, as shown in Fig. 5. If an event is discovered, it triggers the next event. This architecture can flexibly decide the subsequent process according to the information content in the future, and it is more suitable for the increasingly large and complex logical structure.

Fig. 5

The process of event processing.

As shown in Fig. 6, in complex event processing, in order to achieve low latency, high utilization, and high throughput, event stream processing is used. It can conveniently provide complex logic processing for event streaming, so that event streaming can do pattern comparison processing in memory and query actions. Therefore, this process is performed in the memory and does not need to go through the storage device.

Fig. 6

Event flow.

According to the format of data collected by this system, complex event processing models can be divided into three layers. As shown in Fig. 7, the three layers are the raw data layer, the basic event layer and the abstract event layer.

Fig. 7

RFID data complex event processing model.

The teacher module includes classroom monitoring and behavior correction functions. After the teacher logs in to the system, a unified view of classroom monitoring is displayed, including basic classroom information, information on teaching management information, and dynamic values such as classroom performance score, attendance rate, pass rate, and teaching evaluation score, and a visual statistical view. The statistical view and the classroom teaching view area can be switched back and forth, and the teaching management message is the classroom intervention prompt message sent by the role of the teaching management staff. After clicking on the desk in each view, the teacher can view the details of the student’s performance and take pictures of the negative behavior. Moreover, the system supports behavior correction reminders and prompt information entry, as shown in Fig. 8.

Fig. 8

Unified view of teacher classroom monitoring.

Facial expression perception is the feature extraction and facial expression change perception of the students’ facial expressions during the class to collect the students’ behaviors during the class. During the class, the terminal camera is always in the open camera mode. The system monitors the whole process and performs real-time facial analysis to capture the obvious facial expression characteristics, including yawning, closing eyes, chewing mouth, leaving the video area, and bowing. Furthermore, the captured facial expression features are sent to the facial recognition module for facial expression recognition. At the same time, the continuously recognized expression will be recorded and calculated by the behavior score of the scoring component, and the performance score and graded evaluation will be obtained in real time. Facial expression perception will be carried out using facial feature recognition methods. It is necessary to simultaneously extract facial features and dynamically compare context information to achieve facial expression change perception. Moreover, it defines the sensed expressions to perform behavior scoring. In terms of expression definition, in order to accurately recognize the expressions of different individuals, it is necessary to collect each person’s expressions and establish a complete personal expression information database, which is difficult to work and practice. Therefore, this article will adopt the method of abstract features, and do not analyze the individual facial features, and only abstract the common features to compare with the standard template to achieve the definition process of the expression. This sacrifices a certain accuracy, but the difficulty of implementation is reduced. Moreover, through the law of large numbers, this loss of accuracy can be compensated to a certain extent in the performance score. An expression recognition process is shown in Fig. 9. After the recognized new expression is sent to the scoring module for processing, the system automatically loops into the next recognition process until the end of the course.

The system starts the whole process of monitoring. Face positioning and face recognition are performed first, and the detected face is compared with the currently logged in user template. If the verification fails, the detection is continued, and if the verification is passed, the expression feature information extraction and storage process is performed.

Definition of instant facial expression features. The features of the current facial expression are abstract extracted and stored, and after comparison with the expression template library, the current facial expression is defined. Moreover, the defined data will also be sent to the scoring module synchronously to calculate the instant score.

Context awareness. After collecting the image at the next moment for feature extraction, it is compared with the instant expression. For the face that has not changed, the feature information is continuously extracted, and the latest information is temporarily stored. For the case where the face is away, the face positioning and recognition process needs to be re-run. For those who perceive a change, a new expression is recorded, and the expression is defined.

New expression definition. After perceiving a new expression, it is necessary to repeat the process of feature abstraction extraction and expression definition and send it to the scoring library for scoring. At the same time, the new expression is stored in the instant expression library, and the facial perception process is repeated.

Fig. 9

Facial expression perception process.

The system adopts the design concept of separating business logic, data and interface display, so the coupling between modules is low, which makes the interface development of each module less difficult and easy to compile and maintain. At the same time, it can be developed in parallel to speed up development efficiency. The system is implemented in the layered mode of interface layer, business logic layer, and data access layer. Each layer is implemented in the form of components. It mainly includes three functional components: application operation component, face recognition component, and scoring component.

The data access layer is the part of the system that provides data processing. The main functions of the scoring component will be implemented in this layer, and this layer provides service support for other layer. It can be mainly divided into two parts: Provide face recognition component recognition data: generate and manage face sample library and behavior sample library, provide face recognition and facial expression recognition standard library for face recognition component, and according to the collection, Identify and other application results, and conduct dynamic maintenance and management of the sample library. This part is mainly for the processing of unstructured graphics data and structured feature data.

Provide scoring data for application operation components, scoring strategy models for student scoring and classroom scoring, and collection and storage of evaluation data. Mainly for the storage and processing of structured scoring data, and scoring model strategies. The interface layer is the part that deals with the interface and display in the system. The application operation component is where the main operation interface and element display page of the system in this article are located. It will be mainly implemented at this layer to obtain operation instruction information for users. According to user role and operation process, it can be divided into three parts:

Student operation application: The student-side operation module of the e-learning classroom system provides functions such as student check-in, class behavior monitoring and recognition, student class behavior prompts, student classroom performance feedback and recording, and classroom comments. It is mainly responsible for the collection of student data.

Teacher operation application: The teacher-side operation module of the e-learning classroom system provides classroom evaluation feedback, student attendance and behavior data feedback, student behavior correction and other functions, assists the teacher in the course of class, and collects teacher classroom management behavior data.

Teaching administrator operation application: It uses the background monitoring module to carry out unified management and unified monitoring of the classrooms of the whole school, intervene and correct students’ behavior in real time, perform real-time management of classroom effects and quantitative evaluation after class to meet the needs of teaching management.

The business logic layer performs logical processing and result feedback on user task instructions, interactive operations, business logic, and data information. Face recognition components, and task processing of other interactive instructions of application components will be deployed at this layer. The functions of this layer are deployed by the core business logic, which can be divided into three parts according to the functional classification:

Face recognition: It is mainly aimed at signing in and monitoring the identity of the subject and uses the basic recognition function provided by the seetaface engine and the backup recognition algorithm optimized for the physiological changes of young students to achieve the most basic face recognition function.

Facial recognition: It sets typical facial expressions that are closely related to positive and negative behaviors in class, and captures and records specific expressions through full-time monitoring, and passes the data to the scoring model component for scoring processing.

Business logic: It is mainly related to system application related business process control and result feedback.

The system adopts B/S system architecture for development. B/S is the browser/server architecture. The user interface operation part is deployed through a web page, and most of the processing logic is handed over to the back-end server. Interface development is relatively simple in this mode. The user accesses the page through the browser to perform operations and sends operation instructions to the server. The server receives and analyzes the task instructions, starts related services, calls the terminal device to obtain data and performs task processing, and then displays the results on the feedback page. At this point, a complete interaction and task processing is completed.

6 Model performance analysis

Based on the model constructed above, the performance of the model constructed in this study is analyzed. This research system mainly recognizes the emotions of students to verify the status of students. This article is based on the neural network structure, so this article uses the neural network algorithm as a comparison algorithm to verify the performance of the model constructed in this study. First of all, in this study, the accuracy of student emotion recognition is studied and analyzed, and 66 sets of data are identified. The results are shown in Table 1 and Fig. 10.

Table 1
Comparison table of accuracy rate of students’ emotion recognition

Neural network model (%) Improve the model (%) Neural network model (%) Improve the model (%)

1 83.6 98.4 34 64.8 91.1

2 83.0 98.3 35 64.4 91.0

3 82.8 98.2 36 64.1 90.7

4 82.0 97.9 37 64.1 90.6

5 81.9 97.9 38 63.8 90.6

6 79.5 97.7 39 63.7 90.2

7 79.0 97.4 40 63.1 90.2

8 78.8 97.2 41 61.1 90.0

9 77.3 97.1 42 61.1 89.9

10 77.1 97.0 43 61.0 89.9

11 76.5 97.0 44 60.7 89.8

12 74.9 96.8 45 59.5 89.2

13 74.3 96.5 46 58.3 89.2

14 74.1 96.5 47 58.3 88.9

15 73.6 95.9 48 57.6 88.6

16 73.3 95.9 49 56.8 88.5

17 72.9 95.7 50 56.8 88.4

18 72.2 95.6 51 55.7 88.2

19 71.8 95.3 52 55.6 88.2

20 71.5 94.8 53 54.9 87.9

21 71.5 94.6 54 54.9 87.1

22 71.4 94.5 55 54.6 87.0

23 71.0 94.4 56 54.0 86.9

24 70.4 93.9 57 53.8 86.7

25 69.1 93.7 58 53.7 86.6

26 68.7 93.4 59 50.7 86.4

27 68.7 92.8 60 48.8 86.3

28 67.9 92.4 61 47.7 86.3

29 66.5 92.1 62 47.2 86.2

30 66.0 91.7 63 46.8 86.0

31 65.9 91.7 64 46.0 85.5

32 65.8 91.7 65 45.8 85.4

33 65.3 91.5 66 45.2 85.2

	Neural network model (%)	Improve the model (%)		Neural network model (%)	Improve the model (%)
1	83.6	98.4	34	64.8	91.1
2	83.0	98.3	35	64.4	91.0
3	82.8	98.2	36	64.1	90.7
4	82.0	97.9	37	64.1	90.6
5	81.9	97.9	38	63.8	90.6
6	79.5	97.7	39	63.7	90.2
7	79.0	97.4	40	63.1	90.2
8	78.8	97.2	41	61.1	90.0
9	77.3	97.1	42	61.1	89.9
10	77.1	97.0	43	61.0	89.9
11	76.5	97.0	44	60.7	89.8
12	74.9	96.8	45	59.5	89.2
13	74.3	96.5	46	58.3	89.2
14	74.1	96.5	47	58.3	88.9
15	73.6	95.9	48	57.6	88.6
16	73.3	95.9	49	56.8	88.5
17	72.9	95.7	50	56.8	88.4
18	72.2	95.6	51	55.7	88.2
19	71.8	95.3	52	55.6	88.2
20	71.5	94.8	53	54.9	87.9
21	71.5	94.6	54	54.9	87.1
22	71.4	94.5	55	54.6	87.0
23	71.0	94.4	56	54.0	86.9
24	70.4	93.9	57	53.8	86.7
25	69.1	93.7	58	53.7	86.6
26	68.7	93.4	59	50.7	86.4
27	68.7	92.8	60	48.8	86.3
28	67.9	92.4	61	47.7	86.3
29	66.5	92.1	62	47.2	86.2
30	66.0	91.7	63	46.8	86.0
31	65.9	91.7	64	46.0	85.5
32	65.8	91.7	65	45.8	85.4
33	65.3	91.5	66	45.2	85.2

Fig. 10

Comparison diagram of accuracy rate of student emotion recognition.

As shown in Fig. 10, the accuracy rate of the model proposed in this article is as high as 85% –95%, while the accuracy rate of traditional neural network model recognition is less than 60% on average. It can be seen that the model proposed in this paper has a better effect on the accuracy of student emotion recognition. On the basis of the above analysis, the student status recognition is compared, and the results are shown in Table 2 and Fig. 11.

Table 2

Comparison table of accuracy rate of student status recognition

	Neural network model (%)	Improve the model (%)		Neural network model (%)	Improve the model (%)
1	64.3	84.9	34	53.3	77.6
2	64.2	84.8	35	53.2	77.5
3	63.8	84.5	36	52.8	77.2
4	63.7	84.0	37	52.3	77.2
5	63.6	83.9	38	51.3	76.3
6	63.0	83.8	39	51.3	76.1
7	62.7	83.5	40	51.2	75.8
8	62.4	83.4	41	51.2	75.5
9	61.7	83.1	42	51.1	75.4
10	61.5	83.1	43	51.0	75.4
11	60.4	83.0	44	51.0	74.3
12	60.2	83.0	45	49.9	74.0
13	60.1	83.0	46	49.5	73.9
14	59.6	82.9	47	49.3	73.7
15	58.4	82.9	48	48.9	73.7
16	58.0	82.9	49	48.3	73.2
17	57.6	82.7	50	47.6	73.1
18	57.4	82.5	51	47.4	72.9
19	57.0	82.4	52	46.8	72.8
20	56.2	82.2	53	46.2	72.4
21	56.2	81.9	54	46.2	71.9
22	56.1	81.9	55	45.5	71.8
23	56.0	81.6	56	45.5	71.6
24	55.8	81.5	57	44.9	71.4
25	55.7	81.3	58	43.3	71.1
26	55.4	81.3	59	42.8	70.9
27	55.2	81.0	60	42.8	70.9
28	55.1	80.3	61	42.7	70.5
29	54.7	80.2	62	42.7	70.3
30	54.5	79.6	63	42.3	70.3
31	54.3	79.6	64	41.9	70.2
32	54.2	78.5	65	40.8	70.2
33	53.7	77.8	66	40.6	70.1

Fig. 11

Comparison diagram of accuracy rate of student status recognition.

From the comparison results of student status recognition in Fig. 11, the model proposed in this paper achieves an accuracy rate of 70% –85% in student status recognition, while the accuracy rate of traditional neural network model recognition is less than 50% on average. It can be seen that the model proposed in this paper has a better effect on student status recognition.

7 Conclusion

The purpose of this study is to establish a set of face-to-face classroom intelligent management system. Based on the realization and test application of face recognition algorithm, the system can realize comprehensive applications such as classroom check-in, classroom discipline monitoring, classroom effect evaluation, and student classroom behavior evaluation, which provides system assistance for teacher classroom management and teaching management, and effectively improves classroom efficiency and management quality. The learning algorithm design of the quantitative assessment model mainly introduces the algorithm design of the scoring model, as well as the model training method and related algorithms. Moreover, the machine learning method is used to design the quantitative evaluation index system, as well as the logistic regression scoring algorithm and model training algorithm. The system adopts B/S system architecture for development. B/S is the browser/server architecture. The user interface operation part is deployed through a web page, and most of the processing logic is handed over to the back-end server. This research system mainly recognizes the students’ emotions to verify the students’ status, uses the neural network algorithm as the comparison algorithm to verify the performance of the constructed model, and analyzes the results through chart comparison. The results of the research show that the model proposed in this paper has good performance and can be applied to practical classrooms.

Footnotes

Acknowledgment

This paper was supported by fund as Research on the integration and innovation of Cultural Classics Courses Based on Core Literacy (Research Project of teaching reform in Hunan Province colleges and universities in 2020).

References

Wang

, Shi

, Shu

, et al., Embedded Manifold-Based Kernel Fisher Discriminant Analysis for Face Recognition, Neural Processing Letters 43(1) (2016), 1–16.

Chuk

, Chan

A.B.

and Hsiao

J.H.

, Understanding eye movements in face recognition using hidden Markov models, Journal of Vision 14(11) (2014), 8–8.

Bouchech

, Selection of optimal narrowband multispectral images for face recognition, Monthly Notices of The Royal Astronomical Society 402(4) (2015), 2140–2186.

Ramachandra

and Busch

, Presentation Attack Detection Methods for Face Recognition Systems: A Comprehensive Survey, ACM Computing Surveys 50(1) (2017), 8.1–8.37.

Huang

, Shan

, Wang

, et al., A Benchmark and Comparative Study of Video-Based Face Recognition on COX Face Database, IEEE Transactions on Image Processing 24(12) (2015), 5967–5981.

Wei

C.P.

and Wang

Y.C.F.

, Undersampled Face Recognition via Robust Auxiliary Dictionary Learning, Image Processing IEEE Transactions on 24(6) (2015), 1722–1734.

Ajit Krisshna

N.L.

, Deepak

V.K.

, Manikantan

, et al., Face recognition using transform domain feature extraction and PSO-based feature selection, Applied Soft Computing 22(4) (2014), 141–161.

Weng

, Lu

and Tan

Y.P.

, Robust Point Set Matching for Partial Face Recognition, Image Processing IEEE Transactions on 25(3) (2016), 1163–1176.

Cui

, Chang

, Shan

, et al., Joint sparse representation for video-based face recognition, Neurocomputing 135(6) (2014), 306–312.

10.

Phillips

P.J.

and Otoole

A.J.

, Comparison of human and computer performance across face recognition experiments, Image and Vision Computing 32(1) (2014), 74–85.

11.

Tang

, Zhu

, Yu

, et al., A novel sparse representation method based on virtual samples for face recognition, Neural Computing & Applications 24(3) (2014), 513–519.

12.

Mehta

, Yuan

and Egiazarian

, Face recognition using scale-adaptive directional and textural features, Pattern Recognition 47(5) (2014), 1846–1858.

13.

Raghavendra

, Raja

K.B.

and Busch

, Presentation Attack Detection for Face Recognition Using Light Field Camera, IEEE Transactions on Image Processing 24(3) (2015), 1060–1075.

14.

Yang

, Zhu

, Liu

, et al., Joint representation and pattern learning for robust face recognition, Neurocomputing 168(30) (2015), 70–80.

15.

Yan

, Wang

and Suter

, Multi-subregion based correlation filter bank for robust face recognition, Pattern Recognition 47(11) (2014), 3487–3501.

16.

Fan

, Ni

, Zhu

, et al., Weighted sparse representation for face recognition, Neurocomputing 151(1) (2015), 304–309.

17.

Tang

, Feng

and Cai

, Weighted group sparse representation for undersampled face recognition, Neurocomputing 145(6) (2014), 402–415.

18.

Al-Arashi

W.H.

, Ibrahim

and Suandi

S.A.

, Optimizing principal component analysis performance for face recognition using genetic algorithm, Neurocomputing 128(27) (2014), 415–420.

19.

Morelli Andr’´es

, Padovani

, Tepper

, et al., Face recognition on partially occluded images using compressed sensing, Pattern Recognition Letters 36(1) (2014), 235–242.

20.

Ding

and Tao

, Pose-invariant face recognition with homography-based normalization, Pattern Recognition 66(5) (2017), 144–152.

21.

Yang

, Feng

, Shiu

S.C.K.

, et al., Fast and robust face recognition via coding residual map learning based adaptive masking, Pattern Recognition 47(2) (2014), 535–543.

22.

Zhu

and Li

, A Kernel-based sparse representation method for face recognition, Neural Computing & Applications 24(3) (2014), 845–852.

23.

Chen

W.S.

, Zhao

, Pan

, et al., Supervised kernel nonnegative matrix factorization for face recognition, Neurocomputing 205(12) (2016), 165–181.

24.

, Gong

, Li

, et al., Learning Compact Feature Descriptor and Adaptive Matching Framework for Face Recognition, Image Processing, IEEE Transactions on 24(9) (2015), 2736–2745.

25.

Lan

H.-You.

and Nieto

J.J.

, On a System of Semilinear Elliptic Coupled Inequalities for S-Contractive Type Involving Demicontinuous Operators and Constant Harvesting, Dynamic Systems and Applications 28 (2019), 625–649.

26.

Yang

and Zhang

, Chaotic Behavior of Co-Semigroups of Operators, Dynamic Systems and Applications 29(2) (2020), 281–287.

27.

Rathore

M.M.

, Paul

, Hong

W.H.

, Seo

H.C.

, Awan

and Saeed

, Exploiting IoT and big data analytics: Defining smart digital city using real-time urban data, Sustainable Cities and Society 40 (2018), 600–610.

28.

Paul

, Internet of Things: A primer’, R Jeyaraj Human Behavior and Emerging Technologies 1(1) (2019), 37–47.

29.

Paul

, Daniel

, Ahmad

and Rho

, Cooperative cognitive intelligence for internet of vehicles, IEEE Systems Journal 11(3) (2017), 1249–1258.

30.

Paul

, Ahmad

, Mazhar Rathore

and Jabbar

, Smart buddy: defining human behaviors using big data analytics in social internet of things, IEEE Wireless Communications 23(5) (2016), 68–74.

31.

Paul

, Jiang

Y.C.

, Wang

J.F.

and Yang

J.F.

, Parallel reconfigurable computing-based mapping algorithm for motion estimation in advanced video coding, ACM Transactions on Embedded Computing Systems (TECS) 11(S2) (2012), 1–18.