Simulation of English classroom effectiveness based on human-computer interaction and facial identification

Abstract

A variety of factors affect English classroom teaching, which prevents teachers from effectively grasping students’ learning status and learning situation. In particular, classroom management is more difficult during online teaching. In order to improve the effectiveness of English classroom teaching, based on the human-computer interaction algorithm and facial identification algorithm, this paper effectively recognizes the human-computer interaction process and classroom learning status of students in online teaching, and eliminates the image background according to the actual teaching needs. Moreover, by extracting and fusing the time sequence information and spatial information of the motion in the video, a spatiotemporal feature image capable of expressing a dynamic typical feature is obtained. In addition, this paper uses system algorithms to judge the status and feed back the identification results to the teacher’s teaching terminal equipment, which is convenient for timely teaching adjustment. Finally, this paper analyzes the effectiveness of the algorithm through simulation experiments. The research results show that the algorithm constructed in this paper has good performance.

Keywords

Human-computer interaction facial identification English classroom teaching simulation

1 Introduction

Today’s university teaching is completely different from the past. The campus multimedia teaching equipment is distributed in every classroom, and the campus network is connected to every office, classroom and student apartment. We use multimedia systems and the Internet to obtain more and more timely information. In particular, the Internet has almost become a major tool for students to acquire knowledge. Teachers’ teaching content and teaching methods have also undergone tremendous changes with the birth of the Internet. The teaching content may come from the Internet at any time, so as to ensure the novelty of the course and keep pace with the times.The way of teaching is not limited to forty minutes in the classroom and began to use modern information technology to extend after class. As a result, various teaching aid systems were born. The English online auxiliary teaching system is also one of them, which is used to help students self-study English courses [1].

The research of English online auxiliary teaching system aims to develop and implement an auxiliary system for English after-class teaching and teaching management according to the characteristics of English course learning. This auxiliary system can meet the needs of most schools for English teaching, to ensure that students use more of their spare time to learn English, communicate with teachers online, and increase their interest in English learning. The research of English online auxiliary teaching system has the following significance. In any school, the guarantee of the teaching quality cannot be achieved by 40 minutes in the classroom. A large amount of students’ extra-curricular time requires us to make full use of management. It can be said that whoever makes good use of extra-curricular time will ensure the quality of teaching. The purpose of developing English online auxiliary teaching system is to make full use of students’ spare time. Colleges and universities have always attached great importance to the cultivation of students’ self-learning ability. By cultivating students’ self-learning ability, students can master the way of survival in the future. The cultivation of self-learning ability does not depend solely on the students themselves, but also requires guidance from school teachers.Through English online auxiliary teaching system, it can help students develop self-study ability and develop a good language content learning method [2]. The level of modernization of teaching has always been a reflection of the comprehensive level of schools. Therefore, major universities attach great importance to the construction of modernization of teaching, and scientifically manage the school curriculum through the modernization of teaching.The purpose of the development of English online auxiliary teaching system is to provide a means for the modernization of English courses.

The teaching process of online education is difficult to control, and teachers cannot observe dozens of students at the same time. Therefore, it is necessary to cut off human interaction and facial identification technology to recognize the status of students and improve the effectiveness of online English teaching [3].

2 Related work

The most fundamental purpose of human-computer interaction evaluation technology research is to realize the usability of system engineering.Usability is the effectiveness, efficiency, and subjective satisfaction of a product for a specific user for a specific use under a specific use environment.Usability emphasizes the behavior of products in real life and the way they are used by users. It is not simply to see at a product from the inside, but from the outside to see how users use and interact with it [4]. All along, in product development, people usually focus on what the product is used for rather than how to use the product to do things.Therefore, from the beginning of product design, we must seriously consider the user experience, not only focus on the factors of technological innovation and ignore the human factor.Products with good usability not only enable developers to reduce development costs, increase credibility and improve product competitiveness, but also improve user efficiency and satisfaction [5]. There are many studies on usability abroad, and they started early. Usability plays a very important role in the product development process.The usability evaluation research of the virtual simulation human-computer interaction interface is inextricably linked with various industrial departments of a country.The UK Department of Industry and Trade (DTI) has a research and development plan called “Usability: Start Now!”. IBM, Microsoft, etc. have dedicated usability labs or usability research groups. Studies have shown that investment in usability accounts for 10% of the investment, and usability of systems redesigned based on usability studies has increased by an average of 135%. A hard-to-use system will keep users away, and once the user has a bad experience in the use of the system, it is difficult to accept the use of the system again.Many mission tests indicate that 35% of the operation fails when users use the system. The operation when the user uses the system is to solve the problem. If the efficiency of the user to solve the problem is greatly improved, the possibility that the user will use the system again will increase. The usability of the human-computer interaction interface is a problem that runs through the design, and the user-centered design method should be used in the design [6]. One of the most direct results of improved usability is the increased utilization of the system, and the efficiency of users using the system to complete tasks will also increase, and user satisfaction will increase. To measure the usability of the human-computer interaction interface of the system, we need to determine the usability index through evaluation. This index can be a specific quantitative value or a satisfaction evaluation [7]. Therefore, usability research must study usability assessment. From the perspective of the user interface development process, human-computer interaction interface evaluation can be roughly divided into two categories: one is evaluation in the design process, called formative evaluation, and the other type is the final evaluation made after the user interface is completed, called summative evaluation.Both types of evaluation play an important role in the development process and are an integral part of the entire human-computer interaction interface design. Among them, the formative evaluation emphasizes the use of open methods in the development process, such as interviews, questionnaires, attitude surveys and scale technology [8]. However, most of the summative assessments use strict quantitative assessments, such as response time and error rate. Foreign countries have started earlier in the study of system human-computer interaction usability evaluation, and there are many researches on human-computer interaction usability evaluation of the system, such as the usability evaluation of digital libraries [9]. Related research has also achieved rapid development in recent years, and there has been a study on the usability evaluation of digital museums and e-commerce systems [10]. The literature [11] identified the central task as improving the availability of IT and the market impact of human-computer interaction products. The research that has been done includes includes the comparative evaluation,cross-cultural user research, mobile business, and mobile phone company research of the relevance, page coverage, dead link rate, timeliness, professional segmentation, and classification functions of results under the standard search methods of search engines such as Google, hooChina, Baidu, Zhongsou. The literature [12] conducted usability testing of Microsoft’s Chinese products and cooperated with Microsoft to explore ways to train usability professionals.The China EU Usability Research Center promotes usability concepts and usability engineering practices in China and provides more usable products to users in China and around the world [13]. Its customer base is huge, including many well-known domestic and foreign companies such as IBM, HP, eBay, Sina, Tsinghua Tongfang, Germany Volkswagen, etc.The center offers many courses including human-computer interaction technology, usability engineering, user interface design, accessibility and common design, etc. Jinshan Company established the first software usability research laboratory in China, and the China Branch of Usability Experts Association was established [14]. The article [24] addresses the issue such as enormous volume of bigdata and come up with the concept of SmartBuddy to form brilliantly and savvy environment utilizing human practices and human elements.The article [25] talks almost the development of coordinated non-cyclic chart for video coding calculations for movement estimation in parallel reconfigurable computing frameworks. Moreover, the partitioning algorithm plays a major part to speed up the video processing.The article [26] dealt exploiting IoT and BigData Analytics utilizing Hadoop environment in genuine time situations. Execution of IoT-based Smart City is accomplished by the above-mentioned processes.The article [27] centers around IoT and its significant job in sophisticating the human practices and endeavors. This paper additionally managed the assortment of different information from different assets that are associated with the web.The literature [28] addresses the different issues within the field of vehicle communication with the recommendation of a common bound together and scattered range detecting demonstrate [29]. The application of the shared cognitive paradigm minimizes struggle and different obscure problems [30].

3 Summary of image foreground segmentation technology

The direct subtraction method is one of the simplest foreground segmentation methods. It assumes that the background is stationary and does not change with time. At the same time, it does not consider the change of background light caused by the entry of the foreground. The advantages of this method are simple and fast, but the disadvantages are also obvious, and the scope of application is limited. When the background objects and the ambient light are constant, this method can achieve good results. When using this method for foreground segmentation, there are generally two steps [15]:

1. The background is sampled. When sampling the background, there is no foreground object in the environment at this time. I_t (x, y) is used to represent the pixel value at point (x, y) at time t. then the image at this time is expressed as I_a (x, y), and the background image is expressed as: $B (x, y) = I_{a} (x, y)$ (1)

2. Background subtraction. When a foreground object enters and the background needs to be removed to segment the foreground, the time is set to b at this time, then the segmented foreground is: $F_{b} (x, y) = {\begin{matrix} I_{b} (x, y) - B (x, y) < th 0 \\ I_{b} (x, y) - B (x, y) ⩾ th I_{b} (x, y) \end{matrix}$ (2)

Based on the above two steps, this paper did a gesture foreground segmentation experiment based on direct subtraction. The experimental results are shown in Fig. 1.

Fig. 1

Experimental results of gesture foreground segmentation based on direct subtraction.

The median and mean background model is also a simpler background modeling method, but compared to the direct subtraction method, it takes the dynamic change of the background into account. When performing real-time background segmentation, the background model is also updated in real time. Its basic idea is to count the M-frame video sequences before the current moment, and calculate the background model based on this. The background model is shown in formula (3) and formula (4) [16]:

${VB}_{t} (x, y) = \frac{1}{M} \sum_{i = t - M}^{y - 1} I_{i} (x, y)$ (3) $B_{t} (x, y) = \underset{i \in [t - M, T - 1]}{median} (I_{i} (x, y))$ (4)

Among them, formula (3) is the mean background model, formula (4) is the median background model, and I_i (x, y) represents the gray value at the moment i at (x, y). At the same time, median () is the function of taking the median value of the pixels within a certain time range. Single Gaussian modeling is generally divided into three stages:

1. Training phase. At this stage, the background features are extracted by learning a sequence of video frames, and on this basis, a mathematical statistical model that can describe the background features is established. When modeling a single Gaussian background model, the distribution of pixel values at fixed locations in the background image conforms to the Gaussian distribution, that is [17]: $B (x, y) \sim N (μ, σ^{2})$ (5)

Among them, the values of μ and σ² are: $μ (x, y) = \frac{1}{T} \sum_{i = 1}^{T} f_{i} (x, y)$ (6) $σ^{2} (x, y) = \frac{1}{T} \sum_{i = 1}^{T} {[f_{i} (x, y) - μ (x, y)]}^{2}$ (7)

μ and σ² are the average gray value and gray value variance of the point with coordinate (x, y) in a certain period of time T, respectively.

2. Detection stage. At this stage, the foreground needs to be segmented in real time according to the background model established in the training stage. For each pixel, the background foreground judgment formula is generally as formula (8): $(x, y) = {\begin{matrix} background | F_{t} (x, y) - B_{t} (x, y) | ⩽ th \\ prospect, | F_{t} (x, y) - B_{t} (x, y) | > th \end{matrix}$ (8)

Among them, th is the threshold. the threshold is not fixed, but may be different for each pixel, which often takes three times the variance, that is 3σ².

3. Update stage. Since the background is not constant, it is likely to change or shift with the increase of time, ambient light, background objects, etc. Therefore, with the increase of time, the background model needs to be updated according to adapt to the new background environment. The update formula of the background model is as formula (9) [18]: $B_{t} (x, y) = (1 - ρ) \cdot B_{t - 1} (x, y) + ρ \cdot F_{t} (x, y)$ (9)

B_t-1 (x, y) represents the background image at time t–1, F_t (x, y) represents the current frame, and ρ represents the background update rate, which is used to reflect the background image update rate. The larger the value, the faster the background image update.

Based on the above three steps, this paper did a gesture foreground segmentation experiment based on a single Gaussian background model. The experimental results are shown in Fig. 2.

Fig. 2

Single Gaussian modeling experiment results.

The four figures in Fig. 2 are screenshots of real-time gesture segmentation video frames. Among them, figure (a) and figure (c) are Gaussian background modeling gesture segmentation original image and segmented image respectively, figure (b) and figure (d) are the original image and segmented image when the background environment object moves. It can be seen from the experimental results that single Gaussian background modeling can overcome the problem that the background model cannot be updated in time when the background environment changes simply. However, because the light in the shadow area under the hand is too dark, the segmentation effect of this area is still not ideal [19].

4 Summary of skin color modeling technology

The most common use of the RGB color space is the display system. The color raster graphic display uses different R, G, and B values to drive the corresponding electron guns to emit electrons and excite the corresponding phosphors of the three channels on the display screen, thereby emitting different colors of light. The scanner is also based on the RGB color space to collect color information of color manuscripts. Different components in the scanner will absorb the color components from different channels in the RGB space and use it to represent the color of the manuscript.

As shown in Fig. 3, each point in the cube represents a color, which is represented by the parameter values of the three dimensions of R, G, and B, respectively. The coordinate at the origin is (0, 0, 0), which is represented by the parameter value.

Fig. 3

RGB color space color cube.

The correlation between the three components of the RGB color space is relatively high, and the distance between different colors is larger than the human visual perception, so it is not suitable for color segmentation. This section explores another color space, that is, YCbCr color space, it is the color model of CCIR601 encoding. Meanwhile, it has a wide range of applications in the field of color TV displays. In the YCbCr color space, Y represents the luminance component, Cb is the blue chrominance component, and Cr is the red chrominance component. The color conversion formula from RGB space to YCbCr color space is shown in formula (10) [20]: $[\begin{matrix} Y \\ Cb \\ Cr \\ 1 \end{matrix}] = [\begin{matrix} 0.2990 & 0.5870 & 0.1140 & 0 \\ - 0.1687 & - 0.3313 & 0.5000 & 0.5 \\ 0.5000 & - 0.4187 & - 0.0813 & 0.5 \\ 0 & 0 & 0 & 1 \end{matrix}] [\begin{matrix} R \\ G \\ B \\ 1 \end{matrix}]$ (10)

The YCbCr color space can separate the brightness information and chroma information of the color. On the Cb-Cr plane, the skin color can be well gathered in a small area. Converting the color to the YCbCr color space has the following benefits:

In the YCbCr color space, the brightness is separately divided into one dimension. When analyzing the color information without considering the brightness information, only two dimensions of Cb and Cr need to be analyzed in this space, which reduces the dimension of the color space and reduces the computational complexity;

The composition principle of YCbCr color space is similar to human visual perception;

Skin color has a good clustering in the YCbCr color space. In this space, the skin color is not scattered, and can be well gathered in a small area;

The YCbCr color format is widely used in many fields such as displays, and is also widely used in the field of video compression coding.

In order to better cluster human skin color, we exclude the interference of brightness, only consider the chroma information, and find the skin color clustering area in the Cb-Cr plane. To this end, the color values of a large number of skin regions need to be counted, and then the skin color regions in the Cb-Cr plane are counted on this basis. In order to explore the clustering performance of YCbCr color space for skin color, a total of 88 hand images under different environments were collected, and 113 pieces of human skin color area were manually cropped, which totaled 6007466 pixels. Through calculation and statistics, the average gray value of these pixels is 128.95, and the average values of the three RGB channels are 149.23, 124.21, and 99.90. There is R, G, B ∈ [0, 255]. This result is converted to YCbCr space, the brightness information Y of the color is discarded, and the distribution of skin color in the Cb-Cr plane is shown in Fig. 4 [21]:

Fig. 4

The distribution of skin color pixels collected in this paper in the Cb-Cr plane.

It can be seen from Fig. 4 that the skin color has a good clustering in the Cb-Cr plane, which is mainly distributed in the range of 80–130 on the Cb axis and in the range of 125–170 on the Cr axis. Its shape is similar to an irregular quadrilateral. In order to represent this area more accurately, this paper uses quadrilateral to fit it, as shown in Fig. 5 [22]:

Fig. 5

Fitting skin color area with quadrilateral.

In this paper, the quadrilateral area of the four vertices with the coordinate (65, 152) , (118, 115) , (135, 130) , (111, 175) is selected to represent the skin color area, so the expression of the quadrilateral area is: ${\begin{matrix} y + 0.69 x > 196.85 \\ y + 1.87 x < 382.57 \\ y - 0.5 x < 119.5 \\ y - 0.88 > 11.6 \end{matrix}$ (11)

Among them, x represents the Cb axis, y represents the Cr axis. According to this skin color area, this paper establishes the corresponding skin color model and tests the skin color area segmentation effect on this basis, as shown in Fig. 6 [23]:

Fig. 6

The effect of segmentation based on YCbCr skin color model.

In order to explore the effect of the skin color model on removing the shadow under the hand after modeling the single Gaussian background in the previous section, this paper uses the diagram d in Fig. 2 as the input image and uses the skin color model to perform the skin color region segmentation experiment. The experimental results are shown in Fig. 7:

Fig. 7

Further extraction of gesture foreground based on skin color segmentation in YCbCr color space.

Color images are usually represented by three components, R, G, and B, but there is a high correlation between them. When modeling color, if you use the RGB model directly, you often cannot obtain the desired effect. In contrast, the correlation between the three components of the HSV (Hue, Saturation, Value) color space is relatively low, and is closer to human visual perception. The HSV color model is a color space created by AR Smith in 1978. As shown in Fig. 8, the angle between the radius of the cone bottom and the 0° line represents Hue, the radius of the cone bottom represents saturation, and the height of the cone represents the value of Value, their value ranges are respectively 0∼360, 0∼255, 0∼255.

Fig. 8

HSV color space.

Each color in the HSV space corresponds to a relatively consistent Hue, so that the Hue can be used to separate color regions. Compared with RGB space, HSV space can well meet the characteristics of color uniformity and integrity. The transformation from RGB space to HSV space is as follows:

First, RGB is normalized, there is R, G, B ∈ [0, 1]. The conversion formula from RGB space to HSV space is: $V = max (R, G, B)$ (12) $S = {\begin{matrix} 0 if V = 0 \\ \frac{V - min (R, G, B)}{V} else \end{matrix}$ (13)

We set: $r = \frac{V - R}{V - min (R, G, B)}$ (14) $g = \frac{V - G}{V - min (R, G, B)}$ (15) $b = \frac{V - B}{V - min (R, G, B)}$ (16) $6 H = {\begin{matrix} NaN if S = 0 that is V = 0 \\ 1 - g if V = R, min (R, G, B) = B \\ 5 + b if V = R, min (R, G, B) = G \\ 3 - b if V = G, min (R, G, B) = R \\ 1 + r if V = G, min (R, G, B) = B \\ 5 - r if V = B, min (R, G, B) = G \\ 3 + g if V = B, min (R, G, B) = R \end{matrix}$ (17)

Normally, the V value of a color image will not be zero to make H meaningless, and the H channel determines the hue information of an object. When detecting skin color, the saturation S and brightness V have little effect on skin color. Therefore, we do not consider the information of these two dimensions, only analyze the information of the H dimension.

In order to establish a good skin color model, some scholars collected several skin color images of Asian and European and American races from the Internet, and counted the H value distribution of these pixels, and then established a skin color model on this basis. A good segmentation effect has been achieved for the face, hands, and bare body parts. In this paper, the segmentation object is limited to the hands in a specific environment. Therefore, in order to achieve a more accurate skin color segmentation effect, the skin color image collection objects in this paper are also collected in a specific environment, and only the hand skin color image is collected, such as different gestures, different light changes (morning, noon, and evening), different Experimental environment (change of background desktop), etc. In this paper, a total of 88 hand images in different environments were collected, and 113 pieces of human skin color regions were manually cropped, which totaled 6007466 pixels. Through calculation and statistics, the average gray value of these pixels is 128.95, and the average values of the three RGB channels are 149.23, 124.21, and 99.90. There is R, G, B ∈ [0, 255]. This result is converted to HSV spatial image smoothing and morphological filtering.

During the acquisition or transmission process, the image is often affected by various factors, which inevitably affect the image quality, such as noise and distortion. Therefore, in order to improve image quality, we often do some processing on noisy images, such as image smoothing, morphological filtering, etc.

Smoothing is also called Blurring. It can suppress high-frequency signals in the image and highlight low-frequency signals in the image. It is a simple and common image processing method. Image smoothing has many uses, and the most common is to reduce noise or distortion on the image. The image smoothing process is often completed by different filters. Common filters include mean filtering, Gaussian filtering, and median filtering.

The neighborhood operator is an operator that determines the output value of a pixel by the pixel value around a given pixel of the input image, and the output value of the pixel is determined by the weighted sum of surrounding pixels. The specific calculation process is shown in formula (18). $g (i, j) = \sum_{m, n} f (i + m, j + n) h (m, n)$ (18)

Among them, f (x, y) , h (x, y) , g (x, y) represents input image, filter, and output image, respectively. The above process can be simply written as: $g = f \otimes h$ (19)

Mean filtering is the simplest filtering operation. The output is the average value of the corresponding pixels in the input kernel window, that is, the weighting coefficients at different positions in the filter are fixed values: $h (m, n) = \frac{1}{m * n}$ (20)

The mean filtering is to average the pixels in the neighborhood, so that it can suppress the high frequency part of the image and make the image blur and smooth, but the mean filtering also has inherent defects. If the noise pixel value is very different from the nearby pixel value, the output pixel value after averaging the area near the noise will also be larger, it also destroys some details in the image while denoising, and cannot remove the noise well.

Gaussian filtering is a linear smoothing filter that is commonly used to eliminate Gaussian noise. The Gaussian filter is a filter that selects weights according to the shape of the Gaussian function. The one-dimensional zero-mean Gaussian function is as follows: $G (X) = exp (- x^{2} / 2 {sigma}^{2})$ (21)

Among them, the parameter sigma determines the height of the Gaussian function. For two-dimensional images, a two-dimensional zero-mean Gaussian function is commonly used as a filter. The two-dimensional zero-mean Gaussian function is as follows: $G_{0} (x, y) = Ae \frac{- {(x - u_{x})}^{2}}{2 σ_{x}^{2}} + \frac{- {(x - u_{y})}^{2}}{2 σ_{y}^{2}}$ (22)

The specific steps of Gaussian filtering are similar to the mean filtering, but the filter is different. It is very effective for suppressing the noise that follows the normal distribution.

Mathematical morphology is an image analysis discipline based on Glenn and topology. Common operations include: corrosion, expansion, opening and closing operations, skeleton extraction, and so on. This section briefly introduces the most basic dilate and erode operations of morphology. The main functions of corrosion and dilation are as follows:

The noise is removed;

The maximum and minimum regions of the image are searched;

The gradient is calculated.

Dilation is the process of convolving an image with a convolution kernel. The convolution kernel can be square or circular, etc., which can be used as a mask or a template. Dilation is the operation of finding the maximum value of all pixels in the kernel. The specific steps are:First, the kernel is used to traverse the original image, then the local maximum pixel value of the original image in the region of the kernel is obtained as the output pixel value at that point, and finally the traversal is completed to obtain the output image. Repeated dilation operations will cause the highlighted areas in the image to grow outward along the border, similar to the “dilation” effect, so it is called dilation operation. The calculation formula of image expansion is shown in formula (23). Among them, src (x, y) is the original image, dilate (x, y) is the expanded image, and max() is the operation of maximizing the pixels in the core window area. $dilate (x, y) = max_{(x^{'}, y^{'}) \in ker nel} src (x + x^{'}, y + y^{'})$ (23)

Corrosion operation is just the opposite of dilation operation. The process is the same, and it takes the convolution kernel to traverse the original image. The difference is that the corrosion operation seeks the local minimum pixel value of the original image in the kernel area, and then this value is used as the output. Repeated corrosion operations will cause the highlighted areas in the image to shrink continuously along the border, similar to the effect of “corroded”, so it is called corrosion. The calculation formula of image erosion is shown in the following formula. Therefore, src (x, y) is the original image, erode (x, y) is the expanded image, and min() is the minimum value operation for the pixels in the core window area. $erode (x, y) = min_{(x^{'}, y^{'}) \in ker nel} src (x + x^{'}, y + y^{'})$ (24)

5 Experimental verification

Based on the system constructed above, this study conducts a system effect verification.This article takes an English classroom teaching as an example to carry out research and uses an online identification system as an experimental platform in actual teaching. Moreover, this paper combines the human-computer interaction and facial identification algorithms proposed in this paper to test the effectiveness of the system algorithm by artificially setting the interpersonal interaction and student status.Firstly, the human-computer interaction actions are identified and compared with the human identification method. The results obtained are shown in Table 1 and Fig. 9.

Table 1
1 Statistical table of human-computer interaction action identification

No. Human identification (%) System identification (%) No. Human identification (%) System identification (%) No. Human identification (%) System identification (%)

1 55.0 97.4 20 52.3 95.1 39 50.1 97.0

2 49.7 97.0 21 49.5 97.1 40 48.4 96.4

3 56.6 98.8 22 47.6 97.2 41 49.9 97.3

4 62.1 97.7 23 59.6 95.1 42 62.2 98.6

5 46.5 98.4 24 50.1 97.5 43 62.8 97.0

6 59.1 98.2 25 53.5 97.6 44 55.2 96.6

7 59.3 95.7 26 51.6 97.3 45 54.2 96.7

8 48.1 98.8 27 61.7 97.3 46 52.3 95.8

9 50.3 95.4 28 61.0 98.3 47 59.1 97.5

10 58.2 97.4 29 64.8 98.1 48 46.0 98.1

11 56.3 97.9 30 51.4 98.1 49 45.9 96.1

12 60.2 98.2 31 57.8 98.8 50 64.7 98.3

13 64.9 95.2 32 47.3 97.3 51 60.1 97.8

14 61.3 98.7 33 45.9 96.9 52 49.1 95.4

15 56.7 98.0 34 63.5 96.7 53 60.8 97.8

16 55.0 97.5 35 55.4 97.6 54 54.8 95.4

17 50.0 97.9 36 56.0 96.2 55 60.1 96.2

18 54.0 96.4 37 58.6 98.0 56 49.9 97.7

19 46.9 97.5 38 54.7 98.9

No.	Human identification (%)	System identification (%)	No.	Human identification (%)	System identification (%)	No.	Human identification (%)	System identification (%)
1	55.0	97.4	20	52.3	95.1	39	50.1	97.0
2	49.7	97.0	21	49.5	97.1	40	48.4	96.4
3	56.6	98.8	22	47.6	97.2	41	49.9	97.3
4	62.1	97.7	23	59.6	95.1	42	62.2	98.6
5	46.5	98.4	24	50.1	97.5	43	62.8	97.0
6	59.1	98.2	25	53.5	97.6	44	55.2	96.6
7	59.3	95.7	26	51.6	97.3	45	54.2	96.7
8	48.1	98.8	27	61.7	97.3	46	52.3	95.8
9	50.3	95.4	28	61.0	98.3	47	59.1	97.5
10	58.2	97.4	29	64.8	98.1	48	46.0	98.1
11	56.3	97.9	30	51.4	98.1	49	45.9	96.1
12	60.2	98.2	31	57.8	98.8	50	64.7	98.3
13	64.9	95.2	32	47.3	97.3	51	60.1	97.8
14	61.3	98.7	33	45.9	96.9	52	49.1	95.4
15	56.7	98.0	34	63.5	96.7	53	60.8	97.8
16	55.0	97.5	35	55.4	97.6	54	54.8	95.4
17	50.0	97.9	36	56.0	96.2	55	60.1	96.2
18	54.0	96.4	37	58.6	98.0	56	49.9	97.7
19	46.9	97.5	38	54.7	98.9

It can be seen from the statistical diagram of human-computer interaction action identification shown in Fig. 9 that the system has a better identification effect on human-computer interaction actions and a higher accuracy rate, while the accuracy rate of human-based identification is lower. Moreover, in the actual teaching, when only the teacher recognizes the student status, not only the efficiency is poor, but also the teaching effect is affected. It can be seen that the algorithm proposed in this paper has a certain effect in the teaching system.

Fig. 9

Statistical diagram of human-computer interaction action identification.

Next, this paper analyzes the accuracy of facial identification, recognizes facial expressions through the system, and also conducts comparative verification by combining human identification methods. The results are shown in Table 2 and Fig. 10.

Table 2

Statistical table of facial identification accuracy

No.	Human identification (%)	System identification (%)	No.	Human identification (%)	System identification (%)	No.	Human identification (%)	System identification (%)
1	16.9	95.4	20	24.1	96.6	39	22.5	95.9
2	22.1	98.8	21	23.1	98.6	40	30.9	97.8
3	18.4	96.4	22	34.2	98.4	41	26.6	95.1
4	32.8	96.7	23	20.9	97.3	42	19.8	98.1
5	33.7	97.5	24	31.1	95.9	43	19.4	96.7
6	28.0	98.0	25	33.9	95.7	44	26.6	96.2
7	31.2	97.5	26	34.0	96.0	45	20.8	96.0
8	21.8	97.3	27	32.6	97.1	46	26.4	96.7
9	20.0	95.6	28	20.4	96.3	47	15.8	95.8
10	17.6	95.9	29	25.0	96.4	48	16.5	97.7
11	19.3	96.8	30	15.9	95.6	49	18.3	95.2
12	19.6	98.0	31	26.9	98.4	50	21.6	98.2
13	28.2	96.9	32	19.0	98.0	51	30.2	96.7
14	24.8	98.2	33	18.3	97.3	52	30.9	98.9
15	24.5	96.2	34	21.2	97.7	53	33.4	98.0
16	29.3	98.7	35	22.1	96.8	54	25.6	98.5
17	16.7	96.0	36	23.6	97.9	55	31.3	96.4
18	15.8	98.7	37	34.7	95.9	56	25.3	96.4
19	16.5	99.0	38	34.8	98.4

Fig. 10

Statistical diagram of facial identification accuracy.

It can be seen from Fig. 10 that the algorithm’s statistics on the accuracy of facial identification are close to 100% (red part in the figure), while the accuracy rate of human observation is lower (yellow part in the figure). It can be seen that the algorithm constructed in this paper has good performance and can effectively improve the efficiency of English teaching.

6 Conclusion

As the teaching scale of colleges and universities continues to expand, the task of teaching management is also increasing.English courses are public courses for the whole school, so English teachers have a heavy teaching task. How to achieve the teaching purpose within the stipulated course hours and improve the teaching quality of English courses has always been a difficult problem to be solved in college teaching management. This forced colleges and universities to take further measures to improve their own teaching management system to meet the needs of college English teaching management. This paper proposes a face identification method for human-computer interaction in a complex background environment. By extracting and fusing the time sequence information and spatial information in the action in the video, a spatiotemporal feature image that can express a dynamic typical feature is obtained.Different actions have very different spatiotemporal feature images, so we can use different feature images to express different actions.Finally, deep learning is used to construct a convolutional neural network to train feature images to achieve the purpose of understanding and recognizing different actions and facial expressions.The research results show that the algorithm constructed in this paper has good performance.

Footnotes

Acknowledgment

The research in this paper was supported by Social and Science Fund of Hunan Province: Research on categorization Perception of wh-words under the influence of linguistic experience (NO. 18WLH21).

References

Anjos

, Chakka

M.M.

and Marcel

, Motion-based counter-measures to photo attacks in face recognition, Biometrics, IET 3(3) (2014), 147–158.

Zhuang

, Chan

T.H.

, Yang

A.Y.

, et al., Single-Sample Face Recognition with Image Corruption and Misalignment via Sparse Illumination Transfer, Proceedings / CVPR, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society Conference on Computer Vision and Pattern Recognition 114(2-3) (2014), 272–287.

Bouchech

, Selection of optimal narrowband multispectral images for face recognition, monthly notices of the royal astronomical society 402(4) (2015), 2140–2186.

Huang

, Shan

, Wang

, et al., A Benchmark and Comparative Study of Video-Based Face Recognition on COX Face Database, IEEE Transactions on Image Processing 24(12) (2015), 5967–5981.

Wei

C.P.

and Wang

Y.C.F.

, Undersampled Face Recognition via Robust Auxiliary Dictionary Learning, Image Processing IEEE Transactions on 24(6) (2015), 1722–1734.

Ajit Krisshna

N.L.

, Deepak

V.K.

, Manikantan

, et al., Face recognition using transform domain feature extraction and PSO-based feature selection, Applied Soft Computing 22(4) (2014), 141–161.

Weng

, Lu

and Tan

Y.P.

, Robust Point Set Matching for Partial Face Recognition, Image Processing IEEE Transactions on 25(3) (2016), 1163–1176.

Cui

, Chang

, Shan

, et al., Joint sparse representation for video-based face recognition, Neurocomputing 135(11) (2014), 306–312.

Phillips

P.J.

and Otoole

A.J.

, Comparison of human and computer performance across face recognition experiments, Image and Vision Computing 32(1) (2014), 74–85.

10.

Tang

, Zhu

, Yu

, et al., A novel sparse representation method based on virtual samples for face recognition, Neural Computing & Applications 24(3-4) (2014), 513–519.

11.

Mehta

, Yuan

and Egiazarian

, Face recognition using scale-adaptive directional and textural features, Pattern Recognition 47(5) (2014), 1846–1858.

12.

Raghavendra

, Raja

K.B.

and Busch

, Presentation Attack Detection for Face Recognition Using Light Field Camera, IEEE Transactions on Image Processing 24(3) (2015), 1060–1075.

13.

Khan

M.A.

, Multiresolution coding of motion capture data for real-time multimedia applications, Multimedia Tools & Applications 76(15) (2016), 1–16.

14.

Puupponen

, Wainio

, Burger

, et al., Head movements in Finnish Sign Language on the basis of Motion Capture data: A study of the form and function of nods, nodding, head thrusts, and head pulls, Sign Language & Linguistics 18(1) (2015), 41–89.

15.

Lee

and Yoo

, Low-Cost 3D Motion Capture System Using Passive Optical Markers and Monocular Vision, Optik International Journal for Light & Electron Optics 130(22) (2016), 1397–1407.

16.

Rahman

M.M.

, Analysis of Finger Movements of a Pianist Using Magnetic Motion Capture System with Six Dimensional Position Sensors, Transactions of the Virtual Reality Society of Japan 15(4) (2017), 243–250.

17.

Park

S.W.

, Park

H.S.

, Kim

J.H.

, et al., 3D displacement measurement model for health monitoring of structures using a motion capture system, Measurement 59(11) (2015), 352–362.

18.

Giannetti

, Petrella

, Bach

, et al., Bone Position Measurement Using High-Frequency Ultrasound Validated with 3-D Optical Motion Capture Systems: A Feasibility Study, Journal of Medical & Biological Engineering 37(7) (2017), 1–8.

19.

Zhang

, Wang

, Chu

, et al., Application of Optical Motion Capture Technology in Power Safety Entitative Simulation Training System, Optics & Photonics Journal 06(8) (2016), 155–163.

20.

Wang

S.K.

, Xie

and Hu

, Research on Protection Property of Running Sportswear Fabrics Based on 3-D Motion Capture System, Textiles and Light Industrial Science and Technology 13(4) (2014), 57–62.

21.

Miura

, Kaiga

, Shibata

, et al., Low-dimensional Feature Vector Extraction from Motion Capture Data by Phase Plane Analysis, Journal of Information Processing 25(2) (2017), 884–887.

22.

Frost

D.M.

, Beach

T.A.C.

, Campbell

T.L.

, et al., Can the Functional Movement Screen™ be used to capture changes in spine and knee motion control following 12 weeks of training? Physical Therapy in Sport 23(2) (2016), 50–57.

23.

Zhang

, Yan

X.K.

and Liu

A.G.

, An Introduction to A Newly-developed “Acupuncture Needle Manipulation Training-evaluation System” Based on Optical Motion Capture Technique, Acupuncture Research 41(6) (2016), 556–559.

24.

Paul

, Ahmad

, Rathore

M.M.

and Jabbar

, Smartbuddy: defining human behaviors using big data analytics in social internet of things, IEEE Wireless communications 23(5) (2016), 68–74, November 2016.

25.

Subedi

and Vatsala

A.S.

, Quenching Problem for Two Dimensional Caputo Time-Fractional Reaction- Diffusion Equation, Dynamic Systems and Applications 29(1) (2020), 26–52.

26.

Paul

, Jiang

Y.C.

, Wang

J.F.

and Yang

J.F.

, Parallel reconfigurable computing-based mapping algorithm for motion estimation in advanced video coding, ACM Transactions on Embedded Computing Systems (TECS) 11(S2) (2012), 1–18.

27.

Rathore

M.M.

, Paul

, Hong

W.H.

, Seo

H.C.

, Awan

and Saeed

, Exploiting IoT and big data analytics: Defining smart digital city using real-time urban data, Sustainable cities and society 40 (2018), 600–610.

28.

Paul

, “Internet of Things: A primer’, R Jeyaraj Human Behavior and Emerging Technologies 1(1) (2019), 37–47.

29.

Peng

and Wang

, Lyapunov-Type Inequalities for a Class of Linear Sequential Fractional Differential Equations, Dynamic Systems and Applications 28 (2019), 859–867.

30.

Paul

, Daniel

, Ahmad

and Rho

, Cooperative cognitive intelligence for internet of vehicles, IEEE Systems Journal 11(3) (2017), 1249–1258.