The construction of multimodal smart classroom teaching evaluation indicator system for higher vocational English based on hierarchical analysis approach

Abstract

Along with the continuous promotion of curriculum reform, English has become a very important course content in the stage of higher vocational education, which is an important path to cultivate English-speaking professional talents and integrate them into social development. At the same time, the continuous promotion of educational reform, but also the diversified teaching methods into the English curriculum, the application of multimodal teaching mode in higher vocational English teaching, is bound to enhance the quality of English teaching to a greater extent. Based on this, this paper uses the hierarchical analysis method to construct a multimodal intelligent classroom teaching evaluation index system for higher vocational English, and teachers, media, and students are assigned three indicators, of which teachers occupy 20%, teaching media 60%, and students 20%. And puts forward the corresponding teaching optimization strategy, from adopting multimodal English teaching resources, creating a multimodal English teaching atmosphere, developing multimodal English teaching activities, and implementing multimodal English teaching evaluation, to put forward corresponding teaching suggestions. And puts forward the corresponding teaching optimization strategy, from adopting multimodal English teaching resources, creating a multimodal English teaching atmosphere, formulating multimodal English teaching activities, and implementing multimodal English teaching evaluation, so as to provide certain reference for the improvement of higher vocational English teaching level and the application of multimodal intelligent classroom teaching mode.

Keywords

higher vocational English multimodal teaching teaching mode wisdom teaching English teaching hierarchical analysis method

Introduction

Multimodal teaching is an emerging teaching mode, the core content of which lies in fully mobilizing the learners’ physical senses such as hearing, vision, taste, touch, smell, and other senses to participate in learning, communicating through a variety of means and symbolic resources such as language, image, sound, and movement, broadening the perception pathway of the learners’ knowledge, competence, and skills, and applying a variety of teaching methods to mobilize the learners’ multiple senses to operate synergistically, and then enhance the learners’ English multimodal learning ability and cross-cultural communicative competence. The study used a hierarchical analysis method (Anatomy of a hierarchical analysis) to determine the effectiveness of the teaching and learning of English.¹

Researchers at home and abroad through empirical studies have concluded that multimodal teaching is significantly better than unimodal teaching. 1996, the New London Group opened the precedent of multimodal language teaching, emphasizing the importance of multilingual symbols for language learning, advocating the use of multilingual literacy to carry out language teaching and guiding the educators to cultivate the learners’ multimodal competence through the generation of meaning of the linguistic symbols. American scholar Edgar Dale’s “pyramid theory” of language learning holds that the degree of language mastery is closely related to the symbolic modality, and that the average retention rate of learners’ learning content is “5% listening to lectures, 10% reading, 20% audio-visual, 30% demonstration, 50% discussed, 75% practiced, and 90% taught to others.” It can be seen that the more diversified the modes of language teaching, the more optimistic the learners’ knowledge acquisition and retention rate will be.

In 2024, Yu et al. introduced multimodal teaching into teaching for the first time.² Through the conceptual analysis of multimodal teaching theory, focusing on the theoretical issues of multimodal conversion, multimodal collocation, and multimodal filters, and analyzing the difference between multimedia teaching and multimodal teaching, he constructed a multimodal learning model based on the students’ learning situation to strengthen the cultivation of the learners’ comprehensive English application ability. Shou et al. pointed out that the multimodal college English smart learning teaching mode can break through the limitations of time and space through the multimedia platform, and mobilize students to participate in teaching and learning activities in multiple angles and in an all-round way, which extends from inside the classroom to outside the classroom.³ Duan et al. pointed out that the intelligent learning teaching mode can break through the traditional teaching time limitations and take the time of after-class learning as a teaching supplement, which in turn creates a long time learning efficiency for students to construct English thinking.⁴ However, from the examination of the current situation of the research on multimodal intelligent classroom teaching in higher vocational English, it is found that there are very few relevant research discourses, and only a preliminary discussion on the use of multimodal teaching in the English classroom, with insufficient systematic and scholastic research, and a lack of empirical research on the effectiveness of multimodal teaching and English teaching in secondary vocational education. Based on the analysis of the above scholars as well as the collation and study of related theories, this paper innovatively combines multimodal intelligent classroom teaching with the evaluation of secondary English classroom teaching to explore the construction of evaluation indexes and the optimization strategy of teaching mode. On the one hand, it can provide certain reference for the improvement of related theories, and on the other hand, it can also provide certain support for the improvement of classroom teaching efficiency.

This study adopts the Analytic Hierarchy Process (AHP) to study the influencing factors of multimodal classroom teaching effect in higher vocational English and explore the establishment of a teaching effect evaluation model. In this way, it provides certain suggestions for the efficient implementation of the evaluation of multimodal intelligent teaching of English in higher vocational education as well as the improvement of teaching quality.

Determination of evaluation indexes

Check all kinds of literature on multimodal classroom teaching and evaluation of higher vocational English, and organize various influencing factors of multimodal classroom teaching through brainstorming. Prepare the Questionnaire for Evaluating the Effectiveness of Multimodal Classroom Teaching in Higher Vocational English, which includes teachers engaged in higher vocational English education and higher vocational students. The questionnaire is designed around the evaluation of multimodal teaching effect of higher vocational English, and the questionnaire content is centered around multimodal teaching from these three aspects, investigating the application and effect of multimodal teaching of teachers, the knowledge, defects and needs of students on the effect of multimodal teaching, and the weighting and use of various modes of the medium. Weighting and use, among other aspects. A total of 55 teachers and supervisors and 200 students from three higher education institutions were surveyed to obtain ratings for each indicator. A hierarchical analysis evaluation model was established.

Determination of evaluation index weights

Due to the wide range and complexity of the situation involved in determining the weights of the evaluation system indicators, a two-by-two comparison consultation table was designed to reduce the number of experts consulted. In the previous round of teaching managers in the institutions based on each institution proportionally selected to invite the 10 experts engaged in teaching management for the longest number of years to participate in determining the weights of indicators at all levels in the form of anonymity, and get 10 valid questionnaires. Table 1 shows the indicators.

Table 1.

Findings from the identification of primary and secondary indicators.

Primary indicators	Secondary	Number of people in favor	Proportion (%)
Teacher	Teacher quality	23	92
	Instructional strategies	23	92
	The use of multimodal	2	8
Intermediary	Content is suitable	25	100
	Scale relation of each mode	20	80
	Media interactivity	25	100
Student	Teaching participation	25	100
	Learning ability	25	100
	Learning state	25	100

Ideas for calculating the weights of indicators

Based on the hierarchical structure of the evaluation system and the principle of AHP, the weights of the indicators are calculated.

(1) Construct judgment matrix. Using the 1–9 comparison scale will be used for each expert’s feedback results of the consultation form for the construction of the current elements of the previous level of each element of the two-two comparison judgment matrix; (2) Calculate the matrix square root method to determine the relative weights. Determine for each element of the previous level, the ranking of the importance of the current level with its related elements. (3) Consistency test. After obtaining the weight coefficients given by each expert for the evaluation indicators, test whether the weights obtained by the calculation are logical. Usually, the consistency test index CI tests the relative priority of each index.

To prioritize the order of logical confusion, it is generally believed that CI < 0.1, the weight is acceptable; average random consistency ratio CR value measure whether the judgment matrix has a satisfactory consistency, CR <0.1, it is generally believed that the judgment matrix has a satisfactory consistency. (4) Calculation of the integrated weights. According to the results of the test to remove the CR >0.1, the weight of the experts on the same indicator for the arithmetic average.

According to the idea of calculating the weight of the indicator, the following formula is used for each expert’s consultation results for constructing and calculating the judgment matrix A, the largest characteristic root λ max and its corresponding eigenvector W, and then the consistency test.

Ⅰ. Multiply the elements of each row in A and divide by m times to get the vector where

W^{*} = {(w_{1}^{*}, w_{2}^{*}, . . ., w_{m}^{*})}^{T}, W^{*} i = m \sqrt{\prod_{j = 1}^{m} a_{i j}}

Ⅱ. Normalize $W^{*}$ to obtain the weight vector $W = {(w_{1}, w_{2}, . . ., w_{m})}^{T}$ , where $W_{i} = W_{i}^{*} / \sum_{i = 1}^{m} w_{i}^{*}$ .

Ⅲ. Calculate the value of $λ_{\max}$ , ${λ \sum_{i = 1}^{m} s_{i} w_{i} \frac{1}{m} \sum_{i = 1}^{m} \frac{{(A W)}_{i}}{W_{i}}}_{\max}$ , where the vector $S = (s_{1}, s_{2}, . . ., s_{m}), s_{j} = \sum_{i = 1}^{m} a_{i j}$ is obtained by summing the elements of each column of A.

Ⅳ. Consistency index of judgment matrix: $C I = \frac{λ_{\max}}{n - 1}$ , random index $R I$ , and consistency ratio $C R = \frac{C I}{R I}$ .

Realization of indicator weight calculation

(1) Calculate the weights of the first-level indicators based on the results of each expert’s consultation.

Ⅰ. Through the results of the consultation table, establish the judgment matrix table of the first-level indicators, see Table 2.

Ⅱ. Construct judgment matrix A0 = according to the judgment matrix table $A_{0} = [\begin{array}{l} 1 & 1 / 3 & 1 \\ 3 & 1 & 3 \\ 1 & 1 / 3 & 1 \end{array}]$ . Calculate the weight vector of first-level indicators $W_{0} = {(0.200, 0.600, 0.200)}^{T} a n d λ_{\max}$ .

Ⅲ. Judgment matrix consistency index: $C I = \frac{λ_{\max}}{n - 1}$ , check the table to get the average stochastic consistency index RI = 0.58 < 0.1, consistency ratio $C R = \frac{C I}{R I} = \frac{0}{0.58} = 0 P ⟨ 0.1$ , the judgment matrix has a satisfactory consistency, which indicates that the judgment matrix is based on the $W_{0} = (0.200, 0.600, 0.200) τ$ each component as a first-level indicator of each factor weight is reasonable, $W_{0}$ can be used as a weight vector.

(2) Calculate the weights of the second-level indicators based on the results of each expert’s consultation.

Table 2.

Matrix.

	Teacher	Intermediary	Student
Teacher	1	1/3	1
Intermediary	3	1	3
Student	1	1/3	1

The second-level indicator level contains a total of eight factors, according to the expert’s opinion formation of the corresponding judgment matrix of the second-level indicators are

A_{1} = [\begin{array}{l} 1 & 2 / 3 \\ 3 / 2 & 1 \end{array}], A_{2} = [\begin{array}{l} 1 & 8 & 8 \\ 1 / 8 & 1 & 1 \\ 1 / 8 & 1 & 1 \end{array}], A_{3} = [\begin{array}{l} 1 & 1 & 3 / 4 \\ 1 & 1 & 3 / 4 \\ 4 / 3 & 4 / 3 & 1 \end{array}]

Using the above steps Ⅱ, Ⅲ in the calculation of the weights of the first-level indicators (1) were calculated for the second-level indicators, the results are as follows:

A_{1} = [\begin{array}{l} 1 & 2 / 3 \\ 3 / 2 & 1 \end{array}], {λ 1 (0.399, 0.598)}_{\max}

Similarly, the weight of the secondary indicators, W₂ = (0.800, 0.100, 0.100), W₃ = (0.300, 0.300, 0.400) and CR are less than 0.1, indicating that the weight coefficients of the indicators, W₁, W₂, W₃ allocation are reasonable, to determine the weight of the results of the expert consultation.

Indicator weighting results

According to the arithmetic average of the weight of each expert on the same indicator (excluding the case of CR >0.1), the comprehensive weights representing the views of the expert group, as Table 3. Similarly, the weights of each indicator in the secondary indicator system are obtained weights, see Table 4.

Table 3.

Determination of the weights of primary indicators.

Expert serial number	$W_{1}$	$W_{2}$	$W_{3}$
1	0.200	0.600	0.200
2	0.311	0.562	0.127
3	0.114	0.691	0.195
4	0.467	0.405	0.128
5	0.079	0.701	0.220
6	0.073	0.650	0.277
7	0.334	0.447	0.219
8	0.179	0.608	0.213
9	0.090	0.702	0.208
10	0.151	0.633	0.216
Average value	0.1998	0.5999	0.2003

Table 4.

Determination of the weights of secondary indicators.

Indicator	Teacher		Intermediary			Student
Indicator	$W_{11}$	$W_{12}$	$W_{21}$	$W_{22}$	$W_{23}$	$W_{31}$	$W_{32}$	$W_{33}$
Average	0.3989	0.6011	0.8001	0.0995	0.1004	0.2986	0.3002	0.4012

Construction of evaluation index system

According to the results of AHP calculation and the status quo of higher vocational colleges, the evaluation index system of students’ comprehensive academic performance is formulated, see Table 5. In order to verify the scientificity and operability of the index system, further verification research will be conducted on the evaluation index system in the future.

Table 5.

Evaluation system of English multimodal intelligent classroom teaching for higher vocational students.

Primary indicators	Secondary	Full credit
Teacher	Teacher quality	8
Teacher	Instructional strategies	12
Intermediary	Content is suitable	48
	Scale relation of each mode	6
	Media interactivity	6
Student	Teaching participation	6
	Learning ability	6
	Learning state	8

As can be seen in Table 5, teacher indicators, medium of instruction indicators, and student indicators occupy 20%, 60%, and 20% of the indicators, respectively. The importance of the medium of instruction is self-evident, and the scientific nature of the teaching content occupies 48%, indicating that the planning of the teaching content should be improved. The second is the application of teaching strategies, which occupies 12%, indicating that diversified teaching methods should be used to improve the efficiency of multimodal teaching.

Strategies for applying multimodal teaching model in higher vocational English teaching

Adopting multimodal English teaching resources

In higher vocational English teaching, in order to better apply the multimodal teaching mode, the first thing teachers should do is to integrate more teaching resources in the classroom. Teaching activities are knowledge transfer activities in the final analysis, so teachers must first fully prepare for classes, try to find resources suitable for multimodal teaching through a variety of channels, and make full preparation for the classroom in order to better practice the classroom effect. In the choice of teaching resources, teachers should choose classroom resources that can be integrated into this teaching mode according to the characteristics of multimodal teaching and students’ learning.^5,6 Generally speaking, common classroom teaching resources are as follows: text, video, and audio. Among them, multimedia equipment is a very important teaching carrier, through multimedia equipment, the above several kinds of teaching resources can be shown to students, so that the classroom is more intuitive and effective. At the same time, audio and video are more expressive than text, and are more frequently used in multimodal teaching, which meets the needs of multimodal teaching mode. In terms of content selection of specific teaching resources, English teaching resources should be close to real life and also have learning value. For example, through the video to understand the culture of the English-speaking countries and the basic customs of life, which can effectively improve the effect of higher vocational English teaching.^7,8

Create a multimodal English teaching atmosphere

In the process of teaching activities, the teaching atmosphere needs to be cultivated. So, while using the multimodal teaching mode, higher vocational English teachers should also actively create a teaching atmosphere that is suitable for this, in order to make the effect of this teaching mode better. Moreover, the lack of enthusiasm of higher vocational students is mainly due to the poor foundation of their past learning, which makes them easy to experience acquired helplessness when learning new knowledge. At this time, if teachers can create a positive and good teaching atmosphere, it will reduce the students’ anorexia. At the same time, in the learning of knowledge, teachers should be gradual, slowly lead students to build a solid foundation for learning.^9,10 For example, in teaching, you can take the group cooperative learning approach, as a way to stimulate students’ initiative and participation, at the same time can make the classroom atmosphere more enthusiastic, so as to avoid the drawbacks of the teacher unilaterally imparting knowledge, or carry out some interesting small activities to enhance the interest of the classroom. Through the above means, to cultivate a good English teaching atmosphere, so that the multimodal teaching mode can be better applied.

Second, creating valuable teaching situations is an important pursuit of teaching reform and an important means to stimulate students’ learning interests. In the multimodal teaching of higher vocational English, teachers must focus on the synergistic participation of students’ multimodality and pay attention to the creation of high-quality and valuable teaching situations. Then, teachers should be good at exploring contextual materials according to the requirements of teaching objectives, creating communicative modes between students, guiding students to participate in classroom interactions, and generating energetic classroom contexts. For example, teachers can create a my birthday drama performance situation, let the students perform a drama about the birthday in the form of a group, students should do a good job of role positioning, and according to the role requirements for situational dialogues, to realize the multimodal communicative interaction. At the same time, teachers can also create multimodal communicative scenarios using teaching carriers such as microclasses, catechisms, and PPTs, introduce video, audio, or pictures, create communicative situations, and guide students to carry out listening and speaking competence training around the teaching theme in the teaching context. Of course, in order to improve the students’ practical application of English, teachers can also create simulated teaching situations that are connected to the workflow, and guide students to carry out multimodal communicative interactions in the workplace, so as to “apply what they have learned” and expand their vocational abilities.

Developing multimodal English teaching activities

In the process of China’s continuous integration into the world, the demand for international talents is also increasing. This puts forward the requirements for education in China, which requires the strengthening of English practical application ability in teaching.^11,12 Therefore, the teaching of English in higher vocational education should also put forward higher requirements on the quality of teaching. Specifically, when applying the multimodal teaching mode, this feature should be reflected more in the teaching activities. Teachers should keep abreast of the times and change their teaching methods and means in time in order to better realize the effect of English teaching. Therefore, when designing English teaching activities, teachers should make them more inclined to multimodal co-development. For example, playing English radio or other works in the classroom can be used to fully exercise students’ English auditory modality. Then students can imitate the content of the video, compete in groups, and select the best group at the end. In this way, not only do students fully participate in the classroom, but also effectively practice the multimodal teaching mode and innovate the teaching method.

Implementing multimodal English teaching evaluation

In addition to classroom teaching activities, the teaching evaluation mechanism can also profoundly affect students’ English learning. Under the background of innovative teaching mode, the teaching evaluation mechanism should also be changed to better adapt to the changes of the times. Therefore, in higher vocational English teaching, it is necessary to establish a teaching evaluation mechanism based on multimodal evaluation in order to more accurately judge the learning effect of students and provide students with positive feedback.^13,14 When students study hard, they can receive positive feedback, which can play a very big role in motivating students to encourage them to learn English better. Meanwhile, for teachers, a perfect teaching evaluation mechanism can also enable them to better grasp the direction of teaching, discover their own shortcomings and deficiencies, and then consciously improve and enhance. Overall, a teaching evaluation mechanism applicable to the multimodal teaching mode can be more effective in assisting teaching activities.

Conclusion

The traditional multimodal teaching evaluation of higher vocational English is to analyze the problem from a qualitative point of view. And as the multimodal teaching mode is more and more widely used in higher vocational English teaching, the establishment of the evaluation mode of multimodal higher vocational English teaching effect is very necessary, as well as quantitative and objective, which is of great significance for the scientific development of the subsequent educational work. However, in the research on multimodal English wisdom teaching, most of the scholars’ views focus on the advantages of multimodal teaching and its application in English teaching. But there are relatively few studies on the evaluation system of multimodal wisdom teaching in higher vocational English. Based on this, this paper constructs a multimodal senior vocational English wisdom classroom teaching evaluation index system based on hierarchical analysis, discovers the weaknesses of senior vocational English wisdom teaching under the current multimodal approach, and puts forward corresponding optimization suggestions, so as to provide certain references for the efficient application of multimodal wisdom teaching of senior vocational English.

Based on the results of this research and combined with other similar studies, the following suggestions are put forward for the multi-modal classroom teaching of higher vocational English: (1) the organic combination of teachers, students, and multi-modal media. (2) To strengthen teachers ‘attention to students’ learning experience. (3) Improve the interactivity of multimodal teaching media. (4) Encourage students’ willingness to learn to improve the teaching effect. (5) Pay attention to the process evaluation of English practice. (6) Strengthen the teaching effect of multimodal mode through analysis. (7) Create a multimodal environment in the classroom context. Classroom teaching is a creative labor, any class is brand-new, even if the same content, but also to have a positive and innovative attitude. The research results can be used as a reference basis for the evaluation of the multimodal classroom teaching effect in higher vocational English.

Although the research in this paper has achieved certain results, there are still some limitations in the results obtained due to the relatively small sample content and the short research time. In the process of future research, the scope of the research sample will be expanded, more theories will be collected as reference, and multimodal intelligent classroom teaching evaluation will be taken as the direction to propose a wider range of indicators, so as to improve the accuracy of the research.

Footnotes

Funding

The author disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: A Project Supported by Scientific Research Fund of Zhejiang Provincial Education Department (Y202353310).

Declaration of conflicting interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

Shu

. An empirical study of a smart education model enabled by the edu-metaverse to enhance better learning outcomes for students. Systems 2023; 11(2): 75.

Guo

. Sustainability in English language teaching: strategies for empowering students to achieve the sustainable development goals. Sustainability 2024; 16(8): 3325.

Shou

, et al. Research on knowledge tracing-based classroom network characteristic learning engagement and temporal-spatial feature fusion. Electronics-Switz 2024; 13(8): 1454.

Duan

Dong

Wang

. Student-centered assessment research on holographic learning paradigm based on intelligent analytic hierarchy process in teaching of bridge engineering course. Sustainability 2024; 16(6): 2430.

Sapena

Onaindia

. Multimodal classification of teaching activities from university lecture recordings. Appl Sci-Basel 2022; 12(9): 4785.

Sezerer

Tekir

. Incorporating concreteness in multi-modal language models with curriculum learning. Appl Sci-Basel 2021; 11(17): 8241.

Liu

, et al. The cultural sustainability in English as foreign language textbooks: investigating the cultural representations in English language textbooks in China for senior middle school students. Front Psychol 2022; 13: 944381.

Muth

Lüftenegger

. Associations between autonomy-supportive teaching, the use of non-academic ICTs, and student motivation in English language learning. Sustainability-Basel 2024; 16(3): 1337.

Wang

. On the role of EFL/ESL teachers’ emotion regulation in students’ academic engagement. Front Psychol 2021; 12: 758860.

10.

Hubálovský

Hubálovská

Matoušová

. A new hybrid particle swarm optimization–teaching–learning-based optimization for solving optimization problems. Biomimetics 2024; 9(1): 8.

11.

Xia

Feng

Yang

, et al. MFIRA: multimodal fusion intent recognition algorithm for AR chemistry experiments. Appl Sci-Basel 2023; 13(14): 8200.

12.

Tan

Thomson

Koh

JHL

, et al. Teaching multimodal literacies with digital technologies and augmented reality: a cluster analysis of Australian teachers’ TPACK. Sustainability 2023; 15(13): 10190.

13.

Tang

. Understanding the role of digital immersive technology in educating the students of English language: does it promote critical thinking and self-directed learning for achieving sustainability in education with the help of teamwork? BMC Psychol 2024; 12(1): 144.

14.

Huertas

Ruipérez-Valiente

García Clemente

, et al. A scalable architecture for the dynamic deployment of multimodal learning analytics applications in smart classrooms. Sensors-Basel 2020; 20(10): 2923.