Video Asset Library and Its Management for Higher Education Department: An Evaluation Study

Introduction

DIGITAL inclusion in education is achieved through digital platforms. It is commonly used to bring classroom education to the doorstep of students. Many renowned global educational institutes have started different initiatives for free universal learning using ICT tools. Stanford University has offered learning opportunities through free online courses, online degrees, graduate and professional certificates, e-learning and open courses through a variety of massive open online courses since 2013. There are many global examples, but India also started its free e-learning initiatives in the year 2003 through the National Programme on Technology Enhanced Learning, along with seven Indian Institute of Technologies and the Indian Institute of Science, which is funded by the Ministry of Education Government of India.

Under Rashtriya Uchchatar Shiksha Abhiyan, the Virtual Tutorial Project (VTP), an e-learning platform (https://vtputkal.odisha.gov.in) has been created to establish equal learning opportunities among urban and rural students of Odisha in all disciplines through a digital platform. This initiative aims to compensate the inadequate teaching assistants in remote areas and bridge the digital divide.

Although the e-learning platform experienced significant growth during the COVID-19 pandemic, not much research has focused on the measurement of e-learning systems service quality, which is crucial for platform users. The work considered in this study includes selecting an e-learning model for measurement, designing a questionnaire for user feedback, conducting data analysis to measure the system’s service quality, drawing conclusions and discussing the results and recommendations. The remainder of the article is organised as follows: The Conceptual Background of Service Quality and the SERVQUAL Model, E-learning Platform Architecture and Feedback Analysis and Google Analytics, Research Methodology, Results of the Data Analysis and finally the Conclusion and Findings of the Research.

Conceptual Background

Providing quality service to customers is essential in today’s competitive business environment. Customers will come back when good service is guaranteed. However, measuring performance against customer satisfaction is paramount. It is also necessary to measure and evaluate the service quality of the e-learning system. The SERVQUAL model, feedback analysis and Google Analytics for measuring the service quality of e-learning systems are the objectives of this study.

Utkal University E-learning Platform Architecture

The e-learning architecture is designed by keeping a search engine in the core, thus the system delivers the content according to the user requirements. It can find the required content quickly, concerning user preferences and choices.

The architecture presented in Figure 1 consists of the different stakeholders in the proposed system and dataflow among entities: Utkal University DDCE Studio, DDCE Pedagogical Team (Creator, Evaluator), Administrator (Web, Media files), Command Control Centre National Informatics Centre (NIC) New Delhi (video on demand [VOD], Media Storage server), Odisha State Data Centre (Web server, Database Server), User (Students, Lecturers and others).

OPEN IN VIEWER Figure 1.

DDCE Utkal University E-learning Platform Architecture.

SERVQUAL Model

The SERVQUAL model is a product of marketing research used to measure customer satisfaction with the quality of service provided. The model is a 5-dimension, 22-item framework. Figure 2 SERVQUAL claims are the five elements called RATER Figure 3 represents 22 items of the SERVQUAL model.

OPEN IN VIEWER

Figure 2.

Source: Parasuraman et al. (1988) and Ali Ramezani Ghtbabadi (2015).

OPEN IN VIEWER

Figure 3.

Source: marketingstudyguide.com.

SERVQUAL model with 5-dimension, 22-item framework.

The following equation expresses the SERVQUAL model quality function;

S Q i = Σ W j ( E i j − P i j )

where SQi—perceived dimension quality; Wj—attribute importance factor; Pij—perception of dimension i in relation to attribute j; Eij—expected level of the attribute; and j, which is a normative of dimension i.

Modified SERVQUAL model; the concept of zone of tolerance (ZOT) (which stands for ZOT) to represent customer expectations.

Customers do not have a single level of expectations, but rather a range of expectations, called tolerance (Figure 4), where desired service is provided at the upper end of the scale and adequate service is provided at the lower end. If the service is acceptable, the customer is satisfied with the service, if the service he desires is above the level, he is satisfied and if it is below the acceptable level, the customer is not satisfied with the service.

OPEN IN VIEWER

Figure 4.

Source: Adopted from Zeithaml et al. (1993, p. 5)/Halil Nadiri and Seyed Muhammad Ali Mayboudi.

It is important to design and create a suitable questionnaire, to obtain the client’s psychometric score. Parasuraman, Zeithaml and Beny developed the three alternative questionnaire formats, as shown in Table 1 for measurement and validated these questionnaires. In his 1997 publication, they introduced the ZOT SERVQUAL model into the Information System field and defined the model in terms of four dimensions and 18 items: Reliability (5), Responsiveness (4) Assurance (4) and Empathy with (5).

OPEN IN VIEWER

Table 1.

Three-column Format Questionnaire Sample.

Attribute	The Level of Services Received from the E-learning Platform on Regular Us								The Level of Services You Desire from the E-Learning Platform on Regular Ise Is							The Minimum Level of Service You Would Accept from the E-learning Platform on Regular Use
	Low				High				Low				High			Low				High
Performance of e-learning portal	1	2	3	4	5	6	7	N	1	2	3	4	5	6	7	1	2	3	4	5	6	7

Research Methodology

Data Analysis and Results

Hitherto 40 thousand users have visited the site and 15 thousand students’ downloaded e-lecturers. Alongside different states of India, the site also receives hits from different corners of the globe, USA, Australia, Singapore and other countries (Figure 5). Therefore, to further improve service quality, it is urgent to measure user satisfaction.

OPEN IN VIEWER Figure 5.

Visitors Statistics Across the World.

Expectations, perceptions and reasonable averages for various respondent attributes within each dimension are shown in Table 2.

OPEN IN VIEWER

Table 2.

Expectation, Perception and Adequate Averages.

	Dimension	Expectation Averages	Perception Averages	Adequate Averages
1	Reliability	6.14	5.88	5.83
2	Reliability	6.56	5.64	5.61
3	Reliability	6.81	5.89	5.33
4	Reliability	6.66	5.84	5.21
5	Reliability	6.45	6.01	5.76
6	Responsiveness	6.44	5.93	5.64
7	Responsiveness	6.34	5.87	5.21
8	Responsiveness	6.41	6.21	5.64
9	Responsiveness	6.33	6.03	5.45
10	Assurance	6.41	5.84	5.21
11	Assurance	6.36	5.78	5.26
12	Assurance	6.41	5.88	5.46
13	Assurance	6.21	5.77	5.33
14	Empathy	6.27	5.83	5.35
15	Empathy	6.25	5.88	5.11
16	Empathy	6.22	5.86	5.23
17	Empathy	6.21	5.73	5.28
18	Empathy	6.24	5.51	5.12
	Average	6.17	5.69	5.39

We analysed the data obtained and examined the difference between expected and perceived service for each attribute. Table 2 shows the top 10 traits with the largest gaps between expectations and perceptions. The student/teacher expectations related to questions 5, 4 and 3 of the Reliability dimensions need to be improved by the organisation. In the response dimension for questions 6 and 8, customers expect feedback on their complaints. The gap is large because they expect immediate feedback. In the Assurance aspect of question 10, customers expect the behaviour of each employee dealing with their services to inspire trust, and in the Empathy aspect of question 16, respondents believe that each employee should understand and provide a solution. Reducing these gaps can improve service quality.

Table 3 shows the quality of service gap for each dimension SQi = ΣWj (Eij − Pij). Reliability has the highest gap, followed by Assurance, Responsiveness and Empathy.

OPEN IN VIEWER

Table 3.

Expectation, Perception and Service Quality.

Dimension	Expectation Mean (E)	Perception Mean (P)	Service Quality (E–P)
Reliability	6.52	5.85	0.67
Responsiveness	6.42	5.89	0.53
Assurance	6.34	5.8	0.54
Empathy	6.38	5.86	0.52

Zone of Tolerance

Expected and reasonable averages for various dimensions are shown in Table 4. Averages of perception are acceptable. The tolerances (ZOT) for the dimensions of Responsiveness (0.94) and Reliability (1.08) are narrower than the ZOTs for Assurance (0.96) and Empathy (1.01).

OPEN IN VIEWER

Table 4.

Expectation, Perception and Adequate Average.

Dimension	Expectation Average (E)	Perception Average (P)	Adequate Average	ZOTSQi = ΣWj (Eij − Pij)
Reliability	6.52	5.85	5.54	1.08
Responsiveness	6.38	6.01	5.48	0.94
Assurance	6.42	5.89	5.46	0.96
Empathy	6.38	5.86	5.40	1.01
Average	6.42	5.9	5.47	1

Respondents are less tolerant of Responsiveness and Assurance compared to other aspects, with respect to tolerance gaps. Also, the Responsiveness and Assurance dimensions ranked higher than the other dimensions. This means that respondents place more importance on these two dimensions. The tolerance range was found to be narrow, less than 1.5% of the scale used (7-point Likert scale). The service is acceptable, the customer is satisfied with the service, if the service he desires is above the adequate average level, he is satisfied and if it is below the acceptable level, the customer is not satisfied with the service.

Feedback Analysis

From the user’s point of view, the e-learning system focuses on the service quality of the system rather than the quality of the software product itself, so it is necessary to measure and evaluate the service quality of the e-learning system through Feedback analysis. The purpose of the feedback form is to collect customer satisfaction surveys. The feedback form is available at https://vtputkal.odisha.gov.in/feedback/. The users selected for this study are divided into four different groups (Ben, G. Customer Feedback Analysis: A How-To Guide)

Hierarchical Coding Framework

Figure 6 shows, that hierarchical frameworks help organise code based on their relationships to each other.

OPEN IN VIEWER

Figure 6.

Source: Alyona Medelyan, PhD, Coding Qualitative Data: How to Code Qualitative Research.

Table 4 depicts, that feedback is collected from the direct users of the platform, that is, students and teachers. It is analysed manually with a pool of 250 feedback data, with a Sentiment score of 1 –5. 1 represents Angry/Negative and 5 represents Happy/Positive (Ben, G. Best Practice for Building a Tagging Taxonomy).

OPEN IN VIEWER

Table 5.

Feedback on Different Attributes.

Attribute	Feedback Categories	Sentiment Score
Website accessibility	Performance issue	5	Happy	Positive
Search operation	Performance issue	5
Performance across devices	Responsive design issue	4	Neutral
Content quality	Quality issue	4		Negative
Customer feedback response	Trust issue	3	Angry
Downloads	Performance issue	2

Google Analytics and Service Quality

Google Analytics provides a solid amount of data about the website (https://vtputkal.odisha.gov.in) and offers numerous reports that provide insight into the performance of the website and the quality of services provided.

Site Users’ Data

Google Analytics classifies users into two categories: new and returning users. From Figure 7 it is evident that 75% are new visitors and 25% are returning visitors.

OPEN IN VIEWER

Figure 7.

Source: Google Analytics.

After the early days of hype of the platform, users’ statistics went down sharply and returning visitors reduced to 25%. User negative feedback was reviewed, and it was found larger media files, are one of the major drawbacks of the system. All file sizes were reduced and made available. An awareness campaign was initiated in all colleges and universities to increase student participation.

Device Type Used to Access the E-learning Platform

It was presumed that the platform is to be used by the young population and mobile penetration is very high in this age group compared to other devices. To maintain serviceable quality, Mobile First design and responsive web design were adopted to make the platform accessible to all kinds of devices. Also, ‘WCAG 2.0 “AA” (Web Content Accessibility Guidelines) by W3C (World Wide Web Consortium) (https://www.w3.org/TR/WCAG20/) and GIGW (Govt. of India Guidelines for Websites) (https://guidelines.india.gov.in/)’ is adopted for making the platform accessible for differently abled persons. Concerning Figure 8, it is evident that 87.6% of users of the platform access through Mobile devices.

OPEN IN VIEWER

Figure 8.

Source: Google Analytics.

Active Concurrent User and Quality of Service Delivery

Google Analytics defines a session as ‘a series of user interactions with your website that occur within a specific time frame’. It is a common problem of most web-based applications that the quality deteriorates with the increase of concurrent users reference Figure 9. To mitigate that scenario, session data is analysed and VM server configuration (number of CPU and RAM sizes) is enhanced (Gaur et al. 2016).

OPEN IN VIEWER

Figure 9.

Source: Google Analytics.

Google Analytics Users Acquisition Overview

The Google Analytics Sources metric relates to where the traffic to your website is coming from. The three traffic media in Google Analytics are measured by default at None, Referrer and Organic. Figure 10 shows traffic through social media is the lowest. A page poster was designed and published through all social media to attract users through the social media channel.

OPEN IN VIEWER

Figure 10.

Source: Google Analytics.

Conclusion and Future Scope

E-learning videos published on YouTube face several issues, including copyright concerns, instances of bullying and inappropriate content. To tackle these challenges, the VTP e-learning portal was created, where all video files are hosted on the Government VOD server of the NIC. This initiative fulfils the mandate of delivering quality education to underprivileged rural areas and bridging the digital divide among rural students in Odisha across various disciplines and universities.

To gauge student satisfaction with the e-learning systems, the ZOT SERVQUAL model, in conjunction with Feedback Analysis and Google Analytics, is employed. Reducing the gaps between user expectations and their perceptions can enhance service quality. Regularly reviewing Feedback Analysis and Google Analytics data will consistently support quality improvement efforts. The feedback received regarding the enhancement of the quality of digital study materials underscores the need for DDCE Utkal University to strengthen the engagement process with proficient educators. Based on feedback and Google Analytics data, the platform is accessed from diverse locations and by individuals from various social backgrounds. Currently, the https://vtputkal.odisha.gov.in site delivers video learning content, but shortly, evaluation and assessment modules will be integrated into the system.