Abstract
Massive open online courses are one of the most prominent trends in higher education in recent years. Instructional videos play a significant role in the massive open online courses platform. This study tested the impact of sending danmaku related to video content versus sending danmaku not related to video content versus not sending danmaku in instructional video. We assessed students’ achievement, learning satisfaction, social presence, and cognitive load. Adopting a quasi-experimental design, we collected data from 137 participants. Result revealed that the danmaku related to video content improved social presence, learning satisfaction, and learning achievement but created more cognitive load.
Keywords
Introduction
Massive open online courses (MOOCs) are one of the most prominent trends in higher education in recent years. MOOCs represent open access, global, free, video-based instructional content, videos, problem sets, and forums released through an online platform to high volume participants aiming to take a course or to be educated (Baturay, 2015). Despite their momentum, the dropout rate is still high: Less than 10% of students in MOOCs complete the course, and only about 4.3% receive MOOCs certification (Gütl, Rizzardini, Chang, & Morales, 2014; Xing, Chen, Stein, & Marcinkowski, 2016) . Accumulative research showed that lack of social interaction was an important factor leading to high level of dropout rate . Moreover, many learners believed that social interaction improved learning performance and satisfaction and helped them completing the whole courses (Chen, Gao, & Yuan, 2017).
One emerging technology of such user-generated content is a real-time commentary system called danmaku or bullet screen, which enables users to view and add commentary subtitles on videos. Danmaku became popular on a Japanese platform called Niconico and made its way to China, where it started out as a subculture phenomenon for fans of Japanese anime and games (Doland, 2015). As shown in Figure 1, the most notable feature of danmaku video sharing websites is scrolling marquee comments, which are overlaid directly onto the video and flying on the screen and synchronized to a specific playback time (Shen, Chan, & Hung, 2014). Danmaku has begun to enter the mainstream and capture the attention of teens via cinema, advertising, and social media in China (Doland, 2015). Although danmaku is still not widely used in the online learning environment, it has a lot of potential advantages in facilitating learning performance. For example, the commenting feature of danmaku can be used with both asynchronous and synchronous content, that is, videos that are posted at one point of time on which people can then later comment and live broadcasts with which people can engage in real time. With danmaku, even asynchronous experiences can feel like synchronous ones (Chen, Gao, & Rau, 2015). It is a potential useful tool for online learning, as learners feel that they learn with peers and instructors (Xie & Lei, 2015).
Danmaku.
Why danmaku might improve learners’ performance in video? Social constructivist theory might provide a theoretical explanation. The theory suggested that learning is the process of constructing one’s own knowledge structure within the social interaction (Qiu, 2005). In online learning, danmaku can create a real time between online and off-line social network. In the social network environment, danmaku may allow learners to cocreate learning content with their instructors and peers by adding comments that are helpful explaining the learning materials (Yao, Bort, & Huang, 2017). Therefore, the learners’ behavior affects other learners and instructors, and at the same time, the behavior of peers and instructors is also a part of regulating learners’ behavior; as a result, the influence is two way.
Recent studies on danmaku evidenced that there are two main lines of research, one focusing on media communication of the danmaku and the other focusing on applying danmaku in online learning. Research on danmaku in online learning and the educational function of the danmaku has been excavated by analyzing the danmaku in the aspect of user interaction (Li, 2015). It is solving existing problems of MOOC to apply the advantage of danmaku to the design of the MOOC resources and activities (Li et al., 2016). Similarly, in order to explore danmaku’s potential in online video learning, Yao et al. (2017) used the properties of the danmaku to build a new design for modifying online learning environments. There are few empirical studies about danmaku. Researchers have verified that building network learning community based on the danmaku, to a certain extent, can improve the learning efficiency by analyzing learning achievement of learners who watch the video or video with danmaku (Xie & Lei, 2015). In the recent study, data analysis and text mining of danmaku found that danmaku helped to promote the emotional exchange between instructors and learners and learners and learners, narrow the distance between learners and instructors, enhance their social telepresence, and reduce the loneliness produced in the process of online learning (Zhang, Yang, & An, 2017).
However, most studies focused on whether danmaku was beneficial to learning and paid little attention to the boundary conditions about the effects of danmaku. While viewing the video lectures with danmaku, learners were required not only to process the learning content current presented but also to process the messages sent by peers and instructors. According to Mayer’s cognitive theory of multimedia learning, learners’ capacity of working memory was limited. As a consequence, they could allocate their attention only to a small portion of incoming information at once (Baddeley, 1992) . In fact, the danmaku can be divided into danmaku related to the content and danmaku not related to the content in video lecture. Therefore, when danmaku is not related to the content in video lecture, it distracts learners?
In summary, the existing research has shown that danmaku in an instructional video may be helpful, but it is important to understand how instructional videos can be further improved. This study attempts to explore the effect of danmaku on learning. Specifically, this study compared the effects of the danmaku related to video content or the damaku not on cognitive load, social presence, learning satisfaction, and learning achievement. Based on the aforementioned literature review, the following hypotheses were formulated:
Method
Participants
This study used a convenience sample of 58 undergraduates (56 females and 2 males) from a general course class named “application of modern educational technology” from a Chinese university (18- to 22-years-old; M = 20.16, SD = .702). None of the participants majored educational technology or had learned the content of the instructional video. Each participant provided informed consent before the experiment and received ¥10 for their participation. The study protocol was approved by the Academic Committee of the School of Psychology at Central China Normal University.
Instructional Materials
Instructional materials included online learning teaching video titled the basic concepts of education technology. The main content of the teaching video covers the definition of education, the understanding of the technology, the development of education technology stage, Instructional Technology 94' definition, and Instructional Technology 05' definition. The content of the video references book of Instructional Technology (He, 2009) . The duration of the video is 8 minutes and 2 seconds.
Instruments
Prior knowledge test
Prior knowledge questionnaire was designed, with reference to Instructional Technology (He, 2009), to test participants’ prior basic knowledge in Instructional Technology. The test included seven multiple-choice questions. For example, the general scientific theoretical basis of education technology includes (a) system theory, (b) communication theory, (c) learning theory, and (d) teaching theory. Every question worth 1 point, and wrong or unanswered questions scored as 0. On tests, there was a significant difference between the top 27% and the bottom 27% in the order from large to small, t(73) = 14.043, p < .05, Meaning Difference (MD) = 2.317. Therefore, the prior knowledge test has good discrimination. Higher scores indicated more prior knowledge.
Cognitive load questionnaire
This questionnaire was the Cognitive Load Self-Rating scale, internal consistency coefficient of which was .74 (Paas & van Merriënboer, 1993). The scale included two dimensions: mental effort and intuitive perception of difficulty of material. Nine-point Likert-type scale questions (1 = very, very low mental effort and very, very easy, 9 = very, very high mental effort and very, very hard) assessed the mental effort involved in learning and the degree of task difficulty. Higher scores indicated higher cognitive load.
Social presence questionnaire
We adopted Chinese version of social presence questionnaire revised by Yang (2014), which contains four dimensions: teacher teaching, classroom content, teacher–student interaction, and learning environment and equipment. This questionnaire was based on the Social Presence Self-Rating scale (Kim & Biocca, 1997), which has been widely used in the measurement of social presence in online learning (Homer, Plass, & Blake, 2008) with good reliability and validity (Coyle & Thorson, 2001) (internal consistency coefficient: 0.91). The questionnaire had eight 7-point Likert-type scale questions (1 = absolutely disagree, 7 = absolutely agree), with three reverse scored. Higher scores indicated higher social presence.
Learning satisfaction questionnaire
This questionnaire was based on the Video Course Learning Satisfaction Questionnaire (Wang, 2013), which has had good reliability and construct validity (internal consistency coefficient was .923). The original 20-item scale was modified such that items that did not fit the context of this study were deleted (e.g., items about teacher-student interaction). The modified scale had 5-point Likert-type scale items (1 = absolutely disagree, 5 = absolutely agree) on four dimensions: teacher teaching had two items, classroom content had one item, and learning environment and equipment had two items. Higher scores indicated higher learning satisfaction.
Learning achievement test
This test was designed, with reference to Ke-Kang He’s “Instructional Technology” (He, 2009), to test participants’ learning achievement of memory and understanding of video content. The test included five multiple-choice questions, For example, education technical field evaluation includes the following content: (a) problem analysis, (b) standard reference measurement, (c) formative assessment, and (d) summative evaluation. Every question worth 1 point, and wrong or unanswered questions scored as 0. On tests, there was a significant difference between the top 27% and the bottom 27% in the order from large to small, t(73) = 19.801, p < .05, MD = 3.077. Therefore, the learning achievement test has good discrimination. Higher scores indicated higher learning achievement.
Design
This study adopted the between-subject designs, the independent variable was the danmaku with three levels: the experimental group (SDC=instructors send danmaku related to video content, SNDC= instructors send danmaku not related to video content) and the control group (NSD= instructors do not send danmaku). Four dependent variables were measured: cognitive load after learning, social presence, learning satisfaction, and learning achievement.
Procedures
The study was carried out in a multimedia classroom, with 30 participants in the SDC, 28 participants in SNDC, and 122 participants in the NSD. Finally, eliminating participants who close the danmaku and data missed, we collected data from 79 participants in the NSD.
When the participants entered the classroom, they were taught how to use danmaku and interface of TUCAO (http://www.tucao.one/) video network. Prior to learning, the basic concepts of education technology video, each participant completed the prior knowledge test.
When participants were watching the video in the control group, instructors did not send danmaku and the participants watched video autonomously, within the limiting time. When participants were watching the video in the SDC, instructors sent danmaku related to video content. The point-in-time and content of the danmaku be sent can be seen in Table 1. When participants were watching the video in the SNDC, instructors sent danmaku not related to video content. The point-in-time and content of the danmaku be sent can be seen in Table 2. For the three groups, participants were free to send danmaku or not to send danmaku while watching video. Participants next completed the cognitive load, social presence, and learning satisfaction questionnaires followed by the recognition and transfer tests.
The Point-in-Time and Content of the Danmaku be Sent.
The Point-in-Time and Content of the Danmaku be Sent.
Results
For all variables, we analyzed descriptive statistics and intercorrelations for the learner’s prior knowledge, the cognitive load, social presence, learning satisfaction, and learning achievement in turn.
Prior Knowledge
First, we compared the difference between the SDC, SNDC, and NSD on prior knowledge. There was no significant difference between the three groups, F(2, 134) = 0.110, p > .05. It meant that three groups had similar lever of prior knowledge.
Cognitive Load
To test H1, we analyzed the relevant data. The results showed that there was a significant difference in the impact of the type of danmaku on students' cognitive load, F(2, 134) = 8.279, p < .05. Subsequently, we carried out multiple comparisons. As shown in Figure 1, an Least Significant Difference (LSD) follow-up procedure confirmed that SDC reported significantly more cognitive load (M = 11.033) than SNDC (M = 8.786) and the NSD (M = 9.278). These results confirmed H1.
Social Presence
To test H2, we analyzed the relevant data. The results showed that there was a significant difference in the impact of the type of danmaku on students’ social presence, F(2, 134) = 3.400, p < .05. Subsequently, we carried out multiple comparisons. As shown in Figure 2, an LSD follow-up procedure confirmed that SDC reported significantly more cognitive load (M = 29.80) than SNDC (M = 26.54). There was no significant difference between the SDC and the NSD on students’ social presence (p > .05, MD = 1.800). The SNDC and NSD on students’ social presence was the same (p > .05, MD = 1.464). These results confirmed a part of H2.
The Multiple Comparison on Cognitive load. SDC = ■; SNDC = ■; NSD = ■; MD = ■.
Through the danmaku related to video content, learners can communicate issues and learning content together and feel strong social presence. In Experiment B, the danmaku not related to video content may interfere in learning, resulting in lower social presence.
Learning Satisfaction
Same as the aforementioned method
The Levene’s test for homogeneity of the variance for cognitive load score indicated that there was no evidence of heterogeneity in the sample at p > .05. There was a significant difference in the impact of the type of danmaku on students’ social presence, F(2, 134) = 3.296, p < .05. As shown in Figure 3, an LSD follow-up procedure confirmed that SDC reported significantly more learning satisfaction (M = 3.233) than SNDC (M = 2.679) and NSD (M = 2.785). These results confirmed H3. Compared with the other two groups, learners feel strong sense of learning satisfaction in the Experiment A. There is no doubt that learning satisfaction of the Experiment B is the lowest (Figure 4).
The Multiple Comparison on Social presence. SDC = ■; SNDC = ■; NSD = ■; MD = ■.
The Multiple Comparison on Learning satisfaction. SDC = ■; SNDC = ■; NSD = ■; MD = ■.
Learning Achievement
We use the single variable of the general linear model group as the independent variable, with previous knowledge as the covariate, academic performance as the dependent variable, analyzing learner learning achievement. There was a significant difference in the impact of the type of danmaku on students’ achievement on the recognition test, F(2, 133) = 4.601, p < .05. Main effects of the types of danmaku were significant. As shown in Tables 3 and 4, an LSD follow-up procedure confirmed that SDC reported significantly more learning achievement (M = 2.967) than SNDC (M = 1.964) and NSD (M = 2.443). These results confirmed H4.
The Multiple Comparisons on Learning Achievement.
Note. Data were based on 137 participants.
Descriptive Statistics of Learning Achievement and Prior Knowledge Score.
Note. Data were based on 137 participants.
In summary, these results show that danmaku related to video content facilitates learner learning. Compared with other groups, learners showed higher cognitive load, social presence, learning satisfaction, and learning achievement in SDC. Although learners feel higher cognitive load, we know that cognitive load of moderate overload can improve learners’ learning.
Discussion
The study investigated the effect of content of guiding danmaku on learning. The results confirmed that relative to learners in the SNDC condition, learners in the SDC condition performed better and experienced higher social presence, cognitive load, and learning satisfaction. Besides, compared with learners in the NSD condition, learners in the SDC condition performed better and experienced higher cognitive load and learning satisfaction. Overall, the results of this study confirmed that SDC facilitates learning. In the remainder of this discussion, we discuss relevant results, limitations of this study, future research, and so on.
Why SDC facilitates learning? The aim of SDC is to guide learners to deeper the understanding of learning materials through interaction between each other. The interaction is developed through iterative coconstruction and plays a role in the learning process. (Damşa & Ludvigsen, 2016). It is the meaningful interaction that promotes the learning achievement and improves the learning satisfaction. The interactivity of danmaku is being associated with the learners’ cognitive processes. From SNDC to SDC, interactivity levels or degree of interaction related to video content, starting from the lowest and subsequently moving toward the highest, reflect the amount of mental engagement of learners, and reflect that learners are inducted to be an active learner (Patwardhan & Murthy, 2015). Similarly, through meaningful interaction and iterative coconstruction, danmaku related to video content increases learning achievement and cognitive load for learners.
Using the platform of the danmaku, the instructors sent danmaku related to video content, which guides the learner to focus on the content and sending the danmaku related to video content and to create useful collective intelligence (CI) jointly. CI, also known as universally distributed wisdom, is continuously accumulated and tuned by the participants in online communities at all times (Levy, 1999) . The power of Internet has made CI more easily accessible (Hung, Kinshuk & Chen, 2018). CI is also a useful content source that can extract valuable information to improve learning (Anderson, 2016; Dron & Anderson, 2014; Flanagin & Metzger, 2000; Hung et al., 2018; Tylor, 2015). That is the reason why SDC is better than the other groups in the learning achievement, learning satisfaction, and social presence. However, in cognitive load, the more CI adds related content, the more the learner received information at the same time. That will inevitably cause the learner’s cognitive overload.
Interpretation of the results must be viewed within the limiting condition of this study. The learning achievement measure was essentially a measure of content recognition and did not attempt to measure higher order thinking skills from the danmaku intervention. In addition, for a 7-minute learning process by video, the social presence and cognitive load tests were less useful in identifying meaningful statistics or conclusions. The results of this study should not be generalized outside of the population of undergraduate students in higher education or populations with similar demographics. We did not collect any qualitative data in our data collection procedures and, consequently, connect triangulate our data sources. This video content belongs to declarative knowledge.
In summary, this study provides a reference design strategy for the application of danmaku in declarative knowledge instructional videos. The results suggest that when using danmaku in instructional videos, sending danmaku related to video content by instructors may be better for learning. After all, the danmaku entertainment interactive tool is applied to education. The interactive features of which allow learners and teachers to communicate online. Content of danmaku is the flexibility and nonlimitation and need teachers’ control. Sending danmaku related content by instructors can create atmosphere and contribute collective wisdom to facilitate learning. To ensure that meaningful learning experiences can occur regardless of the nature of the content, future research can examine different types of knowledge. In addition, the future research should focus on the effects of position, velocity, color, and other factors of danmaku on learners.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was financially supported by self-determined research funds of Central China Normal University from the colleges’ basic research and operation of MOE (CCNU15Z02004) and humanities and social science research plan fund project of Ministry of Education (17YJAZH104).