Interactive Computing Technology in Anti-Bullying Education: The Effects of Conversation-Bot’s Role on K-12 Students’ Attitude Change Toward Bullying Problems

Abstract

The aim of this study was to examine the effects of an anti-bullying activity that utilizes conversational virtual agents (called conversation-bots or chatbots) on students’ attitudes toward bullying problems. An experimental pre- or posttest design with a three-group setting was used. Eighty-nine fifth-grade students were assigned to one of three groups: Conversation with a virtual agent of (a) bully’s role, (b) victim’s role, and (c) teacher’s role. All agents are conversation-bots designed to support learner–computer interactions. The bully agent defends the notion that bullying behaviors are acceptable whereas the victim agent argues that bullying behavior cannot be tolerated. The teacher agent teaches students the types of bullying and its negative aspects. The participants completed an anti-bullying attitude test at pre- and posttest, which included students’ anti-bully, intention, pro-victim, behavior, and self-efficacy factors. The results show that students’ attitudes toward bullying problems changed to more positive responses after the implementation that used the conversation-bot. In addition, the results revealed that the agent’s role had an impact on the students’ attitudes toward the anti-bully factor. Implications and future research regarding the use of conversation-bots in education are discussed.

Keywords

bullying school bullying anti-bullying programs conversational methods virtual agent system conversation-bots

School bullying has been recognized as a serious problem that causes negative impacts on students’ school lives. A body of research over the past decades has shown that both bullies and victims are at risk for different types of severe predicaments, such as poor academic achievement (Glew, Fan, Katon, Rivara, & Kernic, 2005), school avoidance (Hutzell & Payne, 2012), social relationship difficulties (Lambe, Hudson, Craig, & Pepler, 2017), mental disorders (Evans-Lacko et al., 2017), and suicidal ideation (Holt et al., 2015). Given the negative effects of bullying, it is imperative to design an effective educational program for preventive purposes.

It has been reported that anti-bullying education diminished the prevalence of bullying, and the reductions have been modest in size (Rigby, 2011). A meta-analysis of the published research shows that anti-bullying programs in schools decreased bullying incidents by 20% to 23% and victimization 17% to 20% (Ttofi & Farrington, 2011). Still, there are mixed results in previous studies (Swearer, Espelage, Vaillancourt, & Hymel, 2010). Thus, there is room for improvement in conventional anti-bullying approaches.

Along with the traditional methods of anti-bullying programs, which usually include direct instructions and self-report questionnaires, conversational approaches such as role-playing have shown their effectiveness (e.g., Bhukhanwala, 2014). Through having a conversation with bullies and victims of bullying, students would experience their responses to bullying incidents and understand their own attitudes toward bullying.

This article explicates the possibilities and challenges of an anti-bullying program that incorporates different types of bullying issues and pays particular attention to a conversational approach. Specifically, this study focuses on the use of conversational virtual agent systems (called conversation-bots or chatbots) through looking at the effects of agent’s different roles by exploring the changes of students’ attitudes toward bullying problems and their intention to tackle bully-victim issues.

Literature Review

Bullying has been regarded as a serious matter, specifically in K-12 schools. The growing number and the higher level of severity of bullying problems call for educators to take action. Because it is natural for young students to experiment with dominating others, timely interventions are essential before aggressive behaviors evolve into bullying (Frey, Hirschstein, Edstrom, & Snell, 2009). Most anti-bullying programs in school settings adopt the establishment of school policies and classroom rules to reduce school bullying incidents. These types of preventive and reactive school policies and anti-bullying programs have shown to be an effective approach (Stevens, Van Oost, & De Bourdeaudhuij, 2000). However, as our society changes, we notice that new types of bullying emerge and bullying problems become varied (Sokol, Bussey, & Rapee, 2016). Consequently, traditional and comprehensive anti-bullying approaches are not as effective as expected because of the varied bullying problems (Eden, Heiman, & Olenik-Shemesh, 2013; Swearer et al., 2010; Ttofi & Farrington, 2011). Thus, each program requires to be individualized and personalized for each school, each student, and each bullying case.

In Dire Need of Peer Support

To address the diversified bullying problems and accommodate the students’ needs for individualized anti-bullying approaches, the bullying context in school settings should be particularly considered. This is because bullying occurs in a social context involving peers or bystanders as well as bullies and victims of bullying (Trach, Hymel, Waterhouse, & Neale, 2010). The results of school bullying entail victims’ painful distress and damage to social relationships (Nicolaides, Juvonen, & Witkow, 2002). Importantly, a type of victims’ anguish is that there lacks support from their peers and they are isolated from peers, which causes mental disorder such as depression, anxiety, and psychosomatic symptoms (Kaltiala-Heino, Rimpelä, Rantanen, & Rimpelä, 2000). The negative consequences are also applicable to bullies because they are frequently and indirectly excluded from their peers and have mental problems such as antisocial and mental disorder as well (Boulton & Smith, 1994). Thus, the peer group in a social context is one of the key contextual factors when anti-bullying programs are designed.

Although bullies must take full responsibility regarding bullying problems, we cannot ignore the bystander effect. Although a bullying problem involves at least two students (i.e., bully and victim), their peers are also involved in the issue indirectly as a witness of the bullying, which makes them be afraid of being also bullied if they interfere (Campbell, 2005). Bystanders who are aware of bullying but are not helping a victim of bullying can even encourage and prolong bullying (Swearer et al., 2010). Unfortunately, students’ reluctance to help victims of bullying and fear of bullying is prevalent in school settings (Unnever & Cornell, 2004). Thus, in the context of bullying preventions, it is of interest to focus on the change of peers’ attitudes toward bullying problems.

When designing an anti-bullying activity for the purpose of attitude change, the target students’ age should be considered. Trach et al. (2010) conducted a study on students’ peer support behaviors in bullying problems. From 28 elementary schools and 10 secondary schools in Western Canada, more than 9,000 students completed a survey. The results show that younger students were more likely to tell bullies to stop or talk to an adult than older students. Most importantly, the researchers pointed out that the move from elementary to secondary school is an important period of students’ attitude change (Trach et al., 2010). Thus, anti-bullying programs that focus on students’ attitude change are required especially for young children. Specifically, peer norms should be established and peer support systems should be built by improving social-emotional and behavior skills (Frey et al., 2009).

Conversational Programs

Most of the conventional anti-bullying programs consist of direct instruction, workshops, anonymous self-report surveys, questionnaires, or interviews to monitor and evaluate school bullying issues (Rigby, 2011). Along with instructional methods, indirect ways such as conversations through role-playing have shown its effectiveness as an anti-bullying program. Gillespie, Brown, Grubb, Shay, and Montoya (2015) report qualitative findings from a role-play interaction addressing bullying behaviors at two colleges. Sixty-five nursing students participated in role-play conversations during two semesters. Through focus group interviews after the implementation, the researchers identified that participants were able to successfully understand the emotions of being bullied and the context of bullying. Bhukhanwala (2014) conducted a qualitative exploratory study on the effect of conversations on the change of students’ attitudes toward bullying problems. A program was implemented as an after-school program in a middle school. Thirteen students participated in this program and met once a week for 90 minutes, 10 sessions over a semester. Participants had a conversation and enacted bullying situations in a role-play format. After the initial role-play, other students (i.e., audience or spectators) were also invited to be an actor, which called a spect-actor. The results show that the participants were able to report their bullying-related experiences by identifying the issues, taking ownership of it, and raising their awareness of their own and others’ roles of bullies, victims, and bystanders. The participants were able to clearly indicate the negative consequences of bullying including physical and psychological effects through the interaction with the bully, victim, and bystander roles and understanding others’ feelings through the conversation with the other roles. The participants reported that they were able to understand how bullying occurred and, specifically, how it escalated in the presence of bystanders (Bhukhanwala, 2014). These studies show a potential for conversational or interactional methods for the purpose of students’ attitude change toward bullying problems. Through interactions with someone whose role is a bully, victim, or bystander, students can experience others’ responses to bullying and understand their own stance and position, which would ultimately support them to recognize the negative consequences of bullying. It is expected that this recognition leads young students to their attitude change toward bullying problems in a positive way.

Whom do students need to talk with?

As conversational methods showed its effectiveness, an upcoming research agenda could be about conversation partners. Is the conversation itself effective as an anti-bullying program, or do conversation partners have an impact on the effectiveness? It seems that students’ conversation with their peers rather than with teachers or adults could be beneficial. Through the survey of eighth graders, Harris and Petrie (2002) reported that 47% of the participants did not tell anyone when they were bullied. Among the students who told, 21.5% reported that they tell a friend, 12.5% their mother, and only 2.5% their father or teacher (Harris & Petrie, 2002). The results of students’ attitudes show that their peer group is the best to talk about bullying problems.

Direct interactions and conversations with actual bullies and victims might be effective in changing students’ attitudes toward bullying (Rigby, 2011). However, our knowledge about how to conduct successful prevention has been limited due to practical difficulties. Obviously, to implement a conversation activity, a school must reveal bullies and victims of bullying for real interaction. An indirect way of interaction and conversation in a role-play setting also requires time-intensive preparation, such as scripts, coaches, practice, stage, and so on (Bhukhanwala, 2014). Hence, the conversational method has been tested in only a handful of studies, and the impact of conversation partners’ roles has not been fully addressed in the educational research field. Therefore, a more convenient and efficient way to utilize the benefits of conversational methods is needed. As Nocentini, Zambuto, and Menesini (2015) suggested in their review research and Limper (2000) utilized a computer program to ask students to anonymously report their bullying issues in the classroom, which significantly reduced the preparation workload and provided affordances, a conversational strategy that utilizes technology deserves investigation.

Conversation-Bots

In the human–computer interaction research field, a type of conversational software, called conversation-bots or chatbots, has been utilized for user-system interactions. A conversation-bot conducts an interaction task through conversation with its users by simulating human’s dialog patterns and behavior. In the education field, conversation-bot systems have been adopted as a form of educational or pedagogical agent or intelligent agent system, which refers to “a computing system designed to realize a set of goals or tasks while inhabiting, sensing and acting autonomously in a complex dynamic environment on behalf of a person or organization” (Xu & Wang, 2006, p. 828). Educational agent systems are educational software with human characteristics or appearances for the purpose of supporting the learner (Krämer & Bente, 2010). Initially, the agent systems delivered instructional contents, and currently, the systems have been evolved to interact with learners through conversation. There have been research attempts to utilize conversational agent systems in education, such as literacy tutoring (Cole et al., 2003), formative assessment (Graesser & McDaniel, 2007), and self-reflection (Song, Rice, & Oh, 2019). A few studies were conducted to examine the use of conversational agent systems for the purpose of students’ attitude changes, such as general attitudes toward mathematics (Arroyo, Woolf, Cooper, Burleson, & Muldner, 2011; Kim & Wei, 2011). However, the use of conversational systems in anti-bullying education has been underinvestigated. Given the feasibility of conversation that agent systems can provide, it is expected that anti-bullying programs would be able to benefit from conversation-bot systems to utilize the conversational approach for bullying prevention if appropriately adopted.

This Study

The purpose of this study is to explore changes in students’ attitudes toward bullying problems by examining the effect of using a conversation-bot. Specifically, this study pays attention to the agent’s role when K-12 students have a conversation with a virtual bully, victim, or teacher agent. Research questions are as follows:

Research Question 1. How do students’ attitudes towards bullying problems change when they have a conversation with a virtual agent? Research Question 2. How are the changes in students’ attitudes towards bullying problems different from each group when they have a conversation with a virtual bully, victim, or teacher agent?

Methods

An experimental pretest and posttest design including three groups was used to evaluate the anti-bullying program.

Participants and Procedure

Ninety-two students initially participated in this study. They are fifth graders at an elementary school in an urban city in South Korea, where bullying has been recognized as a problem at the individual level, and it has not been fully adopted as a school district’s critical agenda. The reason for selecting this age group is that normative beliefs about aggressive behaviors would stabilize around this age-group (Huesmann & Guerra, 1997). Participants were assigned to one of three groups: Conversation with a virtual agent of (a) bully’s role (Bully Group), (b) victim’s role (Victim Group), and (c) teacher’s role (Teacher Group). The researchers did not inform the participants of their group or condition. Because three students (two from Bully Group and one from Victim Group) could not complete the posttest, they were excluded from the analysis. Finally, 89 students participated in this study (28 Bully Group, 30 Victim Group, and 31 Teacher Group students).

Anti-bullying program

An anti-bullying program was designed to change students’ attitudes toward bullying problems and their normative beliefs that support anti-bullying through having a conversation with the virtual agent, and ultimately, to encourage peer involvement to reduce bullying problems. The program consists of five modules: physical, verbal, social, cyber, and money bullying. As shown in Figure 1, each module starts by chatting with a conversation-bot. The agent asks a student to watch animation videos about bullying cases (see Figure 2), which are embedded within the conversation system (i.e., 5–6 minutes per video). The animation videos created by the research team of this study show hypothetical bullying scenarios with a relevant bullying issue to each module, specifically with one day of school life regarding a bullying incident.

Figure 1.

Students are having a conversation with the conversation-bot.

Figure 2.

Screenshots of the animation videos: (a) social bullying and (b) cyberbullying.

As shown in Figure 3, an existing chatbot development framework was adopted in this bullying prevention program. The framework was initially designed to support online students’ conversation with a computer agent (Song, Oh, & Rice, 2017). The authors of this study adopted and modified this development framework for the implementation. Depending on their group, participants had a conversation with the agent whose role was the bully or victim of the animation videos, or a counseling teacher. In the conversation, attention was particularly given to the role of the agent. The bully agent plays in reinforcing pro-bully behaviors, the victim agent pro-victim attitudes, and the teacher agent anti-bullying intentions. Specifically, the bully agent defends the notion that bullying behaviors are acceptable, whereas the victim agent argues that bullying behaviors will not be tolerated. The teacher agent teaches different types of bullying and the negative effects of each bullying type.

Figure 3.

Screenshots of the conversation-bot program: (a) bully agent and (b) victim agent. (c) Translation of bully agent and victim agent.

The participants also completed a self-report questionnaire at pretest and posttest via an online tool. Anonymity was guaranteed, and participants were assured that the test and conversation data would be used for research purposes only and not by the school. After the implementation, five students from each group were interviewed by the researchers of this study. From the pretest and five modules of implementation (i.e., two modules per week) to the final interview took 3 weeks.

Data Collection and Analysis

Pretest and posttest

An anti-bullying attitude test was adopted in this study. Through the literature review, repetitive reliability tests, and factor analyses, Stevens et al. (2000) developed a list of 25 questions to measure students’ attitudes toward bullying and their intention to the involvement in solving bullying problems. Of the 25 questions, Items 1 to 9 reflect students’ pro-bully factor (e.g., “It is fun to sit by and watch bullying”), 10 to 12 intention factor (e.g., “I have an intention to react against bullying”), 13 to 19 pro-victim factor (e.g., “I'm upset when another student is being bullied”), 20 to 22 behavior factor (e.g., “I will support victims of bullying”), and 23 to 25 self-efficacy factor (e.g., “I have self-efficacy to seek teacher's help”; Stevens et al., 2000). The test has the 5-point Likert type alternatives (1 = Totally Disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, and 5 = Totally Agree). Because the pro-bully items measure students’ justification of students who bullied others, in this study, the result scores of the pro-bully factor were reversed (e.g., 5 to 1, 4 to 2, and vice versa) and renamed “anti-bully factor” in order for the factor to indicate the anti-bullying measurement. A reliability test was conducted for the test items from pretest results. Cronbach’s alpha values are .825 (anti-bully), .895 (intervention), .895 (pro-victim), .948 (behavior), .943 (self-efficacy), and .941 (total), which indicate a high level of internal consistency for the scale with the participants.

Student interview and conversation data

After the implementation, five students from each group were interviewed by the researchers. The interviewees were selected based on participants’ voluntary intention for the interview. Interview questions asked students’ experiences of this implementation focusing on the agent’s role, students’ emotions during the conversation and feeling about the bullying problems that were addressed in the implementation program. The audio-recorded interview data were transcribed by the researchers. In addition, students’ conversation data were collected through the conversation-bot system. The system collected each student’s session identification number, timestamp, and dialog text. The student interview and conversation data were qualitatively analyzed.

Following thematic analysis (Braun & Clarke, 2006), each data were coded to determine major themes and categories that would emerge from the data through a process of reading and re-reading the data by the researchers. Meaningful statements regarding student experiences were initially highlighted in the interview and conversation data. The coding scheme was composed of text segments, such as the bully, the victim of bullying, negative aspects of bullying, the participants’ intention to help victims, and so on. Then, codes were grouped into subcategories (e.g., anti-bully and pro-victim aspects). Similar subcategories were integrated by comparing and contrasting the properties of each subcategory, and constantly refining and collapsing the subcategories as stronger themes emerged. The researchers summarized similar ideas in statements to present common themes and insights from the implementation experience. The common issues were addressed using the analyzed themes. After the analysis, the themes and examples were translated from the Korean language to English.

Results

Research Question 1. How Do Students’ Attitudes Toward Bullying Problems Change When They Have a Conversation With a Virtual Agent?

A paired sample t test was conducted in each group to test the difference between the pretest and posttest scores. As shown in Table 1, using a nominal alpha value of .05, a significant mean difference between pretest and posttest was found for the elementary school students on the total score (t = − 2.568, p = .012), specifically, the pro-victim factor (t = −2.958, p = .004). For Bully Group, there were significant differences between pretest and posttest in the total score (t = −3.530, p = .002), specifically, the anti-bully factor (t = −2.091, p = .046) and pro-victim factor (t = −.3.689, p = .001). For Victim Group, there were significant differences between pretest and posttest in the total score (t = −2.146, p = .040), specifically, the self-efficacy factor (t = −3.039, p = .005) and behavior factor (t = −.2.811, p = .009). For Teacher Group, there was no significant difference between pretest and posttest in the total score (t = −.249, p = .805).

Table 1.

Mean Scores and Standard Deviations of the Attitude Test and t Test Results.

	Bully group Mean (SD) (n = 28)	Victim group Mean (SD) (n = 30)	Teacher group Mean (SD) (n = 31)	Total Mean (SD) (n = 89)
Anti-bully factor
Pre	4.07 (.64)	4.29 (.58)	4.43 (.83)	4.27 (.70)
Post	4.62 (.45)	4.67 (.31)	4.57 (.76)	4.62 (.54)
t value (p)	−2.091 (.046)	.269 (.790)	.768 (.449)	−.582 (.562)
Pro-victim factor
Pre	3.85 (1.13)	4.14 (1.04)	4.48 (.69)	4.17 (.99)
Post	4.24 (.93)	4.35 (.96)	4.51 (.81)	4.37 (.89)
t value (p)	−3.689 (.001)	−1.961 (.060)	−.313 (.756)	−2.958 (.004)
Self-efficacy factor
Pre	3.87 (1.19)	4.21 (1.10)	4.56 (.55)	4.22 (1.01)
Post	3.95 (1.17)	4.50 (.89)	4.45 (.87)	4.31 (1.00)
t value (p)	−1.921 (.065)	−3.039 (.005)	.428 (.672)	−1.735 (.086)
Intention factor
Pre	3.64 (1.24)	4.29 (1.07)	4.37 (.93)	4.11 (1.12)
Post	4.06 (1.10)	4.36 (1.01)	4.37 (.98)	4.27 (1.03)
t value (p)	−.522 (.606)	−.763 (.452)	.278 (.783)	−.526 (.600)
Behavior factor
Pre	3.92 (1.25)	4.34 (.97)	4.62 (.58)	4.31 (.99)
Post	4.15 (1.18)	4.57 (.81)	4.55 (.87)	4.43 (.97)
t-value (p)	−1.918 (.066)	−2.811 (.009)	.194 (.847)	−1.579 (.118)
Total
Pre	3.91 (.78)	4.25 (.66)	4.47 (.56)	4.22 (.70)
Post	4.31 (.70)	4.51 (.58)	4.51 (.60)	4.45 (.63)
t-value (p)	−3.530 (.002)	−2.146 (.040)	.249 (.805)	−2.568 (.012)

Research Question 2. How Are the Changes in Students’ Attitudes Toward Bullying Problems Different From Each Group When They Have a Conversation With a Virtual Bully, Victim, or Teacher Agent?

An analysis of covariance (ANCOVA) was performed on the posttest results, in which the pretest was the covariant (see Tables 2 and 3). The posttest results were the dependent variable, and the three-group setting was the control variable to examine the relationship between the posttest results of the three groups.

Table 2.

The ANCOVA Result of the Attitude Posttest.

Variable	Group	N	Mean (SD)	Adjusted mean (SE)	F(2, 85)
Total	(1) Bully	28	4.31 (.70)	4.55 (.07)	3.023*
	(2) Victim	30	4.51 (.58)	4.49 (.07)
	(3) Teacher	31	4.51 (.60)	4.32 (.07)

*p = .054, partial η²= .066.

Table 3.

The ANCOVA Result of the Anti-Bully Category of the Attitude Posttest.

Variable	Group	N	Mean (SD)	Adjusted mean (SE)	F(2, 85)	Post hoc^a
Anti-bully	(1) Bully	28	4.62 (.45)	4.73 (.07)	3.303*	(1) > (3)
	(2) Victim	30	4.67 (.31)	4.66 (.07)
	(3) Teacher	31	4.57 (.76)	4.48 (.07)

^aAdjustment for multiple comparisons: Bonferroni.

*p = .042, partial η²= .072.

Before analyzing the total attitude result, a Levene’s test of equality of error variances test was conducted (F = .272, p = .762), which shows that the variability in this condition is about the same. As shown in Table 2, using a nominal alpha value of .05, the ANCOVA result shows that the variance of the total attitude score between the three groups was not significant (F = 3.023, p = .054) after the impact of the pretest scores on the posttest was excluded. The posttest scores were not significantly different due to the different experimental processes.

Before analyzing the anti-bully result, a Levene’s test of equality of error variances test was conducted (F = .372, p = .690), which shows that the variability in this condition is about the same. As shown in Table 3, using a nominal alpha value of .05, the ANCOVA result shows that the variance of the anti-bully score between the three groups was significant (F = 3.303, p = .042) after the impact of the pretest scores on the posttest was excluded. The posttest scores of the anti-bully factor were significantly different due to the different experimental processes with the small–medium effect size (partial η²= .072). A post hoc analysis was performed to examine specific differences in the posttest between the groups. A Bonferroni test revealed that the Bully Group scores were significantly higher than those of Teacher Group, comparing the adjusted mean of 4.73 for the Bully Group with the Teacher Group score of 4.48 (p = .047).

There were no significant differences between the three groups regarding other factors: Intention F(2, 85) = .323 (p = .725), Pro-victim F(2, 85) = .587 (p = .558), Behavior F(2, 85) = 1.029 (p = .362), and Self-efficacy F(2, 85) = 2.117 (p = .127).

Qualitative Results

Interview results

To investigate the student experience in each group, the transcribed interview data were coded. Fifty-nine codes were grouped into 24 subcategories. From the subcategories, 11 themes emerged. As shown in Table 4, themes found from Bully Group are Bully’s negative personality and Intention to stop bullying. Three themes found from Victim Group are Understanding about the victim, Victim’s negative personality, and Supportive efforts. One theme found from Teacher Group is Educational benefits. Five themes in common are Understanding about bullying, Attitude change, Benefits of 1 on 1 conversation, Usability, and Limitations.

Table 4.

Students’ Interview Analysis.

Group	Theme	Example^a
Bully	Bully’s negative personality	Ga-hye [the bully] is a horrible person. She’s so brazen, so disgusting. She keeps justifying and justifying her behaviors and keeps saying she was just joking.
Bully	Persuasion effort	I had to step up and say “don’t do that again!”,\| but she [the bully] didn’t even listen to what I said! She just changed the conversation topic.
Victim	Understanding about the victim	I hadn’t thought about how they [victims of bullying] feel, their pain, but now I really think they are having a very difficult time.
	Victim’s negative personality	Honestly, he’s [the victim] so dumb and stupid. I watched the animation video. He just gave his money to Ga-hye [the bully]. He should have said, “stop it!” Why is he whining so much now, after giving the money to her? He should have said no at that time.
	Supportive efforts	He [the victim] kept saying that he’d rather die, so I worried about him quite a lot. I told him to stop. It’s very scary, but I tried to cheer him up.
Teacher	Educational benefits	It’s very educational, I mean, I learned about the different types of bullying, and the bad aspects of bullying, and why we should not do this [bullying].
Common	Understanding about bullying	It helped me understand school bullying.
	Attitude change	I think I’m going to help my friends when it [bullying] happens. Also, I'm not going to become those kinds of guys [both bully and victim].
	Benefits of 1 on 1 conversation	The one-on-one chatting activity was great because nobody would hear our conversation, it was just two of us.
	Usability	I’d like to speak because the typing was hard for me. Speech recognition would be great, like Siri [an intelligent personal assistant, part of Apple’s products].
	Limitations	I cannot feel its [agent’s] emotions, it’s just artificial, dry, and blunt a little bit. I can ask my teacher some questions and she answers my questions, but the agent couldn’t answer my questions.

^aTranslated from the Korean language.

Conversation log analysis

In total, 29,934 lines of conversation were collected from the implementation. The students’ dialog was 9,758 lines and the agent 20,176 lines. Due to the vastness of data, student-agent conversations could not be analyzed line by line manually. Instead, to find supportive evidence that might explain the quantitative results, the researchers analyzed students’ responses to the specific key questions that were asked by the agent. The following conversation examples were translated from the Korean language. Overall, most students showed negative attitudes toward the bully except for a few neutral responses. For example, when Bully Agent asked, “Who’s worse, a person who likes jokes or Pi-hye [the victim] who is a terrible whiner?” a student stated, “I think you are 999,999,999 times worse than him [the victim].” In addition, students were able to differentiate bullying behaviors from playful behaviors for fun. A student from Bully Group mentioned, “because he [the victim] didn’t enjoy it and clearly told you to stop it, it’s not a joke anymore, it’s serious bullying.” These conversations might be a piece of evidence to support the quantitative result that shows the change in students’ anti-bully attitudes.

Although there was no significant attitude change of students’ intention to help victims of bullying, the Victim Group students showed somewhat passive intention during the conversation. When the victim agent said, “I’ve been worried a lot. If I told the teacher about this, everybody would make fun of me,” a student typed, “Just tell her [the teacher]. Or, do you want me to tell the teacher about your problem? What do you want me to do?” In addition, when the victim agent expressed his suicidal feelings, a number of students tried to stop him, saying, “Don't ever think like that again!” Although this aspect does not show their self-efficacy or behavior factors, it might reveal their pro-victim attitudes. From the conversation with the teacher agent, the researchers found that most of the conversations were relatively short and disconnected. Most students simply answered teacher agent’s questions with a short sentence, such as, “yes,” “no,” or “I don’t know.”

Discussion

The aim of this study was to explore the effects of an anti-bullying program that utilizes a conversational virtual agent with different roles on attitude change toward bullying problems. Elementary school students who participated in this study were actively engaged in watching animation videos of bullying cases and having a conversation with the virtual agent. Overall, the results show that students’ attitudes toward bullying problems changed to more positive responses after the implementation. This change means the possibility of taking certain action that helps and supports victims of bullying with a disposition to curb bullies and bullying incidents. Although there was no control group, the t test showed that there was a significant difference between the pretest and posttest scores, which means that conversation with the virtual agent was effective to positively change students’ attitudes toward bullying problems. Supplementing this quantitative investigation, the qualitative data of student interview and conversation with the agent partially supported uncovering the possible reasons for this change. The results of this study are in line with other published studies that claimed the effectiveness of students’ engagement in conversation to address bullying problems (Bhukhanwala, 2014; Gillespie et al., 2015).

This study also aimed at comparing the results of the conversation-bot’s different roles among groups. The present findings showed a partial effect of the agent’s different role on the students’ anti-bullying attitude change. In this study, students who had a conversation with the virtual agent of the bully’s role positively changed their attitudes toward anti-bully and pro-victim factors. The Teacher Group students also changed their attitudes toward the pro-victim factor. These factors include negative viewpoints to students who bullied others and sympathy for victims (Stevens et al., 2000), which are related to the way of thinking or feeling about bullying but not to necessarily their behavioral change. The students of Victim Group changed their attitudes toward self-efficacy and behavior factors. Self-efficacy is an assessment of one’s capabilities to attain the desired goal (Bandura, 1977). It seems that having a conversation with the agent of the victim’s role is more associated with behavioral aspects than with the other roles. However, the ANCOVA result showed that the agent’s role only had an impact on the students’ attitudes toward the anti-bully factor, not toward the other factors; that is, Bully Group showed significantly changed their anti-bully attitudes comparing other groups. It can be argued that the role of conversation-bots could be significant in anti-bullying programs, and the bully’s role might be more effective to change students’ attitudes than the other roles. Still, it requires further investigation.

We cannot expect a sustained anti-bullying atmosphere at school without the students’ attitude change. Students’ positive attitudes toward bullying problems are important because it is expected to reduce morbid-bystander effects and prevent students from becoming passive victims. There is a report that students consider bystanders as partly to blame for bullying problems if they do not help the victim of bullying, even enjoy the bullying (Kyriacou, Mylonakou-Keke, & Stephens, 2016). Having bystanders who have positive attitudes toward bullies would seriously undermine the efforts of educators who try to provide a safe learning environment. Although some of the conventional anti-bullying programs deal with the role of bystanders, changes in students’ norm and attitude in relation to the conversation partner’s role have not been fully addressed, and factors that impact bystanders’ choice of reactions have been underexplored. This study suggests a feasible approach to positively change students’ attitudes toward bullying problems.

The attitude change is also crucial to the victims of bullying. As revealed in a study on teachers’ perspectives (Sokol et al., 2016), it has been perceived that confident victims or victims who ignore bullies would be able to handle bullying situations better than sad or angry victims. Because “assertiveness” can empower students to confront bullies (Kyriacou et al., 2016), students’ attitude change is vital to reduce bullying incidents. Since bullying frequently occurs when a teacher is not present and students cannot always count on a teacher, they should be prepared to hinder bullies or bullying. Therefore, schools should promote students to become a supporter of victims by changing students’ attitudes toward bullying problems, and at the same time, to be confident and assertive enough to confront bullies. Anti-bullying programs need to develop a school climate that encourages prosocial behaviors, specifically, preventive programs should focus on empathy building activities that help all students confront bullying problems (Stephens, 2011). As students’ behavioral changes should be dependent upon their attitudes toward a problem, anti-bullying programs should focus on the attitudes and understanding improvement (Stevens et al., 2000). It is expected that the conversational approach that was used in this study might shed light on students’ behavioral changes in a positive way. This study is also relevant to elementary school teachers and administrators as it examines the impact of the conversation-bot system as a tool for engaging elementary school students in talking about their thoughts and experiences with a computer agent.

Anti-bullying programs need to actively adopt technology and new tools for their effectiveness. As reviewed, conventional school programs consist of direct instruction and self-report measures. However, this approach might not be sufficiently sensitive to detect students’ attitude changes in bullying. As Frey et al. (2009) implemented, the use of real-time observation could be an effective way to monitor potential bullying issues and provide us with contextual information about bullies, victims, and bystanders’ behaviors and attitudes, which is the key for the cessation of bullying. As Rigby (2011) reviewed, prevention methods, such as training the victim directly, providing an opportunity for bullies to reflect their aggressive behavior, or inviting a bullying suspect to a meeting where practitioners share a concern for victims’ troubles, could be effective. Nevertheless, real-time observations and invitation or training methods are time- or cost intensive and require tremendous efforts of teachers and administrators. This study suggests a computerized way of implementing an anti-bullying activity using a conversation-bot. It is expected that intelligent computer systems can be conveniently utilized to monitor and positively change students’ attitudes.

Limitations and Future Directions

There are notable limitations found in this study. Although the results of this study show positive outcomes of the implementation, it is far from obvious how to best conduct anti-bullying conversations. Quantitative and qualitative evidence for effective conversation strategies should be investigated further. In this study, student-agent conversation data could not be analyzed due to the vastness. Textual big data would possibly be analyzed if appropriate learning analytics techniques were properly utilized (Song, 2018). Computational techniques to analyze qualitative data should be investigated further. Another limitation of this study is the short period of implementation. The authors recommend examining long-term effects in future studies. Longitudinal studies replicating these findings are also an important area for future research. In addition, more studies are required in determining when and how conversation-bots can be utilized most effectively. Although a chatting-based conversational method has been investigated in a counseling area (Althoff, Clark, & Leskovec, 2016), this area requires further investigation. For example, early detection of bullying incidents can be possible through the conversational method. Last, the prevention program consisted of interaction between students and conversation-bots without the involvement of teachers, parents, and administrators for the efficiency of implementation. Note that previous studies showed that anti-bullying programs are particularly effective when implemented with whole-school supports (Rigby, 2011). Bullying must be recognized as a problem by all stakeholders, so it should be condemned unequivocally; accordingly, schools must adopt anti-bullying programs and interventions; and teachers and counselors must be involved to take actions to bullying problems (Limper, 2000). Further investigation is required to explore possible and effective ways for teacher, parent, counselor, and administrator involvement in this program.

Conclusions

Problems of school bullying, especially in K-12 environments, remain serious. The linkage between research and practice could be the key to successfully reduce bullying problems. Theoretically driven and data-based model of bullying preventions are significant to reduce bullying incidents. As reviewed in this article, anti-bullying programs need to be designed to raise awareness regarding bullying, which would increase student reports of bullying by changing their attitudes toward bullying problems. Concurrently, practical aspects such as affordance, convenience, and user-friendliness should not be ignored when adopting anti-bullying programs. This study shows a feasible approach to an easy-to-use method for anti-bullying in real-classroom settings. This program could be especially effective when a school has not enough infrastructure for developing anti-bullying programs and appropriate counselors because this program only requires a web browser and the Internet connection to be implemented. As revealed in this study, there are indications that the use of conversation-bots might be successfully applied in anti-bullying education in the hope of creating meaningful and sustainable environments that change students’ attitudes.

Footnotes

Author Note

This work was carried out at the Donggwang Elementary School, Jeju, South Korea.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

Althoff

Clark

Leskovec

(2016) Large-scale analysis of counseling conversations: An application of natural language processing to mental health. Transactions of the Association for Computational Linguistics 4(1): 463–476.

Arroyo, I., Woolf, B. P., Cooper, D. G., Burleson, W., & Muldner, K. (2011, July). The impact of animated pedagogical agents on girls’ and boys’ emotions, attitudes, behaviors and learning. Proceedings of the 2011 IEEE 11th International Conference on Advanced Learning Technologies (pp. 506–510). Athens, GA: IEEE.

Bandura

(1977) Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review 84(2): 191–215.

Bhukhanwala

(2014) Theater of the oppressed in an after-school program: Middle school students’ perspectives on bullying and prevention. Middle School Journal 46(1): 3–12.

Boulton

M. J.

Smith

P. K.

(1994) Bully/victim problems in middle-school children: Stability, self-perceived competence, peer perceptions and peer acceptance. British Journal of Developmental Psychology 12(3): 315–329.

Braun

Clarke

(2006) Using thematic analysis in psychology. Qualitative Research in Psychology 3(2): 77–101.

Campbell

M. A.

(2005) Cyber bullying: An old problem in a new guise? Australian Journal of Guidance and Counselling 15(1): 68–76.

Cole

van Vuuren

Pellom

Hacioglu

Movellan

(2003) Perceptive animated interfaces: First steps toward a new paradigm for human computer interaction. Proceedings of the IEEE 91(9): 1391–1405.

Eden

Heiman

Olenik-Shemesh

(2013) Teachers’ perceptions, beliefs and concerns about cyberbullying. British Journal of Educational Technology 44(6): 1036–1052.

10.

Evans-Lacko

Takizawa

Brimblecombe

King

Knapp

Maughan

Arseneault

(2017) Childhood bullying victimization is associated with use of mental health services over five decades: A longitudinal nationally representative cohort study. Psychological Medicine 47(1): 127–135.

11.

Frey

K. S.

Hirschstein

M. K.

Edstrom

L. V.

Snell

J. L.

(2009) Observed reductions in school bullying, nonbullying aggression and destructive bystander behavior: A longitudinal evaluation. Journal of Educational Psychology 101(2): 466–481.

12.

Gillespie

G. L.

Brown

Grubb

Shay

Montoya

(2015) Qualitative evaluation of a role play bullying simulation. Journal of Nursing Education and Practice 5(7): 73–80.

13.

Glew

G. M.

Fan

M. Y.

Katon

Rivara

F. P.

Kernic

M. A.

(2005) Bullying, psychosocial adjustment, and academic performance in elementary school. Archives of Pediatrics & Adolescent Medicine 159(11): 1026–1031.

14.

Graesser

A. C.

McDaniel

(2007) Conversation agents can provide formative assessment, constructive learning, and adaptive instruction. In: Dwyer

C. A.

(ed) The future of assessment: Shaping teaching and learning, Mahwah, NJ: Erlbaum, pp. 85–112.

15.

Harris

Petrie

(2002) A study of bullying in the middle school. NASSP Bulletin 86(633): 42–53.

16.

Holt

M. K.

Vivolo-Kantor

A. M.

Polanin

J. R.

Holland

K. M.

DeGue

Matjasko

J. L.

Reid

(2015) Bullying and suicidal ideation and behaviors: A meta-analysis. Pediatrics 135(2): 1–14.

17.

Huesmann

L. R.

Guerra

N. G.

(1997) Children’s normative beliefs about aggression and aggressive behavior. Journal of Personality and Social Psychology 72(2): 408–419.

18.

Hutzell

K. L.

Payne

A. A.

(2012) The impact of bullying victimization on school avoidance. Youth Violence and Juvenile Justice 10(4): 370–385.

19.

Kaltiala-Heino

Rimpelä

Rantanen

Rimpelä

(2000) Bullying at school—An indicator of adolescents at risk for mental disorders. Journal of Adolescence 23(6): 661–674.

20.

Kim

Wei

(2011) The impact of learner attributes and learner choice in an agent-based environment. Computers & Education 56(2): 505–514.

21.

Krämer

N. C.

Bente

(2010) Personalizing e-learning. The social effects of pedagogical agents. Educational Psychology Review 22(1): 71–87.

22.

Kyriacou

Mylonakou-Keke

Stephens

(2016) Social pedagogy and bullying in schools: The views of university students in England, Greece and Norway. British Educational Research Journal 42(4): 631–645.

23.

Lambe

L. J.

Hudson

C. C.

Craig

W. M.

Pepler

D. J.

(2017) Does defending come with a cost? Examining the psychosocial correlates of defending behaviour among bystanders of bullying in a Canadian sample. Child Abuse & Neglect 65(1): 112–123.

24.

Limper

(2000) Cooperation between parents, teachers, and school boards to prevent bullying in education: An overview of work done in the Netherlands. Aggressive Behavior 26(1): 125–134.

25.

Nicolaides

Toda

Smith

P. K.

(2002) Knowledge and attitudes about school bullying in trainee teachers. British Journal of Educational Psychology 72(1): 105–118.

26.

Nocentini

Zambuto

Menesini

(2015) Anti-bullying programs and Information and Communication Technologies (ICTs): A systematic review. Aggression and Violent Behavior 23(1): 52–60.

27.

Rigby

(2011) What can schools do about cases of bullying? Pastoral Care in Education 29(4): 273–285.

28.

Sokol

Bussey

Rapee

R. M.

(2016) Teachers’ perspectives on effective responses to overt bullying. British Educational Research Journal 42(5): 851–870.

29.

Song

(2018) Learning analytics as an educational research approach. International Journal of Multiple Research Approaches 10(1): 102–111.

30.

Song, D., Oh, E., & Rice, M. (2017, July). Interacting with a conversational agent system for educational purposes in online courses. Proceedings of the 10th International Conference on Human System Interaction (HSI) (pp. 78–82). Ulsan, South Korea: IEEE.

31.

Song, D., Rice, M., & Oh, E. (2019). Participation in online courses and interaction with a virtual agent. The International Review of Research in Open and Distributed Learning, 20(1), 43–62.

32.

Stephens, P. (2011). Preventing and confronting school bullying: A comparative study of two national programmes in Norway. British Educational Research Journal, 37(3), 381–404.

33.

Stevens

Van Oost

De Bourdeaudhuij

(2000) The effects of an anti-bullying intervention programme on peers’ attitudes and behaviour. Journal of Adolescence 23(1): 21–34.

34.

Swearer

S. M.

Espelage

D. L.

Vaillancourt

Hymel

(2010) What can be done about school bullying? Linking research to educational practice. Educational Researcher 39(1): 38–47.

35.

Trach

Hymel

Waterhouse

Neale

(2010) Bystander responses to school bullying: A cross-sectional investigation of grade and sex differences. Canadian Journal of School Psychology 25(1): 114–130.

36.

Ttofi

M. M.

Farrington

D. P.

(2011) Effectiveness of school-based programs to reduce bullying: A systematic and meta-analytic review. Journal of Experimental Criminology 7(1): 27–56.

37.

Unnever

J. D.

Cornell

D. G.

(2004) Middle school victims of bullying: Who reports being bullied? Aggressive Behavior 30(5): 373–388.

38.

Wang

(2006) Intelligent agent supported personalization for virtual learning environments. Decision Support Systems 42(2): 825–843.