Evidence Based Bullying Prevention in Turkey: Implementation of the ViSC Social Competence Program

Abstract

The ViSC program is the first evidence based anti-bullying program implemented in Turkey. A quasi-experimental longitudinal control group design comprising six schools and 26 classes was realized to examine the program effectiveness regarding different forms of perpetration and victimization. The effectiveness of two dosages of interventions – school and class level vs. school level – was investigated. In total, 642 grade 5 students (227 in school and class level, 201 in school level, 214 in control group, 49% girls, M_age = 10.06) participated. Multilevel growth models revealed baseline effects indicating that the two intervention groups had lower levels of perpetration and victimization at pre-test than the controls. Perpetration and victimization increased in the two intervention groups compared to control group between pre- and post-test, but also decreased between post-test and follow-up indicating a sensitizing effect of the program. Results are important for both intervention research and social policy in Turkey.

Keywords

Anti-bullying program bullying victimization aggressive behavior intervention prevention

Bullying is a form of aggressive behavior performed repeatedly and over time by one or more students against another student who cannot easily defend him- or herself (Olweus, 1993). Country-comparative studies consistently show that bullying is a prevalent problem in schools all over the world (Currie et al., 2008, 2012). As a result, several European countries started to tackle school bullying by large scale implementing whole school prevention programs (e.g., Norway, see Roland, 2011; or Finland, see Kärnä et al., 2011). In Turkey, 12 to 42% of children and adolescents are involved in bullying as a bully, victim, or both (Burnukara & Ucanok, 2012; Dogan-Ates & Yagmurlu, 2010; Erdur-Baker, 2009; Soydas & Ucanok, 2014). Therefore, there is an urgent need to design, implement, and evaluate preventive intervention programs in Turkey. To promote the evidence-based practice movement in Turkey (Spiel & Strohmeier, 2012), such programs should ideally fulfill the standards of evidence, that is they should be efficacious, effective, and ready for dissemination (Flay et al., 2005).

The ViSC Social Competence program was the first evidence based whole-school anti-bullying program that was adapted, implemented, and evaluated in Turkish schools because this was feasible given the existing personal and economic resources. The ViSC Social Competence program was originally developed and large scale implemented in Austria (Spiel & Strohmeier, 2011); later the program was also implemented and evaluated in Cyprus and Romania (Solomontos-Kountouri et al., 2016; Trip et al., 2015). Because there is an increased awareness regarding the challenges of the transferability of existing evidence-based programs to other national contexts (e.g. Malti, Noam, Beelmann & Sommer, 2016; Nocentini & Menesini, 2016), cross national dissemination studies of existing evidence based anti-bullying programs are important.

The current research had two main goals: (1) to adapt and implement the ViSC Social Competence program in Turkey, and (2) to compare the effectiveness of two program dosages (school and class level intervention vs. school intervention).

Bullying Prevention within the Socio-Ecological Framework

From a socio-ecological perspective, aggressive behaviour and bullying are understood as complex systemic problems with mechanisms operating on several interacting levels (Bronfenbrenner, 1979; Swearer & Espelage, 2004). While aggressive behavior comprises intentional harm doing; bullying also includes repetition and imbalance of power (Olweus, 1993; Roland & Idsøe, 2001). Both bullying and aggressive behaviour are expressed directly and indirectly and include a variety of negative forms, like physical attacks, verbal insults, relational harassments (Olweus, 1993) or offenses via electronic means (Smith et al., 2008). It is important to differentiate these different forms of bullying and aggressive behaviors as maladjustment differs for direct and indirect aggressive behaviour (Card, Stucky, Sawalani, & Little, 2008), for bullies and victims (Stefanek, Strohmeier, & Yanagida, 2017); and for traditional bullying and cyber bullying (Gradinger, Strohmeier, & Spiel, 2009; Solomontos-Kountouri, Tsagkaridis, Gradinger, & Strohmeier, 2017). Research also demonstrated that age is an important variable to consider. International comparison studies show that the prevalence rates of bullying are highest among 13 year olds compared to 11 or 15 year olds (Currie et al., 2008, 2012). This age pattern has been explained by socio-cognitive developmental changes taking place in early adolescence (Malti, Strohmeier, & Killen, 2015) and by group dynamics going on after youth experienced the transition between primary to secondary school (Pellegrini & Bartini, 2000; Salmivalli, 2010). It was also shown that the same whole school anti-bullying program is more effective in late childhood compared to adolescence (Kärnä et al., 2011) indicating a developmental pattern that has not been fully understood yet (Yeager, Fong, Lee, & Espelage, 2015).

The ViSC Social Competence Program

The ViSC program is a socioecological anti-bullying program; therefore, it consists of measures on the school, class, and individual level (Strohmeier, Hoffmann, Schiller, Stefanek, & Spiel, 2012). The ViSC program is designed for secondary schools because prevalence rates of bullying and victimization are related with school transitions (Pellegrini & Long, 2002) and peak during pre-adolescence (Currie et al., 2008, 2012). The program defines the prevention of aggressive behavior and victimization as a whole school task and the initial implementation of the program lasts one school year. Because the main goal is to enable a school transformation process, the program not only aims to change the behavior of the students (e.g., individual and class level), but also aims to foster knowledge and competences among teachers to be able to initiate several change processes on the school level. During program implementation, several routines on the school level (e.g., the handling of acute bullying cases) are targeted. The program is implemented via several in-school-teacher trainings and a class project for students (Strohmeier et al., 2012).

The main goal on the school level is to commit as many teachers as possible to work “together against violence” (Spiel & Strohmeier, 2011). Thus, on school level teachers are the main change agents. Therefore, the program aims to foster the shared responsibility between them which in turn implies that as many teachers as possible in the school have worked out a common understanding of aggressive behavior and bullying, agreed on procedures of how to tackle acute cases, and jointly implement preventive measures (Schultes, Stefanek, van de Schoot, Strohmeier, & Spiel, 2014). On school level a series of in-school trainings for teachers are organized by so called ViSC coaches; these are school psychologists and other professionals who attended a training offered by the program developers.

On the class level, a 13-unit class project is implemented by teachers in their classes. The fully manualized class project subsequently trains a broad spectrum of competencies among students considered important for the prevention of aggressive behavior and victimization. To begin with, in many classes there is a rather large group of students who neither feel responsible for what happens around them nor intervene in critical situations (Craig, Pepler, & Atlas, 2000). Therefore, all students are taught (a) to feel responsible when something negative is going on and (b) to react in a way which is likely to improve the situation. Second, in secondary school there is still a group of adolescents who is not able to manage their negative emotions in a non-aggressive way (Salmivalli & Nieminen, 2002). Therefore, all students are trained (a) to recognize their own emotions and the emotions of others and (b) to cope with these emotions in a positive, non-aggressive way. Third, it is necessary to empower students who might get victimized easily to avoid that they are attacked because of their non-assertive behavior (Veenstra et al., 2007). Thus, all students are trained how to best react when getting picked on by others. Besides reacting in an assertive way, students are advised to tell somebody (ideally an adult) if a student is victimized in the class. These competences are trained during the 13 units by applying a large variety of didactic methods, e.g. role plays, small group work, or whole class discussions. All units comprise single work sheets, group activities, interactive games and a summary sheet (for more details regarding the units and measures see Strohmeier et al., 2012).

On the individual level, the program assumes that aggressive behavior has two underlying functions (Card & Little, 2006). Reactive aggression is theoretically grounded in the frustration-aggression model; instrumental aggression has its roots in social cognitive learning theory describing a planned behavior controlled by external rewards (Vitaro, Brendgen, & Barker, 2006). While anger is the central emotion for reactive aggression, instrumental aggression is characterized by positive or neutral emotions. In addition, the program acknowledges that victimized students are heterogeneous (Yang & Salmivalli, 2013). While victims of bullying usually cannot easily defend themselves, reactive aggressive students are also often victims and thus labelled as bully-victims. The knowledge regarding the individual level mechanisms of reactive and instrumental aggressive behavior is important for the teachers who are trained (1) to recognize and differentiate bullies, victims, and bully-victims; and (2) to conduct structured conversations with them. These state-of-the-art conversations are based on the approach suggested by Roland and Vaaland (2006) and they differ for bullies, victims, and bully-victims.

To date, the ViSC program has been implemented in Austria, Cyprus, and Romania. In Austria, the effectiveness of the ViSC program has been demonstrated in various studies and it has been shown that the program is effective in reducing victimization, cyber-victimization, and cyberbullying (Gollwitzer, Eisenbach, Atria, Strohmeier, & Banse, 2006; Gollwitzer, Banse, Eisenbach, & Naumann, 2007; Gradinger, Yanagida, Strohmeier, & Spiel, 2015; 2016; Yanagida, Strohmeier, & Spiel, 2016). In Cyprus, evaluation results revealed that grade 7 students profited more from the program compared to grade 8 students (Solomontos-Kountouri et al., 2016). In Romania, where only the class project was implemented by external research assistants and no teacher trainings were provided, no intervention effects on victimization and bullying were found but changes regarding dysfunctional cognitions and emotions were observed (Trip et al., 2015).

The Current Study

The current study is the first investigation of the effectiveness of the ViSC program in Turkey and it is the first study that investigated the effectiveness of two program dosages (school and class level intervention vs. school level intervention). Exactly these two conditions were compared because they represent the only two theoretically meaningful subtypes of interventions of this particular anti-bullying program when the program is implemented by teachers and not by external research assistants. In both conditions, all teachers participate in a general training to learn what bullying is and how to detect and handle it (e.g., the school level intervention). However, only some teachers participate in an additional training and implement the 13-unit social competence training in their classes (e.g., school and class level intervention). This kind of implementation implies that all students in the intervention schools get in touch with teachers who know what bullying is and how to detect and handle it (e.g., school level intervention), while only some students receive the additional 13-unit social competence training delivered by their class teachers (e.g., school and class level intervention).

The ViSC program was chosen to be implemented in Turkey because it was already successfully implemented in three other countries and the current implementation was feasible given the limited resources in Turkey. The program was implemented in grade 5 classes. This grade level was chosen, because of the school transition that takes place between grade 4 (considered as elementary school) and grade 5 (considered as secondary school) in Turkey. There were two main reasons for this decision. First, the literature consistently shows that the bullying behaviors start at the beginning of middle school and peak towards the end of middle school. Second, one of the main goals of the ViSC program is to prevent bullying behaviors and victimization before the problems escalate. Therefore, the program included the 5th graders as they are the newcomers. The program implementation started three months after the beginning of the school year, when students were already familiar with each other. We differentiated various forms of perpetration and victimization in the present study, because perpetration and victimization include a variety of negative forms, like physical attacks, verbal insults, or relational harassments (Olweus, 1993) as well as offenses via electronic means (Smith et al., 2008). We formulated the following two main research questions:

Hypothesis 1: We expected that the program would be effective in the two intervention groups (school intervention and school and class level intervention) compared to the control group. Program effectiveness is indicated by a steeper decrease or lower increase in the different forms of perpetration and victimization when comparing the two intervention groups to the control group. We also expected differential training effects depending on the form of perpetration and victimization. While we expected that the program is effective regarding bullying and victimization, no training effects on cyberbullying and cyber-victimization were expected. This is because only few ten year olds have their own mobile phones in Turkey (Strohmeier & Dogan, 2012) and the ViSC program does not contain specific measures to tackle cyberbullying and cyber-victimization for students. When comparing physical, verbal, and relational forms of perpetration and victimization, we speculated that the program might be more effective for relational aggression and victimization, as the program might raise awareness for these indirect forms of aggression that are related to the collectivistic culture in Turkey (Hofstede, 2001).

Hypothesis 2: We expected that the school and class level intervention would be more effective than the school level intervention only. This hypothesis was formulated because our implementation model implies that all students in the intervention schools get in touch with teachers who know what bullying is and how to detect and handle it (= the school level intervention), while only some students receive the additional 13 units social competence training delivered by their class teachers (= school and class level intervention). Because the latter group is more directly exposed to the program content, it is reasonable to expect differences between these two conditions.

Method

Implementation Process

The implementation model of the ViSC program in Turkey included several steps. First, permission was obtained from the ethical committee of Ege University and the local branch of the Ministry of Education to implement the program at secondary schools. Second, a consortium including the principal investigator and representatives of the local branch of the Ministry of Education was established. This consortium started to meet around nine months before the program implementation and was in charge of all organizational details. The consortium selected the ViSC coaches as well as the intervention and control schools based on a list provided by the local branch of the Ministry of Education. The research team visited all schools in order to meet school officials and psychological counselors to talk about the data collection and implementation process. Third, the ViSC manual, materials, school posters, and surveys were translated from German to Turkish by professional bilingual psychologists. Minor modifications were made to adapt some of the games or vignettes to Turkish culture (e.g., the names in the vignettes). Fourth, the implementation model of the ViSC program consists of a cascaded train-the-trainer model in which the Austrian program developers together with the Turkish principal investigator trained the ViSC coaches who, in turn, trained teachers, and then teachers trained their students. Based on this model, we organized three ‘ViSC Coaches Training Workshops’ at Ege University. Ten experienced professionals (e.g., school psychological counsellors, educational psychologists) attended these workshops and were certified as ViSC coaches. Training 1 (14 hours, April 2015) focused on the definition and recognition of bullying; tackling acute bullying incidents; and preventive measures at the school level; Training 2 (14 hours, May 2015) focused on how to implement the ViSC class project and class activities. Training 3 (4 hours, October 2015) focused on the nature of cyberbullying and the feedbacks regarding challenges and successes in implementing the program. Fifth, the certified ViSC coaches organized a teacher-training seminar (3 hours) for all the teachers of the intervention schools in October/November 2015. At the same time, parent meetings (one-hour) were organized to explain the details of the project and active parental consent was obtained during these meetings. Sixth, volunteer classroom teachers (3 teachers in each school) were selected to implement the ViSC class project between December 2015 and May 2016. ViSC coaches provided one-to-one trainings for the class teachers to introduce the materials and activities. Then, teachers implemented the class project in every other week during two-class periods (a total of 90 minutes). During classes 1–8, class teachers worked on how to prevent bullying in their classroom and during classes 9–13 students worked together on creating a common activity (e.g., puppet show, designing musical instruments, and theatre). Because the class units comprise very detailed timetables and four undergraduate research assistants were also present during the program implementation, all class units were implemented by teachers with high fidelity. Because the research assistants did not act as co-trainers, they did not intervene during the class trainings, but they helped the teachers to prepare all worksheets and materials before the training.

Study Design and Procedure

A quasi-experimental longitudinal control group design with three measurement points was realized comprising six schools (3 intervention and 3 control). Data were collected at three time points: pre-test (November 2015), post-test (June 2016), and follow-up (November 2016). The school and class level intervention was implemented in nine grade 5 classes of three intervention schools (3 classes per school). The school level intervention was implemented in eight grade 5 classes of the same three intervention schools (2-3 classes per school). Nine classes out of three control schools served as control group (3 classes per school).

Randomization is considered as the gold standard of evidence-based prevention (Flay et al., 2005; Gottfredson et al., 2015); however, because the program was implemented in three schools only, it was not feasible to realize a cluster randomized control study on school level. Within three intervention schools, however, classes were randomly assigned into two intervention conditions. Thus, it was only possible to utilize a small scale quasi-experimental longitudinal control group design in the present study.

Participants

In total, 642 students (227 in school and class level intervention, 201 in school level intervention, 214 in control group) nested in 26 classes (9 in school and class level intervention, 8 in school level intervention, 9 in control group) and 6 schools participated in at least one measurement and were included. At wave 1 (pretest), the sample comprised 602 students (52% girls) with a mean age of 10.06 years (SD = 0.24, Min = 10, Max = 12). Table 1 provides a description of the sample. The groups were compared regarding demographic characteristics and there were no statistically significant differences between the groups.

Table 1

Demographic Characteristics of the Sample at Pretest

	School and Class Level Intervention (n = 212)	School Level Intervention (n = 191)	Control Group (n = 199)
Gender (% female)	53.8	53.9	49.2
Age, M (SD)	10.08 (0.28)	10.03 (0.18)	10.06 (0.26)
Married parents (%)	88.7	92.1	89.4
Perceived financial situation (%)
very bad or bad	2.4	2.1	6.1
neither bad nor good	21.2	26.2	20.7
good	44.3	39.3	37.9
very good	32.1	32.5	35.4
Parent’s working status (%)
Working father	97.6	94.7	95.9
Working mother	39.1	34.0	31.8
Father’s education level (%)
Elementary school or less	25.9	26.8	28.8
Middle school	35.8	41.5	29.8
High school or higher	38.3	31.7	41.4
Mother’s education level (%)
Elementary school or less	37.7	37.6	32.8
Middle school	30.2	33.9	30.8
High school or higher	32.1	28.5	36.4

Note. There were no statistically significant differences between groups.

Measures used in this study are described below. The same scales were used when evaluating the program effectiveness in Austria, Cyprus, and Romania (for details see Solomontos-Kountouri et al., 2016; Trip et al., 2015; Yanagida et al., 2016a).

Demographic Information

Children reported their gender, age as well as the marital status, the perceived financial situation, the working status, and the educational level of parents.

Aggressive behavior and victimization. Each of these constructs was measured with five scales, (1) bullying perpetration/victimization, (2) cyberbullying perpetration/victimization (3) physical aggression/victimization, (4) relational aggression/victimization, and (5) verbal aggression/victimization. Cross-national scalar measurement invariance of these scales could be established (for more details see Yanagida et al., 2016b). The items cover specific aggressive behavior and victimization incidents during the last two months. Answers to all questions were given on a five-point response scale ranging from 0 (never), 1 (once or twice), 2 (two or three times a month), 3 (once a week), 4 (nearly every day).

Bullying Perpetration and Victimization

The self-report scales consist of one global item, and three specific items covering different forms (physical, relational, and verbal) of bullying and victimization. In the global item, students were asked “How often have you insulted or hurt other students during the last two months?” and “How often have others insulted or hurt you in the last two months?”. The three specific items were similar to the global ones, except that they described specific forms of bullying and victimization. Cronbach’s α coefficients were 0.71/0.66/0.74 (pretest/posttest/follow up) for the bullying perpetration scale and 0.79/0.77/0.80 (pretest/posttest/follow up) for the victimization scale.

Cyberbullying and Cyber-Victimization

Both of these scales contain one global and seven specific items related to different electronic means based on Smith and colleagues (2008). The different electronic means were calls, text messages, emails, chat contributions, discussion board, instant messages, and videos or photos. Cronbach’s α coefficients for the cyberbullying scale were 0.84/0.76/0.76 (pretest/posttest/follow up) and 0.83/0.83/0.81 (pretest/posttest/follow up) for the cyber victimization scale.

Physical Aggression and Physical Victimization

The peer nomination measure developed by Crick and Grotpeter (1995) was modified into a self-report questionnaire and comprised three items, e.g., “How often did you hit one or more classmates?” or “How often have you been hit by one or more classmates?”. Cronbach’s α coefficients were 0.75/0.64/0.63 (pretest/posttest/follow up) for the physical aggression scale and 0.76/0.77/0.76 (pretest/posttest/follow up) for the physical victimization scale.

Relational Aggression and Relational Victimization

These five items were also adapted from the peer nomination measure originally developed by Crick and Grotpeter (1995), e.g., “How often did you left out other kids on purpose when it was time to play or do an activity?” or “How often were you excluded from play or another activity by one or more classmates?”. Cronbach’s α coefficients were 0.84/0.62/0.75 (pretest/posttest/follow up) for the relational aggression scale and 0.88/0.81/0.87 (pretest/posttest/follow up) for the relational victimization scale.

Verbal Aggression and Verbal Victimization

These three items cover direct and indirect verbal harassments (Strohmeier, Aoyama, Gradinger, & Toda, 2013), e.g., “How often did you say mean or hurtful things to other classmates?” or “How often did other classmates make fun of you?“. Cronbach’s α coefficients were 0.78/0.68/0.66 (pretest/posttest/follow up) for the verbal aggression scale and 0.78/0.79/0.82 (pretest/posttest/follow up) for the verbal victimization scale.

A series of confirmatory factor analyses (CFA) was conducted with Mplus Version 7.4 (Muthén & Muthén, 1998-2013) using robust maximum likelihood estimator (MLR) to establish measurement models for the five different forms of aggression and victimization. Moreover, all scales were tested for strong longitudinal and between-group measurement invariance (see Little, 2013). Results yield acceptable model fit (see Table 2) indicating sound measurement properties for all scales except for bullying, relational aggression, relational victimization, and cyberbullying victimization. In addition, the CFA model for cyberbullying did not converge due to estimation problems stemming from low prevalence of this type of aggressive behavior. Because of the extremely low frequency (see Table 3), we decided to exclude the cyberbullying and cybervictimization from further analyses. At pretest, scales for perpetration had an intercorrelation between 0.532 and 0.694; and scales for victimization had an intercorrelation between 0.573 and 0.685. Thus, as expected, the scales are correlated but discriminant validity can be assumed (see Brown, 2015, p. 116).

Table 2

Model Fit of the Measurement Models

	χ ²	df	p	CFI	RMSEA	SRMR
Perpetration
Bullying	297.71	189	<0.001	0.889	0.052	0.105
Cyberbullying	/	/	/	/	/	/
Physical Aggression	112.21	95	0.110	0.972	0.029	0.073
Relational Aggression	632.90	310	<0.001	0.816	0.070	0.129
Verbal Aggression	124.611	95	0.022	0.954	0.038	0.093
Victimization
Victimization	216.57	189	0.083	0.981	0.026	0.071
Cyberbullying Victimization	2207.18	835	<0.001	0.585	0.088	0.165
Physical Victimization	111.54	95	0.118	0.982	0.029	0.055
Relational Victimization	524.39	310	<0.001	0.885	0.057	0.097
Verbal Victimization	97.59	95	0.407	0.997	0.011	0.054

Note. The measurement model for cyberbullying could not be estimated.

Table 3

Descriptive Statistics for the Criterion Variables: Means and Standard Deviations

Criterion	School and Class Level Intervention			School Level Intervention			Control Group
	Wave 1	Wave 2	Wave 3	Wave 1	Wave 2	Wave 3	Wave 1	Wave 2	Wave 3
	(n = 212)	(n = 210)	(n = 187)	(n = 191)	(n = 177)	(n = 166)	(n = 199)	(n = 157)	(n = 182)
Perpetration
Bullying	0.16 (0.37)	0.30 (0.43)	0.24 (0.43)	0.23 (0.40)	0.37 (0.52)	0.31 (0.52)	0.38 (0.56)	0.35 (0.49)	0.32 (0.52)
Cyberbullying	0.03 (0.19)	0.04 (0.11)	0.06 (0.16)	0.05 (0.20)	0.07 (0.24)	0.05 (0.25)	0.06 (0.22)	0.08(0.25)	0.06 (0.21)
Physical Aggression	0.22 (0.46)	0.28 (0.37)	0.25 (0.43)	0.22 (0.37)	0.34 (0.53)	0.26 (0.44)	0.43 (0.74)	0.40 (0.65)	0.34 (0.54)
Relational Aggression	0.09 (0.32)	0.16 (0.31)	0.08 (0.21)	0.11 (0.36)	0.18 (0.33)	0.13 (0.31)	0.18 (0.46)	0.15 (0.33)	0.15 (0.40)
Verbal Aggression	0.17 (0.43)	0.21 (0.49)	0.18 (0.40)	0.15 (0.36)	0.22 (0.46)	0.19 (0.37)	0.30 (0.62)	0.30 (0.51)	0.28 (0.53)
Victimization
Victimization	0.48 (0.70)	0.74 (0.82)	0.56 (0.76)	0.42 (0.58)	0.69 (0.89)	0.52 (0.73)	0.71 (0.92)	0.66 (0.79)	0.57 (0.83)
Cybervictimization	0.10 (0.33)	0.15 (0.29)	0.11 (0.27)	0.10 (0.26)	0.18 (0.40)	0.11 (0.27)	0.15 (0.33)	0.14 (0.36)	0.11 (0.29)
Physical Victimization	0.53 (0.72)	0.58 (0.75)	0.47 (0.67)	0.51 (0.75)	0.61 (0.86)	0.55 (0.79)	0.68 (0.93)	0.72 (0.94)	0.59 (0.83)
Relational Victimization	0.16 (0.42)	0.35 (0.57)	0.18 (0.49)	0.22 (0.56)	0.38 (0.68)	0.22 (0.48)	0.37 (0.78)	0.37 (0.72)	0.36 (0.80)
Verbal Victimization	0.41 (0.73)	0.60 (0.85)	0.53 (0.93)	0.43 (0.77)	0.59 (0.92)	0.46 (0.77)	0.57 (0.93)	0.59 (0.91)	0.60 (1.02)

Note. Sample size slightly differs by scale and wave due to missing values. Answers to all questions were given on a five-point response scale ranging from 0 (never), 1 (once or twice), 2 (two or three times a month), 3 (once a week), 4 (nearly every day).

Missing Data

In total, 232 records (74 in school and class level intervention, 63 in school level intervention, 95 in control group) were incomplete. 195 students showed wave non-response: 17 students were missing at wave 1, 69 students were missing at wave 2, 59 students were missing at wave 3, 2 students were missing at wave 1 and 2, 21 students were missing at wave 1 and 3, and 27 students were missing at wave 2 and 3. Because active parental consent was obtained from all students, these students missed one or more wave because of three reasons: They were (1) absent at the day of data collection, (2) completed the questionnaire in an invalid way or (3) the child moved to another school (especially between T2 and T3). The remaining 37 students had a general missing data pattern with omitted items on single scales. The percentage of missing values across the 142 variables varied between 0% and 16.98%.

A series of two-sample Welch t-tests with Bonferroni-Holm correction for multiple comparisons was conducted to compare students with complete and incomplete data. There were no statistically significant differences between students with complete data and students with missing values (effect sizes ranged between d = 0.00 to d = 0.27). These results can be found in the supplementary material (Table S1) and indicate that missing data is not systematically related to the study variables. Full information maximum likelihood (FIML) under the MAR assumption was used to deal with missing data (Enders, 2010).

Analytic Strategy

Multilevel growth modelling (level 1: time, level 2: student, level 3: class) was conducted with SPSS Version 24 to test program effectiveness. Maximum likelihood was used as estimation procedure. This analysis adequately considers the nested data structure, were time is nested in students, and students are nested in classes taking into account the dependencies between observations (i.e., design effect, see Snijders & Bosker, 1999). The data met all necessary conditions for all the analyses that we conducted. Two sets of multilevel growth models were run to investigate the research questions.

The first set of models was conducted to investigate whether the control group differs from the two intervention groups (= Hypothesis 1). Program effectiveness regarding four perpetration and four victimization scales was investigated based on the following cross-level interactions Time x Control vs. School and Class Level Intervention, Time x Control vs. School Level Intervention and Time² x Control vs. School and Class Level Intervention, Time² X Control vs. School Level Intervention (see Table 4).

Table 4

Multilevel Growth Curve Model Results: Comparison of the Control Group and the Intervention Groups (School and Class Level vs. School Level Intervention) for Different Forms of Perpetration and Victimization

	Perpetration				Victimization
	Bullying	Physical Aggression	Relational Aggression	Verbal Aggression	Victimization	Physical Victimization	Relational Victimization	Verbal Victimization
Level 1: Measurement
Intercept	0.385^***	0.421^***	0.186^***	0.300^***	0.718^***	0.690^***	0.388^***	0.586^***
Time	–0.085	–0.049	–0.039	0.007	–0.063	0.013	–0.003	–0.021
Time²	0.025	0.004	0.010	–0.008	–0.005	–0.028	–0.003	–0.014
Level 3: Class: Unstandardized Estimates
Baseline Effects
Control vs. School and Class Level Intervention	–0.228^***	–0.202^***	–0.101^*	–0.131^*	–0.237^**	–0.166	–0.230^***	–0.185^*
Control vs. School Level Intervention	–0.152^**	–0.204^***	–0.072	–0.151^**	–0.294^***	–0.172	–0.159^*	–0.159
Intervention Effects
Time X Control vs. School and Class Level Intervention	0.334^***	0.155	0.183^*	0.066	0.540^***	0.123	0.376^***	0.349
Time X Control vs. School Level Intervention	0.325^***	0.268^**	0.158^*	0.130	0.567^***	0.198	0.315^*	0.338
Time² X Control vs. School and Class Level Intervention	–0.127^***	–0.048	–0.083^*	–0.027	–0.208^***	–0.048	–0.177^***	–0.144
Time² X Control vs. School Level Intervention	–0.129^***	–0.104^*	–0.066	–0.051	–0.223^***	–0.063	–0.155^*	–0.165
Level 3: Class: Standardized Estimates
Baseline Effects
Control vs. School and Class Level Intervention	–0.48^***	–0.39^**	–0.29^*	–0.28^*	–0.30^*	–0.21	–0.37^**	–0.21^*
Control vs. School Level Intervention	–0.32^*	–0.39^**	–0.21	–0.32^**	–0.37^**	–0.21	–0.25^*	–0.18
Intervention Effects
Time X Control vs. School and Class Level Intervention	0.71^***	0.30	0.53^*	0.14	0.68^**	0.15	0.60^**	0.40
Time X Control vs. School Level Intervention	0.68^***	0.52^*	0.46^*	0.27	0.72^**	0.25	0.50^*	0.39
Time² X Control vs. School and Class Level Intervention	–0.27^**	–0.09	–0.24^*	–0.06	–0.26^**	–0.06	–0.28^**	–0.17
Time² X Control vs. School Level Intervention	–0.27^**	–0.20^*	–0.19	–0.11	–0.28^**	–0.08	–0.25^*	–0.19

Note.^*p < 0.05; ^**p < 0.01; ^***p < 0.001.

The second set of models was conducted to investigate whether the two intervention groups (school and class level intervention vs. school level intervention) differ from each other (= Hypothesis 2). Program effectiveness regarding four perpetration and four victimization scales was investigated based on the following cross-level interactions Time x School vs. School and Class Level Intervention and Time² x School vs. School and Class Level Intervention (see Table 5).

Table 5

Multilevel Growth Curve Model Results: Comparison of the Intervention Groups (School Level vs. School and Class Level Intervention) for Different Forms of Perpetration and Victimization

	Perpetration				Victimization
Bullying	Physical Aggression	Relational Aggression	Verbal Aggression	Victimization	Physical Victimization	Relational Victimization	Verbal Victimization
Level 1: Measurement
Intercept	0.232^***	0.217^***	0.113^***	0.148^***	0.421^***	0.517^***	0.227^***	0.429^***
Time	0.240^***	0.218^**	0.118^*	0.136	0.509^***	0.211	0.314^***	0.315^*
Time²	–0.104^**	–0.100^**	–0.055^*	–0.058	–0.230^***	–0.092	–0.158^***	–0.150^*
Level 3: Class: Unstandardized Estimates
Baseline Effects
School Level Intervention vs. School and Class Level Intervention	–0.077	0.001	–0.028	0.019	0.060	0.007	–0.068	–0.025
Intervention Effects
Time X School Level Intervention vs. School and Class Level Intervention	0.010	–0.109	0.026	–0.061	–0.036	–0.076	0.056	0.009
Time² X School Level Intervention vs. School and Class Level Intervention	0.002	0.054	–0.017	0.023	0.018	0.015	–0.020	0.021
Level 3: Class: Standardized Estimates
Baseline Effects
School Level Intervention vs. School and Class Level Intervention	–0.173	0.002	–0.090	0.046	0.079	0.010	–0.126	–0.029
Intervention Effects
Time X School Level Intervention vs. School and Class Level Intervention	0.021	–0.251	0.083	–0.146	–0.048	–0.100	0.102	0.003
Time² X School Level Intervention vs. School and Class Level Intervention	0.001	0.123	–0.055	0.055	0.024	0.020	–0.036	0.027

Note.^*p < 0.05; ^**p < 0.01; ^***p < 0.001.

The quadratic effect of time was included in the analyses to examine whether the dependent variables would change differently between pretest-posttest and posttest-follow-up-test. We also computed standardized estimates for each effect to be able to estimate effect sizes. In order to compute these standardized estimates we re-run all analyses with standardized dependent variables again. We added the standardized estimates in the tables only for the relevant effects.

Results

Descriptive Statistics for Outcome Variables

As a first step, the means and standard deviations of perpetration and victimization for three groups are reported by wave of data collection (pretest, posttest, follow up) in Table 3.

Examining the means, perpetration and victimization scores are higher in the control group compared with the two intervention groups. Furthermore, perpetration and victimization increased in the two intervention groups between pretest and posttest, and then did not change or decreased again between posttest and follow-up. Changes between waves appeared to be bigger in the two intervention groups compared with the control group.

Baseline Effects in Perpetration and Victimization

Both intervention groups differed from the control group regarding bullying perpetration, physical aggression, verbal aggression, bullying victimization, and relational victimization. The school and class level intervention group also differed from the control group regarding relational aggression and verbal victimization (see Table 4). When comparing the two intervention groups (school and class level intervention vs. school level intervention) with each other, no baseline effects were detected (see Table 5).

Intervention Effects on Perpetration

As shown in Table 4, the overall pattern of results revealed that in both intervention groups bullying and relational aggression increased stronger compared with the control group that did not change over time (Time x Control vs. School and Class Level Intervention, Time x Control vs. School Level Intervention). However, these effects were qualified by a quadratic intervention effect (Time² x Control vs. School and Class Level Intervention,) indicating that the intervention groups had a steeper decrease after posttest compared with the control group. For bullying and physical aggression, this pattern was also found for the school level intervention group compared with the control group (Time² x Control vs. School Level Intervention). As shown in Table 5, results in the two intervention groups did not differ from each other. No dosage effects were found for all four forms of perpetration.

Bullying

There was a steeper increase in bullying over time in the two intervention groups (Time x Control vs. School and Class Level Intervention, b = 0.334, p < 0.001; Time x Control vs. School Level Intervention, b = 0.325, p < 0.001) compared with the control group. Moreover, quadratic effects were found in the two intervention groups (Time² x Control vs. School and Class Level Intervention, b = –0.127, p < 0.001; Time² x Control vs. School Level Intervention, b = –0.129, p < 0.001) indicating a steeper decrease of bullying after posttest compared with the control group.

Physical Aggression

There was a steeper increase in the school level intervention group compared with the control group over time (Time x Control vs. School Level Intervention, b = 0.268, p < 0.01) which was qualified by a quadratic effect (Time² x Control vs. School Level Intervention, b = –0.104, p < 0.05), indicating a steeper decrease after posttest compared with the control group.

Relational Aggression

There was a steeper increase in relational aggression over time in the two intervention groups (Time x Control vs. School and Class Level Intervention, b = 0.183, p < 0.05; Time x Control vs. School Level Intervention, b = 0.158, p < 0.05) compared with the control group. Moreover, quadratic effect was found in the school and class level intervention group (Time² x Control vs. School and Class Level Intervention, b = –0.083, p < 0.05) indicating a steeper decrease of relational aggression after posttest compared with the control group.

Verbal Aggression

No intervention and dosage effects were found.

Intervention Effects on Victimization

As shown in Table 4, the overall pattern of results revealed that in both intervention groups victimization and relational victimization increased stronger compared with the control group that did not change over time (Time x Control vs. School and Class Level Intervention, Time x Control vs. School Level Intervention). However, these effects were qualified by a quadratic intervention effect (Time² x Control vs. School and Class Level Intervention, Time² x Control vs. School Level Intervention) indicating that both intervention groups had a steeper decrease after posttest compared with the control group. As shown in Table 5, results in the two intervention groups did not differ from each other. No dosage effects were found for all four forms of victimization.

Victimization

There was a steeper increase in victimization over time in the two intervention groups (Time x Control vs. School and Class Level Intervention, b = 0.540, p < 0.001; Time x Control vs. School Level Intervention, b = 0.567, p < 0.001) compared with the control group. Moreover, quadratic effects were found in the two intervention groups (Time² x Control vs. School and Class Level Intervention, b = –0.208, p < 0.001; Time² x Control vs. School Level Intervention, b = –0.223, p < 0.001) indicating a steeper decrease of victimization after posttest compared with the control group.

Physical Victimization

No intervention and dosage effects were found.

Relational Victimization

There was a steeper increase in victimization over time in the two intervention groups (Time x Control vs. School and Class Level Intervention, b = 0.376, p < 0.001; Time x Control vs. School Level Intervention, b = 0.315, p < 0.05) compared with the control group. Moreover, quadratic effects were found in the two intervention groups (Time² x Control vs. School and Class Level Intervention, b = –0.177, p < 0.001; Time² x Control vs. School Level Intervention, b = –0.155, p < 0.05) indicating a steeper decrease of relational victimization after posttest compared with the control group.

Verbal Victimization

No intervention and dosage effects were found.

Discussion

Previous research indicates that bullying is a prevalent problem in Turkish schools (Dogan-Ates & Yagmurlu, 2010; Kapci, 2004; Pekel-Uludagli & Ucanok, 2005) and that there is an urgent need to implement preventive intervention programs in schools (Strohmeier & Noam, 2012). The ViSC program is the first evidence based anti-bullying program that was implemented in Turkey. The current research was designed as a small-scale pilot study in order to adapt and implement the ViSC program in Turkey and to evaluate its effectiveness. Most importantly, the ultimate goal of the present study is to raise public awareness of school bullying, help to change school policies, start a national strategy with the help of the National Ministry of Education, and in the long run, to impact the lives of a large number of students in Turkey.

The present study realized a cascaded train-the-trainer model in order to stimulate a systemic transformation on various systemic levels. To begin with, strong relationships with administrators of local branches of the Ministry of Education and of municipality were established and highly experienced experts participated in an extensive training to be certified as ViSC coaches. Second, the ViSC coaches carefully trained the teachers who in turn delivered the class project to their students. Thus, the present study demonstrates that it is possible to implement the ViSC program with high fidelity in Turkey.

Program Effectiveness on Perpetration and Victimization

Because the implementation of a whole school systemic anti-bullying program is a big effort for all parties involved, it is important to find out whether all elements are necessary to obtain training effects. We hypothesized that the school and class level intervention would be more effective than the school level intervention, because of the intense exposure to the training material of the teachers including the 13-units class activities for the students. Contrary to our expectations, no differences regarding program effectiveness between the two intervention groups were detected. Instead, analyses revealed that the three intervention schools had lower levels of all forms of perpetration and victimization compared with the control schools at baseline indicating a selection effect on school level. Most likely because of these very low baseline prevalence rates in the three intervention schools, we observed a significant increase in both intervention groups between pre-test and post-test in bullying and victimization as well as in relational aggression and victimization which could be interpreted as a statistical problem (e.g., bottom effect). When examining the quadratic intervention effect, however, analyses demonstrated that both intervention groups again decreased between post-test and follow-up test compared with the control group. In both intervention groups, quadratic effects were found for bullying and victimization as well as relational aggression and victimization. These results with small to medium effect sizes might be interpreted as a promotion of sensitization for bullying and aggression among the students. The ViSC coaches and classroom teachers observed that the students not only were introduced to the definition of bullying but also were more aware of different forms of perpetration and victimization during the class project. As a result, it is possible that the students reported higher prevalence rates at post-test which then decreased again at follow-up test.

We expected differential training effects depending on the form of perpetration and victimization. While we expected that the program is effective regarding bullying and victimization, no training effects on cyberbullying and cyber-victimization were expected. Our analyses revealed that prevalence rates of cyberbullying and cyber-victimization were extremely low; therefore, measurement models could not be estimated. Because measurement invariance is a pre-condition to make valid comparisons between intervention and control groups (van de Schoot, Lugtig, & Hox, 2012), no further analyses were conducted. Effects were found for bullying and victimization as well as relational aggression and victimization, however they were not in the desired direction because they increased between pre-test and post-test and decreased again at follow-up.

Limitations

The pilot implementation and evaluation of ViSC program in Turkey provides important insights despite the limitation that we implemented the program only in three schools. Because of this small sample size, it was not feasible to realize a cluster randomized control study on school level. Therefore, the two intervention groups systematically differed from the control group already at baseline. Although we picked grade 5 classes for good reasons, this decision turned out to be a limitation. Probably because of the recent school transition, the baseline rates for perpetration and victimization in the present sample were extremely low. Furthermore, we relied on self-assessments only. Self-report measures are often used because they are easy to apply and are reliable given multiple items are used to measure a construct (Yanagida et al., 2016b). The strengths and weaknesses of self-report measures in studies about aggressive behavior including bullying have already been discussed extensively in the literature (e.g., Solberg & Olweus, 2003). Likely, aggressive behavior is underestimated using self-reports because perpetrators might not report the “true” frequency of their behavior but might underestimate it. Thus, self-report measures should be interpreted with caution.

Implications for Research, Policy, and Practice

To replicate the present study with a large-scale cluster randomized longitudinal control study in Turkey is an important next step towards “evidence-based practice”. Such a study would be important, because baseline effects that were observed in this study would be avoided. Moreover, future studies should differentiate subgroups of children (e.g., bullies, victims, bully-victims, and uninvolved) based on a person-centered analytic approach (see von Eye & Spiel, 2010). To apply person-oriented approaches for data analysis in future studies is also important to be able to test risk x intervention effects as it was shown that a similar whole school program implemented in Finland was particularly beneficial among youth who were most victimized before the program implementation (Juvonen, Schacter, Sainio, & Salmivalli, 2016). However, it is also important to investigate the psychosocial adaptation of those youth who remain victims even after the program implementation, as it is possible that their situation in the class gets worse. Lastly, side effects of the program (e.g., academic motivation) should be investigated in future studies. Most importantly, effort is needed to sustainably implement evidence-based programs into educational systems (Spiel, Schober, & Strohmeier, 2016; Spiel & Strohmeier, 2011, 2012; Spiel, Wagner, & Strohmeier, 2012). To reach this goal, intensive collaboration and communication between policy makers, practitioners, and researchers are needed.

Author’s Note

We are very grateful to the whole ViSC project team consisting of ViSC coaches, classroom teachers who implemented the program in classes, and psychology graduate and undergraduate students at EGE University for their invaluable work during the intervention study. We also want to thank the municipality, administrators at the local branch of Ministry of Education, school principals and all students who participated in this study. Special thanks to Merve Balkaya for the help with translating the materials.

The implementation and evaluation of the ViSC program in Turkey was funded by the Jacobs Foundation between 2014 and 2017 (PI: Aysun Doğan). Dagmar Strohmeier was a visiting researcher at Ege University (February – June 2015) with the support fellowship awarded by the Scientific & Technological Research Council of Turkey (Tübitak, 2221). The data analyses were funded by the Platform for Intercultural Competences, University of Applied Sciences Upper Austria in 2017 (PI: Dagmar Strohmeier).

Footnotes

The supplementary information is available in the electronic version of this article: .

References

Bronfenbrenner

(1979). The ecology of human development: Experiments by nature and design. Cambridge, MA: Harvard University Press.

Brown

T. A.

(2015). Confirmatory factor analysis for applied research (2nd ed.). New York: Guilford Press.

Burnukara

, & Uçanok

(2012). İlk ve orta ergenlikte akran zorbaliği: Gerçekleştiği yerler ve baş etme yollari, [Bullying during early and middle childhood: Where it happens and coping strategies]. Türk Psikoloji Yazilari, 15, 68–82.

Card

N. A.

, & Little

T. D.

(2006). Proactive and reactive aggression in childhood and adolescence: A meta-analysis of differential relations with psychosocial adjustment. International Journal of Behavioral Development, 30, 466–480. doi: 10.1177/0165025406071904

Card

N. A.

, Stucky

B. D.

, Sawalani

G. M.

, & Little

T. D.

(2008). Direct and indirect aggression during childhood and adolescence: A meta-analytic review of gender differences, intercorrelations, and relations to maladjustment. Child Development, 79, 1185–1229. doi: 10.1111/j.1467-8624.2008.01184.x

Craig

, Pepler

, & Atlas

(2000). Observations of bullying in the playground and in the classroom. School Psychology International, 21, 22–36.

Crick

N. R.

, & Grotpeter

J. K.

(1995). Relational aggression, gender, and social-psychological adjustment. Child Development, 66, 710–722. doi: 10.1111/j.1467-8624.1995.tb00900.x

Currie

, Gabhainn

S. N.

, Godeau

, Roberts

, Smith

, Currie

, Picket

, Richter

, Morgan

, & Barnekow

(Eds.). (2008). Inequalities in young people’s health: HBSC international report from the 2005/06 survey. Health Policy for Children and Adolescents, No. 5. Copenhagen: WHO Regional Office for Europe.

Currie

, Zanotti

, Morgan

, Currie

, de Looze

, Roberts

, Samdal

, Smith

O. R. F.

, & Barnekow

(Eds.). (2012). Social determinants of health and well-being among young people: Health behaviour in school-aged children (HBSC) study: International report from the 2009/2010 survey. Health Policy for Children and Adolescents, No. 6. Copenhagen: WHO Regional Office for Europe.

10.

Dogan-Ates

, & Yagmurlu

(2010). Examining victimization in Turkish schools. European Journal of Educational Studies, 2, 31–37.

11.

Enders

(2010). Applied missing data analysis. New York, NY: Guilford.

12.

Erdur-Baker

Ö.

(2009). Peer victimization, rumination, and problem solving as risk contributors to adolescents’ depressive symptoms. The Journal of Psychology, 143, 78–90.

13.

Flay

B. R.

, Biglan

, Boruch

R. F.

, Castro

F. G.

, Gottfredson

, Kellam

, Mościcki

E. K.

, Schinke

, Valentine

J. C.

, & Ji

(2005). Standards of evidence: Criteria for efficacy, effectiveness, and dissemination. Prevention Science, 6, 151–175. doi: 10.1007/s11121-005-5553-y

14.

Gollwitzer

, Banse

, Eisenbach

, & Naumann

(2007). Effectiveness of the Vienna social competence training on implicit and explicit aggression. Evidence from an Aggressiveness IAT. European Journal of Psychological Assessment, 23, 150–156. doi: 10.1027/1015-5759.23.3.150

15.

Gollwitzer

, Eisenbach

, Atria

, Strohmeier

, & Banse

(2006). Evaluation of aggression-reducing effects of the “Viennese social competence training”. Swiss Journal of Psychology, 65, 125–135. doi: 10.1024/1421-0185.65.2.125

16.

Gottfredson

D. C.

, Cook

T. D.

, Gardner

F. E. M.

, Gorman-Smith

, Howe

G. W.

, Sandler

I. N.

, & Zafft

K. M.

(2015). Standards of evidence for efficacy, effectiveness and scale-up research in prevention science: Next generation. Prevention Science, 16, 893–926. doi: 10.1007/s11121-015-0555-x

17.

Gradinger

, Strohmeier

, & Spiel

(2009). Traditional bullying and cyberbullying: Identification of risk groups for adjustment problems. Journal of Psychology, 217, 205–213. doi: 10.1027/0044-3409.217.4.205

18.

Gradinger

, Yanagida

, Strohmeier

, & Spiel

(2015). Prevention of cyberbullying and cybervictimization: Evaluation of the ViSC Social Competence Program. Journal of School Violence, 14, 87–110. doi: 10.1080/15388220.2014.963231

19.

Gradinger

, Yanagida

, Strohmeier

, & Spiel

(2016). Effectiveness and sustainability of the ViSC Social Competence Program to prevent cyberbullying and cyber-victimization: Class and individual level moderators. Aggressive Behavior, 42, 181–193. doi: 10.1002/ab.21631

20.

Hofstede

(2001). Culture’s consequences: Comparing values, behaviours, institutions, and organizations across nations. Thousand Oaks, CA: Sage.

21.

Juvonen

, Schacter

H. L.

, Sainio

, & Salmivalli

(2016). Can a school-wide bullying prevention program improve the plight of victims? Evidence for risk×intervention effects. Journal of Consulting and Clinical Psychology, 84, 334–344. doi: 10.1037/ccp0000078

22.

Kärnä

, Voeten

, Little

T. D.

, Poskiparta

, Alanen

, & Salmivalli

(2011). Going to scale: A nonrandomized nationwide trial of the KiVa antibullying Program for grades 1–9. Journal of Consulting and Clinical Psychology, 79, 796–805. doi: 10.1037/a0025740

23.

Kapci

E. G.

(2004). İlköğretim öğrencilerinin zorbaliğa maruz kalma türünün ve sikliğinin depresyon, kaygi ve benlik saygisiyla ilişkisi [The relationship between the prevalence of bullying and depression, anxiety and self-esteem among elementary school children]. Ankara Üniversitesi Eğitim Bilimleri Fakültesi Dergisi, 37, 1–13.

24.

Little

T. D.

(2013). Longitudinal structural equation modeling. New York: Guilford press.

25.

Malti

, Noam

G. G.

, Beelmann

, & Sommer

(2016). Toward dynamic adaptation of psychological interventions for child and adolescent development and mental health. Journal of Clinical Child & Adolescent Psychology, 45, 827–836. doi: 10.1080/15374416.2016.1239539

26.

Malti

, Strohmeier

, & Killen

(2015). The impact of on-looking and including bystander behavior on judgments and emotions regarding peer exclusion. British Journal of Developmental Psychology, 33, 295–311. doi: 10.1111/bjd12090

27.

Muthén

L. K.

, & Muthén

B. O.

(1998–2012). Mplus user’s guide. Los Angeles, CA: Muthén & Muthén.

28.

Nocentini

, & Menesini

(2016). KiVa anti-bullying program in Italy: Evidence of effectiveness in a randomized control trial. Prevention Science, 17, 1012–1023. doi: 10.1007/s11121-016-0690-z

29.

Olweus

(1993). Bullying at school: What we know and what we can do. Oxford, UK: Blackwell.

30.

Pellegrini

A. D.

, & Bartini

(2000). Longitudinal study of bullying, victimization, and peer affiliation during the transition from primary school to middle school. American Educational Research Journal, 37, 699–725. doi: 10.3102/00028312037003699

31.

Pellegrini

A. D.

, & Long

J. D.

(2002). A longitudinal study of bullying, dominance, and victimization during the transition from primary school through middle school. British Journal of Developmental Psychology, 20, 259–280. doi: 10.1348/026151002166442

32.

Roland

(2011). The broken curve: Effects of the Norwegian manifesto against bullying. International Journal of Behavioral Development, 35, 383–388. doi: 10.1177/0165025411407454

33.

Roland

, & Idsøe

(2001). Aggression and bullying. Aggressive Behavior, 27, 446–462. doi: 10.1002/ab.1029

34.

Roland

, & Vaaland

(2006). ZERO teacher's guide to the Zero Anti-Bullying Programme. Stavanger: Centre for Behavioural Research, University of Stavanger.

35.

Salmivalli

(2010). Bullying and the peer group: A review. Aggression and Violent Behavior, 15, 112–120. doi: 10.1016/j.avb.2009.08.007

36.

Salmivalli

, & Nieminen

(2002). Proactive and reactive aggression among school bullies, victims, and bully-victims. Aggressive Behavior, 28, 30–44. doi: 10.1002/ab.90004

37.

Schultes

M.T.

, Stefanek

, van de Schoot

, Strohmeier

, & Spiel

(2014). Measuring implementation of a school-based violence prevention program on two levels: Fidelity and teachers’ responsiveness as predictors of proximal outcomes. Zeitschrift für Psychologie/Journal of Psychology, 222, 49–57. doi: 10.1027/2151-2604/a000165

38.

Smith

, Mahdavi

, Carvalho

, Fisher

, Russell

, & Tippett

(2008). Cyberbullying: Its nature and impact on secondary school pupils. The Journal of Child Psychology and Psychiatry, 49, 376–385. doi: 10.1111/j.1469-7610.2007.01846.x

39.

Snijders

, & Bosker

(1999). Multilevel analysis: An introduction to and advanced multilevel modeling. London, UK: SAGE.

40.

Solberg

, & Olweus

(2003). Prevalence estimation of school bullying with the Olweus Bully/Victim Questionnaire. Aggressive Behavior, 29, 239–268. doi: 10.1002/ab.10047

41.

Solomontos-Kountouri

, Gradinger

, Yanagida

, & Strohmeier

(2016). The implementation and evaluation of the ViSC Program in Cyprus: Challenges of cross-national dissemination and evaluation results. European Journal of Developmental Psychology, 13, 737–755. doi: 10.1080/17405629.2015.1136618

42.

Solomontos-Kountouri

, Tsagkaridis

, Gradinger

, & Strohmeier

(2017). Academic, socio-emotional and demographic characteristics of adolescents involved in traditional bullying, cyberbullying, or both: Looking at variables and persons. International Journal of Developmental Science, 11, 19–30. doi: 10.3233/DEV-17219

43.

Soydaş

D. K.

, & Uçanok

(2014). Do the characteristics for the adolescent use of information and communication technologies explain involvement in cyberbullying?Çocuk ve Gençlik Ruh Sağlği Dergisi, 21, 17–32.

44.

Spiel

, Schober

, & Strohmeier

(2016). Implementing intervention research into public policy - the “I-Approach” . Prevention Science. Advance online publication. doi: 10.1007/s11121-016-0638-3

45.

Spiel

, & Strohmeier

(2011). National strategy for violence prevention in Austrian schools and kindergarten: Development and implementation. International Journal of Behavioral Development, 35, 412–418. doi: 10.1177/0165025411407458

46.

Spiel

, & Strohmeier

(2012). Evidence-based practice and policy: When researchers, policy makers, and practitioners learn how to work together. European Journal of Developmental Psychology, 9, 150–162. doi: 10.1080/17405629.2011.616776

47.

Spiel

, Wagner

, & Strohmeier

(2012). Violence prevention in Austrian schools: Implementation and evaluation of a national strategy. International Journal of Conflict and Violence,, 6, 176–186. doi: 10.4119/ijcv-2910

48.

Stefanek

, Strohmeier

, & Yanagida

(2017). Depression in groups of bullies and victims: Evidence for the differential importance of peer status, reciprocal friends, school liking, academic self-efficacy, school motivation and achievement. International Journal of Developmental Science, 11, 31–43. doi: 10.3233/DEV-160214

49.

Strohmeier

, Aoyama

, Gradinger

, & Toda

(2013). Cybervictimization and cyberaggression in Eastern and Western countries: Challenges of constructing a cross culturally appropriate scale. In Bauman

, Cross

, & Walker

(Eds.), Principles of cyberbullying research: Definitions, measures, and methodology (p. 202–221). New York and London: Routlege.

50.

Strohmeier

, & Dogan

(2012). Emotional problems and victimization among youth with national and international migration experiences living in Austria and Turkey. Emotional & Behavioural Difficulties, 17, 287–304. doi: 10.1080/13632752.2012.704311

51.

Strohmeier

, Hoffmann

, Schiller

E. M.

, Stefanek

, & Spiel

(2012). ViSC Social Competence Program. New Directions for Youth Development, 133, 71–84. doi: 10.1002/yd.20008

52.

Strohmeier

, & Noam

G. G.

(2012). Bullying in schools: What is the problem, and how can educators solve it?New Directions for Youth Development, 133, 7–14. doi: 10.1002/yd.20003

53.

Swearer

S. M.

, & Espelage

D. L.

(2004). Introduction: A social-ecological framework. of bullying among youth. In Espelage

D. L.

, & Swearer

S. M.

(Eds.), Bullying in American schools: A social-ecological perspective on prevention and intervention (pp. 1–12). Mahwah: Lawrence Erlbaum Assoc Publ.

54.

Trip

, Bora

, Sipos-Gug

, Tocai

, Gradinger

, Yanagida

, & Strohmeier

(2015). Bullying prevention in schools by targeting cognitions, emotions and behaviour: Evaluating the effectiveness of the REBE-ViSC Program. Journal of Counseling Psychology, 62, 732–740. doi: 10.1037/cou0000084

55.

Van de Schoot

, Lugtig

, & Hox.

(2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9, 486–492. doi: 10.1080/17405629.2012.686740

56.

Veenstra

, Lindenberg

, Zijlstra

B.J.H.

, De Winter

A. F.

, Verhulst

F. C.

, & Ormel

(2007). The dyadic nature of bullying and victimization: Testing a dual-perspective theory. Child Development, 78, 1843–1854. doi: 10.1111/j.1467-624.2007.01102.x

57.

Vitaro

, Brendgen

, & Barker

E. D.

(2006). Subtypes of aggressive behaviors: A developmental perspective. International Journal of Behoral Development, 30, 12–19. doi: 10.1177/0165025406059968

58.

von Eye

, & Spiel

(Eds.). (2010). Conducting person-oriented research [Special issue]. Zeitschrift für Psychologie/Journal of Psychology, 218(3).

59.

Yanagida

, Strohmeier

, & Spiel

(2016a). Dynamic change of aggressive behavior and victimization among adolescents: Effectiveness of the ViSC Program. Journal of Clinical Child and Adolescent Psychology1–15. doi: 10.1080/15374416.2016.1233498

60.

Yanagida

, Gradinger

, Strohmeier

, Solomontos-Kountouri

, Trip

, & Bora

(2016b). Cross-national prevalence of traditional bullying, traditional victimization, cyberbullying and cyber-victimization: Comparing single-item and multiple-item approaches of measurement. International Journal of Developmental Science, 10, 21–32. doi: 10.3233/DEV-150173

61.

Yang

, & Salmivalli

(2013). Different forms of bullying and victimization: Bully-victims versus bullies and victims. European Journal of Developmental Psychology, 10, 723–738. doi: 10.1080/17405629.2013.793596

62.

Yeager

D. S.

, Fong

C. J.

, Lee

H. Y.

, & Espelage

D. L.

(2015). Declines in efficacy of anti-bullying programs among older adolescents: Theory and a three-level meta-analysis. Journal of Applied Developmental Psychology, 37, 36–51. doi: 10.1016/j.appdev.2014.11.005