Psychometric Properties of the Cyberbullying Test,a Screening Instrument to Measure Cybervictimization,Cyberaggression,and Cyberobservation

Abstract

The purpose of the study was to analyze the psychometric properties of the Cyberbullying Test. The sample included 3,026 participants from the Basque Country (northern Spain), aged 12 to 18 years. Results confirmed high internal consistency and moderate temporal stability. Exploratory factor analysis yielded three moderately correlated factors (cyberobserver, cyberaggressor, and cybervictim). Confirmatory factor analysis ratified adequate model fit of the three factors. Convergent and discriminant validity were confirmed: (a) cybervictims use a variety of conflict resolution strategies, scoring high in neuroticism, openness, antisocial behavior, emotional attention, school-academic problems, shyness-withdrawal, psychopathological disorders, anxiety, and psychosomatic complaints, and low in agreeableness, responsibility, self-esteem, and social adjustment and (b) cyberaggressors use many aggressive conflict resolution strategies, scoring high in neuroticism, antisocial behavior, school-academic problems, psychopathological and psychosomatic disorders, and low in empathy, agreeableness, responsibility, emotion regulation, and social adjustment. The study confirms the test’s reliability and validity.

Keywords

cyberbullying adolescence assessment instrumental study

Introduction

The most basic definition of cyberbullying, and the one most researchers agree on, is that it is a form of bullying others by using electronic communication technologies (Kowalski, Giumetti, Schroeder, & Lattanner, 2014). In other words, it consists of using information and communication technologies (ICT)—mainly Internet (e-mail, short text message [SMS], websites, blogs, online videogames, etc.) and mobile phones—to carry out psychological peer harassment. According to Smith et al. (2008), cyberbullying is an aggressive and intentional behavior repeated frequently over time by means of the use—by an individual or a group—of electronic devices targeting a victim who cannot easily defend himself or herself.

Cyberbullying is similar to bullying in that it is a premeditated, intentional, and repetitive violent behavior, based on an asymmetric relationship of power submission with another person. Nevertheless, cyberbullying presents some peculiarities that differentiate it from presential bullying, for example, the victims cannot escape (because they are constantly receiving messages on their mobile or computer), the breadth of the audience (it reaches an infinite number of people), the bullies’ invisibility, the duration (the harassment content may be permanent), as well as the speed and ease with which it is carried out. Cyberbullying is a type of bullying but it should be noted that the harm caused by the use of ICT is different from that produced by traditional violence. One of the differences is that a single behavior (e.g., uploading a photo or video on the web) can greatly harm a person because, among other reasons, a photo or a video can be instantaneously sent to a large number of individuals with a single click (Garaigordobil & Martínez-Valderrey, 2015) and may be durable. The rapid development and growth of this new form of harassment has generated the urgent need for its study (Garaigordobil, 2011).

Review of studies analyzing the prevalence of cyberbullying showed that violence using ICTs—and within it, the phenomenon of cyberbullying—has recently become a severe problem, present in all developed countries (Barlett, 2015; Garaigordobil, 2011, 2015; Kowalski & Limber, 2007; Tokunaga, 2010). Although still in its formative stages, a large amount of literature has reported the pervasive nature of cyberbullying. As with traditional bullying, the prevalence of cyberbullying is difficult to estimate because the measures used until now have varied significantly; however, research indicates that approximately 30% to 70% of children report having been victimized by a cyberbully (Fenaughty & Harré, 2013; Stewart, Drescher, Maack, Ebesutani, & Young, 2014; Wade & Beran, 2011; Walrave & Heirman, 2011).

Results of a prevalence study carried out recently in the Basque Country are similar to the data obtained in other countries: 69.8% of the sample was involved in cyberbullying (suffered, performed, or observed it once or more in the past year), 30.2% were cybervictims, 15.5% were cyberaggressors, and 65.1% were cyberobservers (Garaigordobil, 2015). The Basque Country (northern Spain), 1 of the 17 autonomous regions of Spain, was granted powers of self-governance through its Autonomy Statute of 1979. It is a small region with an area of 7,234.8 km² and a population of 2,174,033 inhabitants, located near the French border. According to the Basque Statistics Institute, it has an advanced industrial and technological development and a high rate of human development. The majority of the population belongs to the middle socioeconomic-cultural level, without extremes of poverty or wealth.

Emotional harm arising from cyberbullying is very important. Reviews (Garaigordobil, 2011; Hinduja & Patchin, 2010; Soler, Paretilla, Kirchner, & Forns, 2013; Stewart et al., 2014) have shown that (a) cybervictims have feelings of social anxiety, depression, suicidal ideation, stress, fear, low self-esteem, anger and frustration, helplessness, nervousness, irritability, somatization, sleep disorders, concentration difficulties affecting academic performance, and so on and (b) cyberaggressors are more likely to display moral disengagement, lack of empathy, difficulties following rules, problems due to their aggressive behavior, delinquent behavior, alcohol and drug consumption, dependence on technologies, truancy, and so on. In addition, victims and aggressors are at risk for developmental problems that can persist into adulthood. At its most extreme, cyberbullying can lead to suicide and youth violence.

The high prevalence and the negative consequences of cyberbullying reveal the need for systematic assessments to detect and intervene in these situations having negative impacts on human development, thus requiring the use of assessment instruments. During the past decade, many investigations have focused on the analysis of cyberbullying, generating some instruments for its assessment and identification. Measuring cyberbullying is difficult because there are few valid and reliable instruments, and there is an ongoing debate about the most appropriate methodological approaches. There are currently two approaches: (a) assessing cyberbullying as a function of the means used, for example, asking the frequency with which certain behaviors were suffered or carried out through Internet, e-mail, cellphone, and so on and (b) measuring certain behavioral categories regardless of the means used, for example, asking about behaviors such as lying, stealing someone’s password, humiliating, and so on (see Menesini & Nocentini, 2009; Nocentini, Menesini, & Calussi, 2009).

Cyberbullying assessment instruments have recently undergone systematic reviews (Berne et al., 2013; Stewart et al., 2014). Among the first instruments, we note the 88-item Cyberbullying Questionnaire (Smith, Mahdavi, Carvalho, & Tippett, 2006). The questionnaire was applied to 92 students, aged 11 to 16 years, from 14 schools in London. It comprised multiple-choice questions, with some qualitative sections. It examined the incidence of cyberbullying in and out of school, distinguishing seven types: text message bullying, picture/video clip bullying (via mobile phone cameras), phone call bullying, email bullying, chat room bullying, bullying through instant messaging, and bullying via websites.

Researchers generally use instruments that were developed for their specific studies, which have hindered the generalization of the nature and frequency of peer victimization across samples. Moreover, many measures have not been adequately researched in terms of their psychometric properties (Berne et al., 2013). Nevertheless, a few recent works have performed psychometric analyses, providing support for the reliability and validity of some questionnaires.

Some cyberbullying questionnaires are unifactorial whereas others explore various factors. Some were designed to assess the frequency with which the informer is the aggressor or the victim of violence through cellphones or Internet. Among them, we note the Berlin Cyberbullying-Cybervictimization Questionnaire (BCCQ; Schultze-Krumbholz & Scheithauer, 2009), the European Cyberbullying Intervention Project Questionnaire (ECIPQ; Brighi et al., 2012), the Cyberbullying Questionnaire (CBQ; Gámez-Guadix, Villa-George, & Calvete, 2014), and the Cyberbullying Scale (CS; Menesini, Nocentini, & Calussi, 2011). Another group of questionnaires specifically measure cybervictimization. Among them, the Cybervictimization Scale (Akbulut, Levent-Sahin, & Eristi, 2010), the E-Victimization Scale (E-VS; Lam & Li, 2013), the Cyberbullying Scale (CBS; Stewart et al., 2014), the Cybervictimization Scale of the Revised Cyberbullying Inventory (RCBI; Topcu & Erdur-Baker, 2010), or the Cybervictimization Questionnaire (CBV; Álvarez-García, Dobarro, & Nuñez, 2015) are unifactorial. Among the most recent unidimensional instruments, the CBS (Stewart et al., 2014) is notable for its psychometric properties. The authors examined the factor structure and reliability of the CBS in 736 sixth to twelfth graders in six northern Mississippi schools. Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) indicated that a one-factor model best represented the CBS structure. The CBS displayed strong psychometric properties, including excellent internal consistency (Cronbach’s α = .94) and significant positive correlations with the related constructs of anxiety, depression, and loneliness. Results support the CBS as a measure of cybervictimization among adolescents.

Among the multifactorial instruments are the two-factor Adolescent Victimization through Mobile Phone and Internet Scale (CYBVIC; Buelga, Cava, & Musitu, 2012); the three-factor Cybervictim and Bullying Scale (CVBS; Çetin, Yaman, & Peker, 2011), measuring verbal cyberbullying, concealing identity, and cyberfalsification; or the four-factor Online Victimization Scale (OVS; Tynes, Rose, & Williams, 2010), measuring general victimization, sexual harassment, individual racial discrimination, and vicarious racial discrimination.

Despite an increasing body of research on cyberbullying, there is no consensus about the best way to define and measure it. The construct of cyberbullying is problematic because it is complex, difficult to operationalize, and subject to varying interpretations across populations. Its classification becomes almost immediately obsolete due to the complexity and fast evolution of new technologies. In view of these conceptual and methodological difficulties, the measurement of cyberbullying demands improvement. In general, theoretical and empirical efforts are needed to overcome these difficulties and to directly capture the meaning of cybernetic problems for children, adolescents, and youth in the digital era.

Within this context, we designed the Cyberbullying Test to assess 15 behavioral categories regardless of the means used, collecting three types of information: Adolescents and youth report their own experiences of cybervictimization, their own cyberaggressive behaviors, and the cyberbullying behaviors they observe in others. The instrument’s triangular perspective is a novelty compared with previous assessment tools, allowing the appraisal of each individual’s degree of cybervictimization, cyberaggression, and cyberobservation.

The goal of this investigation was to perform psychometric analyses providing data about the instrument’s reliability (internal consistency, test–retest reliability) and validity (EFA and CFA, intercorrelations, convergent and discriminant validity). Within this framework, we hypothesize that the Cyberbullying Test will present strong psychometric guarantees of reliability and validity according to the standards of test construction, including an acceptable model fit for the three factors (cybervictimization, cyberaggression, and cyberobservation).

Method

Participants

The sample comprised 3,026 participants from the Basque Country (northern Spain), aged 12 to 18 years, 1,469 (48.5%) males and 1,557 (51.5%) females. The participants were secondary education (75.4%) and high school (24.6%) students enrolled in various public (45.6%) and private (54.4%) schools in the Basque Country. The distribution of the sample by sex and age is shown in Table 1. To obtain a representative sample of the Basque Country, we consulted the latest population survey of the Basque Statistical Institute, confirming a population of 101,757 students enrolled in Compulsory Secondary Education and High School. Using a 99% confidence level and a sample error of 0.024 for a population variance of 0.50, a representative sample should include 2,802 students. To select a representative sample of students from the Basque Country, we used a stratified, proportional, and randomized sampling technique, taking into account the proportionality of the schools in each province and balancing the diverse conditions (socioeconomic-cultural levels and school types of public–private, urban–rural, secular–religious, etc.).

Table 1.

Description of the Sample: Frequency and Percentage of Males and Females in the Three Age Groups.

	12-13 Years	14-15 Years	16-18 Years	Total
Males	543 (51.2%)	536 (49%)	390 (44.8%)	1,469 (48.5%)
Females	518 (48.8%)	558 (51%)	481 (55.2%)	1,557 (51.5%)
Total	1,061 (100%)	1,094 (100%)	871 (100%)	3,026 (100%)

Instruments

The Cyberbullying Test was administered along with seven assessment instruments to determine its reliability and validity. The Cyberbullying Test assesses 15 cyberbullying behaviors (see the Appendix) through 45 items, grouped around the role performed in the situation of aggression: cybervictim, cyberaggressor, and cyberobserver. Participants read the statements describing the behaviors and report the frequency with which these behaviors were suffered, performed, or observed during the past year. Each behavior is scored (never = 0, sometimes = 1, several times = 2, always = 3), and a direct global score is obtained for each role, respectively. The test provides information on four indices of cyberbullying: level of cybervictimization, cyberaggression, cyberobservation, and aggressive cybervictimization. These indices provide the mean score of these behaviors suffered, performed, and witnessed in the past year. The test also provides cutoff points to determine whether the respondent has “no problems,” is “at risk,” or has a “problem” in the four indices. The establishment of the two cutoff points delimiting these three score ranges is based on statistical criteria. Respondents with scores equal to or greater than percentile 85 (about 1 standard deviation above the mean) are considered to be within the “at risk” range of the Cyberbullying Test, and respondents with scores equal to or greater than percentile 95 (2 standard deviations above the mean) are considered to be within the “problem” range.

In addition, the following seven assessment instruments with psychometric guarantees (see manuals) were also administered. The Questionnaire Measure of Emotional Empathy (QMEE; Mehrabian & Epstein, 1972) assesses the capacity to cognitively and affectively respond to other people’s emotions. The Conflictalk measures three conflict management styles in youth and adolescents (Kimsey & Fuller, 2003): aggressive or self-oriented (wanting to do everything one’s own way, being aggressive and authoritarian when dealing with conflict), cooperative or problem-oriented (seeking the cause of conflict and specifically identifying the problem in collaboration with the other to find the best solution and cooperative action), and avoidant or other-oriented (thinking that conflict is always bad, dealing passively with it). The NEO Five-Factor Inventory (NEO-FFI; Costa & McCrae, 1992; Spanish adaptation by Cordero, Pamos, & Seisdedos, 1999) measures five big personality factors: (a) Neuroticism (maladapted, emotionally unstable, with a tendency to experience negative feelings such as fear, melancholy, shame, anger, guilt, etc.), (b) Extraversion (outgoing, sociable, assertive, active, talkative, likes excitement and stimulation, cheerful, energetic, and optimistic), (c) Openness (open, unconventional, given to questioning authority and willing to accept new ethical, social, and political ideas), (d) Agreeableness (friendly, altruistic, sympathetic toward others, willing to help), and (e) Responsibility (responsible, strong willed, goal-oriented, tending toward good academic-professional performance, conscientious, punctual, and reliable). The Antisocial-Delinquent Behavior Questionnaire (AD; Seisdedos, 1995) assesses antisocial behaviors such as visiting a banned site, painting graffiti, breaking or throwing other people’s possessions on the floor, fighting with others, hitting, insulting, or using offensive language. The Rosenberg Self-Esteem (RSE; Rosenberg, 1965) Scale assesses general self-esteem with statements focusing on global feelings of self-appraisal. The Trait Meta-Mood Scale (TMMS24; Salovey, Mayer, Goldman, Turvey, & Palfai, 1995; Spanish adaptation by Fernández-Berrocal, Extremera, & Ramos 2004) assesses intrapersonal emotional intelligence with three factors: Attention to Feelings is the amount of attention paid to one’s emotional states; Emotional Clarity refers to understanding one’s emotional states; and Emotional Repair is the ability to regulate one’s emotional states (the belief in one’s ability to release and regulate negative emotional states and to extend positive ones). The Behavioral Problems Scale (parent assessment; BPS; Navarro, Peiró, Llácer, & Silva, 1993) includes 99 items grouped into seven scales: School-Academic Problems (related to low academic performance), Antisocial Behavior (behaviors that can be classified as aggressive and behaviors that, while not aggressive, might impair social relationships), Shyness-Withdrawal (tendency to solitude and susceptibility in social relationships), Psychopathological Disorders (serious problems which generally have a depressive component), Anxiety Problems (behaviors expressing fear and/or generalized anxiety), Psychosomatic Disorders (physical disorders without a medical cause), and a Positive Scale of Social Adjustment (adjustment to social rules).

Procedure

This instrumental study was carried out using a prospective, single-group, ex post facto, cross-sectional design. The procedure established the following phases: (a) we sent a letter to the directors of the schools selected randomly from the list of schools in the Basque Country, explaining the project and requesting their collaboration; (b) we interviewed those who agreed to collaborate, introducing the project and distributing the informed consent forms for the participants’ parents to sign (if a director of a selected school refused to collaborate, the procedure was repeated with the next school on the list, taking into account the type [public–private] and/or the socioeconomic-cultural level of the excluded school); (c) after receiving the parents’ consent, the research team (psychology graduates and PhD students) administered the assessment instruments. The Ethics Committee of the University of the Basque Country approved the study.

Analysis

Reliability

To analyze internal consistency, Cronbach’s alpha was calculated for the 45 items of the Cyberbullying Test and each of its factors. As the test provides ordinal scores (no problem, at risk, problem), to calculate test–retest reliability, we used the ordinal gamma statistic, a measure of rank correlation indicating the strength of the association between ordinal variables. Like the Pearson correlation coefficient, its values range between −1 and +1.

Validity

After randomly dividing the sample into two groups, EFA was used to analyze the first group, and CFA used for the second group. In addition, the correlations of the items of each factor with the total score were calculated for cybervictimization, cyberaggression, and cyberobservation, respectively. Subsequently, to analyze convergent and discriminant validity, partial correlation coefficients were calculated between cybervictimization and cyberaggression and diverse variables (empathy, conflict resolution, personality traits, self-esteem, emotional intelligence, antisocial behavior, and behavioral problems), while controlling for the effects of sex and age. To assess whether the characteristics of cybervictims of this study converge with those obtained in previous studies, we performed ANOVA of the scores obtained in cybervictimization and cyberaggression with regard to the rest of the variables. The analyses were carried out with the SPSS 21.0 and EQS 6.1 programs.

Results

Reliability: Internal Consistency and Test–Retest Reliability

The Cronbach alpha coefficients obtained for the 45 items were high (α = .91), as were those obtained for its three factors, cybervictimization (α = .82), cyberaggression (α = .91), and cyberobservation (α = .87), showing evidence of the test’s internal consistency. To calculate test–retest reliability, we used a sample of 83 adolescents aged 12 to 16 years, who completed the test two times, with a 3-month interval. The results of the correlation (ordinal gamma) between the scores at both applications (see Table 2) showed moderate values of temporal stability, suggesting that adolescents who suffer, perform, and observe cyberbullying behaviors are fairly likely to continue doing so 3 months later.

Table 2.

Test–Retest Reliability.

	Test		Retest		Gamma
	M	SD	M	SD	Gamma
Cybervictimization	0.90	3.41	1.51	3.80	.63
Cyberaggression	0.25	0.93	0.70	1.77	.74
Cyberobservation	2.60	2.94	3.37	5.21	.80
Aggressive cybervictimization	1.16	3.69	2.20	5.43	.77

EFA and Interscale Correlations

First, principal component analysis was conducted to examine test dimensionality. Previously, the Kaiser–Meyer–Olkin (KMO) sample adequacy measurement and Bartlett’s sphericity test were calculated. The KMO index yielded a value of .93, which can be considered adequate, and Bartlett’s test was statistically significant (χ₉₉₀ = 40959.99, p < .001), indicating that principal component analysis was appropriate. We used Varimax factor rotation method. Taking into account the Kaiser factor-extraction criterion, we extracted three factors with eigenvalues greater than 1, explaining 42.39% of the variance (see Table 3). Considering the value .30 as the cutoff point to assign an item to a factor, the structure of each of the three obtained factors was very clear. The first factor comprised 15 items concerning the role of cyberaggressor, the second one had 15 items referring to the role of cyberobserver, and the third included 15 items referring to the role of cybervictim. These data confirmed the expected factor structure of the test.

Table 3.

Rotated Factor Matrix.

Item	Factor 1 Cyberaggressor	Factor 2 Cyberobserver	Factor 3 Cybervictim
13.	.814	.080	.098
14.	.811	.135	.067
7.	.762	.129	.127
11.	.752	.088	.152
8.	.718	.151	.028
5.	.709	.099	.191
12.	.703	.179	.123
2.	.679	.157	.065
4.	.678	.130	.055
9.	.673	.093	.175
3.	.666	.080	.083
15.	.636	.197	.118
1.	.589	.211	.149
10.	.587	.161	.129
6.	.586	.247	.143
11	.042	.681	.098
9.	.126	.665	.112
6.	.079	.662	.113
7.	.052	.646	.140
1.	.061	.640	.126
12.	.178	.635	.059
15.	.052	.628	.093
10.	.094	.622	.116
13.	.175	.606	.153
2.	.100	.601	.132
4.	.166	.592	.117
5.	.185	.557	.133
8.	.283	.533	.172
3.	.127	.528	.178
14.	.269	.500	.184
2.	.146	.105	.622
1.	.009	.109	.589
14.	.416	.021	.580
15.	.033	.257	.576
13.	.144	.126	.573
7.	.122	.144	.571
6.	.037	.129	.546
12.	.084	.102	.538
11.	.019	.064	.534
10.	.025	.184	.509
5.	.299	.141	.507
4.	.168	.083	.492
9.	.075	.240	.484
8.	.288	.071	.415
3.	.222	.061	.411
%Variance explained	26.22	9.68	6.49

Note. Extraction method: Maximum likelihood.

p < .001.

Pearson’s correlations between the 15 items of cybervictimization and the total cybervictimization score were calculated, as well as correlations between cyberaggression and cyberobservation items with the total scores of these scales, respectively. The results revealed moderate and high correlations (p < .001) between the items and their respective scales (cybervictimization, r = .45-.63; cyberaggression, r = .66-.78; cyberobservation, r = .55-.68). Moderate correlations were found between cybervictimization and cyberaggression (r = .48), between cybervictimization and cyberobservation (r = .44), and between cyberaggression and cyberobservation (r = .39).

CFA

Second, the fit of the three-factor model was examined with CFA, showing a good statistical fit, χ² = 4604.73 (942), p < .000, χ²/df = 4.88, Satorra–Bentler χ²/df = 1.28, comparative fit index (CFI) = .91, non-normed Fit Index (NNFI) = .90, goodness of fit (GFI) = .92. The model had a root mean square error of approximation (RMSEA) value of .056, with an adequate 90% confidence interval (CI) = [0.056, 0.063], and standardized root mean square residual (SRMR) was .050. Overall, the fit indices suggested acceptable fit of the model, and the NNFI and CFI indicated that the model fits the data well.

Convergent and Discriminant Validity: Relations Between Cybervictimization/Cyberaggression and Behavioral, Cognitive, Emotional, and Social Variables

Partial correlation coefficients (controlling for the effects of sex and age) were calculated between cybervictimization and cyberaggression and numerous variables. The results (see Table 4) showed positive correlations between cybervictimization and the use of conflict resolution strategies (aggressive, passive, cooperative), neuroticism, openness, antisocial behavior, emotional attention, and diverse behavioral problems assessed by parents (school-academic problems, antisocial behavior, shyness-withdrawal, psychopathological disorders, anxiety problems, psychosomatic disorders). Negative correlations were found between cybervictimization and agreeableness, responsibility, self-esteem, and social adjustment.

Table 4.

Partial Correlations Between Cybervictimization and Cyberaggression With Behavioral, Cognitive, Emotional, and Social Variables.

	Cybervictimization	Cyberaggression
Empathy	−.02	−.14***
Conflict resolution
Cooperative	.09***	.00
Aggressive	.11***	.15***
Avoidant	.13***	.03
Personality dimensions
Neuroticism	.18***	.08***
Extraversion	−.00	−.01
Openness	.06***	.01
Agreeableness	−.12***	−.16***
Responsibility	−.08***	−.10***
Antisocial behavior	.13***	.17***
Self-esteem	−.12***	−.07***
Emotional intelligence
Emotional attention	.06***	−.03*
Emotional clarity	.00	−.06***
Emotional repair	.01	−.08***
Behavioral problems
School-academic problems	.17***	.14***
Antisocial behavior	.18***	.17***
Shyness-withdrawal	.10***	.05*
Psychopathological disorders	.19***	.12***
Anxiety problems	.09***	.04*
Psychosomatic disorders	.13***	.11***
Social adjustment	−.12***	−.12***
Behavioral problems total	.20***	.15***

p < .05. **p < .01. ***p < .001.

Regarding cyberaggression, the coefficients obtained confirmed significant positive relationships with aggressive conflict resolution, neuroticism, antisocial behavior, and behavioral problems (school-academic problems, antisocial behavior, shyness-withdrawal, psychopathological disorders, anxiety problems, psychosomatic disorders). Negative correlations were found between cyberaggression and empathy, agreeableness, responsibility, self-esteem, emotional intelligence (attention, clarity, repair), as well as with social adjustment.

Convergent and Divergent Validity: Profiles of Cybervictims and Cyberaggressors

To ratify validity, the participants were divided into cybervictims (they had received some aggression through electronic means in the past year) and non-cybervictims (they had not suffered any cyberbullying behavior), as well as cyberaggressors (they had performed cyberbullying behaviors in the past year) and non-cyberaggressors (they had not performed any cyberbullying behavior).

To assess whether the characteristics of cybervictims and cyberaggressors of this study were similar to those obtained in prior studies (confirming test validity), we analyzed the participants’ profiles, that is, the characteristics of those who had been cybervictims and cyberaggressors versus those who had not. For this purpose, we performed ANOVA with the scores obtained in diverse behavioral, cognitive, emotional, and social variables, the results of which are presented in Table 5.

Table 5.

Non-Cybervictim, Cybervictim, Non-Cyberaggressor, and Cyberaggressor Profiles in Behavioral, Cognitive, Emotional and Social Variables.

	Non-Cybervictim (n = 2,114)	Cybervictim (n = 912)	F(1, 3024)_{victimization}	d	Non-Cyberaggressor (n = 2,557)	Cyberaggressor (n = 469)	F(1, 3024)_aggression	d
	M (SD)	M (SD)	F(1, 3024)_{victimization}	d	M (SD)	M (SD)	F(1, 3024)_aggression	d
Empathy	16.11 (3.74)	16.38 (3.67)	3.10 ns	0.07	16.38 (3.67)	15.13 (3.83)	44.36***	0.33
Conflict resolution
Cooperative	15.94 (5.95)	17.13 (5.95)	54.54***	−0.19	16.24 (5.97)	16.55 (6.04)	1.01 ns	−0.05
Aggressive	9.81 (3.57)	10.59 (3.94)	27.43***	−0.20	9.72 (3.45)	11.83 (4.39)	130.98***	−0.53
Avoidant	11.56 (3.68)	12.67 (3.81)	24.64***	−0.29	11.80 (3.74)	12.42 (3.81)	10.52***	−0.16
Personality dimensions
Neuroticism	19.71 (7.24)	23.56 (7.49)	159.07***	−0.52	20.51 (7.47)	22.85 (7.75)	34.38***	−0.30
Extraversion	33.37 (6.74)	33.54 (7.16)	0.36 ns	−0.02	33.37 (6.82)	33.68 (7.13)	0.71 ns	−0.04
Openness	24.95 (6.87)	26.38 (7.07)	24.89***	−0.20	25.36 (7.02)	25.49 (6.62)	0.12 ns	−0.01
Agreeableness	29.39 (5.87)	27.76 (6.15)	43.92***	0.27	29.46 (5.86)	25.80 (5.87)	138.41***	0.62
Responsibility	28.40 (7.06)	26.78 (6.78)	31.09***	0.23	28.41 (6.95)	25.05 (6.71)	84.23***	0.49
Antisocial behavior	7.18 (5.18)	9.07 (5.40)	78.83***	−0.35	7.16 (5.14)	11.07 (5.05)	217.81***	−0.76
Self-esteem	30.15 (5.26)	28.55 (5.10)	55.76***	0.30	29.88 (5.22)	28.50 (5.34)	25.23***	−0.26
Emotional intelligence
Emotional attention	24.72 (7.04)	26.37 (7.56)	30.89***	−0.22	25.19 (7.11)	25.32 (7.91)	0.11 ns	−0.01
Emotional clarity	25.03 (6.72)	24.90 (6.78)	0.23 ns	0.01	25.05 (6.75)	24.64 (6.68)	1.32 ns	0.06
Emotional repair	26.29 (6.46)	25.96 (6.76)	1.49 ns	0.04	26.37 (6.47)	25.20 (6.96)	11.50***	0.17
Behavioral problems
School-academic problems	5.57 (5.32)	7.68 (6.42)	45.94***	−0.35	5.81 (5.55)	8.24 (6.30)	37.23***	−0.40
Antisocial behavior	5.11 (4.42)	6.88 (5.31)	47.27***	−0.36	5.26 (4.50)	7.64 (5.62)	52.57***	−0.46
Shyness-withdrawal	6.25 (3.78)	7.07 (3.96)	14.90***	−0.21	6.41 (3.84)	6.91 (3.89)	3.39 ns	0.12
Psychopathological disorders	3.17 (2.91)	4.45 (3.99)	51.41***	−0.36	3.42 (3.19)	4.11 (3.81)	8.94**	−0.19
Anxiety problems	4.32 (2.94)	5.00 (3.19)	16.55***	−0.22	4.46 (3.01)	4.81 (3.01)	2.74 ns	−0.11
Psychosomatic disorders	1.38 (1.87)	2.00 (2.31)	31.75***	−0.29	1.47 (1.96)	2.04 (2.31)	16.45***	−0.26
Social adjustment	25.60 (3.90)	24.76 (4.24)	14.74***	0.20	25.56 (3.84)	24.22 (4.76)	22.86***	0.30
Behavioral problems total	25.79 (15.73)	33.08 (19.09)	62.53***	−0.41	26.81 (16.41)	33.74 (19.33)	34.18***	−0.38

Note. ns = non-significant, d = Cohen’s d.

p < .01. ***p < .001.

Table 5 shows that cybervictims displayed (a) significantly greater use of cooperative, passive, and aggressive responses as a conflict resolution technique, and higher levels of neuroticism, antisocial behavior, school-academic problems, shyness-withdrawal, psychopathological disorders, anxiety, and psychosomatic problems and (b) significantly lower levels of agreeableness, responsibility, self-esteem, and social adjustment. Cybervictims also showed significantly more openness and high emotional attention. Nevertheless, the effect size was low, except for neuroticism. No differences were found in empathy, extraversion, or emotional clarity and repair.

Cyberaggressors (see Table 5) displayed (a) significantly greater use of aggressive and passive conflict resolution techniques, higher neuroticism, antisocial behavior, school-academic problems, psychopathological disorders, and psychosomatic problems and (b) significantly lower levels of empathy, agreeableness, responsibility, self-esteem, emotional repair, and social adjustment. The effect size was medium-high in some variables (aggressive conflict resolution, antisocial behavior, agreeableness). No significant differences were found in the use of cooperative conflict resolution strategies, extraversion, openness, emotional attention and clarity, shyness-withdrawal, and anxiety disorders.

Discussion

The psychometric analyses of the Cyberbullying Test showed a high level of internal consistency. Moreover, moderate values of temporal stability were confirmed, suggesting that adolescents who suffer, perform, and observe cyberbullying behaviors are fairly likely to continue doing so 3 months later, although direct participation, either as a victim or an aggressor, seems more variable over time. These results are similar to those found by other authors analyzing the temporal stability of cyberbullying behaviors (Del Rey, Elipe, & Ortega-Ruiz, 2012). Therefore, the instrument shows evidence of an adequate level of reliability.

The results of the EFA yielded three factors (cyberobserver, cyberaggressor, and cybervictim), ratifying the expected factor structure. In general, high correlations were found between the items and the total score of each of the three factors, as well as moderate correlations between the three scales. The CFA confirmed an adequate fit to a three-factor model. Unlike other assessment instruments that collect information about cybervictims and cyberaggressors, the Cyberbullying Test also identifies cyberobservers, a significant role in the phenomenon of harassment that has rarely been taken into account in other tests. This is important because the role of observers in cyberbullying is crucial. Bullying in all its forms (face-to-face and technological) is largely perpetuated as a result of the silence and inaction of observers (who tend to say nothing due to lack of empathy or out of fear of becoming the target of the aggressors). Therefore, an instrument that identifies observers may be helpful, as it would allow us to implement actions to encourage their active involvement, to denounce what they witness and to support the victim, which would surely inhibit harassment. In addition, this information may lead to the implementation of more qualitative studies on the role of observers, asking them, for example, what actions they undertake when they witness situations of cyberbullying, to whom they communicate what they witnessed, the reasons for their behavior, their feelings when witnessing such situations, and so on. Moreover, the instrument’s emphasis on observers could be disseminated through social media to raise awareness about the importance of their role in eradicating bullying in all its forms.

The results of the analyses of convergent and discriminant validity suggest that cybervictims tend to use a variety of conflict resolution strategies, obtaining high scores in neuroticism, openness, antisocial behavior, school-academic problems, shyness-withdrawal, psychopathological disorders, anxiety, and psychosomatic complaints. In addition, they pay considerable attention to their emotions and obtain low scores in agreeableness, responsibility, self-esteem, and social adjustment. Cyberaggressors use many aggressive conflict resolution strategies, scoring high in neuroticism, antisocial behavior, school-academic problems, and psychopathological and psychosomatic disorders. They also obtain low scores in empathy, agreeableness, responsibility, emotional repair, and social adjustment. As a whole, the results confirm the validity of the test, as they are consistent with the profiles of cybervictims and cyberaggressors identified in other studies.

This study provides a tool to assess cyberbullying with psychometric guarantees of reliability and validity in a broad, representative sample, and the test is easy to administer, score, and interpret. Compared with previously designed instruments (Álvarez-García et al., 2015; Brighi et al., 2012; Gámez-Guadix et al., 2014; Menesini et al., 2011), (a) it employs a triangular perspective, collecting information about the level of cybervictimization, cyberaggression, and cyberobservation in students aged 12 to 18 years and (b) it has standardized norms, appraising the four indices of cyberbullying (cybervictimization, cyberaggression, cyberobservation, and aggressive cybervictimization).

In view of the high prevalence of cyberbullying (Fenaughty & Harré, 2013; Garaigordobil, 2015; Stewart et al., 2014; Wade & Beran, 2011; Walrave & Heirman, 2011) and its negative effects (Hinduja & Patchin, 2010; Soler et al., 2013; Stewart et al., 2014), these results have practical implications: (a) for assessment, by emphasizing the importance of assessing cyberbullying systematically in all schools and (b) for intervention, by identifying relevant variables that the programs should include to prevent and intervene in cyberbullying (Garaigordobil & Martínez-Valderrey, 2014a, 2014b, 2015), which should promote socioemotional development by stimulating social adjustment, self-esteem, prosociability, comprehension and expression of emotions, empathy, agreeableness, and so on. Applied in educational settings, these programs can help to decrease cybervictimization and cyberaggression.

As a limitation of the study, we note the use of self-reports, due to their inherent social desirability. Hence, in the future, it is recommended to contrast the results of self-reports, for example, with data obtained from sociometric or hetero-report techniques by peers, teachers, and parents. The study also has the intrinsic limitations of anonymous survey-based studies in general, as well as the specific sociolinguistic or ethnographic limitations of this particular survey group. Cross-cultural validation of the test could be another future line of research. In this sense, we note that the test is currently being applied in various countries (Argentina, México, Colombia) wherefore data will be available in the near future to analyze its cross-cultural validity. Furthermore, another interesting future line of research could be to apply the test to children aged 10 to 11 years. The test is currently applied to a representative sample of 1,993 fifth and sixth graders of primary education in the Basque Country. The work has revealed the suitability of the test for application during late childhood, and in the future, the norms established with this sample will be included in the test manual.

In addition, it would be useful to carry out qualitative studies asking victims, aggressors, and observers open questions. For example, asking (a) the victims what actions they performed, to whom they reported the situation, what they felt as a result of the situation, the effects of the experience, and so on; (b) the aggressors how long they have bullied, who they bully, whether they bully individually or in groups, why they bully, what they feel when they bully others; and (c) the observers what they do or whom they inform when they observe a classmate bullying others, how they feel when witnessing these behaviors, and so on. Collecting information on the known strategies for dealing with this situation, whether as victims or as observers, would also have practical implications for intervention.

Footnotes

Appendix

Cyberbullying Behaviors Explored by the Cyberbullying Test.

1. Have they ever sent you offensive and insulting messages by cellphone or Internet?

2. Have you ever received offensive and insulting calls on your cellphone or by Internet (Skype . . . )?

3. Have you ever been assaulted to tape the assault and hang it on the Internet?

4. Have they ever diffused your private or compromising pictures or videos by Internet or cellphone?

5. Have they ever taken pictures of you without your permission in places such as locker rooms, beaches, or toilets and hung them on the Internet or diffused them by cellphone?

6. Have you ever received anonymous calls to scare or frighten you?

7. Have they ever blackmailed or threatened you with calls or messages?

8. Have they ever harassed you sexually by cellphone or on the Internet?

9. Has anybody ever signed your blog, pretending to be you, making slandering comments, lying, or revealing your secrets?

10. Have they ever stolen your password to prevent your access to your blog or e-mail?

11. Have they ever touched up your photos or videos to diffuse them through social networks or YouTube to humiliate you or make fun of you?

12. Have they ever harassed you to isolate you from your social network contacts?

13. Have they ever blackmailed you, making you do things you did not want to do to prevent them from diffusing your intimate matters on the network?

14. Have they ever threatened to kill you or your family by cellphone, the social networks, or any other type of technology?

15. Have they ever slandered you through the Internet, telling lies about you to discredit you? Have they ever spread rumors about you to harm you?

Note. The 15 items of the Appendix are applied in the victim role (participants report whether they have suffered these behaviors in the past year and with what frequency); then, they are asked if they have carried out these behaviors in the past year and with what frequency (the aggressor role), and finally, they are asked if they have seen some classmates performing these behaviors toward other classmates in the past year and with what frequency (the observer role).

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study was financed by the Spanish Ministry of Economics and Competitiveness (MINECO; PSI2012-30956), by the Department of Education of the Basque Government (IT-638-13), and by the Training and Research Unit of the University of the Basque Country (UFI PSIXXI 11/04).

Author Biography

Maite Garaigordobil is Full Professor of Psychological Assessment and Diagnosis in the Psychology Faculty of the University of the Basque Country (Spain). She coordinates the Research Group “Psychological assessment: Assessment instruments and intervention programs”. Nowadays she directs the project “Cyberbullying: Prevalence in the Basque Country, links with personal and family variables, and prevention and intervention program”.

References

Akbulut

Levent-Sahin

Eristi

(2010). Development of a scale to investigate cybervictimization among online social utility members. Contemporary Educational Technology, 1, 46-59.

Álvarez-García

Dobarro

Nuñez

J. C.

(2015). Validez y fiabilidad del cuestionario de cibervictimización en estudiantes de Secundaria [Validity and reliability of the Cybervictimization questionnaire in secondary education students]. Aula Abierta, 43, 32-38. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1016/j.aula.2014.11.001

Barlett

C. P.

(2015). Anonymously hurting others online: The effect of anonymity on cyberbullying frequency. Psychology of Popular Media Culture, 4(2), 70-79. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1037/a0034335

Berne

Frisén

Schultze-Krumbholz

Scheithauer

Naruskov

Luik

. . . Zukauskiene

(2013). Cyberbullying assessment instruments: A systematic review. Aggression and Violent Behavior, 18, 320-334. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1016/j.avb.2012.11.022

Brighi

Ortega

Pyzalski

Scheithauer

Smith

P. K.

Tsormpatzoudis

Barkoukis

(2012). European bullying intervention project questionnaire (ECIPQ). Unpublished manuscript, University of Bologna, Italy. Retrieved from http://www.bullyingandcyber.net

Buelga

Cava

M. J.

Musitu

(2012). Validación de la escala de victimización entre adolescentes a través del teléfono móvil y de Internet [Validation of the scale of victimization among adolescents through mobile phone and internet]. Revista Panamericana de Salud Publica, 32, 36-42. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1590/S1020-49892012000700006

Çetin

Yaman

Peker

(2011). Cyber victim and bullying scale: A study of validity and reliability. Computers & Education, 57, 2261-2271. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1016/j.compedu.2011.06.014

Cordero

Pamos

Seisdedos

(1999). NEO PI-R Inventario de Personalidad NEO Revisado y NEO-FFI. Inventario NEO reducido de 5 factores [NEO PI-R personality inventory revised NEO and NEO-FFI reduced inventory NEO 5-factor]. Madrid, Spain: Técnicos Especialistas Asociados.

Costa

McCrae

(1992). The revised NEO personality inventory (NEO-PI-R) and NEO five-factor inventory (NEO-FFI) professional manual. Odesa, FL: Psychological Assessment Resources.

10.

Del Rey

Elipe

Ortega-Ruiz

. (2012). Bullying y cyberbullying: Overlapping and predictive value of co-occurrence. Psicothema, 24, 608-613.

11.

Fenaughty

Harré

(2013). Factors associated with distressing electronic harassment and cyberbullying. Computers in Human Behavior, 29, 803-811. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1016/j.chb.2012.11.008

12.

Fernández-Berrocal

Extremera

Ramos

(2004). Validity and reliability of the Spanish modified version of the Trait Meta-Mood Scale. Psychological Report, 94, 751-755.

13.

Gámez-Guadix

Villa-George

Calvete

(2014). Psychometric properties of the Cyberbullying Questionnaire (CBQ) among Mexican adolescents. Violence and Victims, 29(2), 232-247.

14.

Garaigordobil

(2011). Prevalence and consequences of cyberbullying: A review. International Journal of Psychology and Psychological Therapy, 11, 233-254.

15.

Garaigordobil

(2015). Cyberbullying in adolescents and youth in the Basque Country: Prevalence of cybervictims, cyberaggressors, and cyberobservers. Journal of Youth Studies, 18, 569-582. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1080/13676261.2014.992324

16.

Garaigordobil

Martínez-Valderrey

(2014a). Cyberprogram 2.0. Un programa de intervención para la prevenir y reducir el ciberbullying [Cyberprogram 2.0. An intervention program to prevent and reduce cyberbullying]. Madrid, Spain: Pirámide.

17.

Garaigordobil

Martínez-Valderrey

(2014b). Effect of Cyberprogram 2.0 on reducing victimization and improving social competence in adolescence. Revista de Psicodidáctica / Journal of Psychodidactics, 19, 289-305.

18.

Garaigordobil

Martínez-Valderrey

(2015). Cyberprogram 2.0: Effects of the intervention on “face-to-face” bullying, cyberbullying, and empathy. Psicothema, 27, 45-51. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.7334/psicothema2014.78

19.

Hinduja

Patchin

(2010). Bullying, cyberbullying, and suicide. Archives of Suicide Research, 14, 206-221. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1080/13811118.2010.494133

20.

Kimsey

W. D.

Fuller

R. M.

(2003). Conflictalk: An instrument for measuring youth and adolescent conflict management message styles. Conflict Resolution Quarterly, 21, 69-78. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1002/crq.49

21.

Kowalski

R. M.

Giumetti

G. W.

Schroeder

A. N.

Lattanner

M. R.

(2014). Bullying in the digital age: A critical review and meta-analysis of cyberbullying research among youth. Psychological Bulletin, 140, 1073-1137. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1037/a0035618

22.

Kowalski

R. M.

Limber

(2007). Electronic bullying among middle school students. Journal of Adolescent Health, 41, 22-30. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1016/j.jadohealth.2007.08.017

23.

Lam

L. T.

(2013). The validation of the E-Victimisation Scale (E-VS) and the E-Bullying Scale (E-BS) for adolescents. Computers in Human Behavior, 29, 3-7. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1016/j.chb.2012.06.021

24.

Mehrabian

Epstein

(1972). A measure of emotional empathy. Journal of Personality, 40, 525-543. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1111/j.1467-6494.1972.tb00078.x

25.

Menesini

Nocentini

(2009). Cyberbullying definition and measurement: Some critical considerations. Journal of Psychology, 217, 230-232.

26.

Menesini

Nocentini

Calussi

(2011). The measurement of cyberbullying: Dimensional structure and relative item severity and discrimination. Cyberpsychology, Behavior, and Social Networking, 14, 267-274. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1089/cyber.2010.0002

27.

Navarro

A. M.

Peiró

Llácer

M. D.

Silva

(1993). EPC. Escala de problemas de conducta [Behavioral problems scale]. In Silva

Martorell

M. C.

(Eds.), Evaluación Infanto-Juvenil (pp. 31-81). Madrid, Spain: Material y Ediciones Psicotécnicas S.A. (MEPSA).

28.

Nocentini

Menesini

Calussi

(2009). Cyberbullying measurement: A comparison between different strategies. In Menesini

Smith

P. K.

Zukauskiene

(Eds.), Cyberbullying: Definition and measurement. Abstract Book (p. 18). Vilnius, Lithuania: Mykolas Romeris University Publishing Center.

29.

Rosenberg

(1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press.

30.

Salovey

Mayer

J. D.

Goldman

S. L.

Turvey

Palfai

T. P.

(1995). Emotional attention, clarity, and repair: Exploring emotional intelligence using the Trait Meta-Mood Scale. In Pennebaker

J. W.

(Ed.), Emotion, disclosure, & health (pp. 125-151). Washington, DC: American Psychological Association.

31.

Schultze-Krumbholz

Scheithauer

(2009). Measuring cyberbullying and cybervictimisation by using behavioral categories—The Berlin Cyberbullying-Cybervictimisation Questionnaire (BCCQ). In Menesini

Smith

P. K.

Zukauskiene

(Eds.), Cyberbullying: Definition and measurement. Abstract Book (p. 17). Vilnius, Lithuania: Mykolas Romeris University Publishing Center.

32.

Seisdedos

(1995). AD. Cuestionario de conductas antisociales-delictivas [Antisocial-delinquent behavior questionnaire]. Madrid, Spain: Ediciones Técnicos Especialistas Asociados (TEA).

33.

Smith

P. K.

Mahdavi

Carvalho

Fisher

Russell

Tippett

(2008). Cyberbullying: Its nature and impact in secondary school pupils. Journal of Child Psychology and Psychiatry, 49, 376-385. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1111/j.1469-7610.2007.01846.x

34.

Smith

P. K.

Mahdavi

Carvalho

Tippett

(2006). An investigation into cyberbullying, its forms, awareness and impact, and the relationship between age and gender in cyberbullying (Research Brief No. RBX03-X06). London, England: Department for Education Services (DfES).

35.

Soler

Paretilla

Kirchner

Forns

(2013). Impact of poly-victimization on mental health: The mediator and/or moderator sole of self-esteem. Journal of Interpersonal Violence, 28, 2695-2712. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1177/0886260513487989

36.

Stewart

R. W.

Drescher

C. F.

Maack

D. J.

Ebesutani

Young

(2014). The development and psychometric investigation of the Cyberbullying Scale. Journal of Interpersonal Violence, 29, 2218-2238. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1177/0886260513517552

37.

Tokunaga

R. S.

(2010). Following you home from school: A critical review and synthesis of research on cyberbullying victimization. Computers in Human Behavior, 26, 277-287. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1016/j.chb.2009.11.014

38.

Topcu

Erdur-Baker

(2010). The Revised Cyber Bullying Inventory (RCBI): Validity and reliability studies. Procedia Social and Behavioral Sciences, 5, 660-664. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1016/j.sbspro.2010.07.161

39.

Tynes

B. M.

Rose

Williams

(2010). The development and validation of the Online Victimization Scale for adolescents. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, 4, Article 2. Retrieved from http://cyberpsychology.eu/view.php?cisloclanku=2010112901&;;article=

40.

Wade

Beran

(2011). Cyberbullying: The new era of bullying. Canadian Journal of School Psychology, 26, 44-61. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1177/0829573510396318

41.

Walrave

Heirman

(2011). Cyberbullying: Predicting cybervictimization and cyberaggression. Children & Society, 25, 59-72. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1111/j.1099-0860.2009.00260.x