Validation of the European Cyberbullying Intervention Project Questionnaire for Colombian Adolescents

Abstract

Cyberbullying is the act of using unjustified aggression to harm or harass via digital devices. Currently regarded as a widespread problem, the phenomenon has attracted growing research interest in different measures of cyberbullying and the similarities and differences across countries and cultures. This article presents the Colombian validation of the European Cyberbullying Intervention Project Questionnaire (ECIPQ) involving 3,830 high school students (M = 13.9 years old, standard deviation = 1.61; 48.9 percent male), of which 1,931 were Colombian and 1,899 Spanish. Confirmatory factor analysis (CFA), content validation, and multigroup analysis were performed with each of the sample subgroups. The optimal fits and psychometric properties obtained confirm the robustness and suitability of the assessment instrument to jointly measure cyber-aggression and cyber-victimization. The results corroborated the theoretical construct and the two-dimensional and universal nature of cyberbullying. The multigroup analysis showed that cyberbullying dynamics are similar in both countries. The comparative analyses of prevalence revealed that Colombian students are less involved in cyberbullying. The results indicate the suitability of the instrument and the advantages of using such a tool to evaluate and guide psychoeducational interventions aimed at preventing cyberbullying in countries where few studies have been performed.

Introduction

The prevalent use of information and communications technologies (ICTs) has significantly transformed interpersonal relationships among adolescents.¹ Although ICTs have certain benefits, they have also given rise to a complex scenario of interactions that require new abilities and social skills to navigate cyberspace successfully.² Research has also shown that the use of ICTs has led to an increase in social problems, including cyberbullying;³ a phenomenon currently regarded as a major public health issue in schools^4,5 given its negative impact on the social and emotional development of children and adolescents.^1,6,7 It is estimated that around 20 percent of young people aged from 10 to 18 have been cyberbullies or cybervictims;^8,9 with puberty and adolescence increasing the risk of becoming involved in cyberbullying.¹⁰

Cyberbullying is an aggressive, intentional act or act of intimidation carried out using electronic media, which creates an imbalance of power between the bully and the victim.^11,12 Some authors regard this phenomenon as an indirect form of harassment, as it is conceived within the definitional framework of traditional bullying. Certain characteristics, however, differentiate cyberbullying from traditional bullying, such as the potential and, in some cases, frequent anonymity of the perpetrator,^13,14 and the duration of an aggressive act in cyberspace, where an image or other humiliating audiovisual material is freely accessible (is public) at any time.¹⁵ Insulting or threatening behavior through text messages or the Internet; spreading rumors about someone on social networks; extracting, disclosing, or publishing personal information; displaying or sending compromising photos of someone; excluding or being excluded from a group or chat; and online identity theft are all actions regarded as cyberbullying.¹⁶ Moreover, it is important to highlight that both bullying and cyberbullying constitute unjustified behavior that involves a certain degree of immorality.¹⁷

Although there have been notable advances in research on cyberbullying over the last decade, important questions remain with regard to the adequate assessment of the phenomenon. This may be due to cyberbullying's multiple manifestations, which render it difficult to develop and validate scales with optimal psychometric properties.¹⁸ A systematic review of 636 studies showed that most of the existing measurement instruments do not take into account the properties and theoretical and structural factors involved, and of the 44 recognized instruments, only 24 reported convergent validity.¹⁹ Other studies have shown that the scales used to measure cyberbullying do not have the same factorial structure, unlike traditional instruments for assessing bullying.²⁰ In developing an instrument to measure cyberbullying, Law et al.²¹ found that the items used to assess traditional bullying were clearly organized into two factors (victimization and aggression), while items related to cyberbullying were organized into just one. The same result was obtained by Menesini et al.,²² who reported a tendency for single factor grouping. In general, there are few instruments that comprehensively identify and measure all the factors related to cyberbullying involvement.^18,23 Indeed, it is still very common to find studies that measure only cyber-aggression^21,24 or cyber-victimization,²⁵ thus increasing the propensity of biased measures.²⁶

The European Cyberbullying Intervention Project Questionnaire (ECIPQ)¹⁸ was designed based on the studies of Dooley et al.^27,28 This rigorous measurement instrument is comparable to other international instruments, and it has been validated in six European countries with optimal psychometric results.^14–30 The ECIPQ includes new definitions of cyberbullying, reflects its various manifestations and recognizes the dynamic structure of the phenomenon by measuring its two principal dimensions: aggression and victimization. It also makes no distinction between digital devices (mobile phones and PCs) and includes the criteria of repetition and the imbalance of power as it assesses the dominion of technology in the aggression and the lack of security measures in victimization.¹⁸

A recent worldwide review of cyberbullying studies has shown that the articles published on the Web of Science are mainly studies from North America (57 percent) and Europe (28 percent), followed by Western Asia (8 percent) and Australia (4 percent), which, and denoting an uneven geographical distribution, highlights the scarce scientific participation of developing countries, such as those in Latin America.²⁹ The small number of studies conducted in these countries tend to report data on prevalence, primarily using instruments lacking rigorous psychometric properties. Other studies report on the type of harassment and the description and characterization of cyberbullying,^30,31 while very few examine the development or validation of instruments.^32–34 This situation, coupled with the scarcity of cross-cultural research, represents important limitations for advancing in the study of cyberbullying in regions such as Latin America, where violence is considered to be even more exacerbated.³⁵ The validation and use of a common instrument with known psychometric properties to compare the phenomenon of cyberbullying would allow us to get further insight into the universality of the phenomenon and examine differences and similarities between countries; additionally comparisons between two different geographical regions, which share a similar culture, history, and same language, are of particular interest as they open up new lines of research into cyberbullying beyond cross-cultural, comparative studies among European countries¹⁸ or between the United States and countries of Asia.³⁶

Owing to the limited availability of adequately validated instruments for Latin American countries and the need for internationally recognized instruments incorporating both cyber-victimization and cyber-aggression, the main objective of this study was to validate the ECIPQ¹⁸ scale in Colombia. The second objective was to compare the prevalence of involvement in cyberbullying in both countries (Colombia and Spain) focusing on the following roles: bullies, victims, bully-victims, and noninvolved students. Our hypothesis was that the instrument would show a two-factor structure with optimal psychometric properties and measurement homogeneity in the Spanish version, while Colombian students would be less involved in cyberbullying based on the assumption that technology is used to a lesser degree in the country.³⁷

Methodology

Participants

The overall sample comprised 3,830 adolescent students (48.9 percent male) aged 10 to 19 (M = 13.95, standard deviation [SD] = 1.61). The Colombian subgroup consisted of 1,931 students (46.9 percent male) of public and private schools in the city of San Juan de Pasto, southern Colombia, aged 10 − 19 (M = 14.9; SD = 1.89); while the Spanish subgroup consisted of 1,899 students (51 percent male) of public and private schools in Andalucia, southern Spain, aged 11 − 18 (M = 14.3; SD = 1.81).

Instrument

We used the ECIPQ,¹⁸ which comprises 22 items (11 for cyber-victimization and 11 for cyber-aggression). The ECIPQ uses a Likert-type scale with five response options ranging from 0 = never, 1 = once or twice, 2 = once or twice a month, 3 = about once a week, and 4 = more than one once a week. An example of an item for cyber-victimization is “Someone said nasty things to me or called me names using texts or online messages,” while “I spread rumours about someone on the Internet” is an example of a cyber-aggression item. The internal consistency of the original test is optimal: α_{cyber-victimization} = 0.97; α_{cyber-aggression} = 0.93; α_total = 0.96.

Procedure

We used a cross-sectional, ex post facto, retrospective, one group, one measurement research design.³⁸ To ensure ethical standards, we first obtained authorization from the school officials and subsequently sent informed consent forms to the students' parents and/or legal guardians. After receiving signed consent from the parents, we then visited the students who had been authorized to take part in the study and asked them to sign a consent form. Before administering the questionnaire to the students, we informed them that participation was anonymous and voluntary and explained the objective of the study. The average time taken to complete the questionnaire was 15 minutes. Convenience sampling was performed owing to accessibility.

The scale content was validated by a panel of six Colombian experts, who were given the Spanish version translated from English to Castilian by Ortega-Ruiz et al.³⁹ with the parallel back-translation procedure.⁴⁰ The panel assessed the compliance of criteria such as adequacy of vocabulary, conceptual clarity, and the consistency and relevance of each item. To this end, a 4-point scale was used where 1 = noncompliance, 2 = low level of compliance, 3 = moderate compliance, and 4 = high level of compliance. Finally, a pilot test was conducted with 60 students to assess their understanding of the items. Any words or terms in the original Spanish text that were not clear to the students were modified. For example, the item “Alguien me ha dicho palabras malsonantes o me ha insultado usando el e-mail o SMS” (Someone used bad words or insulted me using e-mail or SMS) was substituted for “Alguien me ha dicho groserías o insultado por internet (e-mail, redes sociales, llamadas o SMS)” (Someone made rude comments or insulted me on Internet [e-mail, social networks, calls or SMS]). The changes were then incorporated into the Colombian version (Appendix Table A1).

To establish the different roles of involvement, we followed the criteria established by the authors of the ECIPQ scale.¹⁸ For example, to determine the role of cybervictim we took into account subjects with scores ≥2 (once a month) in all the cyber-victimization items, and a score ≤1 (once or twice) in all of the cyber-aggression items. Involvement in the role of cyberbully was measured taking into account subjects with scores ≥2 (once a month) in any of the cyber-aggression items, and scores ≤1 (once or twice) in all the cyber-victimization items. The degree of involvement in the role of bully-victim was obtained with scores ≥2 (once a month) in at least one of the cyber-aggression items and in at least one of the cyber-victimization items. Finally, noninvolvement was measured using scores ≤1 (once or twice) in all cyber-aggression and cyber-victimization items.

Data analysis

A confirmatory factor analysis (CFA) was performed for the structural validation of the scale. Maximum likelihood (ML) estimation with robust correction⁴¹ and polychoric correlation were used, given the categorical nature of the variables.⁴² To assess the suitability of the instrument, we used the Satorra-Bentler scaled chi-square test⁴³ (χ²_S-B), chi-square divided by its degrees of freedom (χ²_S-B/df) (≤5 acceptable ≤3 optimal), the comparative fit index (CFI), and the non-normed fit index (NNFI) (whose values must be ≥0.95).^44,45 We also took into account the root mean square error approximation (RMSEA ≤0.05), the standardized root mean square residual (SRMR ≤0.08 acceptable, ≤0.05 optimal),⁴⁵ and the Akaike information criterion (AIC), which is used to compare models (the lowest value indicates the best model). The analysis was performed using the EQS 6.2 program.⁴⁶

A McDonald's Omega (Ω) test was performed to analyze the internal consistency of the instrument given that the variables were categorical and reflected the absence of multivariante normality.⁴⁷ The analysis was performed using the FACTOR 9.2 program.⁴⁸

To assess the degree of robustness of the factorial structure and hence the degree of generalization of the model in the two countries, a multigroup analysis was performed. This analysis requires a series of sequential comparisons using progressively restricted models starting with a Model 1 to test the configurational invariance, conferring the same factorial structure to the two subgroups. To analyze the measurement invariance, we proposed three models: Model 2, where the covariances in both sample groups were restricted; Model 3, where the factor loadings were made equal; and Model 4 in which the residuals were restricted.⁴⁹ After developing the models, the delta values (Δ) of the NNFI, CFI, RMSEA, and SRMR measures of fit were obtained using a variance of ≤0.01 as the cutoff point to accept the invariance hypothesis.⁵⁰ The chi-square difference test (Δχ²_S-B) was also performed, where nonsignificant differences indicate model invariance.⁵¹ This multigroup analysis was performed using the EQS 6.2 program.⁴⁶

To compare the differences between countries regarding the roles of involvement included in the questionnaire, we performed a chi-square test (χ²), taking into account the values of the adjusted standardized residuals greater than ±1.96 (95 percent confidence interval [CI]) and ±2.58 (99 percent CI).

The level of significance was 0.05.

Results

The validation of the ECIPQ¹⁸ content for Colombia, based on the assessment of the expert panel, showed an adequate degree of agreement (r_k = 0.81).

The CFA performed with the Colombian subsample indicated that the assumptions of multivariate normality were not met, as a Mardia coefficient value = 875.13 was obtained. The original two-factor structure was confirmed by the adequate fit indices (χ²_S-B = 644.97; χ²_S-B/(208) = 3.10; p < 0.001; NNFI = 0.97; CFI = 0.97; RMSEA = 0.047 (90 percent CI [0.043, 0.052]); SRMR = 0.080; AIC = 228.96) (Fig. 1 and Table 1). The total internal consistency and the consistency of each factor were optimal (Ω cyber-aggression = 0.94; Ω cyber-victimization = 0.91; Ω total = 0.95).

FIG. 1.

CFA of the adapted ECIPQ for Colombia (*p < 0.05). CFA, confirmatory factor analysis; ECIPQ, European Cyberbullying Intervention Project Questionnaire.

Table 1.

Polychoric Correlation Matrix: European Cyberbullying Intervention Project Questionnaire

Items	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22
1	1
2	0.77	1
3	0.57	0.57	1
4	0.36	0.40	0.45	1
5	0.37	0.39	0.44	0.80	1
6	0.33	0.40	0.40	0.44	0.55	1
7	0.40	0.43	0.51	0.46	0.49	0.60	1
8	0.32	0.39	0.43	0.41	0.38	0.42	0.67	1
9	0.39	0.38	0.42	0.48	0.41	0.37	0.66	0.64	1
10	0.36	0.39	0.37	0.30	0.30	0.29	0.40	0.44	0.48	1
11	0.59	0.62	0.48	0.36	0.37	0.41	0.58	0.42	0.48	0.40	1
12	0.58	0.50	0.43	0.32	0.32	0.36	0.46	0.40	0.45	0.39	0.52	1
13	0.52	0.47	0.42	0.29	0.36	0.27	0.44	0.44	0.41	0.34	0.51	0.80	1
14	0.39	0.42	0.62	0.35	0.35	0.48	0.54	0.51	0.57	0.42	0.45	0.62	0.65	1
15	0.38	0.35	0.45	0.38	0.35	0.48	0.48	0.44	0.32	0.20	0.29	0.45	0.52	0.64	1
16	0.19	0.34	0.45	0.39	0.41	0.53	0.52	0.55	0.41	0.37	0.33	0.51	0.50	0.67	0.80	1
17	0.27	0.35	0.38	0.36	0.32	0.47	0.42	0.48	0.40	0.38	0.35	0.35	0.46	0.59	0.58	0.71	1
18	0.37	0.43	0.49	0.47	0.43	0.55	0.69	0.73	0.70	0.55	0.56	0.55	0.58	0.71	0.56	0.71	0.69	1
19	0.34	0.42	0.49	0.42	0.40	0.51	0.58	0.72	0.60	0.49	0.48	0.48	0.53	0.61	0.53	0.65	0.62	0.79	1
20	0.29	0.36	0.41	0.31	0.36	0.40	0.47	0.51	0.62	0.44	0.44	0.51	0.48	0.59	0.51	0.66	0.53	0.78	0.74	1
21	0.25	0.34	0.34	0.23	0.28	0.35	0.42	0.37	0.40	0.37	0.40	0.49	0.47	0.56	0.48	0.54	0.42	0.56	0.51	0.57	1
22	0.33	0.43	0.41	0.28	0.33	0.42	0.44	0.60	0.57	0.43	0.60	0.52	0.60	0.52	0.45	0.56	0.46	0.70	0.66	0.65	0.56	1

All correlations with p < 0.01.

The CFA performed with the Spanish subsample confirmed the original two-factor structure (χ²_S-B = 563.07; χ²_S-B/(208) = 2.71; p < 0.001; NNFI = 0.96; CFI = 0.97; RMSEA = 0.031 (90 percent CI [0.028, 0.034]); SRMR = 0.079; AIC = 147.07). The total internal consistency and the internal consistency of each factor were also optimal (Ω cyber-aggression = 0.96, Ω cyber-victimization = 0.94, Ω total = 0.97).

The results of the multigroup analysis were within the established cutoff values (Table 2). In addition, the chi-square differences between models 1 and 2, 1 and 3, and models 1 and 4 were not significant. These results demonstrate the existence of invariance in the factorial structure of the scale, indicating an optimal degree of robustness.

Table 2.

Multigroup Analysis: Configuration and Measurement Invariance

Mod	χ² _S-B	df	NNFI	CFI	RMSEA	SRMR	Δχ² _S-B	Δp	Δdf	ΔNNFI	ΔCFI	ΔRMSEA	ΔSRMR
Mod 1	2,503.97	416	0.96	0.96	0.05	0.08
Mod 2	2,492.47	436	0.96	0.96	0.05	0.08	11.50	0.94 (ns)	20	0.00	0.00	0.00	0.00
Mod 3	2,483.36	417	0.96	0.96	0.05	0.08	20.61	0.97 (ns)	1	0.00	0.00	0.00	0.00
Mod 4	2,537.58	437	0.96	0.96	0.05	0.09	33.61	0.98 (ns)	21	0.00	0.00	0.00	0.01

CFI, comparative fit index; Mod 1, no restrictions; Mod 2, loaded factorial restrictions; Mod 3, covariance factorial restrictions; Mod 4, residual restriction; NNFI, non-normed fit index; ns, not significant; RMSEA, root mean square error approximation; SRMR, standardized root mean square residual.

The chi-square analysis of roles of involvement indicated a statistically significant and directly proportional relationship between the country and cyber-aggression, with the Spanish students being the most involved in this role χ²(1, 1,873) = 21.006; p = 0.000, and between the country and noninvolvement χ²(1, 1,901) = 7.062; p = 0.008, with the Colombian students being the least involved (Table 3).

Table 3.

Percentage of Involvement in Cyberbullying

	Involvement
Country	Cyber-victimization	Cyber-aggression	Cyberbully-victim	Not involved
Colombia	10.7% SR = 0.4	2.5% SR = −4.6	5.5% SR = −1.3	81.0%^** SR = 2.7
Spain	9.3% SR = −0.4	5.3%^*** SR = 4.6	6.4% SR = 1.3	78.6% SR = −2.7

p < 0.01; ^***p < 0.001.

SR, standardized residual.

Discussion

The aim of this study was to validate the ECIPQ scale for Colombia; an internationally recognized measurement instrument of proven psychometric quality.¹⁸ The analysis confirmed the original two-factor structure of the ECIPQ: cyber-aggression and cyber-victimization. Optimal values and fit indices, in addition to good internal consistency were also obtained. These results support the theoretical construct that cyberbullying consists primarily of cyber-aggression and cyber-victimization and that both dimensions are clearly defined as in traditional bullying.¹¹

The results for the configuration and measurement invariance obtained in the multigroup analysis between Colombia and Spain show that (a) the factorial structural of the scale presents good robustness, thus ensuring optimal and rigorous properties, as well as the added benefit of being able to jointly measure the two major dimensions of cyberbullying, cyber-aggression and cyber-victimization,^36,39,52 and (b) despite the differences found in the role of aggressor and no involvement, the dynamics of cyberbullying could be similar. This similarity could be attributed, among other things, to the shrinking technology gap.⁵³ Although this inference should be taken with caution, it could be a new line of research focused on comparing the cultural aspects and the use of new information technologies.

A more detailed analysis of the factorial model validated for Colombia regarding the high factor loadings and saturation in the items relating to rumors (gossip) and misuse of personal data (pictures and videos) as bully or victim suggest both the high tendency to engage intimidating practices using digital devices and the need for more education targeted at the prevention and management of the personal data that adolescents divulge in cyberspace.^54,55

The second hypothesis of this study was confirmed by comparing the prevalence of cyberbullying among Colombian and Spanish students. The results show that the former are less involved, particularly in the role of cyberbully. These results could be attributed to the lower use of information technologies by Colombian youth. Also may be due to specific values regarding attitudes, behaviors, and habits of Colombian adolescents, related to the collectivist and restrictive Colombian school culture,^54,56 characterized by respect for the rules of the institutions, conformity, and obedience.⁵⁷ On the contrary, Spanish school culture promotes individualism and self-assertion,⁵⁸ which leads them to greater use of social networks. It has been shown that a high use increases the risk of involvement in cyberbullying.⁵⁹

In conclusion, owing to its optimal psychometric properties and ability to measure cyber-victimization and cyber-aggression in a comprehensive manner, the ECIPQ is a valuable instrument for gaining greater knowledge of cyberbullying. It could also be of use in other comparative studies on the possible differences in the prevalence of cyberbullying and associated variables or that assess the results of interventions targeted at preventing and reducing cyberbullying.

This study has some limitations, such as the cross-sectional nature of the analysis and the potential social desirability bias common to self-reporting.⁶⁰ Moreover, although the overall sample was large, the Colombian subsample was not representative, which could limit the generalization of the model. As a future line of research, the sample could include other Latin American countries, which would allow for broader cross-cultural studies.

Footnotes

Acknowledgments

We would like to thank members from the schools of San Juan de Pasto, Colombia, and Andalusia, Spain, for participating in the study. This study was performed within the framework of the following projects: Project PRY040/14 funded by the Fundación Pública Andaluza Centro de Estudios Andaluces, Project EDU2013-44627-P funded by the Spanish National R&D Plan, and Project BIL/14/S2/163 funded by the Fundación Mapfre.

Author Disclosure Statement

No competing financial interests exist.

Appendix

Appendix Table A1.

European Cyberbullying Intervention Project Questionnaire for Colombia

Cyber-victimization
ECIPQ 1	Alguien me ha dicho groserías o insultado por internet (e-mail, redes sociales, llamadas o sms) [Someone said nasty things to me or called me names using texts or online messages]
ECIPQ 2	Alguien ha dicho a otros groserías sobre mí usando internet o sms (mensajes de celular) [Someone said nasty things about me to others either online or through text messages]
ECIPQ 3	Alguien me ha amenazado a través de internet o sms (mensajes de celular) [Someone threatened me through texts or online messages]
ECIPQ 4	Alguien ha pirateado mi cuenta de correo y ha sacado mi información personal (por ejemplo, a través de e-mail o red social) [Someone hacked into my account and stole personal information (e.g., through e-mail or social networking accounts)]
ECIPQ 5	Alguien ha pirateado mi cuenta y se ha hecho pasar por mí (a través de las redes sociales o e-mail) [Someone hacked into my account and pretended to be me (e.g., through instant messaging or social networking accounts)]
ECIPQ 6	Alguien ha creado una cuenta falsa en internet para hacerse pasar por mí (facebook, twitter, whatsapp, e-mail, otra) [Someone created a fake account, pretending to be me (e.g., on Facebook or MSN)]
ECIPQ 7	Alguien ha colgado información personal sobre mí en internet [Someone posted personal information about me online]
ECIPQ 8	Alguien ha colgado videos o fotos comprometedoras mías en internet [Someone posted embarrassing videos or pictures of me online]
ECIPQ 9	Alguien ha retocado fotos mías que yo había colgado en internet [Someone altered pictures or videos of me that I had posted online]
ECIPQ 10	He sido sacado (excluido) o ignorado de una red social o de chat [I was excluded or ignored by others in a social networking site or Internet chatroom]
ECIPQ 11	Alguien ha difundido chismes (rumores) sobre mí por internet [Someone spread rumors about me on the Internet]
Cyber-aggression
ECIPQ 12	He dicho groserías o insultado a alguien usando mensajes por internet o sms (mensajes por celular) [I said nasty things to someone or called them names using texts or online messages]
ECIPQ 13	He dicho groserías sobre alguien a otras personas en mensajes por internet o sms [I said nasty things about someone to other people either online or through text messages]
ECIPQ 14	He amenazado a alguien por internet o a través de mensajes de celular (sms) [I threatened someone through texts or online messages]
ECIPQ 15	He pirateado (hackeado) la cuenta de correo o perfil de alguien y he robado su información personal (e-mail, red social) [I hacked into someone's account and stole personal information (e.g., through e-mail or social networking accounts)]
ECIPQ 16	He pirateado la cuenta o perfil de alguien y me he hecho pasar por él/ella a través del chat, mensajes o correos en las redes sociales [I hacked into someone's account and pretended to be them (e.g., through instant messaging or social networking accounts)]
ECIPQ 17	He creado una cuenta falsa para hacerme pasar por otra persona (por ejemplo en facebook, twitter, chat, instagram u otra) [I created a fake account, pretending to be someone else (e.g., on Facebook or MSN)]
ECIPQ 18	He colgado información personal sobre alguien en internet (por ejemplo en redes sociales) [I posted personal information about someone online]
ECIPQ 19	He colgado videos o fotos comprometedoras de alguien en internet [I posted embarrassing videos or pictures of someone online]
ECIPQ 20	He retocado fotos o videos de alguien, que estaban colgados en internet [I altered pictures or videos of another person that had been posted online]
ECIPQ 21	He excluido (sacado) o ignorado a alguien de una red social o de chat [I excluded or ignored someone in a social networking site or Internet chatroom]
ECIPQ 22	He difundido rumores (chismes) sobre otras personas por internet [I spread rumors about someone on the Internet]

ECIPQ, European Cyberbullying Intervention Project Questionnaire.

References

Kubiszewski

, Fontaine

, Potard

, et al. Does cyberbullying overlap with school bullying when taking modality of involvement into account?. Computers in Human Behavior, 2015; 43:49–57.

Hymel

, Swearer

. Four decades of research on school bullying. An introduction. American Psychologist, 2015; 70:293–299.

Modecki KL Minchin

, Harbaugh

, et al. Bullying prevalence across contexts: a meta-analysis measuring cyber and traditional bullying. Journal of Adolescent Health, 2014; 55:602–611.

Aboujaoude

, Savage

, Starcevic

, et al. Cyberbullying: review of an old problem gone viral. Journal of Adolescent Health, 2015; 57:10–18.

Selkie

, Kota

, Chan

, et al. Cyberbullying, depression, and problem alcohol use in female college students: a multisite study. Cyberpsychology, Behavior, and Social Networking, 2015; 18:79–86.

Brewer

, Kerslake

. Cyberbullying, self-esteem, empathy and loneliness. Computers in Human Behavior, 2015; 48:255–260.

Casas

, Del Rey

, Ortega-Ruiz

. Bullying and cyberbullying: convergent and divergent predictor variables. Computers in Human Behavior, 2013; 29:580–587.

Hinduja

, Patchin

. Cyberbullying: neither an epidemic nor a rarity. European Journal of Developmental Psychology, 2012; 9:539–543.

Shin

, Ahn

. Factors affecting adolescents' involvement in cyberbullying: what divides the 20% from the 80%?. Cyberpsychology, Behavior, and Social Networking, 2015; 18:393–399.

10.

Kowalski

, Giumetti

, Schroeder

, et al. Bullying in the digital age: a critical review and meta-analysis of cyberbullying research among youth. Psychological Bulletin, 2014; 4:1073–1137.

11.

Olweus

. Cyberbullying: an overrated phenomenon?. European Journal of Developmental Psychology, 2012; 9:520–538.

12.

Smith

. The nature of cyberbullying and what we can do about it. Journal of Research in Special Educational Needs, 2015; 15:176–184.

13.

Baldry

, Farrington

, Sorrentino

. “Am I at risk of cyberbullying”? A narrative review and conceptual framework for research on risk of cyberbullying and cybervictimization: the risk and needs assessment approach. Aggression and Violent Behavior, 2015; 23:36–51.

14.

Wright

. Predictors of anonymous cyber aggression: the role of adolescents' beliefs about anonymity, aggression, and the permanency of digital content. Cyberpsychology, Behavior, and Social Networking, 2014; 17:431–438.

15.

Menesini

, Nocentini

, Palladino

, et al. Cyberbullying definition among adolescents: a comparison across six European countries. Cyberpsychology, Behavior, and Social Networking, 2012; 15:455–463.

16.

Perren

, Corcoran

, Cowie

, et al. Tackling cyberbullying: review of empirical evidence regarding successful responses by students, parents, and schools. International Journal of Conflict and Violence, 2012; 6:283–292.

17.

Ortega-Ruiz

, Zych

. Cyber-behavior and educational psychology: challenges and risks [in Spanish]. Psicología Educativa, 2016; 22:1–4.

18.

Del Rey

, Casas

, Ortega-Ruiz

, et al. Structural validation and cross-cultural robustness of the European Cyberbullying Intervention Project Questionnaire. Computers in Human Behavior, 2015; 50:141–147.

19.

Berne

, Frisén

, Schultze-Krumbholz

, et al. Cyberbullying assessment instruments: a systematic review. Aggression and Violent Behavior, 2013; 18:320–334.

20.

Schultze-Krumbholz

, Göbel

, Scheithauer

, et al. A comparison of classification approaches for cyberbullying and traditional bullying using data from six European countries. Journal of School Violence, 2015; 14:47–65.

21.

Law

, Shapka

, Hymel

, et al. The changing face of bullying: an empirical comparison between traditional and internet bullying and victimization. Computers in Human Behavior, 2012; 28:226–232.

22.

Menesini

, Nocentini

, Calussi

. The measurement of cyberbullying: dimensional structure and relative item severity and discrimination. Cyberpsychology, Behavior, and Social Networking, 2011; 5:267–274.

23.

Palladino

, Nocentini

, Menesini

. Psychometric properties of the Florence CyberBullying-CyberVictimization Scales. Cyberpsychology, Behavior, and Social Networking, 2015; 18:112–119.

24.

Calvete

, Orue

, Estévez

, et al. Cyberbullying in adolescents: modalities and aggressors' profile. Computers in Human Behavior, 2010; 26:1128–1135.

25.

Tynes

, Rose

, Williams

. The development and validation of the Online Victimization Scale for adolescents. Journal of Psychosocial Research on Cyberspace, 2010; 4:1–15.

26.

Dempsey

, Sulkowski

, Nichols

, et al. Differences between peer victimization in cyber and physical settings and associated psychosocial adjustment in early adolescence. Psychology in the Schools, 2009; 46:962–972.

27.

Dooley

, Pyżalski

, Cross

. Cyberbullying versus face-to-face bullying. Zeitschrift für Psychologie/Journal of Psychology, 2009; 217:182–188.

28.

Pyżalski

. From cyberbullying to electronic aggression: typology of the phenomenon. Emotional and Behavioural Difficulties, 2012; 17:305–317.

29.

Zych

, Ortega-Ruiz

, Del Rey

. Scientific research on bullying and cyberbullying: where have we been and where are we going. Aggression and Violent Behavior, 2015; 24:188–198.

30.

Del Río

, Bringue

, Sábada

, et al. Cyberbullying: a comparative analysis in students from Argentina, Brazil, Chile, Colombia, Mexico, Peru and Venezuela [in Spanish]. In: V Congrés Internacional Comunicació I Realitat. Barcelona; 2010:307–316.

31.

Mura

, Diamantini

. Cyberbullying among Colombian students: an exploratory investigation. European Journal of Investigation in Health, 2013; 3:249–256.

32.

Buelga

, Cava

, Musitu

. Validation of the adolescent victimization through mobile phone and internet scale [in Spanish]. Revista Panamericana De Salud Pública/Pan American Journal of Public Health, 2012; 32:36–42.

33.

Baquero

, Avendaño

PBL

. Design and psychometric analysis of an instrument to detect the presence of cyberbullying in a school context [in Spanish]. Psychology, Society, & Education, 2015; 7:213–226.

34.

Jiménez

, Castillo

, Cisternas

. Validation of the aggression among peers scale and virtual aggression Sub-scale with Chilean students [in Spanish]. Revista Latinoamericana de Ciencias Sociales, Niñez y Juventud, 2012; 10:825–840.

35.

Baek

, Bullock

. Cyberbullying: a cross-cultural perspective. Emotional and Behavioural Difficulties, 2014; 19:226–238.

36.

Barlett

, Gentile

, Anderson

, et al. Cross-cultural differences in cyberbullying behavior: a short-term longitudinal study. Journal of Cross-Cultural Psychology, 2014; 45:300–313.

37.

Said-Hung

. Vulnerable young people and mobile participation in Colombia: a study of the levels of citizen participation and appropriation among beneficiaries of social programs [in Spanish]. Innovar, 2014; 24:31–44.

38.

Montero

, León

. A guide for naming research studies in psychology. International Journal of Clinical and Health Psychology, 2007; 7:847–862.

39.

Ortega-Ruiz

, Del Rey

, Casas

. Assessing bullying and cyberbullying: Spanish validation of EBIPQ and ECIPQ [in Spanish]. Psicología Educativa, 2016; 22:71–79.

40.

Brislin

. (1986) The wording and translation of research instruments. In Lonner

, Berry

, eds. Field methods in cross-cultural research. Beverly Hills, CA: Sage, pp. 137–164.

41.

Bryant

, Satorra

. Principles and practice of scaled difference chi-square testing. Structural Equation Modeling: A Multidisciplinary Journal, 2012; 19:372–398.

42.

Jöreskog

. On the estimation of polychoric correlations and their asymptotic covariance matrix. Psychometrika, 1994; 59:381–389.

43.

Bentler

, Bonett

. Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 1980; 88:588–606.

44.

Satorra

, Bentler

. A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 2001; 66:507–514.

45.

, Bentler

. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 1999; 6:1–55.

46.

Bentler

. (2005) EQS structural equations program manual. Encino, CA: Multivariate Software.

47.

Elosua

, Zumbo

. Reliability coefficients for ordered categorical response scales [in Spanish]. Psicothema, 2008; 20:896–901.

48.

Lorenzo-Seva

, Ferrando

. FACTOR: a computer program to fit the exploratory factor analysis model. Behavioral Research Methods, Instruments and Computers, 2006; 38:88–91.

49.

Byrne

, Shavelson

, Muthén

. Testing for the equivalence of factor covariance and mean structures: the issue of partial measurement invariance. Psychological Bulletin, 1989; 105:456–466.

50.

Dimitrov

. Testing for factorial invariance in the context of construct validation. Measurement and Evaluation in Counseling and Development, 2010; 43:121–149.

51.

Bollen

. (1989) Structural equations with latent variables. New York: Wiley.

52.

Thomas

, Connor

, Scott

. Integrating traditional bullying and cyberbullying: challenges of definition and measurement in adolescents, a review. Educational Psychology Review, 2015; 27:135–152.

53.

International Telecommunication Union (ITU). (2015) Measuring the information society report 2015. Geneve: ITU.

54.

Almansa-Martínez

, Fonseca

, Castillo-Esparcia

. Social networks and young people. Comparative study of Facebook between Colombia and Spain. Comunicar, 2013; 20:127–135.

55.

Festl

, Scharkow

, Quandt

. The individual or the group: a multilevel analysis of cyberbullying in school classes. Human Communication Research, 2015; 41:535–556.

56.

Arango

, Bringué

, Sádaba

. Interactive generation in Colombia: teenagers before the internet, the cell phone, and the videogames [in Spanish]. Anagrama, 2010; 17:45–56.

57.

Lila

, Musitu

, Buelga

. Colombian and Spanish adolescents: differences, similarities and relationships between family socialization, self-esteem and values [in Spanish]. Revista Latinoamericana de Psicología, 2000; 32:301–319.

58.

Arnett

. Broad and narrow socialization: the family in the context of a cultural theory. Journal of Marriage and the Family, 1995; 57:617–628.

59.

Romera

, Herrera-López

, Casas

, et al. Multidimensional social competence, motivation, and cyberbullying: a cultural approach with Colombian and Spanish adolescents. Journal of Cross-Cultural Psychology. In press. DOI: 10.1177/0022022116687854.

60.

Stone

, Turkkan

, Bacharach

, et al. (1999) The science of self report: implications for research and practice. Mahwah, NJ: Lawrence Erlbawn Associates.