Cross-Cultural Validation of the Compulsive Internet Use Scale in Four Forms and Eight Languages

Abstract

The 14-item Compulsive Internet Use Scale (CIUS) is one of the most frequently internationally adapted psychometric instruments developed to assess generalized problematic Internet use. Multiple adaptations of this instrument have led to versions in different languages (e.g., Arabic and French), and different numbers of items (e.g., from 5 to 16 items instead of the original 14). However, to date, the CIUS has never been simultaneously compared and validated in several languages and different versions. Consequently, the present study tested the psychometric properties of four CIUS versions (i.e., CIUS-14, CIUS-9, CIUS-7, and CIUS-5) across eight languages (i.e., German, French, English, Finnish, Spanish, Italian, Polish, and Hungarian) to (a) examine their psychometric properties, and (b) test their measurement invariance. These analyses also identified the optimal versions of the CIUS. The data were collected via online surveys administered to 4,226 voluntary participants from 15 countries, aged at least 18 years, and recruited from academic environments. All brief versions of the CIUS in all eight languages were validated. Dimensional, configural, and metric invariance were established across all languages for the CIUS-5, CIUS-7, and CIUS-9, but the CIUS-5 and CIUS-7 were slightly more suitable because their model fitted the ordinal estimate better, while for cross-comparisons, the CIUS-9 was slightly better. The brief versions of the CIUS are therefore reliable and structurally stable instruments that can be used for cross-cultural research across adult populations.

Introduction

The Compulsive Internet Use Scale (CIUS^1,2) is one of the most used and rigorously validated scales internationally^3–5 to assess problematic Internet use (PIU). The CIUS has received extensive support regarding its reliability and validity from multiple studies.^6,7 Initial psychometric studies on the CIUS relied on confirmatory factor analyses (CFAs) to test its unidimensionality. The measurement invariance (MI) of the CIUS was initially established across times, gender, age, and PIU status.^1,2 Moreover, its construct, concurrent, criterion, and convergent validity was evidenced² through correlations with time spent online, experiencing PIU, and feeling addicted. Thus, it is one of the best psychometric instruments assessing PIU^7–9 in terms of its psychometric properties and consistency of findings across different samples.

The CIUS was developed a decade ago using substance dependence, pathological gambling, and obsessive-compulsive disorder (OCD) criteria,^2,10 and dominant behavioral addiction models.^11–13 The CIUS includes items corresponding to the Internet Gaming Disorder (IGD¹⁴) criteria.¹⁵ Thus, it is well positioned as a contemporary psychometric scale to assess Internet addiction (IA)^16,17 because of its ease and versatility of use, its stability and multiple validity in assessing CIU,^2,18 and its alignment with IGD.^14,19 Furthermore, the CIUS has fewer items compared with most other IA scales^7,9 (i.e., 14 items: CIUS-14⁵) and is thus time-saving in both clinical and epidemiological contexts where time allocated to assessment is limited, although it arguably remains relatively long for public health surveys.

The CIUS was initially validated in adult Internet users^1,2,20 and assessed: loss of control, preoccupation, withdrawal, coping, and conflict. Further developments led to two longer versions (CIUS-16²¹ and CIUS-17²²) and two shorter versions (CIUS-10²⁰ and CIUS-12²³) in adolescents (e.g., the longer versions, extended the CIUS to adapt it to this younger age group^9,24). The use of the CIUS to assess specific online applications has increased since 2000 (e.g., online video games^20,25,26). It has been adapted for two age groups (i.e., adolescents and adults). The original CIUS-14 was validated in several languages (e.g., Arabic,²⁷ French,²⁸ and German^29–31), but with psychometric adjustments. Furthermore, recent shorter versions presenting good psychometric properties include the CIUS-7 and CIUS-5 (translated into German³²) and the CIUS-9 (translated into French³³).

The initial Dutch CIUS-14² was validated with heavy Internet users through CFA with a predefined criterion of finding a one-factor solution, after a set of paired items with correlated errors was adjusted. It resulted in a good fit with standardized factor loadings ranging from 0.48 to 0.69, aligned to factor variances invariant over time. The Arabic CIUS-14²⁷ also reported a single-factor model using exploratory factor analysis (EFA) and CFA. After including the correlation of variance errors of paired items, the results provided an acceptable solution with satisfactory reliability (α = 0.78).

In the French CIUS-14 validation,²⁸ students and volunteers were surveyed, and EFA and CFA were performed. A solution with paired items comprising correlated errors supported a one-factor model. A short French version (i.e., CIUS-9)³³ was validated for high school students and a CFA demonstrated that the unidimensional model was adequate for both genders, and the measure displayed adequate internal consistency.

The first German CIUS-14²⁹ surveyed university students to test model fit using CFA and reported a unidimensional factor structure after correlating errors of paired items. The second German validation³⁰ used CFA with a general population who reported having spent at least one hour online for private purposes per typical working or weekend day (e.g., gaming); a single-factor solution with four correlated errors obtained good fit indices. Moreover, configural invariance (i.e., determining whether groups of heavy Internet users vs. nonheavy users have the same pattern of CIU) across sex, age, education, and Internet use was established. The third German validation³¹ tested the CIUS among adolescents with similar findings, after correlating the error variances of paired items.

Two new shortened German versions with a merged sample of adults were validated.³² The best performing items across samples were combined to form two short versions of the CIUS, and compared with the CIUS-14. Other versions include a Chinese,³⁴ Italian,³⁵ Persian,³⁶ and Japanese³⁶ CIUS-14. The Persian and Japanese adaptations showed three factors (i.e., absorption, difficulty in setting priorities, and mood regulation^36,37).

However, to the present authors' knowledge, the CIUS has not been psychometrically tested in different languages and formats for use in cross-cultural comparisons, a gap that the present study fills. More specifically, despite the good internal consistencies reported in validated adaptations, no studies have compared the averages of the scale in different languages directly. Group comparisons rely on an established MI of the scale, needed to examine the degree to which the CIUS assesses the same construct across linguistic groups.

The primary purpose was to examine eight different linguistic versions of the CIUS (i.e., German, French, English, Finnish, Spanish, Italian, Polish, and Hungarian) and to test four of the existing CIUS versions (i.e., CIUS-14, CIUS-9, CIUS-7, and CIUS-5). More specifically, the study examined the psychometric properties (i.e., reliability and factorial validity) of the four CIUS versions across all eight languages and tested the scale's MI across languages.

Materials and Methods

Participants and procedure

The Tech Use Disorders online survey study data, using a convenience sample recruited via announcements in several universities, were used.^38,39 This sampling strategy is acceptable and widely used for the MI.^40–43 Indeed, because the first aim was validation of the CIUS in different languages, participants needed to have direct experience of using the Internet, which was almost assured by recruiting a self-selected sample of university students. A total of 4,226 or 81.13 percent of the 5,209 participants completed all CIUS-14 items. Table 1 shows sociodemographic information across the eight language samples studied, and scores for the four CIUS versions.

Table 1.

Demographic Information, Compulsive Internet Use Scale Total Scores Across All Eight Adaptations of the Four Versions of the Compulsive Internet Use Scale, and Item Scores in All and Each Language Adaptation

	All	German	French	English	Finnish	Spanish	Italian	Polish	Hungarian
N	4,226	389	1,079	1,188	449	265	284	258	314
Women [N (%)]	2,931 (69.360)	263 (67.610)	833 (77.201)	756 (63.640)	308 (68.596)	186 (70.188)	191 (67.253)	187 (72.481)	207 (65.924)
Age [in years; M (SD)]	24.630 (8.837)	25.244 (6.568)	25.298 (10.125)	19.966 (4.540)	28.300 (9.06)	27.22 (11.88)	28.285 (9.138)	25.279 (6.965)	27.959 (9.185)
Total score [M (SD)]
CIUS-14	16.028 (10.714)	14.015 (9.525)	16.729 (10.625)	19.886 (11.477)	12.4 (8.927)	13.48 (10.37)	11.173 (8.193)	14.574 (8.855)	14.377 (10.248)
CIUS-9	8.121 (7.338)	6.708 (6.362)	7.752 (7.329)	12.150 (7.778)	7.675 (5.773)	8.464 (6.723)	5.104 (5.423)	8.278 (5.741)	8.173 (6.658)
CIUS-7	6.618 (5.942)	5.479 (5.192)	6.320 (5.967)	9.777 (6.239)	6.223 (4.668)	6.902 (5.478)	4.260 (4.540)	6.794 (4.727)	6.964 (5.422)
CIUS-5	5.101 (4.481)	4.256 (3.969)	4.943 (4.594)	7.291 (4.555)	4.984 (3.618)	5.54 (4.263)	3.363 (3.508)	5.347 (3.690)	5.313 (4.104)
Item scores CIUS-14 [M (SD)]
“Stop”	1.823 (1.256)	1.720 (1.158)	1.899 (1.258)	2.195 (1.235)	1.292 (1.08)	1.879 (1.314)	1.549 (1.281)	1.508 (1.148)	1.596 (1.222)
“Continue”	0.601 (1.063)	0.530 (1.014)	0.501 (0.999)	1.001 (1.253)	1.477 (1.079)	1.494 (1.34)	0.215 (0.591)	0.202 (0.610)	0.436 (0.924)
“Others stop”	0.825 (1.111)	0.456 (0.804)	0.683 (1.043)	1.176 (1.265)	0.706 (0.944)	0.981 (1.201)	0.570 (0.947)	0.795 (0.983)	0.745 (1.078)
“Prefer”	0.795 (0.999)	0.718 (0.922)	0.804 (1.011)	1.010 (1.088)	0.78 (0.944)	0.626 (0.925)	0.387 (0.712)	0.632 (0.832)	0.682 (0.949)
“Sleep”	1.100 (1.187)	0.961 (1.079)	1.082 (1.183)	1.503 (1.273)	0.978 (1.065)	0.774 (1.136)	0.511 (0.907)	0.895 (1.002)	0.933 (1.09)
“Think”	0.902 (1.031)	0.951 (1.003)	0.889 (1.032)	1.020 (1.130)	0.893 (0.969)	0.626 (0.905)	0.658 (0.917)	0.775 (0.884)	0.975 (0.956)
“Session”	1.075 (1.109)	0.992 (1.009)	1.054 (1.097)	1.399 (1.210)	0.871 (0.983)	0.589 (0.888)	0.669 (0.926)	0.988 (1.038)	1.115 (1.023)
“Use less”	1.275 (1.168)	1.052 (1.009)	1.297 (1.172)	1.512 (1.201)	1.147 (1.092)	1.208 (1.196)	0.954 (1.069)	1.128 (1.124)	1.274 (1.178)
“Less time”	0.797 (1.070)	0.558 (0.840)	0.782 (1.069)	1.206 (1.205)	0.367 (0.777)	0.774 (1.031)	0.570 (0.928)	0.566 (0.835)	0.646 (0.975)
“Rush”	0.751 (1.038)	0.466 (0.829)	0.919 (1.080)	0.966 (1.149)	0.278 (0.685)	0.762 (1.059)	0.534 (0.835)	0.86 (0.992)	0.481 (0.865)
“Obligations”	0.792 (1.052)	0.749 (0.954)	0.994 (1.152)	0.839 (1.083)	0.608 (0.903)	0.709 (1.092)	0.530 (0.835)	0.733 (0.914)	0.611 (0.993)
“Sad”	1.740 (1.243)	1.492 (1.143)	1.930 (1.300)	1.927 (1.267)	1.401 (1.13)	1.196 (1.203)	1.496 (1.104)	1.81 (1.115)	1.783 (1.166)
“Escape”	1.378 (1.262)	1.319 (1.171)	1.483 (1.317)	1.636 (1.315)	0.949 (1.073)	1.004 (1.181)	1.078 (1.028)	1.434 (1.173)	1.236 (1.229)
“Restless”	1.060 (1.172)	0.835 (1.035)	1.104 (1.143)	1.474 (1.336)	0.673 (0.872)	0.936 (1.062)	0.784 (1.007)	0.961 (1.001)	0.608 (0.967)

CIUS, Compulsive Internet Use Scale; M, mean; N, sample size; SD, standard deviation.

Measures

Sections of the survey analyzed were sociodemographics (age and gender) and the CIUS-14 adapted from English into other languages (except three versions that had previously been validated^2,28,30). The translation/back-translation method⁴⁴ was used (Appendix A1).

The CIUS-14 comprises 14 items rated from 0 “never” to 4 “very often.” Scores range from 0 to 56, with higher scores referring to greater PIU severity. The original CIUS showed adequate factorial, content, and concurrent validity and good reliability (Cronbach's α between 0.89 and 0.90²). The three shortened CIUS versions used in the present study are the CIUS-9 (i.e., α = 0.85³³; score range 0–36), CIUS-7 (i.e., α = 0.82³²; score range 0–28), and CIUS-5 (i.e., α = 0.77³²; score range 0–20). The respective items of these scales are presented in Table 2. The correlation matrix of the 14 CIUS items across all languages is presented in Table 3. Factor loadings of all items across all languages for each of the CIUS versions are presented in Table 4.

Table 2.

Respective Items Included in Each Version of the Compulsive Internet Use Scale

CIUS versions
	CIUS-9	CIUS-7	CIUS-5
(1) “Do you find it difficult to stop using the Internet when you are online?”	X	X	X
(2) “Do you continue to use the Internet despite your intention to stop?”
(3) “Do others (e.g., partner, children, and parents) say you should use the Internet less?”	X	X	X
(4) “Do you prefer to use the Internet instead of spending time with others (e.g., partner, children, and parents)?”	X
(5) “Are you short of sleep because of the Internet”	X	X	X
(6) “Do you think about the Internet, even when not online”
(7) “Do you look forward to your next Internet session?”	X	X
(8) “Do you think you should use the Internet less often?”
(9) “Have you unsuccessfully tried to spend less time on the Internet?”	X	X
(10) “Do you rush through your (home) work to go on the Internet?”
(11) “Do you neglect your daily obligations (work, school, or family life) because you prefer to go on the Internet?”	X	X	X
(12) “Do you go on the Internet when you are feeling down”	X	X	X
(13) “Do you use the Internet to escape from your sorrows or get relief from negative feelings?”
(14) “Do you feel restless, frustrated, or irritated when you cannot use the Internet”	X

Note: Instructions were “How often …”; “Xs refer to items that were included.

Table 3.

Correlation Matrix of the 14 Compulsive Internet Use Scale Items Across All Language Adaptations (i.e., Overall Sample)

	1	2	3	4	5	6	7	8	9	10	11	12	13	14
1	—
2	0.71^***	—
3	−0.46^***	0.49^***	—
4	0.45^***	0.49^***	0.46^***	—
5	0.47^***	0.54^***	0.44^***	0.51^***	—
6	0.42^***	0.45^***	0.38^***	0.49^***	0.49^***	—
7	0.45^***	0.46^***	0.40^***	0.53^***	0.49^***	0.66^***	—
8	0.47^***	0.54^***	0.55^***	0.42^***	0.50^***	0.42^***	0.44^***	—
9	0.52^***	0.58^***	0.51^***	0.46^***	0.54^***	0.43^***	0.48^***	0.67^***	—
10	0.44^***	0.45^***	0.44^***	0.50^***	0.51^***	0.50^***	0.55^***	0.46^***	0.53^***	—
11	0.42^***	0.49^***	0.39^***	0.51^***	0.54^***	0.45^***	0.45^***	0.47^***	0.51^***	0.59^***	—
12	0.44^***	0.46^***	0.35^***	0.44^***	0.47^***	0.46^***	0.50^***	0.43^***	0.44^***	0.50^***	0.48^***	—
13	0.44^***	0.45^***	0.37^***	0.46^***	0.47^***	0.44^***	0.52^***	0.43^***	0.48^***	0.51^***	0.47^***	0.82^***	—
14	0.41^***	0.40^***	0.36^***	0.43^***	0.41^***	0.47^***	0.51^***	0.35^***	0.43^***	0.50^***	0.43^***	0.48^***	0.48^***	—

***

p < 0.001, p values false discovery rate corrected (BH correction⁵⁵) for multiple comparison.

Table 4.

Factor Loadings of Each [Item] Across All Adaptations for Each of the Compulsive Internet Use Scale Versions

	Factor loadings
Item	CIUS-14	CIUS-9	CIUS-7	CIUS-5
(1)	0.72^***	0.67^***	0.67^***	0.66^***
(2)	0.76^***
(3)	0.61^***	0.62^***	0.62^***	0.61^***
(4)	0.66^***	0.70^***
(5)	0.69^***	0.72^***	0.73^***	0.74^***
(6)	0.68^***
(7)	0.71^***	0.71^***	0.68^***
(8)	0.69^***
(9)	0.74^***	0.72^***	0.74^***
(10)	0.70^***
(11)	0.67^***	0.69^***	0.69^***	0.70^***
(12)	0.83^***	0.67^***	0.66^***	0.65^***
(13)	0.84^***
(14)	0.61^***	0.64^***

***

p < 0.001, all are standardized loadings.

Data analytic strategy

MI testing was carried out to determine if and to what extent the four CIUS versions were psychometrically valid and equivalent across eight languages. The procedure comprised a series of increasingly constrained multigroup confirmatory factor analyses (MGCFAs) to establish whether specific levels of the latent mean structure of the CIUS remained stable across multiple groups.

More specifically, the procedure comprised the following steps: (a) individual CFAs were computed for each language to test model fit; (b) a set of constrained and planned models was implemented for each validated version of the investigated measure; (c) a test of configural invariance was performed, which determines whether the number of factors and their respective items were the same across languages; (d) a test of metric invariance was performed, which estimated whether factor loadings were equivalent across languages, meaning that participants understand and respond to items in the same way across languages; (e) a test of scalar invariance was performed, which investigated if group differences in factor means are unbiased,⁴⁵ meaning latent scores can be compared across languages; (f) a test of strict invariance was performed, which estimated whether observed items had the same residuals, meaning that items had the same measurement error terms across languages; and (g) an additional model of strict invariance and equally constrained means was performed, which tested if the entire mean structure was invariant. If supported, this would suggest that the means of the latent and observed variables are invariant across languages.

After these models were estimated, comparison tests were undertaken to determine if reliable differences existed between models.⁴⁶ The difference tests conducted for each CIUS short version were used to establish whether languages varied between one another at that specific level (and if they were comparable).

Because all CIUS items are assessed on ordinal scales, analyses were conducted using RStudio (i.e., a package manager that organizes and centralizes R packages) Version 0.99.89^47,48 using the Lavaan,⁴⁹ Psych,⁵⁰ and semTools⁵¹ for assessing ordinal data within a CFA framework.⁵² Model fit estimations utilized diagonally weighted least squares scale-shifted (DWLSSS) and are recommended over maximum-likelihood robust analyses.⁵³ Correlation matrices were polychoric, and the weighted root mean-square residual (WRMR) was included as an additional measure of model fit due to its suitability for ordinal estimates.⁵⁴

The CFAs applied cutoff values for fit indices as follows: Comparative Fit Index (CFI) >0.93 adequate and >0.95 good, Tucker–Lewis Index (TLI) >0.93 adequate and >0.95 good, root mean square error of approximation (RMSEA) <0.08, pclose >0.05, SRMR <0.08, and WRMR <0.9.^55–59 Although reported here for completeness, χ² statistics were not used to assess model fit because it artificially inflates with increasing sample size.⁵⁴

Recommendations for MI⁶⁰ were as follows: nested models were assessed using differences (Δ) in CFI, RMSEA, and SRMR, with respective cutoffs indicating reliable differences of ≤0.01, ≤0.015, and ≤0.03 for metric invariance, and ≤0.01 for ΔSRMR in scalar invariance.⁶¹ A series of χ² tests were conducted between models (Satorra–Bentler χ² difference test), but these tests can provide unreliable estimates when large sample sizes are present. MGCFAs were subsequently conducted only on the short CIUS versions.

Results

CFA in the four CIUS versions and eight adaptations

Individual CFAs were performed for both the overall sample, individually for each language, and for each CIUS version. To maximize statistical power and because the data were assumed to be missing at random, pairwise deletion was used. The correlation matrix of all items and factor loadings for the CIUS versions across all languages of the overall sample are shown in Tables 3 and 4.

CFAs in the CIUS-14 (Table 5) provided a poor global overall model fit. However, the single factor solution of the short versions had adequate fit in all languages. The CIUS-5 (Table 6) initially returned marginally better-fit statistics than the CIUS-7 across all languages, most notably via lower WRMR rates. However, the inspection of both models demonstrated parity of fit.

Table 5.

Confirmatory Factor Analyses Conducted for Each Language Adaption of the Original Compulsive Internet Use Scale-14 and Short Compulsive Internet Use Scale-9

Version	Model	α	df	χ ²	p	CFI	TLI	RMSEA	RMSEA 90% CI	p_close	SRMR	WRMR
CIUS-14	All languages	0.93	77	7,213.370	<0.001	0.916	0.901	0.148	0.145–0.151	<0.001	0.076	5.942
	German	0.93	77	519.460	<0.001	0.932	0.920	0.122	0.112–0.132	<0.001	0.084	1.710
	French	0.92	77	2,118.691	<0.001	0.918	0.891	0.157	0.151–.163	<0.001	0.093	3.349
	English	0.93	77	2,075.905	<0.001	0.931	0.918	0.148	0.142–0.153	<0.001	0.074	3.228
	Finnish	0.92	77	632.456	<0.001	0.915	0.900	0.127	0.118–0.136	<0.001	0.084	1.847
	Spanish	0.99	77	422.179	<0.001	0.916	0.901	0.130	0.118–0.143	<0.001	0.105	1.644
	Italian	0.90	77	252.882	<0.001	0.907	0.891	0.090	0.078–0.102	<0.001	0.079	1.176
	Polish	0.90	77	351.278	<0.001	0.911	0.895	0.118	0.105–0.130	<0.001	0.092	1.487
	Hungarian	0.93	77	387.082	<0.001	0.945	0.935	0.113	0.102–0.125	<0.001	0.074	1.445
CIUS-9	All languages	0.88	27	390	<0.001	0.987	0.982	0.056	0.051–0.061	0.015	0.031	1.769
	German	0.88	27	53	= 0.002	0.985	0.980	0.050	0.030–0.070	= 0.458	0.046	0.672
	French	0.87	27	122	<0.001	0.983	0.977	0.057	0.047–0.068	= 0.111	0.039	1.025
	English	0.89	27	124	<0.001	0.990	0.987	0.055	0.045–0.064	= 0.194	0.030	1.007
	Finnish	0.88	27	58.544	<0.001	0.986	0.981	0.051	0.033–0.069	= 0.434	0.045	0.732
	Spanish	0.89	27	50.086	= 0.004	0.985	0.979	0.057	0.031–0.081	= 0.298	0.052	0.662
	Italian	0.84	27	42	<0.001	0.976	0.968	0.045	0.014–0.070	= 0.603	0.061	0.642
	Polish	0.84	27	41	= 0.044	0.984	0.979	0.044	0.008–0.071	= 0.602	0.051	0.613
	Hungarian	0.89	27	77	<0.001	0.976	0.967	0.077	0.057–0.098	= 0.014	0.055	0.819

CFI, Comparative Fit Index; CI, confidence interval; p_close, provides a one-sided test of the null hypothesis that the RMSEA is equal to 0.05 in the population; RMSEA, root mean square error of approximation; SRMR, standardized root mean square residual; TLI, Tucker–Lewis index; WRMR, weighted root mean-square residual.

Table 6.

Confirmatory Factor Analyses Conducted for Each Language Adaption of the Compulsive Internet Use Scale-7 and Compulsive Internet Use Scale-5

Version	Model	α	df	χ ²	P	CFI	TLI	RMSEA	RMSEA 90% CI	p_close	SRMR	WRMR
CIUS-7	All languages	0.86	14	194	<0.000	0.991	0.986	0.055	0.048–0.062	= 0.102	0.027	1.516
	German	0.86	14	19	= 0.151	0.996	0.994	0.031	0.000–0.062	= 0.915	0.030	0.476
	French	0.84	14	52	<0.000	0.990	0.985	0.052	0.038–0.067	= 0.399	0.031	0.808
	English	0.86	14	51	<0.000	0.994	0.992	0.047	0.034–0.061	= 0.061	0.024	0.783
	Finnish	0.85	14	14.798	= 0.216	0.997	0.996	0.025	0.000–0.055	= 0.909	0.031	0.481
	Spanish	0.87	14	20.718	= 0.109	0.994	0.991	0.043	0.000–0.079	= 0.585	0.040	0.505
	Italian	0.81	14	22	= 0.083	0.985	0.977	0.044	0.000–0.078	= 0.565	0.054	0.563
	Polish	0.80	14	17	= 0.240	0.994	0.991	0.030	0.000–0.071	= 0.746	0.045	0.482
	Hungarian	0.86	14	44	<0.000	0.979	0.969	0.083	0.055–0.111	= 0.026	0.049	0.748
CIUS-5	All languages	0.80	5	78	<0.000	0.993	0.985	0.058	0.047–0.070	= 0.095	0.024	1.243
	German	0.81	5	6	= 0.280	0.998	0.997	0.026	0.000–0.079	= 0.715	0.023	0.317
	French	0.79	5	35	<0.000	0.987	0.973	0.075	0.053–0.100	= 0.032	0.036	0.841
	English	0.80	5	22	<0.000	0.994	0.988	0.053	0.032–0.077	= 0.361	0.024	0.662
	Finnish	0.79	5	1.023	= 0.961	1.00	1.00	0.000	0.000–0.000	= 0.997	0.009	0.140
	Spanish	0.82	5	10.640	= 0.059	0.992	0.984	0.065	0.000–0.120	= 0.267	0.042	0.465
	Italian	0.74	5	7	= 0.213	0.992	0.983	0.039	0.000–0.097	= 0.549	0.041	0.410
	Polish	0.75	5	9	= 0.112	0.990	0.980	0.055	0.000–0.113	= 0.372	0.041	0.431
	Hungarian	0.81	5	7	= 0.249	0.998	0.996	0.032	0.000–0.090	= 0.620	0.026	0.359

MI in the four CIUS versions and eight adaptations

Configural invariance was supported (Table 7), and all Δ fit-indices returned below the prespecified cutoff values between the configural and metric model, supporting metric invariance. However, the ΔCFI and ΔRMSEA between the metric and scalar model exceeded cutoff thresholds, indicating a lack of scalar invariance. Subsequent levels of MI were not estimated due to insufficient evidence to support scalar invariance. Thus, the factor structure and loading strengths were invariant for the CIUS-9, CIUS-7, and CIUS-5 across the eight languages. However, there was no support for invariance of latent factor means across these languages in any version of the scale.

Table 7.

Measurement Invariance Procedure Conducted Between Compulsive Internet Use Scale-9, Compulsive Internet Use Scale-7, and Compulsive Internet Use Scale-5

Invariance	df	χ²	p	CFI	TLI	RMSEA	RMSEA 90% CI	p_close	SRMR	WRMR	Δχ²	Δdf	ΔRMSEA	ΔCFI	ΔSRMR
CIUS-9
Configural^***	216	550.1	<0.001	0.986	0.981	0.054	0.049–0.060	= 0.109	0.042	2.225
Configural versus metric											42	56	0.007	0.001	0.009
Metric^***	272	592.1	<0.001	0.987	0.986	0.047	0.042–0.052	= 0.805	0.051	2.756
Metric versus scalar											1,011.9	182	0.022	0.035	0.002
Scalar	454	1,604	<0.001	0.952	0.969	0.069	0.066–0.073	<0.000	0.049	4.558
CIUS-7
Configural^*	112	238.3	<0.001	0.992	0.989	0.046	0.038–0.054	= 0.768	0.034	1.757
Configural versus metric^*											108.9	42	0.004	0.004	0.013
Metric	154	347.2	<0.001	0.988	0.987	0.050	0.043–0.057	= 0.486	0.047	2.529
Metric versus scalar^***											902.8	140	0.029	0.044	0.001
Scalar	294	1,250	<0.001	0.944	0.968	0.079	0.074–0.083	<0.000	0.046	4.728
CIUS-5
Configural^*	40	96.44	<0.001	0.994	0.987	0.052	0.039–0.065	= 0.394	0.029	1.401
Configural versus metric^*											81.9	28	0.004	0.006	0.016
Metric	68	178.4	<0.001	0.986	0.985	0.055	0.046–0.065	= 0.174	0.044	2.227
Metric versus scalar^***											756.0	98	0.038	0.075	0.002
Scalar	166	934.4	<0.001	0.912	0.958	0.094	0.088–0.100	<0.001	0.047	4.987

Note: Satorra–Bentler Δχ² tests: configural versus metric: χ²(4.758) = 9.68, p = 0.074, metric versus scalar: χ²(10.132) = 40.39, p < 0.001. Satorra–Bentler Δχ² tests: configural versus metric: χ²(3.713) = 10.59, p = 0.03, metric versus scalar: χ²(8.475) = 35.75, p < 0.001. Satorra–Bentler Δχ² tests: configural versus metric: χ²(2.491) = 7.263, p = 0.04, metric versus scalar: χ²(6.687) = 30.3, p < 0.001, ^*p < 0.05, ^***p < 0.001.

Discussion

The present study examined the psychometric properties and MI of four CIUS versions across eight languages and determined the optimal version for future cross-cultural studies.

Findings suggest that the short CIUS is robust for screening CIU in adults in the various languages tested. However, the CIUS-14 psychometric properties were harder to replicate^2,27–32,34 without pairing several items via error variance correlations. A potential explanation for these findings is that the items comprising the original CIUS were created based on the diagnostic criteria for different disorders (e.g., OCD, substance use, and gambling disorders), and therefore do not necessarily load on a single latent construct.

During the past decade, the CIUS^1,2 has been widely used in Western^{1,2,6,20–23,25,26,28–33,35,62–64} and Eastern^27,34,36,65 cultures, usually translated into different languages, and validated with adequate and stable psychometric properties. However, it is still considered long for public health surveys.⁶⁶ Instruments usually improve psychological assessment by decreasing their length because it reduces variance, and minimizes burden and fatigue, while increasing response rates and representativeness.⁶⁷ Recently, the CIUS-8 has been validated across the languages used in Switzerland,⁶⁸ which tested MI within one sample to validate cross-cultural comparisons.⁶⁹

The present study adds to the literature by establishing language invariance, and found that the short CIUS versions tested had robust psychometric properties for eight languages. The CIUS-5 was shown to be slightly better than the CIUS-7 because its model fitted the ordinal estimate better, but both models showed parity of fit. Furthermore, the WRMR index itself has received limited support to date,⁷⁰ and thus, its cutoff threshold remains preliminary and needs to be further established. Some researchers reported that this index did not behave as hypothesized,⁷¹ warranting further study and caution to not rely heavily on it when selecting a model.

Therefore, the use of the shortened CIUS is recommended, as it assesses a narrower PIU construct, characterized by loss of control, conflict or negative consequences, mood regulation, and preoccupation. As a consequence, the shortened CIUS can be used to assess PIU as an OCD, impulse-control disorder, addictive disorder, or dysfunctional coping behavior, which is useful given the ongoing debate regarding the exact nature of the condition.^72,73 However, an advantage of the CIUS-5 in comparison to the CIUS-7 is that the CIUS-5 does not comprise the item concerning “preoccupation” (item 7), which is related with both problematic and non-problematic Internet use.^74,75

Another novel aspect was that no previous studies have adapted and validated the original CIUS version into Spanish, Finnish, Italian, Polish, and Hungarian languages. Furthermore, using a shorter CIUS version with fewer items appears to avoid the need to pair items to support good model fit via CFA. Moreover, regarding the shortened CIUS reliability, Cronbach's alphas were good (i.e., α from 0.74 [CIUS-5 in Italian] to 0.89 [CIUS-9 in English, Spanish, and Hungarian]).^76,77 These findings are comparable with previous CIUS validation studies examining different languages and versions.^2,27–35 Compared with the previous shortened CIUS validations,^32,33 the reliabilities achieved were higher (e.g., German CIUS-7 α was 0.82³² compared with 0.86 [i.e., German in this study]).

Regarding the MI, the CIUS factor structure appears equivalent across languages, which was untested to date. The findings provide great confidence in future studies assessing cross-cultural differences regarding CIU in the languages tested because comparison data will be more reliable, similar to previous research conducted with the Internet Addiction Test (IAT) in Chinese, Japanese, and Malaysian.⁷⁸ In line with the recommendation made by the World Health Organization,^79(p9) the current study provided improved screening tools for cross-cultural epidemiological studies on CIU.^38,39 Future cross-cultural studies could use these brief forms, preferably the CIUS-9.

All linguistic adaptations tested fitted the proposed model well, and configural and metric models supported the invariance of these eight languages through the equivalence of the one-factor solution and factor loading of all items contained in the three shortened versions. However, scalar and strict invariance with equally constrained means has not been performed due to exceeding the threshold proposed in the literature.^60,61 In relation to strict factorial invariance, it is usually difficult to achieve, and very stringent tests of equivalence for previous studies are still debated.^57,76,77 The IAT has not achieved this either.⁷⁶ Therefore, findings only affected the invariance of latent factor means, and potential equal residuals in the items across these languages (also reported for the IAT's MI⁷⁸).

This study has limitations derived from cross-cultural data collection difficulties.

First, the sample size was reduced by 983 participants who did not complete the survey for several reasons. However, the final sample size was large enough with regard to the study's objectives; all data collection was carried out simultaneously in 2015, using similar strategies to guarantee procedural standardization for collecting reliable data. The sampling strategy did not use a probabilistic method (i.e., the samples were not nationally representative, nor representative of university students or university employees). However, the sample used was adequate for the purpose of the study, because the aim was not to identify the prevalence of PIU (a representative sample would have been necessary for that), but was designed to test the psychometric properties of the CIUS in different languages. Future studies should therefore collect data outside this environment via a randomized sampling strategy to achieve greater external validity.

Second, the study was based on self-reports and open to well-known biases. Third, the MI^80,81 did not support the invariance of latent factor means and residuals of the items across the eight languages tested in all versions. Nevertheless, the invariance of the brief scales established good comparative factor structure and loadings for use in future cross-cultural studies, similar to what has previously been found in studies testing MI with convenience samples.^82,83 Furthermore, additional validation of the short CIUS versions is necessary. CFA and MI are not sufficient to establish psychometric properties. External criteria based on clinical interviews are the best way to establish validity and provide recommendations regarding cutoffs, which was not possible in the present study.

Moreover, convergent validity based on measures of functional impairment would be meaningful, and reliability should be tested via test–retest methods. Finally, another way to validate these versions is to utilize their predictive power on major outcomes (i.e., PIU associated with specific psychopathological symptoms and functional impairment).

In conclusion, the theoretical robustness and psychometric validity of the shortened 5-item, 7-item, and 9-item CIUS versions across eight languages (German, French, English, Finnish, Spanish, Italian, Polish, and Hungarian) have been demonstrated. The present study supports the brief CIUS instruments as being valid and time-saving in screening for PIU.

Footnotes

Acknowledgments

First, O.L-F. acknowledges the support of the European Commission and the Marie Sklodowska-Curie actions (Funding scheme: MC-IEF—Intra-European Fellowships), and Nottingham Trent University (Funding scheme: QR Funding and Staff Development for a competitive Kickstarter research project). Second, the support of Gert-Jan Meerkerk for his permission adapting the CIUS and his suggestions, and to Bettina Besser and Gallus Bischof for sharing the German shortened CIUS (i.e., CIUS-5, CIUS-7). Third, to Kim Hoffman by the “International Center for Advanced Research and Applied Science” (INCAAS) for supporting data collection at “Universidad Antonio Ruiz de Montoya” (Peru), supported by Carmen Margarita Ilizarbe Pizarro. Finally, O.K. acknowledges the support of the New National Excellence Program of the Ministry of Human Capacities. The present study was supported, above all, by the European Commission (“Tech Use Disorders”; FP7-PEOPLE-2013- IEF-627999) through a Marie Curie postdoctoral grant awarded O.L-F. (supervisor: J.B.). Second, by the Psychology Department QR Funding at Nottingham Trent University, through a Kickstarter bid grant (2017) awarded to O.L-F. to develop studies on “Internet and mobile phone addiction: Cross-cultural epidemiological studies.” Third, O.K. acknowledges the support of the Hungarian part of the study was supported by the Hungarian Scientific Research Fund (grant No. K111938; KKP126835) and the ÚNKP-17-4 New National Excellence Program of the Ministry of Human Capacities.

Authors' Contributions

O.L.-F. wrote the first draft, with C.D. and L.J., who supported the statistical analysis and initial interpretation of the data. J.B., D.J.K., H.M.P., M.D.G., and O.L.-F. supported the review of the second draft. J.B., A.S., M.D.G., and O.L.-F. also supported the review of the third draft. All authors reviewed the article adding comments and suggestions and oversaw the second and third drafts. All coauthors contributed to adapting the short version of the CIUS into their languages, collecting data in their respective countries, and revising the subsequent versions until the final write-up of the article.

Author Disclosure Statement

No competing financial interests exist.

Appendix A1

References

Meerkerk

, Eijnden

, Garretsen

. Predicting compulsive Internet use: it's all about sex! CyberPsychology &. Behavior, 2006; 9:95–103.

Meerkerk

, Van Den Eijnden

, Vermulst

, et al. The Compulsive Internet Use Scale (CIUS): some psychometric properties. CyberPsychology & Behavior, 2009; 12:1–6.

Young

. (1998) Caught in the net: how to recognize the signs of internet addiction—and a winning strategy for recovery. New York, NY: John Wiley & Sons.

Young

. Internet addiction: the emergence of a new clinical disorder. CyberPsychology & Behavior, 1998; 1:237–244.

Király

, Nagygyörgy

, Koronczai

, et al. (2015) Assessment of problematic Internet use and online video gaming. In Aboujaoude

, Starcevic

, eds. Mental health in the digital age: grave dangers, great promise. Mental health in the digital age: grave dangers, great promise. New York, NY: Oxford University Press, pp. 46–68.

Guertler

, Rumpf

, Bischof

, et al. Assessment of problematic internet use by the Compulsive Internet Use Scale and the internet addiction test: a sample of problematic and pathological gamblers. European Addiction Research, 2014; 20:75–81.

Lortie

, Guitton

. Internet addiction assessment tools: dimensional structure and methodological status. Addiction, 2013; 108:1207–1216.

Widyanto

, Griffiths

, Brunsden

. A psychometric comparison of the Internet Addiction Test, the Internet-Related Problem Scale, and self-diagnosis. Cyberpsychology, Behavior, and Social Networking, 2011; 14:141–149.

Laconi

, Rodgers

, Chabrol

. The measurement of Internet addiction: a critical review of existing scales and their psychometric properties. Computers in Human Behavior, 2014; 41:190–202.

10.

American Psychiatric Association. (1994) Diagnostic and statistical manual of mental disorders, fourth edition (DSM-IV). Washington, DC: American Psychiatric Association.

11.

Griffiths

. Internet addiction: fact or fiction?. The Psychologist, 1999; 12:246–250.

12.

Marks

. Behavioural (non-chemical) addictions. British Journal of Addiction, 1990; 85:1389–1394.

13.

Brown

. (1993) Some contributions of the study of gambling to the study of other addictions. In Eadington

, Cornelius

, eds. Gambling behavior and problem gambling. Reno, NV: University of Nevada Press, pp. 241–272.

14.

American Psychiatric Association. (2013) Diagnostic and statistical manual of mental disorders, fifth edition (DSM-5). Arlington, VA: American Psychiatric Association.

15.

Lopez-Fernandez

, Honrubia-Serrano

, Gibson

, et al. Problematic Internet use in British adolescents: an exploration of the addictive symptomatology. Computers in Human Behavior, 2014; 35:224–233.

16.

Anderson

, Steen

, Stavropoulos

. Internet use and Problematic Internet Use: a systematic review of longitudinal research trends in adolescence and emergent adulthood. International Journal of Adolescence and Youth, 2017; 22:430–454.

17.

Dhir

, Chen

, Nieminen

. A repeat cross-sectional analysis of the psychometric properties of the Compulsive Internet Use Scale (CIUS) with adolescents from public and private schools. Computers & Education, 2015; 86:172–181.

18.

Kuss

, Shorter

, van Rooij

, et al. Assessing internet addiction using the parsimonious internet addiction components model—a preliminary study. International Journal of Mental Health and Addiction, 2014; 12:351–366.

19.

Feng

, Ramo

, Chan

, et al. Internet gaming disorder: trends in prevalence 1998–2016. Addictive Behaviors, 2017; 75:17–24.

20.

Van den Eijnden

, Meerkerk

, Vermulst

, et al. Online communication, compulsive Internet use, and psychosocial well-being among adolescents: a longitudinal study. Developmental Psychology, 2008; 44:655–665.

21.

van Den Eijnden

, Spijkerman

, Vermulst

, et al. Compulsive Internet use among adolescents: bidirectional parent–child relationships. Journal of Abnormal Child Psychology, 2010; 38:77–89.

22.

Van der Aa

, Overbeek

, Engels

, et al. Daily and compulsive internet use and well-being in adolescence: a diathesis-stress model based on big five personality traits. Journal of Youth and Adolescence, 2009; 38:765–776.

23.

de Leeuw

, de Bruijn

, de Weert-van Oene

, et al. Internet and game behaviour at a secondary school and a newly developed health promotion programme: a prospective study. BMC Public Health, 2010; 10:544.

24.

Davis

. A cognitive-behavioral model of pathological Internet use. Computers in Human Behavior, 2001; 17:187–195.

25.

Van Rooij

, Schoenmakers

, Van de Eijnden

, et al. Compulsive internet use: the role of online gaming and other internet applications. Journal of Adolescent Health, 2010; 47:51–57.

26.

Van Rooij

, Schoenmakers

, Vermulst

, et al. Online video game addiction: identification of addicted adolescent gamers. Addiction, 2011; 106:205–212.

27.

Khazaal

, Chatton

, Atwi

, et al. Arabic validation of the Compulsive Internet Use Scale (CIUS). Substance Abuse Treatment, Prevention, and Policy, 2011; 6:32.

28.

Khazaal

, Chatton

, Horn

, et al. French validation of the Compulsive Internet Use Scale (CIUS). Psychiatric Quarterly, 2012; 83:397–405.

29.

Peukert

, Steffen

, ElKasmi

, et al. Factorial structure of the German version of the Compulsive Internet Use Scale (CIUS) [In German]. Zeitschrift für Klinische Psychologie und Psychotherapie, 2012; 41:101–108.

30.

Guertler

, Broda

, Bischof

, et al. Factor structure of the Compulsive Internet Use Scale. Cyberpsychology, Behavior, and Social Networking, 2014; 17:46–51.

31.

Wartberg

, Petersen

, Kammerl

, et al. Psychometric validation of a German version of the Compulsive Internet Use Scale. Cyberpsychology, Behavior, and Social Networking, 2014; 17:99–103.

32.

Besser

, Rumpf

, Bischof

, et al. Internet-related disorders: development of the short Compulsive Internet Use Scale. Cyberpsychology, Behavior, and Social Networking, 2017; 20:709–717.

33.

Cartierre

, Coulon

, Demerval

. Validation of a short French version for adolescents of the Compulsive Internet Use Scale [In French]. 2011; 59:415–449.

34.

Dhir

, Chen

, Nieminen

. Psychometric validation of the Chinese Compulsive Internet Use Scale (CIUS) with Taiwanese high school adolescents. Psychiatric Quarterly, 2015; 86:581–596.

35.

Siciliano

, Bastiani

, Mezzasalma

, et al. Validation of a new Short Problematic Internet Use Test in a nationally representative sample of adolescents. Computers in Human Behavior, 2015; 45:177–184.

36.

Alavi

, Jannatifard

, Maracy

, et al. Comparison of national and personal identity between person with internet addiction disorder and normal internet users. Journal of Education and Health Promotion, 2014; 3:42.

37.

Yong

, Inoue

, Kawakami

. The validity and psychometric properties of the Japanese version of the Compulsive Internet Use Scale (CIUS). BMC Psychiatry, 2017; 17:201.

38.

Lopez-Fernandez

, Kuss

, Romo

, et al. Self-reported dependence on mobile phones in young adults: a European cross-cultural empirical survey. Journal of Behavioral Addictions, 2017; 6:168–177.

39.

Lopez-Fernandez

, Männikkö

, Kääriäinen

, et al. Mobile gaming and problematic smartphone use: a comparative study between Belgium and Finland. Journal of Behavioral Addictions Forthcoming, 2018: 1–12.

40.

Checa

, Perales

, Espejo

. Measurement invariance of the satisfaction with life scale by gender, age, marital status and educational level. Journal of Behavioral Addictions, 2018; 7:88–99.

41.

Burke

, Tanofsky-Kraff

, Crosby

, et al. Measurement invariance of the eating disorder examination in black and white children and adolescents. International Journal of Eating Disorders, 2017; 50:758–768.

42.

Joshanloo

, Bakhshi

. The factor structure and measurement invariance of positive and negative affect: a study in Iran and the USA. European Journal of Psychological Assessment, 2016; 32:265–272.

43.

Jafari

, Nozari

, Ahrari

, et al. Measurement invariance of the depression anxiety stress scales-21 across medical student genders. International Journal of Medical Education, 2017; 8:116–122.

44.

Brislin

. Back-translation for cross-cultural research. Journal of Cross-Cultural Psychology, 1970; 1:185–216.

45.

Gregorich

. Do self-report instruments allow meaningful comparisons across diverse population groups? Testing measurement invariance using the confirmatory factor analysis framework. Medical Care, 2006; 44:S78–S94.

46.

Van de Schoot

, Lugtig

, Hox

. A checklist for testing measurement invariance. European Journal of Developmental Psychology, 2012; 9:486–492.

47.

Kim

, Yoon

. Testing measurement invariance: a comparison of multiple-group categorical CFA and IRT. Structural Equation Modeling, 2011; 18:212–228.

48.

Team

. (2015) RStudio: integrated development for R. Boston, MA: RStudio, Inc. www.rstudio.com (accessed Mar. 14, 2018).

49.

Rosseel

. Lavaan: an R package for structural equation modeling and more. Version 0.5–12 (BETA). Journal of Statistical Software, 2012; 48:1–36.

50.

Revelle

. (2016) Psych: procedures for personality and psychological research [internet]. Evanston, IL: Northwestern University. https://CRAN.R-project.org/package=psych Version = 1.6.12 (accessed Mar. 14, 2018).

51.

Contributors

semTools: useful tools for structural equation modeling. R package version 0.4–11. https://CRAN.R-project.org/package=semTools (accessed Mar. 14, 2018).

52.

Hirschfeld

, Von Brachel

. Multiple-Group confirmatory factor analysis in R-A tutorial in measurement invariance with continuous and ordinal indicators. Practical Assessment, Research & Evaluation, 2014; 19:1–11.

53.

Mindrila

. Maximum likelihood (ML) and diagonally weighted least squares (DWLS) estimation procedures: a comparison of estimation bias with ordinal and multivariate non-normal data. International Journal of Digital Society, 2010; 1:60–66.

54.

Muthén

, Muthén

. (2010) Mplus users guide. 6th ed. Los Angeles, CA: Muthén & Muthén;. 2010. http://research.socialwork.wayne.edu/pdf/mplus-users-guide.pdf (accessed Mar. 14, 2018).

55.

Flora

, Curran

. An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 2004; 9:466–491.

56.

Hooper

, Coughlan

, Mullen

. Structural equation modelling: guidelines for determining model fit. Electronic Journal of Business Research Methods, 2008; 6:53–59.

57.

, Bentler

. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling, 1999; 6:1–55.

58.

Marsh

, Balla

, McDonald

. Goodness-of-fit indexes in confirmatory factor analysis: the effect of sample size. Psychological Bulletin, 1988; 103:391–410.

59.

Benjamini

, Hochberg

. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 1995; series B:289–300.

60.

Chen

. Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 2007; 14:464–504.

61.

Cheung

, Rensvold

. Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 2002; 9:233–255.

62.

, O'Brien

, Snyder

, et al. Characteristics of internet addiction/pathological internet use in US university students: a qualitative-method investigation. PLoS One, 2015; 10:e0117372.

63.

, O'Brien

, Snyder

, et al. Diagnostic criteria for problematic internet use among US University students: a mixed-methods evaluation. PLoS One, 2016; 11:e0145981.

64.

Snyder

, Li

, O'Brien

, et al. The effect of US university students' problematic Internet use on family relationships: a mixed-methods investigation. PLoS One, 2015; 10:e0144005.

65.

Yong

, Inoue

, Kawakami

. The validity and psychometric properties of the Japanese version of the Compulsive Internet Use Scale (CIUS). BMC Psychiatry, 2017; 17:201.

66.

Aadahl

, Jorgensen

. The effect of conducting a lottery on questionnaire response rates: a randomised controlled trial. European Journal of Epidemiology, 2003; 18:941–944.

67.

Dodeen

, Al-Darmaki

. The application of item response theory in developing and validating a shortened version of the Emirate Marital Satisfaction Scale. Psychological Assessment, 2016; 28:1625–1633.

68.

Gmel

, Khazaal

, Studer

, et al. Development of a short form of the Compulsive Internet Use Scale in Switzerland. International Journal of Methods in Psychiatric Research, 2019; 28:e1765.

69.

van de Vijver

HJR

, Poortinga

. Towards an integrated analysis of bias in cross-cultural assessment. European Journal of Psychological Assessment, 1997; 13:29–37.

70.

DiStefano

, Liu

, Jiang

, et al. Examination of the weighted root mean square residual: evidence for trustworthiness?. Structural Equation Modeling, 2018; 25:453–466.

71.

Muthén

. (1998–2004) Mplus technical appendices. Los Angeles, CA: Muthén & Muthén. www.statmodel.com/techappen.shtml (accessed Apr. 12, 2018).

72.

Kardefelt-Winther

. A conceptual and methodological critique of internet addiction research: towards a model of compensatory internet use. Computers in Human Behavior, 2014; 31:351–354.

73.

Starcevic

, Aboujaoude

. Internet gaming disorder, obsessive-compulsive disorder, and addiction. Current Addiction Reports, 2017; 4:317–322.

74.

Deleuze

, Long

, Liu

, et al. Passion or addiction? Correlates of healthy versus problematic use of videogames in a sample of French-speaking regular players. Addictive Behaviors, 2018; 82:114–121.

75.

Király

, Tóth

, Urbán

, et al. Intense video gaming is not essentially problematic. Psychology of Addictive Behaviors, 2017; 31:807.

76.

Cortina

. What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 1993; 78:98–104.

77.

Schmitt

. Uses and abuses of coefficient alpha. Psychological Assessment, 1996; 8:350–353.

78.

Lai

, Mak

, Cheng

, et al. Measurement invariance of the internet addiction test among Hong Kong, Japanese, and Malaysian adolescents. Cyberpsychology, Behavior, and Social Networking, 2015; 18:609–617.

79.

World Health Organization. Public health implications of excessive use of the internet, computers, smartphones and similar electronic devices meeting report. main meeting hall, foundation for promotion of cancer research. National Cancer Research Centre, Tokyo, Japan, August 27–29, 2014. http://apps.who.int/iris/bitstream/10665/184264/1/9789241509367_eng.pdf (accessed Mar. 14, 2018).

80.

Vandenberg

, Lance

. A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 2000; 3:4–70.

81.

Schmitt

, Kuljanin

. Measurement invariance: review of practice and implications. Human Resource Management Review, 2008; 18:210–222.

82.

Pontes

, Griffiths

. Portuguese validation of the Internet Gaming Disorder Scale–Short-Form (IGD9-SF). Cyberpsychology, Behavior and Social Networking, 2016; 19:288–293.

83.

Ortiz de Gortari

, Pontes

, Griffiths

. The Game Transfer Phenomena Scale: an instrument for investigating the non-volitional effects of video game playing. Cyberpsychology, Behavior and Social Networking, 2015; 18:588–594.