Measurement Invariance of the Internet Addiction Test Among Hong Kong,Japanese,and Malaysian Adolescents

Abstract

There has been increased research examining the psychometric properties on the Internet Addiction Test (IAT) in different populations. This population-based study examined the psychometric properties and measurement invariance of the IAT in adolescents from three Asian countries. In the Asian Adolescent Risk Behavior Survey (AARBS), 2,535 secondary school students (55.9% girls) aged 12–18 years from Hong Kong (n = 844), Japan (n = 744), and Malaysia (n = 947) completed a survey in 2012–2013 school year. A nested hierarchy of hypotheses concerning the IAT cross-country invariance was tested using multigroup confirmatory factor analyses. Replicating past findings in Hong Kong adolescents, the construct of the IAT is best represented by a second-order three-factor structure in Malaysian and Japanese adolescents. Configural, metric, scalar, and partial strict factorial invariance was established across the three samples. No cross-country differences on Internet addiction were detected at the latent mean level. This study provided empirical support for the IAT as a reliable and factorially stable instrument, and valid to be used across Asian adolescent populations.

Introduction

Internet addiction has increasingly become a serious public health concern among the widespread growth of Internet users.¹ The Internet Addiction Test (IAT) was developed by Young as a screening instrument of the disorder,² and later become a widely used assessment of Internet addiction in both Western^3,4 and Chinese societies.^5–7 To adapt its use in different cultures, the IAT has been translated into different languages, including French,⁸ Italian,⁹ Finnish,¹⁰ Korean,¹¹ Malay,¹² and Chinese.¹³ Despite its strong internal consistency, different factor structures have been reported for the construct measured among adult and college students.^8–10,14,15

Using a combined exploratory factor analysis/confirmatory factor analysis approach, Chang and Man Law¹³ successfully replicated an 18-item three-factor structure in a sample of 410 Hong Kong college students. This study was the first to show that the construct of Internet addiction as assessed by the IAT was best represented by a second-order structure. Later, Lai et al.¹⁶ replicated this second-order three-factor structure of the IAT in a sample of 762 Hong Kong adolescents. These findings arguably provide indirect evidence for the stability of the construct of the IAT from adolescence to adulthood.

In cross-cultural population-based studies, it is a common practice for researchers to compare the average or summated score of the scale directly, and assume that individuals who obtained the same observed score are similar in terms of severity on the construct underlying the research instrument. However, group comparisons of mean differences rely on an established measurement invariance of the research instrument.¹⁷ Therefore, it is necessary to examine the degree to which the IAT assesses the same construct across sociocultural groups as an extrapolation of its applications in health studies. Nevertheless, whether the factor structure underlying the IAT construct is equivalent across different sociocultural groups has remained unexplored. This unexplored issue contributes to the importance of this study, as it examines the factor structure and measurement invariance of the IAT across three different Asian adolescent populations, thereby allowing the assessment of cross-sociocultural differences with greater confidence. Comparisons between groups are methodologically valid only after analyzing the equivalence/non-equivalence between factor structure of the instrument across different groups.

With samples from three countries, the present study aimed to extend the previous studies in Hong Kong adolescents,¹⁶ and addressed the following objectives: (a) to examine the factorial validity of the IAT in Japanese and Malaysian adolescents; (b) to examine the measurement invariance of the IAT across Hong Kong, Japanese, and Malaysian adolescents; and (c) to compare the observed and latent means of the IAT between different sociocultural groups, based upon the assumption of measurement invariance.

Methods

Administration

In the 2012–2013 school year, 2,535 secondary school students aged 12–18 years from Hong Kong (n = 844), Japan (n = 744), and Malaysia (n = 947) participated in the Asian Adolescent Risk Behavior Survey (AARBS). The schools were randomly selected from both rural and urban areas: Kowloon and New Territories of Hong Kong; Western Japan, including Shiga; and Central Western Malaysia, including Selayang. Participants were 55.9% female, and had a mean age of 15.6 years (SD = 1.6 years; see Table 1).

Table 1.

Participants' Characteristics and Observed Mean (Standard Deviation) Scores of the Internet Addiction Test by Country

	Hong Kong (n = 844)	Japan (n = 744)	Malaysia (n = 947)	All (N = 2,535)
Age, years	15.9 (3.5)	16.5 (0.5)	14.7 (1.2)	15.6 (1.6)
Sex, n (%)
Male	318 (37.7)	364 (48.92)	433 (45.97)	1,115 (44.09)
Female	525 (62.3)	380 (51.08)	509 (54.03)	1,414 (55.91)
Daily Internet use, hours
Weekdays	2.8 (3.0)	1.42 (1.37)	2.72 (4.37)	2.32 (3.22)
Weekends	4.9 (4.0)	2.25 (2.33)	4.35 (5.14)	3.87 (4.20)
IAT
Withdrawal and social problems	15.20 (6.36)	15.33 (5.98)	15.63 (6.37)	15.40 (6.26)
Time management and performance	12.51 (5.13)	14.12 (5.51)	12.32 (5.45)	12.91 (5.42)
Reality substitute	6.28 (2.84)	5.81 (2.66)	5.92 (2.83)	6.01 (2.79)
Total	33.99 (12.98)	35.26 (12.97)	33.72 (13.48)	34.26 (13.18)
Internal consistency of IAT (Cronbach's α)
Withdrawal and social problems	0.87	0.86	0.82	0.85
Time management and performance	0.86	0.88	0.77	0.84
Reality substitute	0.70	0.73	0.64	0.68
Total	0.93	0.93	0.90	0.92

IAT, Internet Addiction Test.

Participants were asked to complete a 40-minute Internet usage survey that included demographic items, Internet use habits, and Young's Internet Addiction Test (IAT).² The Chinese, Japanese, and Malay versions of the IAT were prepared by a forward and backward translation procedure by two independent bilingual translators. Items were translated with consideration to the relevance and comprehensibility to adolescents. For instance, “household chores” in item 2 was changed to “daily hassles,” and “intimacy with partner” in item 3 was changed to “activities with companions.” The face validity of the translated questionnaire was then assessed for final approval. Participation was voluntary, and the questionnaire was administered by teachers in the classroom. Informed consent was obtained, and ethics approval was granted from the university and hospital ethics committees.

Instruments

Internet use

Participants were asked about their ownership of a personal computer, and their frequency and duration of Internet use. They also reported the average number of hours spent online daily during school days and holidays, with responses on a 6-point scale ranging from 1 = “once a week or less” to 6 = “more than three times a day” on frequency of Internet use, and on a four-point scale (1 = “no personal or shared computer”; 2 = “shared computer with one sibling”; 3 = “shared computer with more than one sibling”; 4 = “own a personal computer”) for the question about computer ownership.

The IAT

The IAT comprises 20 items rated on a 5-point Likert scale.² These items were derived from the DSM-IV-TR¹⁸ diagnostic criteria of pathological gambling, and examine the degree of preoccupation and compulsiveness to go online, and the impact on life related to Internet usage. A high reliability estimate of the IAT has been consistently reported in adolescent samples, with a Cronbach's alpha > 0.80 in previous studies.^19–21

Data analysis

A series of plausible models identified in previous studies were evaluated with EQS v6.1. These models include: (a) a one-factor model (Model 1); (b) Widyanto et al.'s²² three-factor model (Model 3a); (c) Chang and Man Law's¹³ second-order three-factor model (Model 3b); (d) a modified version of Chang and Man Law's¹³ model with three additional pairs of error covariances (Model 3c); (e) a four-factor model modified from Ng's¹² five-factor model by discarding the single-item factor (Model 4); (f) Widyanto and McMurran's¹⁴ six-factor model (Model 6a); (g) Ferraro et al.'s⁹ six-factor model (Model 6b).

Missing answers in any of the IAT items were excluded case-wise, resulting in 79 exclusions in the Hong Kong data and 171 exclusions in the Malaysia data. The normalized Mardia's coefficient of the Hong Kong, Japan, and Malaysia data was 118.97, 104.02, and 255.43, respectively, indicating that the assumption of multivariate normality was not fulfilled. The Satorra–Bentler scaled chi-square (SB χ²) correction²³ was hence used to indicate the overall goodness of fit. By convention, a value of ≥0.90 in the comparative fit index (CFI),²⁴ the normed fit index (NFI), and the non-normed fit index (NNFI),²⁵ and a value <0.08 in the root mean square error of approximation (RMSEA)²⁶ indicate a model that fits the data well.^27,28

Provided that any of the posited models were acceptable in the sample groups, analyses of hierarchical multisample confirmatory factor analyses (CFA) across countries were performed. First, configural invariance was established across groups by imposing the best-fit model previously obtained separately to each group. This model (Model A) was used as the baseline model to which nested models with equality constraints could be directly contrasted. RMSEA <0.05, complemented with CFI >0.90, was used to evaluate the configural model fit.²⁹ The following sequence of increasingly restrictive tests of invariance across groups was implemented: (a) first-order factor loadings (Model B), (b) second-order factor loadings (Model C), (c) intercepts of observed items (Model D), (d) intercepts of the latent first-order factors (Model E), (e) the residual variances and covariances of the observed items (Model F), and (f) disturbances of the latent first-order factors.³⁰ If configural invariance did not exist, the analyses would stop at that immediate level. If any measurement variance were found on any level, the Lagrange multiplier test (LM test) was conducted to search for the cross-group equality constraint that contributed the most to the misfit. The “full” forms of invariance would be relaxed to obtain partial measurement invariance. Since the corrected scaled difference chi-square (SDCS) test²³ has been criticized to be too sensitive to large sample size and non-normality,^29,31 Chen's³¹ suggestion that a difference of >0.01 in CFI and >0.015 in RMSEA were adopted as the criteria for the test of measurement invariance. In the case of incompatible conclusion, the CFI would be chosen as the main criterion.

To evaluate the influence of measurement invariance on cross-group mean comparisons, observed and latent mean differences for each latent construct were computed. Latent mean values were fixed to zero for adolescents in Hong Kong versus Japan, Hong Kong versus Malaysia, and Japan versus Malaysia. Finally, statistical significance associated with differences between the latent means was determined on the basis of the z statistic, and t statistic for the observed means.

Results

Confirmatory factor analysis in Japan and Malaysia data

Similar to the CFA results in Hong Kong adolescents reported elsewhere,¹⁶ Model 3c fitted the data the best in both Japanese and Malaysian adolescents, as supported by the smallest SB χ² to df ratio and Akaike's information criterion value (see Table 2). The factor loadings of Model 3c in Japanese and Malaysian adolescents are shown in Figures 1 and 2. The Cronbach's alpha values of the total IAT, Withdrawal and Social Problems (W&SP), Time Management and Performance (TM&P), and Reality Substitute (RS) were 0.85, 0.84, 0.68, and 0.92 in all participants, respectively.

FIG. 1.

Standardized solution of modified Chang and Man Law's second-order three-factor model (Model 3c) of the Internet Addiction Test (IAT) in Japanese adolescents.

FIG. 2.

Standardized solution of modified Chang and Man Law's second-order three-factor model (Model 3c) of Internet Addiction Test (IAT) in Malaysian adolescents.

Table 2.

Results of Confirmatory Factor Analyses of the Internet Addiction Test

Model^a	χ ²	SB χ ²	df	SB χ²/df	NFI	NNFI	CFI	RMSEA	AIC
Hong Kong
Model 1	1,290.99	764.90	170	4.50	0.79	0.81	0.83	0.07	424.90
Model 3a	1,215.83	721.55	167	4.32	0.80	0.82	0.84	0.07	387.55
Model 3b^b	854.34	502.39	132	3.81	0.85	0.87	0.88	0.06	238.39
Model 3c^b	620.26	367.20	129	2.85	0.89	0.91	0.93	0.05	109.20
Model 4	1,090.79	650.62	146	4.46	0.82	0.82	0.85	0.07	358.62
Model 6a	1,075.72	641.77	155	4.14	0.83	0.83	0.86	0.06	331.77
Model 6b	1,098.28	659.05	155	4.25	0.82	0.82	0.86	0.07	349.05
Japan
Model 1	1,684.46	1,086.41	170	6.39	0.79	0.79	0.81	0.09	746.41
Model 3a	1,394.50	905.11	167	5.42	0.82	0.83	0.85	0.08	571.11
Model 3b^b	1,096.21	537.34	132	4.07	0.88	0.89	0.90	0.06	273.34
Model 3c^b	772.68	375.37	129	2.91	0.91	0.93	0.94	0.05	514.68
Model 4	1,546.02	966.20	146	6.62	0.80	0.79	0.82	0.09	674.20
Model 6a	1,425.01	923.05	155	5.96	0.82	0.81	0.84	0.08	613.05
Model 6b	1,262.57	808.83	155	5.22	0.84	0.84	0.87	0.08	498.83
Malaysia
Model 1	707.38	459.89	170	2.71	0.86	0.90	0.91	0.05	119.89
Model 3a	672.61	437.12	167	2.62	0.87	0.90	0.91	0.05	103.12
Model 3b^b	563.21	331.79	132	2.51	0.89	0.92	0.93	0.04	67.79
Model 3c^b	435.04	256.14	129	1.99	0.91	0.95	0.96	0.04	–1.86
Model 4	626.09	411.22	146	2.82	0.87	0.90	0.91	0.05	119.22
Model 6a	577.69	376.76	155	2.43	0.89	0.91	0.93	0.04	66.76
Model 6b	553.29	364.67	155	2.35	0.89	0.92	0.93	0.04	54.67

Note. Estimation method = maximum likelihood.

Model 1 is the one-factor model; Model 3a was the final factor solution reported in Widyanto et al.; Model 3b was the final factor solution reported in Chang and Man Law; Model 3c was the final factor solution reported in Lai et al.; Model 4 was a 19-item 4-factor model derived from Ng et al.; Model 6a was the final factor solution reported in Widyanto and McMurran; Model 6b was the final factor solution reported in Ferraro et al.

Content of the items refer to Young; comparison of the pattern of factor structure of the above models refers to Lai et al.

Factors of Model 3b and Model 3c were regressed on a second-order factor. Except Model 3b and Model 3c, all models were specified on a first-order structure in which factors were allowed to correlate freely.

SB χ², Satorra–Bentler scaled χ²; df, degrees of freedom; NFI, normed fit index; NNFI, non-normed fit index; CFI, comparative fit index; RMSEA, root mean square error of approximation; AIC, Akaike's Information Criterion.

Measurement invariance across countries

As shown in Table 3, the configural invariance model (Model A) provided a high goodness-of-fit in all samples tested, enabling it to serve as a baseline model for subsequent comparison of the increasingly restrictive invariance models.

Table 3.

Summary of Fit Statistics for Testing Measurement Invariance of Modified Chang and Man Law's Second-Order 3-Factor Model (Model 3c) of the Internet Addiction Test Across the Three Countries

Invariant model		χ ²	SB χ ²	df	SB χ²/df	NFI	NNFI	CFI	RMSEA	90% CI	AIC	ΔCFI	ΔRMSEA
Hong Kong vs. Japan
A	Configural	1,392.94	807.59	258	3.13	0.894	0.910	0.923	0.038	0.035, 0.041	291.59
B	First-order metric	1,529.44	907.49	273	3.32	0.880	0.902	0.913	0.039	0.036, 0.042	361.49	0.010	0.001
C	Second-order metric	1,533.89	909.57	276	3.30	0.880	0.903	0.913	0.039	0.036, 0.042	357.57	0.000	0.000
D	First-order scalar	2,035.00	1,251.92	288	4.35	0.883	0.898	0.913	0.047	0.044, 0.050	675.92	0.000	0.008
E	Second-order scalar	2,221.10	1,375.10	291	4.73	0.882	0.897	0.913	0.050	0.047, 0.052	793.10	0.000	0.003
F	Strict factorial	2,455.11	1,537.28	312	4.93	0.867	0.882	0.898	0.051	0.048, 0.054	913.28	0.015	0.001
G	Partial strict factorial^a	2,371.19	1,467.47	311	4.72	0.874	0.891	0.905	0.050	0.047, 0.052	845.47	0.008	0.000
Hong Kong vs. Malaysia
A	Configural	1,044.22	650.48	258	2.52	0.894	0.920	0.933	0.032	0.029, 0.035	134.48
B	First-order metric	1,130.51	714.84	273	2.62	0.884	0.915	0.924	0.033	0.030, 0.036	168.84	0.009	0.001
C	Second-order metric	1,147.50	727.51	276	2.64	0.882	0.914	0.923	0.033	0.030, 0.036	175.51	0.001	0.000
D	First-order scalar	1,580.85	1,014.08	288	3.52	0.888	0.912	0.925	0.041	0.038, 0.044	438.08	−0.002	0.008
E	Second-order scalar	1,625.55	1,067.95	291	3.67	0.885	0.909	0.923	0.042	0.040, 0.045	485.95	0.002	0.001
F	Strict factorial	2,155.56	1,367.41	312	4.38	0.844	0.863	0.881	0.048	0.045, 0.050	743.41	0.042	0.006
G	Partial strict factorial^b	1,694.67	1,088.27	304	3.58	0.882	0.908	0.921	0.042	0.039, 0.044	480.27	0.002	0.000
Japan vs. Malaysia
A	Configural	1,196.65	727.60	258	2.82	0.898	0.919	0.930	0.035	0.032, 0.038	211.60
B	First-order metric	1,331.68	821.94	273	3.01	0.885	0.910	0.920	0.037	0.034, 0.040	275.94	0.010	0.002
C	Second-order metric	1,346.29	835.01	276	3.03	0.883	0.910	0.918	0.037	0.034, 0.040	283.01	0.002	0.000
D	First-order scalar	2,294.76	1,497.30	288	5.20	0.882	0.896	0.911	0.054	0.051, 0.056	921.30	0.007	0.017
E	Second-order scalar	2,564.53	1,732.03	291	5.95	0.883	0.897	0.912	0.058	0.055, 0.061	1,150.03	−0.001	0.004
F	Strict factorial	3,442.99	1,827.21	312	5.86	0.864	0.877	0.893	0.058	0.055, 0.060	1,203.21	0.019	0.000
G	Partial strict factorial^c	2,610.41	1,774.64	302	5.88	0.881	0.897	0.912	0.058	0.055, 0.060	1,170.64	0.000	0.000

Equality constraint on the residual variances of item 10 was released.

Equality constraints on residual variances of items 13, 15, 16, 17, 18, and 20 and residual covariances between items 1 and 2, and items 16 and 17 were released.

Equality constraint on residual variances of items 10, 12, 13, 15, 16, 17, 18, 19, and 20 and residual covariances between items 1 and 2 were released.

△CFI >0.01 and △RMSEA >0.015 are shown in bold to indicate substantial change in model fitness.

SB χ², Satorra-Bentler scaled χ²; df, degrees of freedom; NFI, normed fit index; NNFI, non-normed fit index; CFI, comparative fit index; RMSEA, root mean square error of approximation; 90% CI, 90% confident interval of the RMSEA; AIC, Akaike's Information Criterion.

Hong Kong versus Japan samples

After constraining the first-order factor loadings, no substantial differences in SDCS between the equal first-order factor loadings model (Model B) and the baseline model (Model A) were observed. Following this, both first-order and second-order factor loadings were constrained to be equal, non-significant SDCS in addition to the small changes in CFI and RMSEA indices provided evidence of second-order metric invariance of Model 3c (Model C) across countries. The scalar invariance of the IAT (Model D and Model E) was tested by imposing equality constraint on the intercepts of the observed items and first-order factors. The less constrained model (Model C) did not provide significantly better fit than the equal item intercepts model (Model D) and the equal first-order factor intercepts model (Model E).

To examine the strict factorial invariance of the IAT, equality constrain was imposed on all factor loadings, item and factor intercepts, and residual variances and covariances of all observed items (Model F). Although the result of the SDCS test was also not significant, the LM test statistics revealed that the residual variances of item 10 (“Block disturbing thoughts about life with soothing thoughts of the Internet”) operated differentially between Hong Kong and Japanese adolescents after Bonferroni correction for the number of tested parameters had been made. Allowing this parameter to vary freely resulted in a comparable model fit relative to the equal factor intercepts model (Model E) on the basis of CFI and RMSEA indices.

Hong Kong versus Malaysia samples

The equal first-order factor loadings model (Model B) was compared with the baseline model, but there was no substantial decrease in CFI and RMSEA, albeit a significant difference in the SDCS test. Once again, the second-order factor loadings (Model C), item intercepts (Model D), and first-order factor intercepts of Model 3c (Model E) were all considered to be invariant across the groups, as evidenced by a small change in CFI and RMSEA indices relative to the less restrictive models. Nonetheless, the ΔCFI of the model with equality constraint on all residual variances and covariances of the observed items (Model F) exceeded the cutoff value. The results of the LM Test with Bonferroni correction suggested that six residual variances (items 13, 15, 16, 17, 18, and 20) and two residual covariances (between items 1 and 2, and items 16 and 17) were different across the groups. Freeing the eight analogous pairs of residual variances and covariances of the IAT items resulted in no substantial difference in CFI and RMSEA indices.

Japan versus Malaysia samples

Given little changes of the practical fit indices in the more restrictive model, first-order factor loadings (Model B), second-order factor loadings (Model C), item intercepts (Model D), and first-order factor intercepts (Model E) were all considered to be invariant across the groups, Although ΔRMSEA slightly exceeded the cutoff value, the respective change in CFI, a critical criterion, was <0.01. The equal item intercepts model (Model D) is thereby considered tenable. Similarly, when analogous pairs of residual variances and covariances were constrained to be equal, substantial degradation in fit was revealed. Releasing equality constraint on the nine residual variances (items 10, 12, 13, 15, 16, 17, 18, 19, and 20) and one pair of residual covariances (between item 1 and 2) suggested by the LM test as the major source of misfit resulted in a substantial improvement; changes in CFI and RMSEA were consistently small in the modified model.

Tests of latent and observed mean differences across countries

The full-scale invariance established above allowed comparison of latent factor mean differences of IAT subdomain scores across the groups (Model G). The Wald test suggested that a significantly higher latent mean of TM&P and lower latent mean of RS in Japanese than Hong Kong adolescents (see Table 4). Consistently, Malaysian adolescents obtained a significantly higher latent mean on W&SP than Hong Kong adolescents in the t test. However, a significantly higher observed mean score on TM&P but not on W&SP was obtained. The comparison between Japanese and Malay adolescents revealed significant differences of the latent means for all three first-order factors of the IAT. Higher latent means on W&SP and RS and a significantly lower latent mean on TM&P were revealed. When observed scores were compared, only the mean of TM&P remained significant. To test for the latent mean difference of the second-order factor between the groups, the equality constraints on the intercept of first-order factors were imposed. No significant latent mean difference across the three samples, although significant observed mean differences between Malay and Japanese samples were found.

Table 4.

Mean Differences Between Country on Latent Constructs and Observed Dimensions of the Internet Addiction Test

	Japan minus Hong Kong^a				Malaysia minus Hong Kong^a				Malaysia minus Japan^a
	Latent		Observed		Latent		Observed		Latent		Observed
	z	M_diff	t	M_diff	z	M_diff	t	M_diff	z	M_diff	t	M_diff
1. Withdrawal and Social Problems	−1.28	−0.04	−0.41	−0.13	3.74^***	0.12	−1.40	−0.43	3.80^***	0.14	−0.98	−0.30
2. Time Management and Performance	7.13^***	0.30	−6.02^***	−1.61	0.30	0.01	0.77	0.19	−10.34^***	−0.44	6.69^***	1.80
3. Reality Substitute	−3.54^***	−0.16	3.39^**	0.47	−1.38	−0.06	2.63^**	0.36	2.85^**	0.13	−0.84	−0.11
Total	0.14	0.01	−1.94	−1.27	1.26	0.06	0.43	0.27	1.22	0.04	2.36^*	1.54

Latent mean values were fixed to 0 to serve as the reference group.

p < 0.05; ^**p < 0.01; ^***p < 0.001.

Discussion

The surge of Internet use over the last two decades has generated an increased need of research instruments assessing Internet addiction across different cultures. Nevertheless, whether the factor structure underlying the IAT construct is equivalent across different sociocultural groups has remained untested until now. With this in mind, the present study examined the factor structure of the IAT in Malaysian and Japanese adolescents. Additionally, the measurement invariance of the IAT was evaluated across three groups of Asian adolescents (from Hong Kong, Malaysia, and Japan), thereby allowing the assessment of cross-sociocultural differences of Internet addiction with greater confidence. Finally, mean differences of the IAT composite score and factor score from the CFA-based Wald test and traditional t test approach were assessed.

With a confirmatory approach, it was found that the construct of the IAT is best represented by a second-order three-factor structure in Malaysian and Japanese adolescents, which replicated an earlier finding in Hong Kong adolescents.¹⁶ This provides evidence for the stability of the construct of Internet addiction as assessed by the IAT. It also suggests that the original 20-item IAT—initially designed as unidimensional—should be reorganized into 18 items underlying three symptom dimensions, including withdrawal and social problems, time management and performance, and reality substitute, which are in turn derived from a higher-order construct—“Internet Addiction.” This hierarchical factor structure lends support for the use of both the composite and factor scores of the IAT. While the former may be used as a criterion to determine the severity of the problem, the latter aids understanding about symptom presentation.

When the IAT is used in practice, each of its items is assumed to make an equal and important contribution to the three latent constructs. No matter whether the factor loadings and/or the intercepts contribute differentially to the factor scores of the test across countries, the results of comparison of mean scores and regression coefficients across countries would be ill-advised. One of the most important findings of this study is that measurement invariance of the IAT was sufficiently upheld. More specifically, factor form, first- and second-order factor loadings, latent factor, and observed item intercepts of the IAT were invariant across the three countries, even though residual variances of some items were not invariant. These residual variances reflect portions of variance of these items were not attributable to the factors of the IAT. A cross-group inequality might be the result of both non-systematic error variance and method variance that cast differential effects on the individual's item response. Future attention could be drawn on the interpretation of item 10 for Japanese adolescents, and items 13, 15, 16, 17, 18, and 20 for Malaysian adolescents, in which residual variances inequality were consistently reported. Researchers and practitioners are urged to interpret cross-sociocultural differences corresponding to these non-invariant items with caution.

In relation to the mean differences, even though all invariant item residual variances were fixed to be equal, the impact of measurement error could still be observed. Apart from the contrast between the Hong Kong and Japanese samples, the conclusions from the latent means and observed mean differences were not entirely consistent. The two methods disagreed, particularly in the comparison between Malaysian and Japanese adolescents. Although both statistical analytical methods tend to reflect the difference of the level of the higher-order construct “Internet Addiction”, the latent mean comparison was commented to be more precise and powerful because measurement errors are adjusted.²⁹

The measurement error inequality revealed in this study advised against group comparison of mean of the IAT with the use of traditional t test. As an illustration, incompatible findings of the two statistical comparison methods were found. On the basis of latent means comparison, no significant differences were obtained on the aggregated IAT score between Hong Kong, Japanese, and Malaysian adolescents, W&SP as it relates to Malaysian adolescents and TM&P as it relates to Japanese adolescents were significantly higher in Hong Kong adolescents; in addition, W&SP as it relates to Malaysian adolescents and RS as it relates to Hong Kong adolescents appeared to be higher in Japanese adolescents. Finally, Malaysian adolescents on average scored higher, in terms of latent mean, on TM&P than Japanese adolescents did. These findings, taken together, suggest that the three countries studied did not differ with respect to the overall severity on Internet addiction, yet were prone to different domains of symptoms associated with Internet addiction.

The present study fails to establish the strict factorial invariance of the IAT across countries. Nevertheless, this last step is generally considered to be difficult to achieve and a very stringent test of equivalence. It is therefore still arguable whether this step is necessary.^17,29,32,33 With the evidence of scalar invariance and partial strict factorial invariance, this study provides evidence for the equality of meaning of the Internet addiction construct in the three Asian countries, giving confidence in future use of the IAT in a cross-national fashion. From a practical standpoint, this study also demonstrates the critical effects of item measurement errors on observed group (country; i.e., socio-cultural backgrounds in this case) mean differences, which reminds future researchers to be careful that latent mean scores should also be taken into account alongside observed mean scores. Test adaptation should not be limited to linguistic analyses of translated instrument, but should also involve psychometric analyses of the instrument in order to assure the inferences derived from the score comparisons were not ill-founded.

This study only focused on the internal validity of the IAT. Other criteria for an effective and useful assessment such as predictive and concurrent validity were beyond the scope of this article. Methodologically, testing the measurement invariance of IAT using item response theory (IRT) analysis could confirm the results generated from CFA, especially when the relationship between the latent construct and the measured scores to which it is theoretically linked is non-linear. However, when multiple latent constructs and multiple populations need to be handled simultaneously, the CFA method is preferred to IRT. In addition to the technical concerns, there are only three Asian countries included in this study. Yet, Hong Kong, Japan, and Malaysia represent three very different cultures in Asia. Hong Kong is a place where East meets West, balancing a modernized way of life with traditional Chinese cultures. Malaysia is a multi-ethnic and multi-religious country, heavily influenced by Chinese and Indian culture. On the other hand, Japanese culture is relatively unique because of the long period of isolation of the island country in the Asia Pacific region. Therefore, caution is warranted when generalizing the present results to other Asian cultures. Furthermore, due to social desirability and peer pressure, adolescents may under-report their problems of Internet addition or overuse. In Caucasian populations, as mentioned in the Introduction, mixed findings concerning the factor structure of the IAT were reported in the past validation studies. This calls the generalization of the current model to non-Asian adolescents into question. To claim the IAT as a “culture neutral” instrument, further studies are needed to explore whether Asian adolescents conceptualize Internet addiction problems in the same way as their non-Asian counterparts.

Conclusions

In conclusion, the results of the present study provide good support for the reliability and factorial validity of the IAT construct. The findings also take a further step in understanding the internal validity of the IAT, namely, confirming its hierarchical three-factor structure to other Asian samples. In addition, using multigroup CFA, the measurement structure of the IAT was shown to be invariant across Hong Kong, Malay, and Japanese adolescents. The major implication is that the IAT generally measures the same construct across samples from different Asian countries in adolescents. This lays the foundation for future cross-sociocultural investigations of Internet addiction. Meaningful comparisons of statistics such as means and regression coefficients could only be made if the scale (i.e., the IAT) is a widely adapted assessment and comparable across different groups. The existence of non-invariant measurement errors of some items suggests that merely interpreting observed mean difference of the IAT could lead to erroneous inferences. After partialling out variance attributable to measurement error, no cross-country comparisons were found to result in significantly different levels of Internet addiction severity.

Footnotes

Author Disclosure Statement

No competing financial interests exist.

References

Block

. Issues for DSM-V: Internet addiction. American Journal of Psychiatry, 2008; 165:306–307.

Young

. (1998) Caught in the Net: how to recognize the signs of Internet addiction and a winning strategy for recovery. New York: John Wiley.

Durkee

, Kaess

, Carli

, et al. Prevalence of pathological Internet use among adolescents in Europe: demographic and social factors. Addiction, 2012; 107:2210–2222.

Johansson

, Götestam

. Internet addiction: characteristics of a questionnaire and prevalence in Norwegian youth (12–18 years). Scandinavian Journal of Psychology, 2004; 45:223–229.

Cao

, Su

. Internet addiction among Chinese adolescents: prevalence and psychological features. Child: Care, Health, & Development, 2007; 33:275–281.

Chou

, Hsiao

M-C

. Internet addiction, usage, gratification, and pleasure experience: the Taiwan college students' case. Computers & Education, 2000; 35:65–80.

Zhou

, Lin

F-C

, Du

Y-S

, et al. Gray matter abnormalities in Internet addiction: a voxel-based morphometry study. European Journal of Radiology, 2011; 79:92–95.

Khazaal

, Billieux

, Thorens

, et al. French validation of the Internet Addiction Test. CyberPsychology & Behavior, 2008; 11:703–706.

Ferraro

, Caci

, D'Amico

, et al. Internet addiction disorder: an Italian study. CyberPsychology & Behavior, 2007; 10:170–175.

10.

Korkeila

, Kaarlas

, Jääskeläinen

, et al. Attached to the web—harmful use of the Internet and its correlates. European Psychiatry, 2010; 25:236–241.

11.

Kim

. (2000) Internet addiction. Seoul, Korea: Nanum Press.

12.

, Isa

, Hashim

, et al. Validity of the Malay version of the Internet Addiction Test: a study on a group of medical students in Malaysia. Asia-Pacific Journal of Public Health, 2005; 27:2210–2219.

13.

Chang

, Man Law

. Factor structure for Young's Internet Addiction Test: a confirmatory study. Computers in Human Behavior, 2008; 24:2597–2619.

14.

Widyanto

, McMurran

. The psychometric properties of the Internet Addiction Test. CyberPsychology & Behavior, 2004; 7:443–450.

15.

Jelenchick

, Becker

, Moreno

. Assessing the psychometric properties of the Internet Addiction Test (IAT) in US college students. Psychiatry Research, 2012; 196:296–301.

16.

Lai

, Mak

, Watanabe

, et al. Psychometric properties of the Internet Addiction Test in Chinese adolescents. Journal of Pediatric Psychology, 2013; 38:794–807.

17.

Schmitt

, Kuljanin

. Measurement invariance: review of practice and implications. Human Resource Management Review, 2008; 18:210–222.

18.

American Psychiatric Association. (2000) Diagnostic and statistical manual of mental disorders. Washington, DC: American Psychiatric Association.

19.

Wang

, Zhou

, Lu

, et al. Problematic Internet use in high school students in Guangdong province, China. PloS One, 2011; 6:e19660.

20.

Bayraktar

, Gün

. Incidence and correlates of Internet usage among adolescents in North Cyprus. CyberPsychology & Behavior, 2007; 10:191–197.

21.

Milani

, Osualdella

, Di Blasio

. Quality of interpersonal relationships and problematic Internet use in adolescence. CyberPsychology & Behavior, 2009; 12:681–684.

22.

Widyanto

, Griffiths

, Brunsden

. A psychometric comparison of the Internet Addiction Test, the Internet-Related Problem Scale, and self-diagnosis. Cyberpsychology, Behavior, & Social Networking, 2011; 14:141–149.

23.

Satorra

, Bentler

. A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 2001; 66:507–514.

24.

Bentler

. Comparative fit indexes in structural models. Psychological Bulletin, 1990; 107:238–246.

25.

Bentler

, Bonett

. Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 1980; 88:588–606.

26.

Steiger

. Structural model evaluation and modification: an interval estimation approach. Multivariate Behavioral Research, 1990; 25:173–180.

27.

Bentler

. On the fit of models to covariances and methodology to the bulletin. Psychological Bulletin, 1992; 112:400–404.

28.

MacCallum

, Browne

, Sugawara

. Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1996; 1:130–149.

29.

Cheung

, Rensvold

. Evaluating goodness-of-fit indexes for testing MI. Structural Equation Modeling, 2002; 9:235–255.

30.

Chen

, Sousa

, West

. Testing measurement invariance of second-order factor models. Structural Equation Modeling, 2005; 12:471–492.

31.

Chen

. Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 2007; 14:464–504.

32.

Byrne

. Testing for the factorial validity, replication, and invariance of a measuring instrument: a paradigmatic application based on the Maslach Burnout Inventory. Multivariate Behavioral Research, 1994; 29:289–311.

33.

Vandenberg

, Lance

. A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 2000; 3:4–70.