Video Game Addiction Test: Validity and Psychometric Characteristics

Abstract

The study explores the reliability, validity, and measurement invariance of the Video game Addiction Test (VAT). Game-addiction problems are often linked to Internet enabled online games; the VAT has the unique benefit that it is theoretically and empirically linked to Internet addiction. The study used data (n=2,894) from a large-sample paper-and-pencil questionnaire study, conducted in 2009 on secondary schools in Netherlands. Thus, the main source of data was a large sample of schoolchildren (aged 13–16 years). Measurements included the proposed VAT, the Compulsive Internet Use Scale, weekly hours spent on various game types, and several psychosocial variables. The VAT demonstrated excellent reliability, excellent construct validity, a one-factor model fit, and a high degree of measurement invariance across gender, ethnicity, and learning year, indicating that the scale outcomes can be compared across different subgroups with little bias. In summary, the VAT can be helpful in the further study of video game addiction, and it contributes to the debate on possible inclusion of behavioral addictions in the upcoming DSM-V.

Introduction

In light of the upcoming DSM-V, Internet addiction (IA) is receiving renewed attention. It is often modeled along the components for addiction proposed by Griffiths.¹ These were also used as input for the 14-item Compulsive Internet Use Scale (CIUS).² However, to fit the specific nature of Internet behavior, the CIUS rephrased the component “relapse” as “loss of control,” while “tolerance” was excluded due to lack of relevance^2,3 (similar to the Internet addiction test^4,5). The CIUS defines IA as an addiction-like behavioral problem comprising: loss of control, conflict, preoccupation/salience, coping/mood modification, and withdrawal symptoms. The CIUS is one of the few validated IA scales, which has been subjected to a confirmatory factor analysis (CFA),^2,6 that is, it supports a one-dimensional measurement structure.

Besides scale construction at the epidemiological level, proposals for a clinical diagnosis of IA are being made.⁷ However, it remains questionable whether people are addicted to the Internet as such. In fact, problematic behavior tends to involve specific applications: online video games,^8,9 online erotica,¹⁰ or instant messaging services.¹¹ Distinguishing between these applications is relevant as the clinical nature of problems seems to vary per application.^12,13 In adolescents, playing online video games was shown to have the strongest relationship with IA.¹⁴

While several studies demonstrate the existence of a small subgroup of problematic gamers^15–17 and there is a clear link with IA, no consensus has been achieved regarding the measurement of video game addiction. We therefore propose to modify the CIUS to refer specifically to gaming, for use in contexts, where problems with games are of particular interest. Through the CIUS, which incorporates established models for defining behavioral addiction,^1,2 video game addiction is more specifically defined as: (a) a loss of control over gaming, (b) conflicts about the gaming behavior and problems associated with gaming, (c) preoccupation with gaming, (d) using gaming as a way to cope with problems or to modify your mood, and (e) withdrawal symptoms if the gamer is forced to quit. In doing so, we broadly define video game addiction as a loss of control (a, c, d, e) over the gaming behavior, which results in significant personal, professional, or social harm (b).^1,18 The present study aims to validate this new Video game Addiction Test (VAT).

Method

Sample

This study uses the 2009 sample of the Monitor Study “Internet and Youth.” The sample includes ten Dutch secondary schools and 171 classes (n=4,074), average per class response rate 92 percent. For details on the procedure, see earlier publications.^14,19 The proposed VAT items were skipped by adolescents who did not play games at all (n=1,024). For the purpose of scale validation, adolescents with less than four of the items on the VAT completed were removed (n=156). As the analyses deal with scale means, missing values do not distort scores. The final dataset has 2,894 cases (62 percent boys), of which 99 percent completed 9 or more VAT items. Overall average age was 14.3 (SD=1.0) years.

Measures

Compulsive Internet Use Scale

The 14-item version of the CIUS² was used to measure CIU, and includes questions, such as “Have you unsuccessfully tried to spend less time on the Internet?” The CIUS contains the following components: loss of control (four items), conflict (four items), preoccupation/salience (three items), coping/mood modification (two items), and withdrawal symptoms (one item). The CIUS has good validity² and internal reliability^2,10,14 (current sample, Chronbach's α=0.89).

Video game Addiction Test

The items that compose the CIUS were rephrased into the 14 proposed VAT items, measured with a with a 5-point scale: 0, never; 1, seldom; 2, sometimes; 3, often; and 4, very often (Table 1). Consequently, the VAT incorporates the same components as the CIUS. While we strongly suspect that online games play a particularly important role,²⁰ playing multiple game types is not unusual¹⁴ and problematic use of other game types cannot be ruled out. Thus, the phrasing was kept universal (i.e., “game” instead of “online game”).

Table 1.

Factor Loadings and Item Phrasing for the 14 Video Game Addiction Test Items

Item ^a	How often …	Standardized factor loadings
1	… do you find it difficult to stop gaming	0.72
2	… do you continue to use the games, despite your intention to stop?	0.71
3	… do others (e.g., parents or friends) say you should spend less time on games?	0.69
4	… do you prefer to game instead of spending time with others (e.g., friends or parents)?	0.72
5	… do you not get enough sleep because of gaming?	0.69
6	… do you think about gaming, even when you're not online?	0.76
7	… do you look forward to the next time you can game?	0.75
8	… do you think you should be gaming less often?	0.65
9	… have you unsuccessfully tried to spend less time on gaming?	0.64
10	… do you feel restless, frustrated, or irritated when you cannot game?	0.78
11	… do you rush through your homework to play games?	0.70
12	… do you neglect to do your homework because you prefer to game?	0.68
13	… do you game because you are feeling down?	0.64
14	… do you game to forget about problems?	0.62

Items are translated from Dutch.

Game Addiction Scale

The seven-item version of the Game Addiction Scale (GAS) was included.²¹ Six of the seven items each measure one of Griffiths' components of behavioral addiction, while one question measures “problems.”⁸ The GAS shows good reliability in this sample (Chronbach's α=0.84).

Weekly hours online, casual browser, and offline gaming

Three types of games are distinguished: multiplayer online games, which are video games played with other people over the Internet; casual browser games, which are simple games that are played online via the web browser; and finally, offline games, which do not utilize the Internet. Hours per week spent on these game types were obtained by multiplication of two questions measuring days per week of gaming and average hours of gaming per day, similar to a previous study.¹⁹

Psychosocial measures

The following psychosocial measures were included: Rosenberg's Negative Self-Esteem Scale^19,22 (Chronbach's α=0.87), the UCLA Loneliness Scale^23,24 (Chronbach's α=0.87), the Depressive Mood List^25–27 (Chronbach's α=0.81), and the Revised Social Anxiety Scale for Children^28–30 (Chronbach's α=0.89). These scales have been used in Dutch studies and have demonstrated good reliability.^3,11,19 For all four scales, a higher score indicates more reported problems.

Strategy of analyses

To confirm if the one-factor structure of the CIUS also holds for the VAT, a 1-factor CFA was applied using Mplus version 5.1.³¹ On grounds of overlapping content, some error terms of items must be correlated to get an acceptable model fit. We correlated the same error terms as reported for the CIUS (item 1 with item 2, 6 with 7, 8 with 9, 11 with 12, and 13 with 14).² Additionally, as the scale is formulated for adolescents, the error term for item 3 was correlated with item 8 on grounds of content overlap. To deal with missing data, the Full Information Maximum Likelihood estimation technique was used. Respondents are nested within schools and classes. Due to software limitations, the COMPLEXTWOLEVEL procedure, which corrects for this nonindependency, did not work. As the differences between separate COMPLEX procedures for schools and classes were negligible, we decided to report the results for classes (most likely point of mutual influence). The COMPLEX procedure uses a robust maximum likelihood estimator resulting in robust χ² values. Model fit is reported in robust χ² values, and estimated degrees of freedom, the Comparative Fit Index (CFI)³² and the root-mean-squared error of approximation (RMSEA).³³

The second step was to test measurement invariance. Various forms of measurement invariance can be distinguished,³⁴ but we restrict ourselves to the three forms proposed by Steenkamp and Baumgartner.³⁵ If the fit of the CFA model is acceptable across different groups, configural invariance is supported. Metric invariance requires that the factor loadings (λ) of corresponding factors are invariant (and thus have the same meaning) across groups. Scalar (or strong) invariance requires that the item intercepts (τ) of corresponding factors are invariant across groups: this makes comparisons of group means meaningful.

The three forms of measurement invariance were tested for three background variables: gender, ethnicity (Dutch or Western European versus non-Western European ethnicity), and learning year. Learning year was operationalized as learning one (average age 13), two, and a mixed group containing years three and four (the fourth learning year has a small sample size; n=120). After fitting a baseline (noninvariant) model (representing configural invariance) with no constraints, metric and scalar invariance are tested by constraining factor loadings (metric invariance) and intercepts (scalar invariance) subsequently across groups. If the increase in χ² (from the baseline model to λ-constrained model or from the λ-constrained model to the τ-constrained model) is nonsignificant, metric and/or scalar invariance is supported. The robust χ² values were rescaled to standard χ² values to compute a correct χ² difference.

As the χ² test is sensitive to sample size,³⁶ we also examined the increment of fit indices (ΔCFI³⁷ and ΔRMSEA) between models. We follow Cheung and Rensvold's rule of thumb that a decrease in CFI of .01 or less indicates that invariance should not be rejected.³⁷ Construct validity is tested by cross comparisons with the GAS,²¹ the CIUS,² measures of psychosocial well-being^19,38,39 and time spent on various game types.

Results

Step 1: confirmatory factor analysis

The fit of the factor model was good: χ² (71)=549.564, RMSEA=0.048, and CFI=0.961 (Table 1), with high reliability (Chronbach's α=0.93). All loadings are >0.62 indicating a good construct validity.

Step 2: measurement invariance

Table 2 summarizes the results for the unconstrained model, the model with constrained factor loadings (metric invariance), and the model with constrained intercepts and constrained factor loadings (scalar invariance) for each of the three background variables. Configural invariance is supported by the high values of CFI (>0.95) and low values of RMSEA (<0.05). Metric invariance shows significant increases in χ² for gender and ethnicity (p<0.05), but not for the learning year. The decreases in the CFI-value were 0.001, 0.001, and 0.000, respectively, indicating that metric invariance is supported to a high degree. With respect to scalar invariance, the χ² difference values have p-values<0.001, supporting scalar noninvariance. However, the decreases in CFI were 0.004, 0.004, and 0.005, respectively, and are lower than the Cheung/Rensvold criterion of 0.01. Our conclusion is that scalar invariance of the VAT should not be rejected.

Table 2.

Tests of Invariance Constraints (Metric and Scalar) for Gender, Ethnicity, and Learning Year

				χ² test of Model Fit			χ² difference
Hierarchical invariance testing steps		CFI	RMSEA	Value	Df	p	Δχ²	Δdf	p
Tests of invariance constraints for male and female respondents
1	Noninvariant	0.961	0.046	578.37	142	0.000
2	λ¹ invariant	0.960	0.045	601.80	155	0.000	24.17	13	0.030
3	λτ invariant	0.956	0.045	661.86	168	0.000	66.91	13	0.000
Tests of invariance constraints for ethnicity (Dutch or Western European vs. Non-Western European)
1	Noninvariant	0.959	0.049	605.12	142	0.000
2	λ constrained	0.958	0.047	629.58	155	0.000	23.58	13	0.035
3	λτ invariant	0.954	0.048	689.69	168	0.000	66.18	13	0.000
Tests of invariance constraints learning year (first year/second year/third and fourth year)
1	Noninvariant	0.961	0.050	726.98	213	0.000
2	λ constrained	0.961	0.047	754.81	239	0.000	29.34	26	0.296
3	λτ invariant	0.956	0.048	843.33	265	0.000	92.07	26	0.000

CFI, Comparative Fit Index; RMSEA, root-mean-squared error of approximation; λ, factor loadings; τ, item intercepts.

Step 3: construct validity

Table 3 demonstrates the construct validity of the VAT: strong relationships are found with related scales, that is, the GAS (r=0.74) and the CIUS (r=0.61), while small, but practically significant relationships⁴⁰ are found with scales measuring psychosocial well-being (depressive mood, negative self-esteem, loneliness, and social anxiety) and the time spent on various game types (r<0.40). Of the various game types, online gaming (hours per week) shows the strongest relationship with the VAT (r=0.37).

Table 3.

Pearson's Correlations Between Game and Internet Addiction Scales, Psychosocial Outcome Measures, and Measures of Time Spent on Various Game Types

	1	2	3	4	5	6	7	8	9
1. Video game Addiction Test	1.00
2. Video game Addiction Scale	0.74^***	1.00
3. Compulsive Internet Use Scale	0.61^***	0.63^***	1.00
4. Depressive Mood List	0.29^***	0.27^***	0.40^***	1.00
5. UCLA Loneliness scale	0.29^***	0.23^***	0.23^***	0.38^***	1.00
6. Social Anxiety for Children Revised	0.22^***	0.20^***	0.22^***	0.45^***	0.47^***	1.00
7. Rosenberg Negative Self-Esteem Scale	0.22^***	0.18^***	0.27^***	0.50^***	0.54^***	0.46^***	1.00
8. Online games (hours per week)	0.37^***	0.38^***	0.37^***	0.03	0.09^**	0.04	0.07^*	1.00
9. Casual browser games (hours per week)	0.19^***	0.17^***	0.19^***	0.09^***	0.12^***	0.07^***	0.07^***	0.08^**	1.00
10. Offline games (hours per week)	0.25^***	0.25^***	0.11^***	0.01	0.01	0.00	0.00	0.22^***	0.24^***

Calculations were performed with scale means, ^*p<0.05; ^**p<0.01; ^***p<0.001.

Discussion

The present study proposes a new, one-dimensional scale to measure video game addiction: the VAT. This scale defines video game addiction as an addiction-like behavioral problem that presents as: a loss of control over gaming, intra- and interpersonal conflict, preoccupation with gaming, using of games for purposes of coping or mood modification, and withdrawal symptoms if the gamer is forced to quit.

The VAT demonstrated excellent reliability, a one-factor model fit, and measurement invariance (configural, metric) across gender, ethnicity, and learning year, indicating that the scale outcomes can be compared across different subgroups with little bias. Scalar invariance is established to a high degree, which means that there are indications that (at comparable test scores) boys, adolescents with a Dutch or Western European ethnicity, and older adolescents might have slightly higher true scores than their counterparts. Additionally, good construct validity was established; the VAT showed a strong relationship with an alternative measure of game addiction (GAS) and it's source—the CIUS, while showing weaker positive relationships with measures of psychosocial health and time spent on various game types.

A limitation of this study is the use of self-report data, which means that it is hard to rule out whether the extreme group is psychologically dependent on games or merely very highly involved.⁴¹ This does not negate the proposed scale: those who score high on VAT indicate considerable problems. Finally, the sample consisted of exclusively of young adolescents. Further validation is needed before applying the VAT to different populations. As the source construct CIUS has been demonstrated to be valid and reliable for adult populations and at least strongly reliable in adolescent populations,³ we do not foresee any problems here.

A variety of attempts have been made to measure video game addiction.^15,21,42 Although this earlier work has been fruitful (i.e., the GAS, which correlates strongly with the VAT), the problem has remained that IA and game addiction are generally studied independently, while it is evident that game-addiction problems are strongly linked to Internet-enabled online games.^17,20 The high correlation between online games (in comparison with other game types) and video game addiction in the current study provides further support for this idea (see Table 3). The VAT has the benefit that it is theoretically and empirically (r=0.61) linked to an IA scale: that is, it's source, the CIUS. As such, the VAT can be employed in further survey research to support theoretical progress in the field of online video game addiction. Moreover, findings provide a starting point for clinical validation of the VAT.

Footnotes

Acknowledgments

The authors thank the following organizations for funding data collection of the Monitor Study “Internet and Youth”: Netherlands Organization for Health Research & Development (ZonMw, project no. 31160208), Volksbond Foundation Rotterdam, Addiction Care North Netherlands, Kennisnet Foundation, Tactus Addiction Care, and De Hoop Foundation. Additionally, we thank Lieneke Spel and Leonard Vanbrabant for their assistance during the data collection phase. Finally, we thank Laraine Visser for her assistance with the English language.

Author Disclosure Statement

No competing financial interests exist.

References

Griffiths

. A “components” model of addiction within a biopsychosocial framework. Journal of Substance Use, 2005; 10:191–197.

Meerkerk

G-J

, Van den Eijnden

RJJM

, Vermulst

et al. The compulsive internet use scale (CIUS): some psychometric properties. CyberPsychology and Behavior, 2009; 12:1–6.

Meerkerk

G-J

. Pwned by the internet. Explorative research into the causes and consequences of compulsive internet use [Ph.D. thesis] Rotterdam: Erasmus University Rotterdam, 2007.

Widyanto

, McMurran

. The psychometric properties of the Internet Addiction Test. CyberPsychology and Behavior, 2004; 7:443–450.

Young

. 1998. Caught in the net: how to recognize the signs of Internet addiction—and a winning strategy for recovery. New York: Wiley.

Caplan

. Theory and measurement of generalized problematic Internet use: a two-step approach. Computers in Human Behavior, 2010; 26:1089–1097.

Tao

, Huang

, Wang

et al. Proposed diagnostic criteria for internet addiction. Addiction, 2010; 105:556–564.

Mehroof

, Griffiths

. Online gaming addiction: the role of sensation seeking, self-control, neuroticism, aggression, state anxiety, and trait anxiety. Cyberpsychology, Behavior, and Social Networking, 2010; 13:313–316.

Wan

C-S

, Chiou

W-B

. Why are adolescents addicted to online gaming? An interview study in Taiwan. CyberPsychology and Behavior, 2006; 9:762–766.

10.

Meerkerk

G-J

, Van den Eijnden

RJJM

, Garretsen

HFL

. Predicting compulsive internet use: it's all about sex! CyberPsychology and Behavior, 2006; 9:95–103.

11.

Van den Eijnden

RJJM

, Meerkerk

G-J

, Vermulst

et al. Online communication, compulsive internet use, and psychosocial well-being among adolescents: a longitudinal study. Developmental Psychology, 2008; 44:655–665.

12.

Bernardi

, Pallanti

. Internet addiction: a descriptive clinical study focusing on comorbidities and dissociative symptoms. Comprehensive Psychiatry, 2009; 50:510–516.

13.

Van Rooij

, Zinn

, Schoenmakers

et al. Treating internet addiction with cognitive-behavioral therapy: a thematic analysis of the experiences of therapists. International Journal of Mental Health and Addiction, 2012; 19,10:69–82.

14.

Van Rooij

, Schoenmakers

, Van den Eijnden

RJJM

et al. Compulsive internet use: the role of online gaming and other internet applications. The Journal of Adolescent Health, 2010; 47:51–57.

15.

Grüsser

, Thalemann

, Griffiths

. Excessive computer game playing: evidence for addiction and aggression? CyberPsychology and Behavior, 2007; 10:290–292.

16.

Peters

, Malesky

. Problematic usage among highly-engaged players of massively multiplayer online role playing games. CyberPsychology and Behavior, 2008; 11:481–484.

17.

Rehbein

, Psych

, Kleimann

et al. Prevalence and risk factors of video game dependency in adolescence: results of a German nationwide survey. Cyberpsychology, Behavior, and Social Networking, 2010; 13:269–277.

18.

West

, Hardly

. 2006. Theory of addiction. Oxford: Wiley-Blackwell, 225.

19.

Van Rooij

, Schoenmakers

, Vermulst

et al. Online video game addiction: identification of addicted adolescent gamers. Addiction, 2011; 106:205–212.

20.

Council on Science and Public Health. 2007. Emotional and behavioral effects, including addictive potential, of video games. Chicago: American Medical Association. www.ama-assn.org/resources/doc/csaph/csaph12a07-fulltext.pdf.

21.

Lemmens

, Valkenburg

, Peter

. Development and validation of a game addiction scale for adolescents. Media Psychology, 2009; 12:77–95.

22.

Rosenberg

. 1989. Society and the adolescent self-image. Revised. Middletown: Wesleyan University Press.

23.

Ghassemzadeh

, Shahraray

, Moradi

. Prevalence of internet addiction and comparison of internet addicts and non-addicts in Iranian high schools. CyberPsychology and Behavior, 2008; 11:731–733.

24.

Russell

, Peplau

, Cutrona

. The revised UCLA loneliness scale: Concurrent and discriminant validity evidence. Journal of Personality and Social Psychology, 1980; 39:472–480.

25.

Engels

RCME

, Finkenauer

, Meeus

WHJ

et al. Attachment and adolescents' emotional adjustment: the associations with social skills and relational competence. Journal of Counseling Psychology, 2001; 48:428–439.

26.

Kandel

, Davies

MNO

. Epidemiology of depressive mood in adolescents: an empirical study. Archives of General Psychiatry, 1982; 39:1205–1212.

27.

Kandel

, Davies

MNO

. Adult sequelae of adolescent depressive symptoms. Archives of General Psychiatry, 1986; 43:255–262.

28.

La Greca

, Dandes

, Wick

et al. Development of the social anxiety scale for children: reliability and concurrent validity. Journal of Clinical Child Psychology, 1988; 17:84–91.

29.

La Greca

, Stone

. Social anxiety scale for children–revised: factor structure and concurrent validity. Journal of Clinical Child Psychology, 1993; 22:17–27.

30.

Gross

. Adolescent Internet use: what we expect, what teens report. Journal of Applied Developmental Psychology, 2004; 25:633–649.

31.

Muthén

, Muthén

. 2007. Mplus user's guide. Fifth. Los Angeles, CA: Muthén & Muthén.

32.

Bentler

. Comparative fit indexes in structural models. Psychological Bulletin, 1990; 107:238–246.

33.

Byrne

. 1998. Structural equation modeling with LISREL, PRELIS and SIMPLIS: basic concepts, applications, and programming. Mahwah, NJ: Lawrence Erlbaum Associates.

34.

Gregorich

. Do self-report instruments allow meaningful comparisons across diverse population groups? Testing measurement invariance using the confirmatory factor analysis framework. Medical Care, 2006; 44:S78–S94.

35.

Steenkamp

JBEM

, Baumgartner

. Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 1998; 25:78–107.

36.

Schermelleh-Engel

, Moosbrugger

, Müller

. Evaluating the fit of structural equation models: tests of significance and descriptive goodness-of-fit measures. Methods of psychological research online [now: Methodology (Gott)], 2003; 8:23–74.

37.

Cheung

, Rensvold

. Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 2002; 9:233–255.

38.

Caplan

. Relations among loneliness, social anxiety, and problematic internet use. CyberPsychology and Behavior, 2007; 10:234–242.

39.

, Kim

, Bae

et al. Depression and internet addiction in adolescents. Psychopathology, 2007; 40:424–430.

40.

Ferguson

. An effect size primer: a guide for clinicians and researchers. Professional Psychology: Research and Practice, 2009; 40:532–538.

41.

Charlton

, Danforth

IDW

. Distinguishing addiction and high engagement in the context of online game playing. Computers in Human Behavior, 2007; 23:1531–1548.

42.

Wölfling

, Müller

. Pathologisches glücksspiel und computerspielabhängigkeit. Bundesgesundheitsblatt—Gesundheitsforschung—Gesundheitsschutz, 2010; 53:306–312.