Internet Gaming Disorder and Well-Being: A Scale Validation

Abstract

The overuse of online games is known to be inversely related to various indicators of well-being. This article validates the DSM-5 criteria of internet gaming disorder (IGD), and analyzes its links with five indicators of well-being: life satisfaction, loneliness, anxiety, depression, and academic performance in a French-speaking sample of 693 gamers. Exploratory and confirmatory factor analyses showed a one-factor structure of IGD criteria. The IGD scale showed satisfactory validity and reliability and was related in a consistent way with well-being measures. The IGD scale appears to be an appropriate measure to assess video game addiction and will contribute to increase the comparability of international research on video game addiction.

Introduction

Video games represent a very popular leisure activity in many countries. However, several early case studies¹ and empirical studies² led scientists and clinicians to wonder whether intense engagement with videogames over a sustained period of time could be associated with psychological and social problems among players and whether these problems shared communalities with other addictions. Studies documented that excessive playing does predict cognitive and behavioral symptoms usually related to drug addiction.^3,4 For example, a minority of heavy players experience a powerful urge to play video games (craving and do not succeed to stop playing despite negative consequences [loss of control]).

Over the last 10 years, research on video game addiction has sharply increased. Most researchers consider video game addiction as the excessive use of video games that cannot be controlled, despite negative consequences for the player.^5,6 Although video game addiction is the popular term, researchers used various terminologies, such as compulsive internet use,⁷ pathological game use,⁸ game addiction,⁸ or problematic video game playing,⁹ to address the same phenomenon.

Several instruments have been recently developed to measure video game addiction.¹⁰ However, the available measures do not allow optimal cross-study comparisons. First, the scales do not use the same diagnostic criteria. Some instruments use diagnostic criteria characteristic of substance addiction and other instruments use diagnostic criteria characteristic of behavioral addiction. One study showed that only one criterion was present in all instruments, and the other criteria were more variable.¹⁰ Researchers have also used different cut-off scores to indicate clinical status. Some instruments consider a person dependent if he/she meets half of the criteria of addiction^9,8 while other instruments consider a person is dependent if he/she meet all of the criteria of addiction.⁶ Hence, studies that examined the prevalence of video game addiction do not agree on the prevalence rate, partly because of differences in assessment tools.

As many professionals and scholars acknowledge, a unified definition of video game addiction and the availability of a unique assessment tool would represent a critical evolution in the field. The Fifth revision of the Diagnostic and Statistical Manual for Mental Disorders, introduced the construct of Internet gaming disorder (IGD), which refers to the problematic use of both online and offline video games over the last 12 months. This disorder was included in the Section 3 of the DSM to foster further studies before being considered a formal disorder. The DSM-5 definition proposes nine criteria that can reflect IGD, but these criteria must still be validated in future studies to test if each criterion is useful for diagnosis. These criteria are: (a) Preoccupation with the game that consists in spending a great amount of time playing video games or of thinking about gaming, even during times of not playing. (b) Withdrawal, which is characterized by symptoms like anxiety or irritability, occurring when playing is impossible. (c) Tolerance, which refers to the need to play more and more each time to experience the same satisfaction as when the player just started playing. (d) Persistence: the impossibility to control, to stop or to reduce gaming. (e) Escape, which refers to playing to avoid negative mood, like anxiety or depression. (f) Problems that indicate the persistence in gaming despite being aware of problems caused by the game. (g) Deception that consists in lying to others about the time played. (h) Displacement, which is the loss of interest in other hobbies or activities. (i) Conflict, which refers to losing a relationship or an important opportunity for employment to play video game. To diagnose IGD on a pathological level, the DSM proposes that at least five of these nine criteria must be present during a period of 12 months.

The inclusion of IGD in the DSM-5 is an important advancement toward a unified definition of IGD. The next step is to validate the criteria proposed by the DSM-5 in a large population and to develop a unique and standardized instrument based on the criteria proposed by the DSM-5 to measure this phenomenon. The DSM does not provide items to measure this disorder. Recently, international experts of video game addiction found a consensus and adapted each criterion of the DSM-5 into several items.¹¹

Although there already exist instruments measuring addiction based on the DSM-5 criteria in other European populations,^12–14 there is still no validated instruments assessing IGD based on the DSM-5 criteria in French. Our aim was, therefore, to validate the scale of IGD based on the DSM-5 criteria in a French population and to investigate its links with relevant well-being measures.

The present research

We used the items reflecting each of the nine criteria, proposed by Petry et al.¹¹ Each item reflects one criterion of the DSM-5, thus the scale we used has a total of nine items. First, we assessed the reliability and factor structure of the IGD scale. Then, we provided the cross-validity, and criterion validity of this measure. We tested cross-validity by splitting our sample in two independent subsamples and testing the reliability and factorial structure across the independent samples. Criterion validity refers to the correlation of the scale with constructs that are theoretically related with the scale. We, therefore, included in our survey several scales measuring psychological constructs related to video game addiction. Based on previous studies,¹² we expected that IGD scale would be positively correlated with time spent playing games, loneliness, anxiety, and depression, and negatively with life satisfaction and academic self-efficacy. We also expected that IGD scale would be related to another measure of video game addiction recently validated although not based on DSM criteria, the Game Addiction Scale (GAS).¹⁵

Methods

Participants

A total of 1,217 French people were recruited through Internet (over Online game forum or Facebook) or approached at the university campus to participate in this study. Two hundred thirty-three questionnaires were incomplete or included inappropriate data, so they were excluded from the analysis. Out of the 984 remaining participants, 70 percent indicated that they practiced video games. The final sample was composed of 693 players, mainly composed of male gamers (63.8 percent, N = 442). The mean age of participants was 22.68 (SD = 5.08).

Measures

Internet Gaming Disorder

Participants playing online or offline video games answered each of the nine items proposed by Petry et al.¹¹ referring to their behavior during the last year (Appendix A). They rated each item on a 6 points scale ranging for 1 (not at all) to 6 (totally). The final score was obtained by calculating the mean score of the nine items for every participant.

Time spent on game

To measure the time spent on the video game, two types of gaming were distinguished: video game online and video game offline. For each type of video game, participants estimated time played during the school day and during the weekend. Weekly time of gaming is obtained by multiplying the days of playing with the number of hours per day spent on gaming.

Life satisfaction

To assess life satisfaction, participants answered the five items (e.g., in most ways my life is close to my ideal) of the Satisfaction with Life Scale¹⁶ (α = 0.80). Answers ranged from 1 (totally disagree) to 7 (totally agree). A high score reflected high satisfaction with life. We used the validated French version of this scale.

Loneliness

Loneliness was measured with the eight items of the French version of UCLA Loneliness Scale¹⁷ (e.g., I feel completely alone) (α = 0.66). For each item, participants answered on a 7-point scale ranging from 1 (totally disagree) to 7 (totally agree). A high score reflected high loneliness.

Academic performances

Participants answered three items measuring academic performances and self-efficacy¹⁸ on a 4-point scale ranking for 1 (totally disagree) to 4 (totally agree).

Anxiety and depression

To measure anxiety and depression, we used the French version of Hospital Anxiety and Depression Scale (HADS)¹⁹. HADS includes 14 items: 7 referring to anxiety symptoms (e.g., I feel tense or edgy) and 7 to depression symptoms (e.g., I feel idling). Reponses ranged from 0 to 3. A high score reflected high anxiety or depression.

French validation of GAS

The GAS is a French questionnaire measuring game addiction¹⁵ and it was added at the end of the questionnaire to test for convergent validity. This measure is not based on DSM-5 criteria and it is validated only on an adolescent population. It includes seven items scored (e.g., Did you spend more and more time on videogame?) (α = 0.84) on a 5-point scale ranking for 1 (never) to 5 (very often).

Sociodemographics

Finally, the questionnaire included items related to gender, age, and two control questions. To ensure that participants paid attention during the whole study, we asked them to give a specific answer to one question at the end of the questionnaire. We excluded from the analysis every participant who gave a different answer to the control questions.

Procedure

We randomly split the original sample in two subsamples. In the first sample, we conducted an exploratory factor analysis (EFA) with Varimax rotation to specify the dimensions of the IGD scale. In the other sample, we performed a confirmatory factor analysis (CFA) using maximum likelihood estimator, to confirm the factor solution found with EFA in an independent sample. The CFA was conducted with R software using “lavaan” library. We used several indices to assess the model fit: chi-square statistic, comparative fit index (CFI), standardized root mean square residual, and Tucker–Lewis Index (TLI). Ideally, CFI values should be greater than 0.95, TLI values should be greater than 0.90, root mean square error of approximation (RMSEA) values must be smaller than 0.08 and the χ² must be nonsignificant. It is important to note that the χ² increases with the sample size, thus in large samples this index is often significant regardless of fit. We used Cronbach's alpha to measure internal consistency. To determine the invariance of internal consistency in different subsamples we performed one Cronbach's alpha in each subsample and we looked at the factor structure. Finally, to determine the criterion validity of the IGD scale we correlated the mean score of the IGD scale with time spent on games, life satisfaction, loneliness, anxiety, and depression, and the score obtained with the GAS.

Results

Table 1 presents sociodemographic information of our sample. The majority of the sample (35.2 percent, N = 244) played 1–2 hours on a week.

Table 1.

Socio Demographic Information of the Sample

Demographics	Total sample
N	693
Age: years mean (SD)	22.68 (5.87)
Time spent playing offline games during the school day
Less than 1 hour	193 (27.8)
1–2 hours	244 (35.2)
2–4 hours	113 (16.3)
4–6 hours	10 (1.4)
6–8 hours	1 (0.1)
More than 8 hours	5 (0.7)
Time spent playing online games during the school day
Less than 1 hour	127 (18.3)
1–2 hours	216 (31.2)
2–4 hours	132 (19.0)
4–6 hours	19 (2.7)
6–8 hours	9 (1.3)
More than 8 hours	10 (1.4)

Exploratory factor analysis

We randomly split the original sample in two parts, to have two independent subsamples. The first sample (N = 332) was subjected to an EFA to determine the underlying factor structure. The EFA was performed using the maximum likelihood estimator and Varimax rotation method. Two indices were used to determine the number of factors: the scree plot and the number of eigenvalues greater than 1. This analysis revealed a single factor solution that explained 35.26 percent of the variance. The factor loadings for every item are greater than 0.50, except for one item that has a factor loading of 0.42 (Table 2). The eigenvalue of the first factor was above 1, while the others were below 1 (see the eigenvalues in Table 3). Thus, the CFA suggests that our scale is one-dimensional.

Table 2.

Mean, Standard Deviation, and Factor Loading of the Two Samples

Item	Overall sample mean (SD)	Sample 1 (N = 332) mean (SD)	Sample 2 (N = 361) mean (SD)	Total sample factor loading	Sample 1 (N = 332) factor loading	Sample 2 (N = 361) factor loading
1	3.05 (1.52)	3.15 (1.55)	2.97 (1.49)	0.58	0.52	0.64
2	1.68 (1.04)	1.73 (1.08)	1.62 (1.01)	0.65	0.64	0.66
3	2.01 (1.45)	1.99 (1.32)	2.03 (1.40)	0.50	0.42	0.57
4	2.08 (1.27)	2.13 (1.50)	2.03 (1.41)	0.73	0.73	0.72
5	1.85 (1.27)	1.88 (1.30)	1.83 (1.26)	0.65	0.65	0.65
6	2.50 (1.60)	2.57 (1.62)	2.45 (1.57)	0.63	0.63	0.54
7	1.67 (1.28)	1.75 (1.33)	1.59 (1.24)	0.63	0.63	0.63
8	2.53 (1.72)	2.63 (1.77)	2.43 (1.67)	0.50	0.50	0.41
9	1.30 (0.90)	1.33 (0.99)	1.26 (0.88)	0.52	0.52	0.46

Table 3.

Eigenvalues

Factor	All sample	Sample 1 (N = 332)	Sample 2 (N = 361)
1	3.8	3.8	3.8
2	0.8	0.9	0.9
3	0.8	0.8	0.8
4	0.7	0.7	0.7
5	0.6	0.6	0.6
6	0.6	0.5	0.5
7	0.5	0.5	0.5
8	0.5	0.5	0.4
9	0.4	0.4	0.3

Confirmatory factor analysis

We performed a CFA on the second sample (N = 361) to confirm the single factor solution found with the EFA in the first sample. We tested a one-factor model. The chi-square was significant, however, it is not unusual to have a significant χ² with large samples.²⁰ Other indices showed acceptable fit (CFI = 0.95, TLI = 0.94, RMSEA = 0.061 with a 90% CI [0.04, 0.07]). Thus, the CFA confirmed the single-factor structure for this scale.

Cross-validation of reliability

The cross-validation of reliability is a critical issue in the problematic online game use scale.²¹ The reliability of the total sample was good (α = 0.82). Moreover, when the two subsamples were separately subjected to the reliability analyses, we found the same values. The first sample has a Cronbach's alpha of 0.82, and the second sample of 0.82. Moreover, the reliability of the IGD scale did not increase when removing one of its items. These results showed that the IGD scale has a good reliability and a good population cross-validity. Cross-validity was also shown by the same factor structure found in both samples. This result shows that the reliability of IGD scale can be generalized beyond the original sample.

Concurrent validity

To examine the concurrent validity, we correlated the IGD score with time spent on video games daily and during the weekend, loneliness, life satisfaction, academic performance, anxiety, and depression. Moreover we tested whether our measure correlates with another validated measure of video game addiction.

Results showed that the IGD scale has a strong correlation with various indicators of well-being. A positive correlation was found between IGD scale and loneliness (r = 0.25, p < 0.001), between IGD scale and depression (r = 0.31, p < 0.001), and between IGD scale and anxiety (r = 0.23, p < 0.001). A negative correlation was found between IGD scale and life satisfaction (r = −0.22, p < 0.001) and between IGD scale and academic performance (r = −0.17, p < 0.001). As displayed in Table 4 these correlations were similar in the two subsamples.

Table 4.

Correlation Analysis Between Internet Gaming Disorder Scale and Related Constructs

Constructs	Total sample (N = 693)	Sample 1 (N = 332)	Sample 2 (N = 361)
Time spent on game	0.42^**	0.38^**	0.46^**
Life satisfaction	−0.22^**	−0.24^**	−0.21^**
Loneliness	0.25^**	0.23^**	0.27^**
Anxiety	0.23^**	0.24^**	0.24^**
Depression	0.31^**	0.35^**	0.28^**
Academic performance	−0.17^**	−0.17^**	−0.17^**
Game Addiction Scale	0.76^**	0.75^**	0.76^**

p < 0.01.

Finally, to identify the predictors of IGD, we performed a stepwise multiple regression. Life satisfaction, loneliness, anxiety, depression, academic performance, and terms of interaction between them were entered as predictors with IGD scale score as criterion. Terms of interaction were not significant, and were removed. With an adjusted R² of 0.11, there were three significant predictors of IGD: depression (β = 0.29; p < 0.001), academic performance (β = −0.09; p < 0.009), and loneliness (β = 0.10; p < 0.02). This analysis confirms the important link between depression and the problematic use of screens.²² Moreover, the high correlation between IGD scale and GAS (r = −0.76, p < 0.001) highlights the convergent validity of these measures.

Discussion and Conclusion

The aim of this study was to evaluate the validity and reliability of items reflecting the DSM-5 criteria and the links between IGD and various well-being measures with a French population. The EFA suggested a one-factor solution, later confirmed by the CFA. These results are consistent with other studies testing factor solutions for other scales measuring video game addiction in other populations.¹³ It is perhaps worth noting, however, that unidimensionality is not a critical issue for many mental health disorders. People can be validly diagnosed with depression despite showing very different patterns of symptoms. Therefore, studies that claim to find different numbers of underlying factors do not truly contradict each other in this domain, which is different from many other areas, where the theoretical assumption is that an issue (e.g., spatial IQ) is unidimensional.

The scale also showed good cross-validity: the two samples we created confirmed that the scale was unidimensional and presented a similar value of Cronbach's alpha. When testing the correlation between our scale and constructs theoretically related to the scale, the scale demonstrated good criterion validity: IGD consistently correlated with life satisfaction, loneliness, anxiety, depression, and academic performance in the theoretically expected directions. These correlations are consistent with other studies testing the validity of IGD in other samples.^13,12 The convergent validity of our scale was found by testing the relation between our IGD questionnaire and the GAS.¹⁵ Results showed that our scale has good convergent validity. Moreover, IGD has similar psychometric properties and similar structural factor to the GAS.

Summarizing, the French IGD scale showed satisfactory validity and reliability and can be used for research and diagnosis aims. However, several limitations have to be acknowledged. First, we did not survey a clinical population, so we could not validate our measure in a clinical setting. To our knowledge, there is currently only one study that has evaluated the diagnostic validity of the IGD criteria,²³ among Taiwanese adolescents. Results of this study showed that most of the items proposed by the DSM-5 have good diagnostic accuracy succeeding in discriminating addicted people from nonaddicted people. It seems important to test the clinical validity of the IGD scale in a French clinical sample. The second limitation of this study is that we did not test reliability over time, meaning that we did not collect longitudinal data. Longitudinal studies are useful to investigate the relationship between IGD and related constructs. Moreover, they can provide information about the temporal process of IGD.

In the French population, about 25 percent of French adolescent players report having problems with family or with school due to their video game use. Despite the growing number of problematic gamers, at this moment, in France, there is no validated instrument that can identify problematic gamers among the adult population. Our study represents the first step toward the validation of an instrument measuring video game disorder based on the DSM-5 criteria in the French language. The important increase in French players makes it important to have an appropriate measure enabling international comparisons. Although some countries are proposing diagnostic instruments, more studies are required to validate a unique and standardized instrument, referring to the same criteria, and capable of measuring IGD across different populations and different cultures. We hope that this study will represent an initial step to increase the comparability of international research on IGD.

Footnotes

Acknowledgment

Funding for this study was provided by the French Ministry of Health (MILDECA). MILDECA has not played any role in the study design, collection, analysis, or interpretation of the data, writing the article, or the decision to submit the article for publication.

Author Disclosure Statement

No competing financial interests exist.

Appendix A

References

Keeper

. Pathological preoccupation with video games. Journal of the American Academy of Child and Adolescent Psychiatry, 1990; 29:49–50.

Young

. Internet addiction: the emergence of a new clinical disorder. Cyber Psychology & Behavior, 1998; 1:237–244.

Hsu

, Wen

M-H

, Wu

M-C

. Exploring user experiences as predictors of MMORPG addiction. Computers & Education, 2009; 53:990–999.

C-H

, Liu

G-C

, Hsiao

, et al. Brain activities associated with gaming urge of online gaming addiction. Journal of Psychiatric Research, 2009; 43:739–747.

Griffiths

. A “components” model of addiction within a biopsychosocial framework. Journal of Substance Use, 2005; 10:191–197.

Lemmens

, Valkenburg

, Peter

. Development and validation of a game addiction scale for adolescents. Media Psychology, 2009; 12:77–95.

Van Rooij

, Schoenmakers

, van de Eijnden

RJJM

, et al. Compulsive Internet use: the role of online gaming and other Internet applications. Journal of Adolescent Health, 2010; 47:51–57.

Gentile

. Pathological video-game use among youth ages 8 to 18 A national study. Psychological science, 2009; 20:594–602.

Tejeiro Salguero

, Morán

RMB

. Measuring problem video game playing in adolescents. Addiction, 2002; 97:1601–1606.

10.

King

, Haagsma

, Delfabbro

, et al. Toward a consensus definition of pathological video-gaming: a systematic review of psychometric assessment tools. Clinical Psychology Review, 2013; 33:331–342.

11.

Petry

, Rehbein

, Gentile

, et al. An international consensus for assessing internet gaming disorder using the new DSM-5 approach: internet gaming disorder. Addiction, 2014; 109:1399–1406.

12.

Lemmens

, Valkenburg

, Gentile

. The Internet Gaming Disorder Scale. Psychological Assessment, 2015; 27:567–582.

13.

Pontes

, Griffiths

. Measuring DSM-5 internet gaming disorder: development and validation of a short psychometric scale. Computers in Human Behavior, 2015; 45:137–143.

14.

Rehbein

, Kliem

, Baier

, et al. Prevalence of internet gaming disorder in German adolescents: diagnostic contribution of the nine DSM-5 criteria in a state-wide representative sample: internet gaming disorder in adolescents. Addiction, 2015; 110:842–851.

15.

Gaetan

, Bonnet

, Brejard

, et al. French validation of the 7-item Game Addiction Scale for adolescents. Revue Européenne de Psychologie Appliquée/European Review of Applied Psychology, 2014; 64:161–168.

16.

Diener

, Emmons

, Larsen

, et al. The satisfaction with life scale. Journal of Personality Assessment, 1985; 49:71–75.

17.

Russell

, Peplau

, Cutrona

. The revised UCLA Loneliness Scale: concurrent and discriminant validity evidence. Journal of Personality and Social Psychology, 1980; 39:472–480.

18.

Enzman

, Marshall

, Killias

, et al. Self-reported youth delinquency in Europe and beyond: first results of the Second International Self-Report Delinquency Study in the context of police and victimization data. European Journal of Criminology, 2010; 7:159–183.

19.

Friedman

. Samuelian

, Lancrenon

, et al. Three-dimensional structure of the Hospital Anxiety and Depression Scale in a large French primary care population suffering from major depression. Psychiatry Research, 2001; 104:247–257.

20.

Byrne

. (2010). Structural equation modeling with AMOS: basic concepts, applications and programming. 2nd ed. New York: Taylor & Francis Group.

21.

Min

, Kim

. Cross-validation of reliability, convergent and discriminant validity for the problematic online game use scale. Computers in Human Behavior, 2010; 26:389–398.

22.

Moreno

, Jelenchick

, Breland

. Exploring depression and problematic internet use among college females: a multisite study. Computers in Human Behaviors, 2015: 49:601–607.

23.

C-H

, Yen

J-Y

, Chen

S-H

, et al. Evaluation of the diagnostic criteria of Internet gaming disorder in the DSM-5 among young adults in Taiwan. Journal of Psychiatric Research, 2014; 53:103–110.

Factor	All sample	Sample 1 (N = 332)	Sample 2 (N = 361)
1	3.8	3.8	3.8
2	0.8	0.9	0.9
3	0.8	0.8	0.8
4	0.7	0.7	0.7
5	0.6	0.6	0.6
6	0.6	0.5	0.5
7	0.5	0.5	0.5
8	0.5	0.5	0.4
9	0.4	0.4	0.3

Factor	All sample	Sample 1 (N = 332)	Sample 2 (N = 361)
1	3.8	3.8	3.8
2	0.8	0.9	0.9
3	0.8	0.8	0.8
4	0.7	0.7	0.7
5	0.6	0.6	0.6
6	0.6	0.5	0.5
7	0.5	0.5	0.5
8	0.5	0.5	0.4
9	0.4	0.4	0.3

Factor	All sample	Sample 1 (N = 332)	Sample 2 (N = 361)
1	3.8	3.8	3.8
2	0.8	0.9	0.9
3	0.8	0.8	0.8
4	0.7	0.7	0.7
5	0.6	0.6	0.6
6	0.6	0.5	0.5
7	0.5	0.5	0.5
8	0.5	0.5	0.4
9	0.4	0.4	0.3