Korean Validation of the Internet Gaming Disorder-20 Test

Abstract

Since “Internet gaming disorder (IGD)” was included as a tentative disorder in Section III of the Diagnostic and Statistical Manual of Mental Disorders-fifth edition (DSM-5), more efforts have been undertaken to provide standardized assessment tools. Among the recently proposed measures of IGD, the IGD-20 (IGD-20) Test was designed to encompass the nine IGD criteria of the DSM-5, and it was subsequently validated into other languages. The present study aimed to validate the Korean version of the IGD-20 Test. In total, 1,403 Korean online game users were recruited for a self-report survey comprising the Korean IGD-20 (K-IGD-20) Test and other measures, including demographics and playing variables. The K-IGD-20 Test had good reliability, and a confirmatory factor analysis revealed six factors, consistent with the original scale. Moreover, the K-IGD-20 Test demonstrated appropriate concurrent and convergent validity with other measures of Internet (game) addiction and psychological problems. Overall, the present results indicate that the K-IGD-20 Test could be used as a reliable and valid tool in IGD research on Korean-speaking populations. Furthermore, this adds empirical evidence to continue the cross-cultural validation of the IGD-20 Test, which could contribute to a more unified, integrative approach in IGD research worldwide.

Introduction

As the Internet has become an essential part of daily life, its adverse effects have emerged as a significant social issue. In particular, Internet addiction is known to have a negative impact on users' academic, social, and occupational functioning,¹ comparable to the cases of substance or alcohol addiction. Internet addiction is an umbrella term that encompasses a wide variety of heterogeneous problems. Among them, Internet game addiction is considered as one of the most serious subcategory that requires therapeutic intervention.² Thus, since the 1990s, considerable scholarly attention has been paid to the association of Internet game addiction with mental health and its social costs.^1–4 However, cumulative, integrative research in this field has been hampered by the absence of consensus on an operational definition and standardized assessment methods.^5,6

In this sense, the addition of “Internet gaming disorder (IGD)” in the latest revision of the Diagnostic and Statistical Manual of Mental Disorders-fifth edition (DSM-5)⁷ provided an impetus for a more unified approach. IGD was listed in Section III (Emerging Measures and Models), as one of the “Conditions for Further Study.” According to the DSM-5, IGD is defined as a “persistent and recurrent use of the Internet to engage in games,” which is indicated by fulfilling five or more of the following nine criteria: (1) preoccupation; (2) withdrawal; (3) tolerance; (4) unsuccessful attempts to control participation in gaming; (5) loss of interests in previous hobbies and other entertainment; (6) continued use despite of psychosocial problems; (7) deception regarding the amount of gaming; (8) escaping or relieving negative mood by gaming; and (9) jeopardy or loss of interpersonal, occupational, or educational opportunities because of gaming. Although IGD is not an official diagnosis yet, this is still a very meaningful milestone in the field, as it has established a basic ground for the conduct of more unified cumulative research on IGD in the future.

Following the DSM-5 conceptualization, several new IGD scales were developed in the hope of providing a standardized as well as psychometrically robust measurement instrument.⁸ Among them, the IGD-20 (IGD-20) Test⁹ seems to be promising. The IGD-20 Test was developed to assess IGD in line with the DSM-5 diagnostic criteria, based on Griffiths' six component model of addiction (salience, mood modification, tolerance, withdrawal, conflict, and relapse).³ In the original development study, the IGD-20 Test exhibited good reliability and validity. The six factors of the IGD-20 Test demonstrated significant positive correlations with corresponding IGD diagnostic criteria.

Subsequently, the IGD-20 Test was validated into Spanish and Arabic.^10,11 The Spanish IGD-20 Test had good reliability, and it was composed of six factors, consistent with the original version. The Arabic IGD-20 Test also demonstrated excellent reliability. However, the Arabic version had a unifactorial structure unlike the previous versions. In addition, a 9-item shortened scale, IGDS9-SF¹² has been developed by the same group. The IGDS9-SF was successfully used on English, Portuguese, Italian, and Slovenian samples,^13–15 showing a potential to be used as a convenient alternative in time-limited settings.

The main objective of the present study was to validate the Korean version of the IGD-20 Test. Although preliminary, the DSM-5 stated that the prevalence of IGD is suspected to be much higher in Asian countries.⁷ Especially, because of world's highest Internet penetration rate and active Internet game industry,¹⁶ Korea may be considered as an ideal place to study IGD. In fact, increasing Internet game addiction is one of the major mental health concerns in Korea.¹⁷ Nevertheless, most of the existing IGD scales in Korea were not developed to accommodate the recently proposed DSM-5 diagnostic criteria,^18–20 which hinders accumulation of converging evidence and comparison across nations. Therefore, the Korean validation of the IGD-20 Test could contribute to fostering a research environment suitable for international standards.

Materials and Methods

Participants

In total, 1,403 Internet game users who had lifetime experience of online gaming participated via online or offline survey. A study advertisement was posted in major online gaming forums in Korea as well as universities, counseling centers, and libraries located in the greater Seoul area, and the Gyeonggi and Chungcheong Provinces in Korea. After informed consent, they answered a survey package containing self-report measures.

All participants completed the Korean version of the IGD-20 Test and a questionnaire on demographics and playing variables. However, because this was part of a large-scale survey conducted in a series, other questionnaires (Internet Game Addiction Diagnostic Scale [IGADS], Korean Internet Addiction Self-Check Scale [K-Scale], Brief Symptom Inventory-18 [BSI-18], and Rosenberg Self-Esteem Scale [RSES]) were administered only to participants who took part in the first round of data collection (n = 625) to reduce respondents' burden.* All procedures were approved by the Institutional Review Board of Seoul National University.

To sample participants who currently engaged in Internet gaming, we excluded those who used Internet gaming less than 1 hour during the previous week of participation. Therefore, data of 1,152 participants were included for subsequent analyses.

Measures

Demographics and playing variables

Current age, gender, age when they first started gaming, weekly amount of time spent in gaming, and number of days of gaming per week were collected.

Korean IGD-20 Test

The IGD-20 Test is a 20-item, 5-point Likert scale (1 = strongly disagree to 5 = strongly agree) self-report questionnaire developed to accommodate the newly proposed DSM-5 diagnostic criteria of IGD. Permission from the original author (H. Pontes) was obtained through personal communication (November 21, 2015). A rigorous approach was adopted for translation. The items were first translated into Korean by clinical psychologists, including the author. Then, the texts were back-translated and two bilingual individuals independently checked if each item retained equivalent meaning until consensus was reached.

Internet Game Addiction Diagnostic Scale

The IGADS¹⁸ is a 25-item self-report questionnaire to measure negative outcomes in psychological, physical, interpersonal, and academic functioning due to Internet game addiction. Each item is rated on a 5-point Likert scale (1 = strongly disagree to 5 = strongly agree), and its internal consistency was excellent (Cronbach's α = 0.92) in the present study.

Korean Internet Addiction Self-Check Scale

The K-Scale¹⁹ was developed to screen Internet addiction, which is not restricted to Internet gaming in scope. The 40 items on this scale were devised to tap maladaptive consequences that could result from Internet use. Each item is rated on a 4-point Likert scale (1 = never to 4 = always), and its internal consistency was excellent (Cronbach's α = 0.94) in this study.

Korean Brief Symptom Inventory

The BSI-18²¹ was originally developed by Derogatis²² for a brief assessment of somatization, depression, and anxiety levels. The 18 items (6 items belonging to each subscale) are rated on a 5-point Likert scale (1 = strongly disagree to 5 = strongly agree). The internal consistency of each subscale was good (somatization: 0.80, depression: 0.86, anxiety: 0.83) in the present study.

Korean Rosenberg Self-Esteem Scale

The 10-item RSES²³ is one of the most widely used measures of global self-esteem.²⁴ The Korean-RSES (K-RSES) is rated on a 5-point Likert scale (1 = strongly disagree to 5 = strongly agree). Consistent with the original version, its internal consistency was good (Cronbach's α = 0.89) in this study.

Data analyses

Internal consistency and item-to-total correlation coefficient were computed as measures of reliability. To investigate the factor structure of the Korean IGD-20 (K-IGD-20) Test, a confirmatory factor analysis was performed using maximum likelihood estimation in analysis of a moment structures. Based on the theoretical rationale upon which the IGD-20 Test was constructed, fit indices for the identical 6-factor-model⁹ were checked. In addition to the chi-square test, the Comparative Fit Index (CFI), the Tucker–Lewis Fit index (TLI), and the Root Mean Square Error of Approximation (RMSEA) were considered for model evaluation. A good fit was defined by a CFI and TLI greater than 0.90 and an RMSEA less than 0.08.^25,26 Finally, to verify concurrent and convergent validity, correlational analyses were conducted.

Results

Descriptive statistics

The mean age of participants was 23.35 years (SD = 5.28 years, range = 13–55 years) and 81.5 percent were male (male = 939, female = 213), consistent with previous research indicating that the male gender is overrepresented in samples of game users.^9,10 The mean age when they first started gaming was 9.26 years. On an average, participants played Internet games for 4.41 days a week and their mean time spent in gaming was 602.80 minutes, which equaled to ∼10 hours per week (range = 60–5,040 minutes).

Reliability

The Cronbach's alpha for all 20 items was 0.85, which was good. The item-to-total correlation coefficient ranged from 0.23 to 0.74 (all p < 0.01), exhibiting relatively low coefficients for item 2 (r = 0.29) and 19 (r = 0.23) (c.f. both are reverse-scored items).

Factor structure of the K-IGD-20 Test

Overall, the fit indices (chi-square, CFI, and RMSEA) of the 6-factor-model were appropriate (Table 1) with the only exception of the TLI, which exhibited marginally good fit (TLI = 0.883). As presented in Figure 1, all factor loadings were significant and there were positive associations between all the six factors, consistent with the previous versions of the IGD-20 Test.^9,10

FIG. 1.

Confirmatory factor analysis of the K-IGD-20 Test (N = 1,152). K-IGD-20 Test, Korean Internet Gaming Disorder-20 Test.

Table 1.

The Model Fit Indices for the K-IGD-20 Test (N = 1,152)

χ ²	df	P	CFI	TLI	RMSEA
779.49	155	<0.0001	0.913	0.883	0.059

CFI, Comparative Fit Index; K-IGD-20, Korean Internet Gaming Disorder-20; RMSEA, Root Mean Square Error of Approximation; TLI, Tucker–Lewis Fit Index.

Concurrent and convergent validity

To verify concurrent validity, the correlations between the K-IGD-20 Test and the existing Internet (game) addiction questionnaires were explored. The K-IGD-20 Test was positively associated with both the IGADS (r = 0.80, p < 0.01) and the K-Scale (r = 0.59, p < 0.01). Specifically, the correlation was relatively stronger with the IGADS, which was intended to measure Internet “game” addiction. In addition, the K-IGD-20 Test showed positive association with weekly gaming hours (r = 0.27, p < 0.01) and days (r = 0.34, p < 0.01).

With regard to convergent validity, the K-IGD-20 Test demonstrated a positive correlation with somatization (r = 0.26, p < 0.01), depression (r = 0.31, p < 0.01), and anxiety (r = 0.23, p < 0.01) assessed by the Korean-BSI-18 (K-BSI-18). Furthermore, it exhibited negative correlation with self-esteem (r = −0.31, p < 0.01). In contrast, the correlations of weekly gaming hours or days with the above-mentioned psychological problems were relatively weak or insignificant (Table 2).

Table 2.

Concurrent and Convergent Validity of the K-IGD-20 Test (n = 445–461)

	1	2	3	4	5	6	7	8	9
1. K-IGD-20 Test	—	0.80^**	0.59^**	0.26^**	0.31^**	0.23^**	−0.31^**	0.27^**	0.34^**
2. IGADS		—	0.62^**	0.40^**	0.37^**	0.35^**	−0.34^**	0.24^**	0.27^**
3. K-Scale			—	0.46^**	0.42^**	0.44^**	−0.29^**	0.23^**	0.15^**
4. Somatization				—	0.54^**	0.72^**	−0.31^**	0.16^**	0.04
5. Depression					—	0.68^**	−0.63^**	0.12^**	0.09^*
6. Anxiety						—	−0.46^**	0.03	0.02
7. RSES							—	−0.09	−0.08
8. Weekly gaming time (minutes)								—	0.46^**
9. Weekly number of days									—
Mean	40.84	44.94	61.44	9.05	11.97	9.43	37.69	503.67	4.02
SD	12.81	14.62	15.90	3.78	5.36	4.04	7.09	789.65	2.19

p < 0.05, ^**p < 0.01.

Anxiety, anxiety subscale of the Korean Brief Symptom Inventory-18; Depression, depression subscale of the Korean Brief Symptom Inventory-18; IGADS, Internet Game Addiction Diagnostic Scale; K-Scale, Korean Internet Addiction Self-Check Scale; RSES, Rosenberg Self-Esteem Scale; SD, standard deviation; Somatization, somatization subscale of the Korean Brief Symptom Inventory-18.

Discussion

This study validated the Korean version of the IGD-20 Test in a fairly large sample of Korean Internet game users. Overall, the current results suggest that the K-IGD-20 Test possesses sound reliability and validity.

First of all, the K-IGD-20 Test was found to have a 6-factor structure (salience, mood modification, tolerance, withdrawal, conflict, and relapse), consistent with the English and Spanish versions.^9,10 This adds empirical evidence from Eastern Asia in favor of the six component model of addiction.⁴ To date, the unifactorial structure was only reported in the Arabic version.¹¹ Although we need more evidence to understand why this difference in the factorial structure emerged, at the moment, it may be partly explained by different sample characteristics. For example, the Arabic sample differed in terms of age and gaming hours. While the three studies supporting 6-factor solution did not specifically limit the age range of participants, only high school students were included in the Arabic validation study. In addition, the average gaming hour was much lower in the Arabic study (1.6 hour on weekdays, 3.8 hour on weekends), compared with the other three studies.

The validity of the K-IGD-20 Test was supported by the significant associations with relevant constructs in the expected direction. The K-IGD-20 Test showed the strongest positive correlation with another measurement of Internet game addiction (the IGADS).¹⁸ In comparison, the positive correlation with a broader construct of Internet addiction (the K-Scale)¹⁹ was still strong, but it was relatively smaller than that with the IGADS. This demonstrates that the K-IGD-20 Test is designed to specifically assess IGD as opposed to Internet addiction in general. The K-IGD-20 Test also exhibited a positive association with psychological symptoms, and a negative association with self-esteem, consistent with extensive literature indicating the harmful consequences of IGD on mental health.^4,27

Moreover, another implication of the present study can be found in the context of the further cross-validation of the IGD-20 Test in Korean-speaking populations. In light of the need for a unified, psychometrically sound instrument to assess IGD,⁹ continuous cross-validation efforts would contribute to fostering a more reliable research environment worldwide. In this respect, Korea can be a very interesting place to study IGD due to its high Internet penetration rate and suspected high prevalence of IGD, as noted earlier. Using a shared measurement tool would facilitate cross-cultural comparisons and the investigation of common and specific factors of IGD across various countries in the nearer future, which is conducive to a more complete understanding and the development of effective prevention/treatment of IGD.

The present results can also be discussed within the conceptual issue of what constitutes behavioral addictions such as IGD, although the primary aim of this study was to test the psychometric properties of the K-IGD-20 Test. Unlike substance addiction, which directly impacts the user on a physiological level and elicits physiological as well as psychological dependence symptoms, behavioral addiction often lacks an objective validation indicator. Furthermore, critical voices do exist regarding whether we overpathologize mostly benign leisure activities by adding a new diagnosis such as IGD.²⁸

Consistent with previous research, the present results confirmed that spending more time on Internet gaming, by itself, is not a sufficient variable to differentiate a form of behavioral addiction.^29,30 The correlation between the time (hours/day) variables and the K-IGD-20 Test was only modest in this study. Furthermore, people who scored higher on the K-IGD-20 Test tended to experience more psychological symptoms and lower self-esteem. On the contrary, the relationship between time variables and maladaptive functioning was not salient enough. This discriminant correlational pattern suggests that the core component of addiction might lie in the dysfunctional rather than excessive nature of engagement.⁹

Certain limitations of this study need to be mentioned as well. First, criterion-related validity was not tested in this study. If diagnostic measures of IGD can be included in the future investigation, it would be meaningful to verify its criterion-related validity and to establish an optimal cutoff score for the K-IGD-20 Test. Second, a convenience sample was used, and it was not a representative sample. Therefore, the generalizability of these findings is limited, and they should be interpreted carefully. Third, due to the cross-sectional, associational design of this study, the present results cannot answer questions of causation or the directionality of the relationship between variables. For instance, it may be questioned whether IGD in fact caused more psychological problems or vice versa. Such questions could be answered by adopting longitudinal or experimental designs.

Fourth, a few items (item 2 and 19) exhibited weaker relationship with the total K-IGD-20 Test score. Both were the reverse-scored items, and this pattern was also reported in the Spanish version. It might be possible to reduce the number of items or to consider if there is a hidden method-factor (comprising reverse-keyed items) in the factor structure of the IGD-20 Test. In addition, the factor loading of item 19 (conflict) was lower (0.30) in the present study, compared to that in the original (0.47) and Spanish (0.46) versions. This discrepancy could be due to different sample characteristics such as gaming hour. Participants' gaming hour duration tended to be higher in previous studies,^9,10 which increases the probability that their main daily activities would be more disrupted by gaming, and that this disruption might lead to conflict. Fifth, correlational analyses took a major part in investigating the relationship of the K-IGD-20 Test and other measures in this study. Although correlations may serve as a useful starting point of such investigation, other higher order statistical analyses would be beneficial to afford stronger and more generalizable evidence.

Nevertheless, this study is still valuable as the first investigation to report the reliability and validity of the Korean version of the IGD-20 Test. In the post-DSM-5 era, when a call for a more unified approach is gaining emphasis, the IGD-20 Test seems to be a potential candidate in terms of sound theoretical basis and practicality. As more cross-cultural validations of the IGD-20 Test accumulate, we would be able to move this field a step forward by overcoming the chronic issue of inconsistent measurement.

Footnotes

Acknowledgments

This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2016S1A5B5A07920123). The survey was conducted in collaboration with Noah Jeong, MA for a thesis project at the Seoul National University (South Korea). Finally, I thank Noah Jeong, MA for his irreplaceable help in preparation and data collection of this study.

Author Disclosure Statement

No competing financial interests exist.

References

Young

. Internet addiction: the emergence of a new clinical disorder. Cyperpsychology & Behavior, 1998; 1:237–244.

Greenfield

. (2010) The addictive properties of internet usage. In Young

, De Abreu

, eds. Internet addiction: A handbook and guide to evaluation and treatment. Hoboken, NJ: Wiley, pp. 135–153.

Griffiths

. A ‘components’ model of addiction within a biopsychosocial framework. Journal of Substance Use, 2005; 10:191–197.

Kuss

, Griffiths

. Internet gaming addiction: a systematic review of empirical research. International Journal of Mental Health and Addiction, 2012; 10:278–296.

Griffiths

, King

, Demetrovics

. DSM-5 internet gaming disorder needs a unified approach to assessment. Neuropsychiatry, 2014; 4:1–4.

Petry

, Rehbein

, Gentile

, et al. An international consensus for assessing internet gaming disorder using the new DSM‐5 approach. Addiction, 2014; 109:1399–1406.

American Psychiatric Association. (2013) Diagnostic and Statistical Manual of Mental Disorders-5th edition (DSM-5). Washington, DC: Author.

Pontes

. Current practices in the clinical and psychometric assessment of Internet gaming disorder in the era of the DSM-5: a mini review of existing assessment tools. Mental Health and Addiction Research, 2016; 1:18–19.

Pontes

, Király

, Demetrovics

, et al. The conceptualisation and measurement of DSM-5 Internet Gaming Disorder: the development of the IGD-20 Test. PLoS One, 2014; 9:e110137.

10.

Fuster

, Carbonell

, Pontes

, et al. Spanish validation of the internet gaming disorder-20 (IGD-20) test. Computers in Human Behavior, 2016; 56:215–224.

11.

Hawi

, Samaha

. Validation of the Arabic Version of the Internet Gaming Disorder-20 Test. Cyberpsychology, Behavior, and Social Networking, 2017; 20:268–272.

12.

Pontes

, Griffiths

. Measuring DSM-5 Internet gaming disorder: development and validation of a short psychometric scale. Computers in Human Behavior, 2015; 45:137–143.

13.

Pontes

, Macur

, Griffiths

. Internet gaming disorder among Slovenian primary schoolchildren: findings from a nationally representative sample of adolescents. Journal of Behavioral Addictions, 2016; 5:304–310.

14.

Monacis

, Palo

, Griffiths

, et al. Validation of the internet gaming disorder scale-short-form (IGDS9-SF) in an Italian-speaking sample. Journal of Behavioral Addictions, 2016; 5:683–690.

15.

Pontes

, Griffiths

. Portuguese validation of the internet gaming disorder scale-short-form. Cyberpsychology, Behavior, and Social Networking, 2016; 19:288–293.

16.

Korea Internet & Security Agency. (2013) Survey on the internet usage. Seoul: Author.

17.

National Information Society Agency. (2016) The survey on internet overdependence. Daegu: Author.

18.

Lee

, Ahn

. Development of the Internet Game Addiction Diagnostic Scale. The Korean Journal of Health Psychology, 2002; 7:211–239.

19.

Kim

, Kim

, Park

, et al. (2002) Internet addiction prevention counseling and program development. Seoul: Korea Agency for Digital Opportunity and Promotion.

20.

Cho

, Kwon

. A validation of the Korean version of the Internet Gaming Disorder Scale (K-IGDS): findings from a community sample of adults. Korean Journal of Clinical Psychology, 2017; 36:104–117.

21.

Park

, Woo

, Chang

. Validational Study of Brief Symptoms Inventory-18(BSI-18) in College Students. Korean Journal of Clinical Psychology, 2012; 31:507–521.

22.

Derogatis

. (2001) Brief Symptom Inventory-18 (BSI-18): Administration, scoring and procedure manual. Minneapolis, MN: NCS Pearson, Incorporated.

23.

Lee

, Won

. An exploratory study on the relationship between paranoid tendency, self-concept, and self-consciousness. A paper submitted to the Annual convention of Korean Psychological Association, 1995; 95:277–290.

24.

Rosenberg

. (1965) Society and the adolescent self-image. Princeton, NJ: Princeton University Press.

25.

, Bentler

. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling, 1999; 6:1–55.

26.

Hong

. The criteria for selecting appropriate fit indices in structural equation modeling and their rationales. Korean Journal of Clinical Psychology, 2000; 19:161–177.

27.

Kim

, Hwang

SSH

, Choi

, et al. Characteristics and psychiatric symptoms of internet gaming disorder among adults using self-reported DSM-5 criteria. Psychiatry Investigation, 2016; 13:58–66.

28.

Pies

. Should DSM-V designate “Internet addiction” a mental disorder?. Psychiatry, 2009; 6:31.

29.

Demetrovics

, Király

. Commentary on Baggio et al. 2016: Internet/gaming addiction is more than heavy use over time. Addiction, 2016; 111:523–524.

30.

Charlton

, Danforth

. Distinguishing addiction and high engagement in the context of online game playing. Computers in Human Behavior, 2007; 23:1531–1548.