An Empirical Review of Internet Addiction Outcome Studies in China

Abstract

Objectives: The authors systematically reviewed the outcomes and methodological quality of 24 Internet addiction (IA) treatment outcome studies in China. Method: The authors used 15 attributes from the quality of evidence scores to evaluate 24 outcome studies. These studies came from both English and Chinese academic databases from 2000 to 2010. Results: Among the 15 attributes, only sequence generation and intention-to-treat were reported by more than 50% of the 24 studies. None of the studies contained treatment adherence ratings or collateral reports. Cognitive behavior therapy combined with family therapy or group therapy emerged as possibly efficacious treatments. Conclusions: More rigorously designed studies, accompanied by transparent reporting of methods and findings are needed to identify promising IA treatments.

Keywords

Internet addiction treatment methodological quality review

Internet addiction (IA; also called pathological Internet use, excessive computer and video game playing, Internet overuse, problematic Internet use, etc.) has become a widespread problem among youth ages 9–23. According to a 2009 national Harris poll survey with a randomly selected sample of American youth ages 8–18, about 8% of video game players exhibited pathological patterns of play (Gentile, 2009), which is also considered a type of IA. In a 2005 survey in Germany, among 323 children from 11 to 14 years of age, 9.3% of the children fulfilled all criteria for excessive computer and video game playing (Gruesser, Thalemann, Albrecht, & Thalemann, 2005). In Australia, excessive computer playing corresponding to addictive behavior was found in 12.3% of school-age children, adolescents, and emerging adults (Batthyany, Muller, Benker, & Wolfling, 2009). This public health problem might be even worse in China. According to the China Youth Internet Association’s 2009 Report on Youth IA Disorder, around 14.3% of youth ages 13–17 and around 15.6% of youth ages 18–23 in China with computer access had IAs (China Youth Internet Association, 2010).

Although there are many cross-sectional studies on the correlates of IA and some review articles, most articles only discuss promising treatments in passing (Brezing, Derevensky, & Potenza, 2010; Chou, Condron, & Belland, 2005; Huang, Li, & Tao, 2010; Young, 2009a, 2009b). Thus, there is no comprehensive review of outcomes of different IA treatments, and there is no review article that focuses on the methodological quality of such studies. Chou, Condron, and Belland (2005) systematically reviewed research on IA definitions, assessments, risk factors, addictive potentials before 2005. However, only two empirical treatment studies, which used cognitive–behavior therapy (CBT) and motivational enhancement therapy, were included in this review. Widyanto and Griffiths (2006) also explored IA research but did not contain detailed information about treatment outcome studies. These above two reviews suggest that cognitive behavioral therapy has been used to treat IA, but with so few studies, conclusions about its effectiveness are premature. In addition, Young (2009) describes a cognitive behavioral therapy model that includes behavior therapy and self-monitoring (i.e., a daily Internet use log) to treat IA. However, this article is only description of potential IA treatments but not actual empirical study.

Although few studies outcome studies exist in the United States, many outcome studies have been conducted in China. Therefore, this review article aims to systematically review IA treatment outcome studies published in English and Chinese language journals that focused on IA treatment in China from 2000 to 2010. Our primary objective is to evaluate the methodological quality of different treatment outcome studies. Our secondary objective is to find out which treatments are empirically supported. Based on evidence-based practice requirements, this systematic review on IA outcome studies will help social workers identify the most effective treatments for IA. Further, by focusing on methodological attributes of these studies, we will identify those practices with the largest effects sizes from the highest quality studies.

Definitions of IA vary considerably among different scholars. Some definitions, and the resulting measures that have been developed like Young’ (2004) IA scales, flow from the conceptualizations of substance use disorders and pathological gambling in the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV; Fisher, 1994; Skoric, Teo, & Neo, 2009; Young, 2004). Similar to these problems, IA is conceptualized as a problem where frequent use leads to significant impairment in one’s life (i.e., urges to play, ignoring personal responsibilities, and loved ones). According to Block (2008), IA appears to be a common disorder that merits inclusion in DSM-V and conceptually the diagnosis is a compulsive-impulsive spectrum disorder that involves excessive online and/or computer usage use, withdrawal, and tolerance. Although a thorough discussion of diagnostic issues surrounding IA is outside of the scope of this article, when we refer to IAhere, we include both problematic Internet and computer use (i.e., video game) that has been diagnosed with the best available, empirically validated measures.

Some of the most commonly used scales in China are Young’s (1995–2004) diagnostic questionnaires. Young developed four questionnaires, based on pathological gambling measurement, called the IA scales (IAS). They range from 7 to 20 items (IAS-7, IAS-8, IAS-10, and IAS-20). Example items include “do you feel the need to use the Internet with increasing amounts of time in order to achieve desired satisfaction?” and “have you repeatedly made unsuccessful efforts to control, cut back, or stop Internet use?” The Chinese Internet Addiction scales (CIAS 1998 and CIAS 2005) are based on Young’s (1995–2004) scales (Yang, Zheng, & Ruan, 2004; Zhang, 2009). The CIAS 1998 include four factors: tolerance, compulsive behavior and time, withdrawal symptoms, and related mental health problems (Yang et al., 2004). The CIAS 2005 were used in a large epidemiological study in China, called the Chinese Adolescent IA Report 2005 (Zhang, 2009). China developed its own diagnostic criteria for IA in late 2008 and currently uses them in treatment settings (Liu, Wang, & Zhuang, 2008). The criteria are as follows: (a) more than 6 hr a day of Internet use; (b) this heavy Internet use persists for more than 3 months; (c) social, study, and communication skills dysfunctions; (d) dependence symptoms (e.g., a strong desire and impulse) are present; and (e) withdrawal (e.g., uncomfortable, easy angered, attention deficit, sleep disorder without Internet use).

In addition to these criteria developed in China, there is evidence that IA is associated with attention deficits, bad academic performance, depression, self-injury, hostility, and violent behaviors (Batthyany et al., 2009; Bernardi & Pallanti, 2009; Bioulac, Arfi, & Bouvard, 2008; Caplan, Williams, & Yee, 2009; Chan & Rabinowitz, 2006; Frolich, Lehmkuhl, & Dopfner, 2009; Gentile, 2009; Ha, Yu, Park, & Lim, 2009; Hwang, Cheong, & Feeley, 2009; Kim, Namkoong, Ku, & Kim, 2008; Ko, Yen, Chen, Yeh, & Yen, 2009; Lam, Peng, Mai, & Jing, 2009; Skoric et al., 2009; Sun, Ma, Bao, Chen, & Zhang, 2008). Therefore, researchers sometimes use the Symptoms Checklist 90 (SCL-90; Derogatis, 1994; Wang, Wang, & Ma, 1999) to evaluate the treatment outcomes of IA besides the IAS and CIAS in hospitals and clinical settings.

Method

Study Selection

An extensive literature search was conducted both in English academic databases (social sciences citation index [SSCI], Social Services Abstracts, Social Work Abstracts, PsycINFO w/ PsycARTICLES) and Chinese academic databases (WanFang Database, China national knowledge infrastructure [CNKI] Chinese academic journal web databases, WeiPu databases) published from 2000 to May 2010. Keywords such as “Internet addiction, pathological Internet use, excessive Internet use, problematic Internet use, video game addiction” were used in the searching combined with keywords of “treatment” or “intervention” or “evaluation”. Based on this search, we selected studies that (a) reported posttreatment outcomes, (b) included participants ages 9–23 that were treated for an IA, (c) were conducted in China, and (d) for which we could obtain the full text report. Review papers and case study reports are not included in this review. We chose a large age rage since we preferred to contain as many studies as possible on IA treatment and usually studies which were done in clinical settings in China had a large age range. Using these criteria, we located 24 studies, which are evaluated in this article.

Selection of Methodological Criteria

Methodological quality attributes of IA outcome studies in this article were established based on a recent review of adolescent substance abuse trials (Becker & Curry, 2008). Becker and Curry reported whether 14 methodological attributes were present in adolescent substance abuse treatment trials. These attributes are consistent with the Consolidated Standards of Reporting Trials (CONSORT statement; Moher, Schulz, & Altman, 2001). The study attributes that were coded included objective, sample size, power, outcome, random sequence generation (i.e., how randomization was achieved), allocation concealment, active comparison, baseline data, manualized treatment (1 = yes, 0 = no), treatment adherence rating, collateral report, collection of an objective measure besides self-report (e.g., collateral report; 1 = yes, 0 = no), intention-to-treat (ITT) analysis, and blind assessment. In our study, we also coded whether the follow-up data were collected at least 30 days after the study treatments ended. Thus, we used 15 attributes to evaluate the methodological strength of the 24 IA treatment outcome studies (see Table 1 ).

Table 1.

Attributes of Methodological Quality

Attributes	Criteria for Rating
Objective	1 = Specific objectives and hypotheses. Hypotheses are amenable to explicit statistical evaluation
Objective	0 = Objectives or hypotheses not explicitly established
Sample size	1 = Process for determining the sample size discussed along with any interim analyses and stopping rules
Sample size	0 = Determination of sample size not discussed
Power	1 = Study is sufficiently powered to detect differences between treatment groups (e.g., at least 27 subjects per condition with passive comparison, at least 71 subjects per condition with active comparison)
Power	0 = Study is not sufficiently powered
Outcome	1 = Established primary and secondary outcome measures. Primary outcome is specified as outcome of greatest importance
Outcome	0 = Primary or secondary outcome measures are not specified
Random sequence generation	1 = Process for generating a random sequence described with sufficient detail to confirm that each participant had an unpredictable, independent chance of receiving each intervention
Random sequence generation	0 = Process was not purely random, unspecified
Allocation concealed	1 = The result of initial assessment was not affected by whether participants in treatment group or not. (e.g., initial assessment was before the random assignment or initial assessment was done by unknown investigator)
Allocation concealed	0 = Process was not concealed, unspecified
Active comparison	1 = At least one active comparison (e.g., alternate model, treatment as usual)
Active comparison	0 = All comparison conditions were passive (e.g., waitlist, no-treatment control, placebo, attention control)
Baseline data	1 = Baseline demographic and clinical characteristic reported by condition
Baseline data	0 = Baseline demographic or clinical characteristic not reported
Treatment manual used	1 = At least one treatment condition was guided by a manual
Treatment manual used	0 = None of the treatments were guided by a manual, unspecified
Treatment adherence ratings	1 = Treatment adherence monitored with scales, checklists, or rating forms completed by therapist, supervisor, independent observer, an/or patients
Treatment adherence ratings	0 = Treatment adherence was not monitored using rating forms, unspecified
Collateral report	1 = At least one outcome is a collateral report (e.g., parent, caregiver, teacher)
Collateral report	0 = No collateral report
Objective measure	1 = At least one outcome is an objective measure (e.g., computer recorded time online, blood samples)
Objective measure	0 = No objective verification
Intention-to-treat	2 = ITT analysis. All subjects analyzed in groups to which they were assigned
	1 = Available cased analysis. Only subjects who completed one or more research assessments were analyzed
	0 = Treated case analysis. Only subjects who completed a portion of the treatment were analyzed
Blind assessment	1 = Follow-up assessments completed by treatment-blind evaluator
Blind assessment	0 = Follow-up assessment not completed by treatment-blind evaluator, unspecified
Follow-up length >30 days posttreatment	1 = At least part of follow-up data were collected at least 30 days after the study treatments ended
Follow-up length >30 days posttreatment	0 = No follow-up data were collected at least 30 days after the study treatments ended

Rating Process

The first and second authors independently reviewed the 24 studies based on the 15 attributes. After that, κ coefficients were used to assess the reliability of each attribute. Discrepancies were resolved through discussion with the third author. All three authors engaged in a discussion until consensus was reached. In conjunction with Becker and Curry’s (2008) article, we also used a composite quality of evidence score (QES) to indicate the number of methodological attributes each study met. We coded each attribute 0 (not met or unclear) or 1 (met), except for the ITT analysis item where items ranged from 0 to 2 (0 = treated case analysis, 1 = available cased analysis, and 2 = full ITT analysis).

We report QES scores separately for three different study designs found in the literature, as some QES items do not apply to some designs. The first type of design is a one group pretest and posttest, called a “preexpeerimental design” study. The second design style is nonrandomized comparison groups design. The third one is the experimental design (Rubin & Babbie, 2011). For preexperimental design studies, some attributes (e.g., random sequence generation, allocation concealment, active comparison, and baseline data) did not apply and were coded as 0 for such studies. After summing items, the range of the total QES is 0–16 for experimental design studies, 0–15 for nonrandomized comparison group design studies while the total QES is from 0 to 12 for preexperimental design studies.

Effect Size Calculation Process

We calculated the effect sizes of the studies mainly by using Cohen’s d (Cohen, 1988). The studies primarily used IAS or SCL-90 as the outcome measurement tools in the 24 studies. According to these two measurement tools, higher scores indicate more severe IA or health problems, respectively. For studies which only used SCL-90, we chose depression factor as the representative outcome factor to calculate effect size, since depression was the most common correlate of IA disorders in China. If the studies used continuous outcome measures and there were control groups, we calculated the effect size by subtracting posttest control group outcomes from those of the experimental treatment group, dividing by the standard deviation of the controls. Similarly, for studies with continuous outcome measures and preexperimental designs, we calculated the effect size by subtracting pretest values of the dependent measures from post treatment values, dividing by the standard deviation of the pretest data wave. Finally, for studies using dichotomous outcome indicators, we used risk ratios as estimates of the effect size. Therefore, the effect sizes are initially expressed as negative values, since posttest scores and treatment group scores were smaller than the pretest scores and control group scores, respectively. So, we transformed negative values to positive ones so that higher positive values indicate larger effect sizes in favor the treatment group (i.e., comparison group design) or posttest wave (i.e., preexperimental designs). In order to prevent outliers from exerting too much influence on analyses, we used winsorizing to compare effectiveness of single treatments and multimodal treatments (Erceg-Hum & Mirosevich, 2008). In this analysis, we also transformed negative effect size scores to positive scores for better interpretation.

Results

Methodological Attributes Across Studies

The κ values for each of the 15 methodological attributes are described in Table 2 . Except for 2 of the 15 attributes, outcome (κ = .57) and active comparison (κ = .52), we reached moderate or high agreement (ranged from 0.63 to 1.00). The average κ value across the studies is .79, indicating a substantial agreement between the two raters, in accord with Landis and Koch’s (1977) benchmark. Discrepancies were resolved through communication with the third author, who had advanced clinical and research experience in outcome study evaluation. The two raters ultimately reached unanimous agreement on each attribute in order to calculate the QES.

Table 2.

Design Characteristics of Internet Use Disorder Outcome Studies

	κ	Preexperimental (n = 8; %)	Nonrandomized Comparison Groups (n = 4; %)	Experimental Design (n = 12; %)
Design Characteristics
Objective	1	12.5	0	8.33
Baseline data	0.75	--	0	41.67
Sample size	1	0	0	8.33
Power	0.63	50	25	16.67
Outcome	0.57	50	50	33.33
Active comparison group	0.52	--	25	58.33
Random sequence generation	0.89	--	--	50
Allocation concealed	0.79	--	0	16.67
Treatment manual used	0.63	12.5	0	16.67
Treatment adherence ratings	1	0	0	0
Collateral report	1	0	0	0
Objective measure	0.63	12.5	25	16.67
ITT	0.92	62.5	50	66.67
Blind assessment	0.65	0	0	16.67
Follow-up length >30 days	0.88	25	0	33.33
Total QES score (M, SD)	--	2.75 (1.58)	2.25 (1.5)	4.58 (2.23)

Note. ITT = intention-to-treat analysis; QES = quality of evidence scores; SD = standard deviation.

The reviewed studies are classified into three categories. We examined how each type of study meets the 16 characteristics separately because some criteria are not applicable to all studies. Over 50% of the studies were methodologically strong in two characteristics, sequence generation, and ITT analysis. However, less than 50% of the studies were strong in the following 13 attributes: objective, baseline data, sample size, power, outcome, active comparison, allocation concealment, manualized treatment, treatment adherence rating, objective measure, blind assessment, and 30 days follow-up.

Most frequently reported attributes. Random sequence generation is one of the most frequently mentioned attributes. It is only applicable to the experimental design studies, as opposed to preexperimental studies and nonrandomized comparison group design studies, which do not need to carry out this process. Half of the experimental design studies (n = 6) mentioned that they implemented the process for generating a random sequence. For example, these studies indicated that they used either a random number table or statistical software to generate random numbers in order to ensure that each participant had an equal chance to get intervention. For the remaining half of the studies, we cannot tell how the randomization was achieved because they did not describe the process explicitly.

The next attribute reported frequently is ITT analysis. Five of the eight preexperimental studies applied true ITT analyses. The remaining three either used treated case analysis (n = 2) or available case analysis (n = 1). Half of the nonrandomized comparison design studies used ITT analyses, and the other half used treated case analyses (n = 2). For the experimental design studies, 66.67% of the studies applied ITT analyses (n = 8), whereas two used available case analyses, and two used treated case analyses. Since sample attrition is inevitable in intervention studies, dealing with missing data in the analysis is a challenge in program evaluation. ITT analysis, in which all participants allocated to the study are included in the analysis, regardless of whether they received the treatment or not, has been established to produce less bias in estimating the true program effect than that produced by other analysis methods, such as available case analysis and treated case analysis (Freedman, 2005). The application of this method in these studies is an indication of researchers’ efforts to address the estimation bias of the treatment.

Least frequently reported attributes. Two of the least frequently reported attributes were treatment adherence ratings and collateral reports. None of the studies reported these two characteristics. These studies did not use scales, checklists, or rating forms to monitor the process of treatment implementation, in the absence of which we can determine neither whether the participants complied with the treatments nor whether the therapists implemented the models as intended. Consequently, it is difficult to tell what treatments were provided. Similarly, none of the studies used collateral reports, Therefore there is no control of potential bias in self-reported outcomes.

The next least frequently reported attributes included baseline data, sample size, allocation concealment, and blind assessment. Only less than 50% of the experimental design studies reported these four attributes. Nonrandomized comparison group design studies ignored these four attributes completely. None of preexperimental design studies reported the baseline data, sample size, and blind assessment. Five of the 12 experimental studies provided information concerning baseline demographics and clinical characteristics of participants in both experimental groups and control groups. For example, they provided t tests or chi-square tests for the demographic characteristics and participants’ Internet use behavior at baseline, through which we established whether there was preexisting selection bias between the two groups. Only one experimental study explicitly showed their power calculation formula to show how the sample size was determined. Two experimental studies specifically mentioned that they collected their baseline data before they randomized the intervention group and control group or that they collected the baseline data without knowing who was in which group. Similarly, two studies conducted the follow-up assessment by means of a treatment-blind evaluator.

Approximately a quarter of the studies reported the following characteristics: objective, manualized treatment, and objective measure. Only one of the eight preexperimental studies and one of the 12 experimental studies specify their study objectives and hypotheses. The lack of information about hypotheses indicates that they have not articulated which outcomes are the most important for their studies. There were few preexperimental studies (n = 1) and experimental studies (n = 2) that mentioned using standardized training manuals to guide their treatments, whereas most of them simply used self-designed manuals or no manual at all. In terms of the measurement of the outcome, less than a quarter of any of these studies used objective measures as opposed to self-report scales. The final block of infrequently encountered attributes included power, outcome, active comparison, and follow-up assessment longer than 30 days. Half of the preexperimental studies (n = 4) achieved adequate statistical power for their design with at least 30 participants for the pretests and posttests. However, studies with comparison groups were less likely to achieve adequate statistical power because at least 71 subjects per condition were needed for active comparison, and 27 subjects per condition were required for passive comparison (Kazdin & Bass, 1989). Half of the preexperimental studies (n = 4) and half of the nonrandomized studies (n = 2) established primary and secondary outcome measures, while only two experimental studies specified which outcome was more important. One quarter of the nonrandomized experimental studies (n = 1) used active comparison involving psychotherapy to treat IA for the control group, and 7 of the 12 experimental studies applied active comparison such as medication or exercise therapy. Finally, two preexperimental studies conducted a follow-up assessment after 30 days to track the lasting effects of the treatment, whereas 4 of the 12 experimental studies did so.

Quality of Evidence and Effect Sizes

Among the 24 studies, the most commonly used treatments for IA are exercise programs (EP), CBT, electroacupuncture (EA), family therapy (FT), group-based treatment (GT), motivational interviewing (MI), and psychotropic medications (M). As indicated in Table 3 , more than half of the studies have multiple treatments. Except for studies by Du, Jiang, and Vance (2010) and Pan and Dai (2010), the rest of 22 studies all have statistically significant effects favoring the treatment. The overall mean effect size is 1.89, indicative of very large effects. Some of the studies which use CBT, M, and FT have extremely high effect size (> 5), such as Shen (2008), Yang, Shao, and Zheng (2005), and Shao, Yang, Luo, and Zheng (2004). However, most of those studies have low quality scores (<5), which means that the quality of the evidence is low.

Table 3.

Effect Sized and Methodological Quality Scores for Chinese Internet Addiction Studies

Author, Year, (Quality Score)	n	Treatment(s)	Outcomes
Author, Year, (Quality Score)	n	Treatment(s)	d (IAS)	d (SCL-90)
Continuous Outcome Measures-Comparison Group Designs
1. Du, Jiang, and Vance, 2010 (10)	56	CBT		0.13
2. Guo, Yu, Chao, and Huang, 2008 (1)	32	GT, CBT	2.08
3. Li, Chen, Zhang, and Jia, 2009 (3)	32	EP	3.26	4.44
4. Liao and Wang, 2009 (1)	50	EP	0.82
5. Lin, Zou, Xie, Lv, and Lin, 2006 (4)	30	M, CBT		0.68
6. Liu and Li, 2008 (5)	60	CBT, FT, GT		2.58
7. Shao, Yang, Luo, and Yang, 2005 (5)	66	M, CBT	2.88	0.54
8. Sun and Wang, 2008 (4)	220	CBT, GT	2.80
9. Wei, 2008a (4)	60	MT, EP, GT, FT, HE		3.83
10. Wu, Yan, and Han, 2007 (3)	27	EA	3.78
11. Wu and Yang, 2008 (3)	126	M, CBT	4.06
12. Zhang, 2009 (4)	163	EP	0.3386
13. Zhan and Li, 2009 (5)	60	M, CBT		1.35
14. Zhong, Tao, Zhu, Sha, and Yang, 2009 (7)	51	MT, CBT, GT		1.27
15. Zhu, Jin, Zhong, Chen, and Li, 2008 (4)	45	EA, CBT	1.86
Continuous outcome measures-preexperimental designs
16. Li, Li, Wang, and Zhang, 2008 (4)	48	M, CBT, FT, GT		0.67
17. Pan and Dai, 2010 (1)	11	EBT	0.29	1.14
18. Shao, Yang, Luo, and Zheng, 2004 (2)	20	CBT	14.5
19. Shek, Tang, and Lo, 2009 (4)	59	CBT, MI, FT, VS	1.45
20. Shen, 2008 (0)	27	CBT, M	10.7	18
21. Wei, 2008b (3)	273	EP, CBT, FT, GT, HE		1.71
22. Yang and Hao, 2005 (4)	52	CBT, FT, SBT	2.69
23. Yang, Shao, and Zheng, 2005 (1)	23	CBT, FT, M	13.43
Dichotomous outcome indicators
24. Liu, Yang, and Yao, 2007 (4)	40	M, CBT	0.875

Note. CBT = cognitive behavior therapy; EA = electroacupuncture; EBT = electroencephalographic biofeedback treatment; EP = exercise program; FT = family therapy; GT = group-based treatment; HE = health education; M = medication; MI = motivational interview; MT = military training; SBT = solution-focused brief therapy; VS = voluntary service.

CBT treatments combined with medications were the most frequently used therapies in the 24 studies. The QES scores for these studies ranged from 0 to 5 (mean = 3.4). The average effect size of CBT treatments combined with medications is 3.93. There are three studies which only used EP as the treatment (shown in Table 4 ). The average effect size of the three studies is 1.47 while the mean QES is 2.7. Two studies only used CBT. For one such study with a high QES (i.e., 10) the effect size is .13 (not statistically significant). However, for the lower quality study (QES = 2), the effect size is 14.5. Two studies used EA. This was a unique treatment in China, which uses electric stimulation to the body to reduce the desire to use the Internet. Since 2009, this method has been suspended in China due to the unclear negative consequences (Wei sheng bu jiao ting dian ji zhi liao wang yin [Chinese Ministry of Health ban on electric stimulus therapy for Internet addition disorder], 2009). One study used EBT which is a new biofeedback machine treatment. However, the findings showed that biofeedback machine therapy did not have statistically significant effects on IA. The combination of CBT, GT, FT, or others have average effect sizes ranging from 1.27 to 5.86, but the methodological quality for these studies is around 4. Based on Tables 3 and 4, when there are extremely large effect sizes (>5) as study 18, 20, 23 the QES falls below 2.

Table 4.

Quality of Evidence Scores and Effect Size by Type of Treatment

Treatment	Study Number	Mean QES (Range)	Mean Effect Size (Range)
EP	12, 3, 4	2.7 (1–4)	1.47 (0.34–3.26)
CBT	1, 18	6 (2–10)	7.3 (0.13–14.5)
EA	10	3	3.78
EBT	17	1	0.29
CBT, M	11, 7, 13, 5, 20, 24	3.4 (0–5)	3.93 (0.68–10.7)
CBT, GT	2, 8	2.5 (1–4)	2.44 (2.08–2.8)
CBT, EA	15	4	1.86
CBT, GT + others	14	7	1.27
CBT, FT + others	22, 23, 19	3 (1–4)	5.86 (1.45–13.43)
CBT, GT, FT + others	6, 21, 16	4 (3–5)	1.65 (0.67–2.58)
MT, BT, GT, FT, HE	9	4	3.83

Note. CBT = cognitive behavior therapy; EA = electroacupuncture; EBT = electroencephalographic biofeedback treatment; EP = exercise program; FT = family therapy; GT = group-based treatment; HE = health education; M = medication; MI = motivational interview; MT = military training; QES = quality of evidence scores; SBT = solution-focused brief therapy; VS = voluntary service.

Single Treatments and Multimodal Treatments

We compared effect sizes of studies which used single treatments with studies which used multiple treatments. The average effect size for single treatments is 1.77, and the average effect size for multiple treatments is 2.3. However, the t test is not statistically significant (t = .39). Using Chambless and Hollons’ (1998) criteria most of the treatments we reviewed would be considered possibly efficacious, with no replication studies for any treatment we reviewed (Chambless & Hollon, 1998).

Discussion and Applications to Social Work

The primary objective of the study was to review the published outcome studies on IA in the Chinese context. Following the procedures of Becker and Curry (2008) we evaluated 24 IA treatment studies. Our main finding was that only 2 of the 15 attributes were reported by more than 50% of the three types of study designs. Although random sequence generation is a necessity for experimental studies, only half of them mentioned how they generated a random sequence, and the other half did not indicate how they randomized participants. In addition, the remaining methodological criteria were poorly met, indicating most research did not establish reporting guidelines for clinical trials (Moher et al., 2001).

None of the studies contained treatment adherence ratings or collateral reports, which indicates that research paid a minimal amount of attention to these areas. Only the experimental studies reported baseline data, sample size, allocation concealment, and blind assessment. Few studies indicated which outcomes were primary. Similarly, few studies used manualized treatments. Replication by outside investigators, a criterion for establishing a treatment as empirically supported (Chambless & Hollon, 1998), would be difficult in the absence of treatment manuals. The lack of objective measures also threatened the reliability of the outcomes. Finally, the small sample sizes of these studies was concerning, as inadequate statistical power limits our abilities to detect true differences if they were indeed present.

Our secondary objective was to determine which treatment was more effective. However, based on low methodological quality scores here and the limited number of studies, this was a difficult task. Thus, we have only described trends in this study, and have not completed formal moderation analyses. In future studies, the QES criteria could be explored as moderators of treatment effects. However, due to the limited number of studies, there was not enough statistical power in this study to enable such an analysis. Another approach is to find studies with high methodological quality, and then to see whether such studies have high effect sizes. Comparatively, in Becker and Curry’s (2008) study, the median QES was 7, reflecting the relatively higher maturity of the evidence-base for that population. That was, in our study, only 1 study of the 24 studies examined that has a QES larger than 7. And this study had a low effect size of .13 without statistically significant differences between treatment conditions. Most of the studies in this review article had a QES lower than 5. Because of this finding, it is important that future trials use more rigorous designs and adhere to established standards of clinical trial reporting. According to Chambless and Hollon (1998), at least two repeated rigorous treatment studies are needed to identify a promising treatment. Using Chambless and Hollons’ (1998) criteria most of the treatments we reviewed would be designated as possibly efficacious, since no replication studies existed for any treatment we reviewed and power was too low in most cases to detect moderate effect sizes. Rigorous designs and reporting can help researchers and social workers to identify whether studies are rigorously replicated. The rigorous design and reporting will also prevent potential biased estimation of effect size for the IA treatment in the future. There is evidence that inadequate methodological approaches and reporting are associated with over-estimation of treatment effects and failure to consider the quality of methodology and reporting limits the ability to detect potentially inflated treatment estimates, identify sources of bias and inform best practices in the field (Juni, Altman, & Egger, 2001; Moyer & Finney, 2005).

There were some limitations for this study. First, we reported low κ values with respect to two attributes, outcome and active comparison, indicating a relatively low interrater reliability for some characteristics. There were high discrepancies initially for these two characteristics. For example, one rater believed the outcome was presented only if the study differentiated which outcome was primary and which was secondary. However, another rater was more generous in coding the outcome as “1” as long as the study presented two or more outcomes in which at least one pertained to IA behavior. Similarly, one rater believed the study applied active comparison only if the comparison group had received evidenced-based intervention to treat Internet addition, whereas the other rater applied a broader rule and coded a “1” if the comparison group received some treatments, no matter whether they were evidenced-based or not. Despite the low interrater reliability for these two attributes, we observed modest or high κ values for the rest of the attributes. In addition, for these two attributes and other attributes having κ values lower than one, unanimous agreement was achieved through dialogue with an experienced colleague.

It was also possible that studies were more rigorous than we have estimated, as we could only score studies based on their reporting of methodological attributes. Thus, it was possible that we underestimated the QES scores for some studies. As long as the authors did not explicitly mention the attributes in the article, we treated them as not having those attributes although they may have simply not reported doing so. The research conventions in the context of Chinese academic language might divert the publication format away from the American academic conventions even further. For example, many articles were between one to four pages in length, which is a highly condensed publication format.

The limited number of studies and nonconsistent outcome measurement tools constrained our ability to fulfill the second main objectives of this review. Thus, we were limited by our small sample size of 24 studies. In addition, these 24 studies used different outcomes measurement tools to report how people recover from IA. Some of the studies only reported changes in symptoms, such as depression or anxiety, without reporting changes in IA criteria. It was problematic to compare effect sizes calculated based on different outcome measurement tools.

Finally, we acknowledge that it is yet unclear how vulnerable and impoverished youth are affected by IA. For example, Eamon’s (2004) analysis of a nationally representative sample found that poor youth had less computer access and were less likely to use computers less for nonacademic purposes. Future clinical research should explicitly report participant demographic characteristics so social workers know if vulnerable youth are affected by IA and represented in these studies.

Given that few of the methodological standards were met, researchers are encouraged to include as many essential elements of design as possible and contribute to establishing reporting guidelines for intervention studies. As the classic experimental design is characterized by criteria such as sequence generation, allocation concealment, and baseline data, these attributes should be reported when they have been implemented. Some criteria such as using objective measures, establishing primary outcome measures, having adequate power, and using manualized treatment, are easier to meet than other criteria; therefore, researchers should report them. For those attributes, including treatment adherence rating, collateral report, objective measure, and blind assessment, as they are embedded in more rigorous designs and are difficult to be fulfilled, researchers might consider including them while designing their studies. The bottom line is that when implementing a design, researchers should not neglect to report them. Otherwise, the audience will assume that the study has not been conducted rigorously.

The overall QES of the 24 studies is relatively low (Mean = 3.58, SD = 2.12, Median = 4, Range = 0–10), suggesting there are many improvements needed for experimental studies on IA treatment. As experimental designs are weighted heavily in the determination of whether practices are empirically supported, it is discouraging that the QES scores for experimental studies in this review are not higher. Indeed, they were lower than those of randomized trials in other reviews (Becker & Curry, 2008), indicating that methodology quality is relatively weak in the area of IA intervention studies. The lack of scientifically established evidence hampers researchers’ abilities to inform practitioners of promising treatments for IA. Therefore, we suggest researchers apply more rigorous experimental designs in their research because only through randomized controlled experiment design can a counterfactual model be set up to determine true treatment effectiveness (Shadish, Cook, & Campbell, 2002).

The second implication of this study is that there exists a need for a more uniform reporting style for outcome studies in China. The CONSORT statement (Moher et al., 2001) is lauded as the gold standard for transparency in clinical trial reporting in the United States. Chinese scholars could develop similar reporting criteria or follow CONSORT criteria to enhance the quality of study and reporting.

Finally, one implication for social work practice pertains to social workers choosing the best evidence about IA treatments. In short, this study highlights how, social workers should not choose treatments only based on simple counts of how many studies have found positive effects for a specific treatment. As we have seen in this study, not all clinical trials are equally rigorous. Thus, criteria for determining whether a particular treatment is empirically supported should consider a more sophisticated weighting scheme to account for studies with low methodological quality, which could inflate effect sizes. To be specific, social workers should continue to receive training in being good consumers of research, with emphasis on identifying attributes of studies that are associated with higher methodological quality. Although the literature is limited with regards to the most promising IA treatments, studies reviewed here with a QES equal or larger than 4 (i.e., the median) have relatively higher methodological quality.

In terms of IA treatment, the most popular treatment is CBT. Unfortunately, replication studies are needed to conclusively determine whether CBT is a promising treatment, since one study had a low QES score, and a more rigorous trial found a weak, statistically insignificant effect (d = .13). With the above evidence, social worker can only treat CBT as a possibly efficacious treatment. CBT combined with medication seems like a promising therapy with a mean effect size of 3.93 and mean QES of 3.4. However, when we looked deeply, we found that five of the CBT plus medication studies used the SCL-90 as the outcome measurement, and did not report actual changes in IA symptoms. Although the studies were using medications that targeted depression, it would also be informative to know if these medications work for IA symptoms. Future trials could use 2 × 2 designs to partition the effects of medications and cognitive behavioral treatments. EPs had a mean effect size of 1.47 and mean QES of 2.7. The effect size was high however the quality of the studies was low. Taking into consideration of effect size inflation, EP should be viewed somewhat skeptically, and studied further in more rigorous trials. Electroencephalographic biofeedback treatment did not have statistically significant effects (d = .29), so social workers should proceed with caution with this treatment until additional studies support its efficacy. CBT, combined with group therapy and military training had a QES of 7 and effect size of 1.27. This treatment fulfilled the criteria with a relatively higher methodological quality and effect size. Therefore, this treatment was the most promising treatment we reviewed in this study. CBTs combined with either group or FT had a mean QES of 4 and mean effect size of 1.65. The average effect size was high, but these studies were only average in terms of methodological quality. Relative to other treatments we studied, these seem promising, with the general caveat that greater specification of these treatments and more rigorously designed studies are needed. Based on this review, possibly efficacious treatments for IA are cognitive behavior therapies that are combined with group therapy, military training, or FT.

Footnotes

Acknowledgments

The authors would like to thank Mary Keegan Eamon and Jun Sung Hong for their helpful comments on an earlier draft of this article.

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

The authors received no financial support for the research, authorship, and/or publication of this article.

References

Batthyany

Muller

K. W.

Benker

Wolfling

(2009). Computer game playing: Clinical characteristics of dependence and abuse among adolescents. Wiener Klinische Wochenschrift, 121, 502–509.

Becker

S. J.

Curry

J. F.

(2008). Outpatient interventions for adolescent substance abuse: A quality of evidence review. Journal of Consulting and Clinical Psychology, 76, 531–543.

Bernardi

Pallanti

(2009). Internet addiction: A descriptive clinical study focusing on comorbidities and dissociative symptoms. Comprehensive Psychiatry, 50, 510–516.

Bioulac

Arfi

Bouvard

M. P.

(2008). Attention deficit/hyperactivity disorder and video games: A comparative study of hyperactive and control children. European Psychiatry, 23, 134–141.

Brezing

Derevensky

J. L.

Potenza

M. N.

(2010). Non-substance-addictive behaviors in youth: Pathological gambling and problematic internet use. Child and Adolescent Psychiatric Clinical of North America, 19, 625–641.

Caplan

Williams

Yee

(2009). Problematic internet use and psychosocial well-being among MMO players. Computers in Human Behavior, 25, 1312–1319.

Chambless

D. L.

Hollon

S. D.

(1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66, 7–18.

Chan

P. A.

Rabinowitz

(2006). A cross-sectional analysis of video games and attention deficit hyperactivity disorder symptoms in adolescents. Annals of General Psychiatry, 5, 16.

China Youth Internet Association. (2010, January). 2009 nian qing shao nian wang yin diao cha bao gao [2009 report on youth internet addiction disorder]. Retrieved from http://www.docin.com/p-46439854.html

10.

Chou

Condron

Belland

J. C.

(2005). A review of the research on internet addiction. Educational Psychology Review, 17, 363–388.

11.

Cohen

(1988). Statistical power analysis for the behavior sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

12.

Derogatis

L. R.

(1994). Symptom Checklist 90-R: Administration, scoring, and procedures manual (3rd ed.). Minneapolis, MN: National Computer Systems.

13.

Y. S.

Jiang

W. Q.

Vance

(2010). Longer term effect of randomized, controlled group cognitive behavioral therapy for internet addiction in adolescent students in Shanghai. Australian and New Zealand Journal of Psychiatry, 44, 129–134.

14.

Eamon

M. K.

(2004). Digital divide in computer access and use between poor and non-poor youth. Journal of Sociology and Social Welfare, 31, 91–112.

15.

Erceg-Hum

D. M.

Mirosevich

V. M.

(2008). Modern robust statistical methods. American Psychologist, 63, 591–601.

16.

Fisher

(1994). Identifying video game addiction in children and adolescents. Addictive Behaviors, 19, 545–553.

17.

Frolich

Lehmkuhl

Dopfner

(2009). Computer games in childhood and adolescence: Relations to addictive behavior, ADHD, and aggression. Zeitschrift Fur Kinder-Und Jugendpsychiatrie Und Psychotherapie, 37, 393–402.

18.

Gentile

(2009). Pathological video-game use among youth ages 8 to 18: A national study. Psychological Science, 20, 594–602.

19.

Gruesser

S. M.

Thalemann

Albrecht

Thalemann

C. N.

(2005). Excessive computer usage in adolescents: A psychometric evaluation. Wiener Klinische Wochenschrift, 117, 188–195.

20.

Guo

Chao

X. L.

Huang

G. M.

(2008). Impact evaluation of group counseling on internet addiction adolescents. Chinese Journal of School Health, 29, 17–19.

21.

Park

D. H.

Lim

(2009). Depression is a major cause of internet addiction in adolescents and young adults: 2 year experience in internet addiction clinic. International Journal of Psychiatry in Clinical Practice, 13, 30–31.

22.

Huang

X. Q.

M. C.

Tao

(2010). Treatment of internet addiction. Current Psychiatry Report, 12, 462–470.

23.

Hwang

J. M.

Cheong

P. H.

Feeley

T. H.

(2009). Being young and feeling blue in Taiwan: Examining adolescent depressive mood and online and offline activities. New Media & Society, 11, 1101–1121.

24.

Block

J. J.

(2008). Issues for DSM-V: Internet addiction. The American Journal of Psychiatry, 165, 306–307.

25.

Juni

Altman

D. G.

Egger

(2001). Systematic reviews in health care: Assessing the quality of controlled clinical trials. British Medical Journal, 323, 42–46.

26.

Kim

E. J.

Namkoong

Kim

S. J.

(2008). The relationship between online game addiction and aggression, self-control and narcissistic personality traits. European Psychiatry, 23, 212–218.

27.

C. H.

Yen

J. Y.

Chen

C. S.

Yeh

Y. C.

Yen

C. F.

(2009). Predictive values of psychiatric symptoms for internet addiction in adolescents: A 2-year prospective study. Archives of Pediatrics & Adolescent Medicine, 163, 937–943.

28.

Lam

L. T.

Peng

Mai

Jing

(2009). The association between internet addiction and self-injurious behavior among adolescents. Injury Prevention, 15, 403–408.

29.

Chen

Zhang

Jia

(2009). Jian shen yun dong chu fang dui da xue sheng wang yin gan yu xiao guo de ying xiang [The effects of fitness exercise prescriptions on intervening college student’s internet addiction]. Journal of Physical Education, 16, 55–58.

30.

G. Y.

Wang

Y. Y.

Zhang

S. Y.

(2008). The effects of residential hospital treatment on 48 internet addiction patients. Journal of Psychiatry, 21, 356–360.

31.

Liao

X. H.

Wang

Z. Z.

(2009). You yang jian shen cao dui wang luo cheng yin da xue sheng gan yu xiao guo de yan jiu [Intervention of aerobic exercise for internet addiction college student]. Journal of Hunan University of Technology, 23, 86–88.

32.

Lin

Z. X.

Zou

X. B.

Xie

Lin

J. D.

(2006). Qing shao nian wang luo cheng yin xin li yao wu lian he zhi liao [Psychological and medical treatments on youth internet addiction disorder]. Nervous Diseases and Mental Health, 6, 127–129.

33.

Liu

Q. S.

Z. Q.

(2008). Zhu yuan wang luo cheng yin huan zhe xin li gan yu de kang fu xiao guo ping jia [Rehabilitation effect of the psychological intervention on internet addiction patients in hospital]. Chinese Journal of School Health, 29, 1117–1119.

34.

Liu

X. Q.

Wang

J. G.

Zhuang

H. H.

(2008, November 11). The Chinese diagnostic criteria for internet addiction was approved by clinic experts. Xin Hua News Agency. Retrieved from http://www.gov.cn/jrzg/2008-11/08/content_1143277.htm.

35.

Liu

Yang

G. D.

Yao

X. M.

(2007). The effect of pharmacotherapy and psychotherapy with 40 IAD cases. Chinese Journal of Drug Abuse Prevention and Treatment, 13, 88–91.

36.

Moher

Schulz

K. F.

Altman

(2001). The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. Lancet,357, 1191–1194.

37.

Moyer

Finney

J. W.

(2005). Rating methodological quality: Toward improved assessment and investigation. Accountability in Research,12, 229–313.

38.

Pan

S. J.

Dai

X. Y.

(2010). Nao dian sheng wu fan kui zhi liao zhong xue sheng wang luo cheng yin de xiao guo guan cha [Efficiency of electroencephalographic biofeedback treatment in middle school students with internet addiction disorder]. Journal of Ningxia Medical University, 32, 71–73.

39.

Rubin

Babbie

E. R.

(2011). Research methods for social work (7th ed.). Belmont, CA: Brooks/Cole.

40.

Shao

Yang

Luo

K. L.

Yang

(2005). Fu xi ting dui ban yi yu de zhong xue sheng wang luo cheng yin zheng de lin chuang jiao zhi zuo yong [The effect of fluoxetine in treating middle school students with depressant internet addiction disorder]. Chinese Journal of Nervous and Mental Disorder, 31, 422–425.

41.

Shao

Yang

Luo

K. L.

Zheng

(2004). Xin li zhi liao zhong xue sheng wang luo cheng yin zheng de lin chuang yan jiu [Clinical research on psychological treatment for adolescent internet addiction]. Chinese Journal of Child Health Care, 12, 548–549.

42.

Shek

D. T. L.

Tang

V. M. Y.

C. Y.

(2009). Evaluation of an internet addiction treatment program for Chinese adolescents in Hong Kong. Adolescence, 44, 359–373.

43.

Shen

Z. M.

(2008). Xi tai pu lan zhi liao zhong du wang luo yi lai 27 li liao xiao fen xi [Treatment effect of anti-depression drug on internet addiction for 27 clients]. Tianjin Pharmacy, 20, 61–74.

44.

Skoric

M. M.

Teo

L. L. C.

Neo

R. L.

(2009). Children and video games: Addiction, engagement, and scholastic achievement. Cyberpsychology & Behavior, 12, 567–572.

45.

Sun

D. L.

Bao

Chen

X. C.

Zhang

D. R.

(2008). Computer games: A double-edged sword? Cyberpsychology & Behavior, 11, 545–548.

46.

Sun

Wang

H. C.

(2008). Evaluation of multiple therapy intervention to internet addiction for college students. Chinese Journal of School Health, 29, 1137–1139.

47.

Wang

X. D.

Wang

X. L.

(1999). Xin li wei sheng ping ding liang biao shou ce [Mental health assessment manual]. Beijing: Chinese Mental Health Journal Institution.

48.

Wei

Q. X.

(2008a). The role of psychological nursing intervention in the treatment of internet addiction. Journal of Heze Medical College, 10, 16–18.

49.

Wei

Q. X.

(2008b). Xing wei jiao zhi yu xin li gan yu dui wang luo cheng yin huan zhe kang fu xiao guo de ying xiang [Affection of behavior correction and mental intervention on the rehabilitation efficacy of internet addicts]. Journal of Military Surgeon in Southwest China, 10, 16–18.

50.

Wei sheng bu jiao ting dian ji zhi liao wang yin [Chinese Ministry of Health ban on electric stimulus therapy for internet addition disorder]. (2009, July 13). Retrieved from http://news.163.com/09/0713/17/5E4D5T7P000120GU.html.

51.

L. Z.

Yan

J. J.

Han

J. S.

(2007). 2/100 Hz jing pi xue zhu dian chi ji dui 27 li qing shao nian wang luo cheng yin zheng de zhi liao zuo yong [Treatment on 27 adolescents with Internet addiction by 2/100 HZ Han’s acupoint nerve stimulation]. Chinese Journal of Drug Dependence, 16, 32-35.

52.

S. L.

Yang

Y. X.

(2008). The evaluation of cognitive behavior therapy on internet addiction. China Medical Herald, 5, 161–162.

53.

Yang

F. R.

Hao

(2005). 52 li wang luo cheng yin qing shao nian xin li she hui zong he gan yu de liao xiao guan cha [The effect of integrated psychosocial intervention on 52 adolescents with internet addiction disorder]. Chinese Journal of Clinical Psychology, 13, 343-346.

54.

Yang

Shao

Zheng

(2005). zhong xue sheng wang luo cheng yin zheng de zong he gan yu [Comprehensive intervention on internet addiction of middle school students]. Chinese Mental Health Journal, 19, 457-459.

55.

Yang

Zheng

Ruan

K. L.

(2004). Wang luo cheng yin shi zheng yan jiu jin zhan [Research progress on internet addiction disorder]. Journal of Southwest China Normal University, 30, 2–5.

56.

Young

(2009a). Internet addiction: Diagnosis and treatment consideration. Journal of Contemporary Psychotherapy, 39, 241–246.

57.

Young

(2009b). Understanding online gaming addiction and treatment issues for adolescents. American Journal of Family Therapy, 37, 355–372.

58.

Young

K. S.

(2004). Internet addiction: A new clinical phenomenon and its consequences. American Behavioral Scientist, 48, 402–415.

59.

Zhan

L. Y.

R. Q.

(2009). Nei guan ren zhi liao fa dui wang yin huan zhe xin li kang fu de zuo yong [Effect of neikan cognitive therapy on psychological status of inpatients with internet addiction]. Journal of Nursing Science, 24, 77–79.

60.

Zhang

Y. H.

(2009). Yi ti yu huo dong wei zhu yao shou duan gan yu da xue sheng wang luo cheng yin de shi zheng yan jiu tan jiu [Experimental study to interfere with college students’ internet addiction by using sports as main means]. Journal of Tonghua Teachers College, 30, 74–76.

61.

Zhong

Tao

Zhu

Sha

Yang

F. C.

(2009). Tuan ti xin li gan yu dui qing shao nian wang luo cheng yin de xiao guo yan jiu [Effect of group psychological intervention in adolescent on internet addiction]. Journal of Capital Medical University, 30, 494–499.

62.

Zhu

T. M.

Jin

R. J.

Zhong

X. M.

Chen

(2008). Dian zhen jie he xin li gan yu dui wang yin huan zhe jiao lv zhuang tai ji xie qing NE bian hua de ying xiang [Effects of electroacupuncture combined with psychological interference on anxiety state and serum NE content in the patient of internet addiction disorder]. Chinese Acupuncture & Moxibustion, 28, 561–564.