Digital Parent Training for Children with Disruptive Behaviors: Systematic Review and Meta-Analysis of Randomized Trials

Abstract

Objectives:

Digital-based parent training (DPT) programs for parents of children with disruptive behaviors have been developed and tested in randomized trials. The aim of this study was to quantitatively assess the efficacy of these programs versus a control condition.

Methods:

We conducted a systematic review and random effects meta-analysis of peer-reviewed randomized studies comparing DPT targeting children with disruptive behaviors versus a control group (wait list or no treatment).

Results:

Altogether, seven studies (n = 718) were meta-analyzed. Compared to the control groups, DPT resulted in significantly greater improvement in child behavior (effect size [ES] = 0.44, 95% confidence interval [CI] = 0.21–0.66, studies = 7), parent behavior (ES = 0.41, 95% CI = 0.25–0.57, studies = 6), and parental confidence (ES = 0.36, 95% CI = 0.12–0.59, studies = 4). The improvement in child behavior was moderated by age group and severity of clinical presentation, which overlapped 100%. While DPT was superior to control conditions in studies of young children (mean age <7 years) with a clinical range of disruptive behaviors (ES = 0.61, 95% CI = 0.40–0.82, studies = 4), results were nonsignificant in studies of older children (mean age >11 years) with a nonclinical range of symptoms (ES = 0.21, 95% CI = −0.01 to 0.42, studies = 3). Analyses yielded similar results of higher ESs favoring studies of young children with clinical range disruptive behaviors for parent behavior and parental confidence, but the differences were not significant. Results further suggested that in studies of younger children, interactive programs (e.g., computerized programs) were more effective in improving child behavior compared to noninteractive programs (e.g., watching video clips) (p < 0.05).

Conclusion:

Although additional studies are needed, DPT holds promise as a potentially scalable evidence-based treatment of children with disruptive behaviors that can save human resources.

Introduction

Disruptive behavior disorders and related difficulties (e.g., aggression, defiance, noncompliance) are among the most prevalent types of psychiatric problems affecting children and adolescents (Comer et al. 2013). In an epidemiological study conducted in 2009, the median 12-month prevalence of conduct disorder or oppositional defiant disorder was 6% (Merikangas et al. 2009), and several studies reported disruptive behaviors to be among the most common reasons youth were referred to mental healthcare (e.g., Egger and Angold 2006; Zisser and Eyberg 2010).

Given the prevalence and wide-ranging consequences of disruptive behavior disorder problems to families and society, research has focused on the development of effective interventions for their prevention and treatment. Among interventions, behavioral parent training (BPT) is considered the first choice treatment (Eyberg et al. 2008; Chorpita et al. 2011). However, barriers to receiving such interventions exist, including the lack of trained staff to provide this evidence-based treatment, stigma related to receiving mental health treatment, and the difficulties to access and engage in treatment in terms of costs, time, and location (Kazdin and Blase 2011; Kazdin and Rabbitt 2013). These barriers lead to poor quality of care for youth with disruptive behaviors. For example, research suggests that psychotropic medications are prescribed for a majority of youth with disruptive behaviors without attempting psychological intervention, such as BPT (Olfson et al. 2012, 2014), exposing youth to the risk of concerning antipsychotic adverse effects, including potentially life-shortening cardiometabolic risks (Correll et al. 2009; Maayan and Correll 2011; Galling et al. 2016).

Technology-based programs offer a host of features that can address most of these barriers and might be particularly useful for parents of youth with relevant disruptive behavior problems. First, technology-based interventions move the intervention outside of traditional clinics and into people's homes, increasing access and reducing stigma. Services can be available 24/7, on demand, which might be useful for people whose schedules are difficult and unpredictable. Furthermore, scalability of technology-based services results in a significant decrease of program costs. Accordingly, the advantages of technology-based solutions have been tested by researchers studying BPT for youth with disruptive behaviors. Specifically, research suggests that technologies could play a significant role in parent training by engaging families into services (Breitenstein et al. 2014; Sourander 2015), training of staff, and in serving as the primary delivery source of BPT (Jones et al. 2013).

These suggestions are in line with the fact that BPT was among the first to test technology in its early and primitive forms (e.g., videotapes) as the primary delivery source of an intervention (Webster-Stratton et al. 1988). Subsequently, digital-based parent training (DPT) programs for disruptive behaviors were developed and tested in randomized trials. There are a few narrative reviews on DPTs (e.g., Breitenstein et al. 2014; Jones et al. 2013), which introduce the ways technology can be used to improve BPT for promoting children's mental health. None of these articles, however, systematically reviewed or meta-analyzed the results of randomized studies of DPT for youth with disruptive behaviors. The aim of this meta-analysis was, therefore, to meta-analytically assess the effects of DPTs for treating youth with disruptive behaviors to inform stakeholders regarding this intervention's potential and to identify main challenges for future consideration.

Methods

This review and following meta-analysis were carried out in line with the Meta-Analysis Reporting Standards (Publications and Communications Board Working Group on Journal Article Reporting 2008) and PRISMA statement (Liberati et al. 2009) for reporting meta-analysis.

Selection of studies

Six criteria were used to select studies for inclusion. First, the study had to report on a parent training intervention targeting their child's disruptive behavior problems (but not targeting criminal activities or child maltreatment), measured using a valid scale with established reliability and validity properties (e.g., Eyberg Child Behavior Inventory [ECBI]; Eyberg and Robinson 1983). Accordingly, we did not include studies of general parent training targeting child rearing, as their focus is not specifically to improve behavioral problems. Attention-deficit/hyperactivity disorder (ADHD) studies were included if the primary goal was to target children's externalizing behavior. Second, the program was designed to use digital media or software programs (e.g., DVD, online based), primarily to replace human support, and not to be primarily used within a therapy setting (e.g., group, face-to-face) or in parallel to sessions with a therapist or a coach. Accordingly, studies investigating the efficacy of therapist-assisted BPT or investigating traditional therapy delivered through teleconference were excluded. Third, the study reported on families in which the targeted children did not have developmental delay or health impairment. Fourth, the study reported meta-analyzable outcomes, as detailed in the coding section, regarding child behavior, parent behavior, or parental confidence in self-efficacy. Fifth, the study had at least one treatment and one control group, drawn from the same population, with at least five participants per group. Sixth, the study was published in English and in a peer-reviewed source.

Computer searches of PubMed, Psychinfo, and Embase databases were conducted for all published studies from 1974 to December 21, 2015. We used a combination of keywords for parent, training, technologies (e.g., online, internet, computerized, mobile), and disruptive behavior problems (see Appendix for complete Psychinfo search terms) and keywords for randomized trials, aiming to start with a broad and sensitive search strategy (Robinson and Dickersin 2002). A search for reference to technology was also conducted by author name, using the names of known experts in the area of BPT programs (e.g., Dumas, Eyberg, Forehand, Jones, Kazdin, Patterson, Sanders, Webster-Stratton). Additional articles were identified by hand searching the references of retrieved articles and previous reviews. The first two authors, a clinical psychologist (A.B.) and psychiatrist (A.P.), independently assessed all potentially relevant articles for inclusion. Any disagreements were resolved through discussion and consensus.

Coding of studies

Eligible studies were reviewed, and data on dependent measures and study characteristics were retrieved in accordance to previous large meta-analysis of parent training (Lundahl et al. 2006) and as detailed below. Individual effect sizes (ESs) were extracted and coded for three dependent outcomes: (i) child behavior was computed based on standardized parent report measures, such as the ECBI (Eyberg and Robinson 1983), (ii) parent behavior was computed based on parent reports on their behaviors and interactions with the child using measures such as the Parenting Scale (Arnold et al. 1993), and (iii) parental confidence (in self-efficacy) was computed based on parental reports using measures, such as the Parenting Sense of Competence scale (Johnston and Mash 1989) or the Parenting Task Checklist, each of which assess parents' task-specific self-efficacy (Sanders and Woolley 2005). In studies with more than one measure for either one of the three dependent outcomes (child behavior, parent behavior, parental confidence), one pooled ES per outcome was computed based on all available relevant measures.

Several study design and participant characteristics were coded and categorized based on conceptual and data distribution aspects. Child's mean age was coded into two groups: young children (37–84 months) and middle school children (138–162 months). Severity level of undesirable child behavior at the beginning of treatment was coded as “clinical” when data indicated a clinical range of disruptive behavior disorders in the sample and was coded as “nonclinical” if data indicated that the study did not include mostly children in the clinical range of disruptive behavior disorders, based on standard measures or inclusion criteria. Family socioeconomic status (SES) was coded into low, medium, and high. We based the coding on data, including parental employment rates, income, and education (e.g., 80% of sample household's income above average was coded as high; sample consisting 73.3% of household income below the median annual income in the United States was coded as low). Interventions were coded into either noninteractive when the intervention was delivered through the use of noninteractive digital videos or audio (e.g., viewing a series of clips) or interactive when the intervention enabled the participants to control pacing, choose different pathways to growth, or receive feedback (Segal et al. 2003; Baumel and Muench 2016). Studies were independently coded by the first and second authors revealing 100% agreement on each of the categories described above; therefore, inter-rater reliability was not computed further.

Assessment of methodological quality and procedures

Methodological quality was assessed with the Cochrane Collaboration “risk of bias” tool that addresses different domains of methodological quality, including random sequence generation, incomplete outcome data, and “other biases” (Higgins et al. 2011). The “other bias” domain was assessed through the following criterion: balance/imbalance of demographic and clinical measurements baseline characteristics across study groups. Since blinding of study participants and study staff is not feasible in these nondrug studies (Higgins et al. 2011), we do not present this assessment item. Two reviewers (A.B., A.P.) independently assessed the methodological study quality. Any disagreements were resolved through discussion and consensus.

Data analysis

We organized our findings in six separate analytic data files, each relating to one of the three examined outcomes (child behavior, parent behavior, parental confidence) and one of the two time frames (i.e., immediate treatment effect and post-treatment follow-up effect). Immediate treatment effect related to the difference in outcome between the intervention and control group immediately after the intervention. Post-treatment follow-up effect related to the within treatment group difference in the intervention arm from the end of treatment until the end of follow-up time.

Cohen's d ESs (Wilson and Lipsey 2001) were calculated using Comprehensive Meta-Analysis software (version 3.3.070) using random-effects models (DerSimonian and Laird 1986). Unlike fixed-effects models, random-effects models assume that the true ES may vary between studies due to different population parameters. Therefore, compared with the fixed effects model, the weights assigned under random effects are more balanced (Borenstein et al. 2007). All ESs are organized in a way that a larger positive effect reflects bigger positive improvement in the relevant outcome, either in the intervention group versus the control condition (treatment effect adjusted for control effect) or at the end of post-treatment follow-up compared to end of the intervention (maintenance effect in the intervention group). Because post-treatment follow-up effects were calculated by examining the difference between outcome measures at the end of the intervention and the outcomes at the end of post-treatment follow-up, a nonsignificant outcome indicates that the effect achieved at the end of treatment was sustained through the follow-up period despite cessation of the intervention. On the contrary, a significant change with a negative effect would mean that the effect achieved at the end of treatment was not sustained through the follow-up period and that prior gains were diminished or lost after the end of the active intervention. We used the means and standard deviations reported in the studies instead of already computed ESs or significance measurements. Z-scores were used to assess the significance of pooled effects sizes, and homogeneities of ESs were assessed with the Q statistic (Hedges and Olkin 2014) and I² to describe the percentage of variation due to heterogeneity (Higgins and Thompson 2002). Potential moderators were evaluated by computing Q_between tests.

Results

Search results

The electronic and manual searches resulted in 7195 hits after removing duplicates. Through the first screening process, 63 articles were identified and retrieved for detailed evaluation (Fig. 1).

FIG. 1.

Flow diagram of study selection.

Characteristics of included studies

A total of seven studies with 718 participants (329 in the intervention condition, 389 in the control condition) met all inclusion criteria. Selected characteristics of these studies are presented in Table 1. In all studies, the control condition groups did not receive any treatment during the intervention time. Four studies were aimed at children with an average age ranging between 3.9 and 6.8 years and three were aimed at children with an average age between 11.8 and 14 years. While all four studies targeting younger children consisted of samples in the clinical range of elevated disruptive behavior symptoms, all three studies targeting older children consisted of samples with data suggesting a magnitude of disruptive behaviors that was mostly below the clinical range of symptoms. Regarding the SES of the samples, two out of the three studies aimed for younger children with SES information included samples within the range of high SES and two out of the three studies aimed for older children had samples within the range of low SES.

Table 1.

Selected Characteristics of Randomized Trials Examining the Effects of Digital Parent Training Targeting Children with Disruptive Behaviors

Study	Child age ^a	Clinical symptoms of the sample	N^b	Intervention–technology implementation	Control	Codes ^c	Fo-up	Measures ^d
Cefai et al. (2010)	9–15 (11.8)	Parent concerns regarding children behavior problems. (ECBI problems mean < 12)	86 (40, 46)	The Parenting Wisely's main aim was to modify parent–child interactions that reinforce antisocial behavior, using refraining and cognitive restructuring methods to foster behavior change (Gordon and Stanar 2003).	WLC	I/ 2/ 2/ 2	3m	ECBI, PSOC
				The intervention consisted of a self-administered software program (CD-ROM) with scenario-based learning design and a supplementary workbook. After selecting a problem, parents viewed a video clip depicting a family struggling with that problem. Parents selected a solution to the problem, viewed a video enactment of their solution, and participated in a critique of that choice. Strategies such as contracting, contingency management, I-statements, active listening, assertive discipline, and praise were presented. After viewing the most effective solution, a series of multiple-choice questions reviewed the concepts and skills depicted in that section. Parents completed the program in a little over 2 hours.
Enebrink et al. (2012)	3–12 (6.8)	1 SD above the mean of ECBI	86 (46, 40)	The intervention was based on a Swedish parent training program, Comet, that targets parents' reactions and activities to promote behavioral change in the parent and then the child (Kling et al. 2010).	WLC	I/1/1/3	—	ECBI, PPI
				The intervention consisted of 7 online interactive sessions distributed over 10 weeks (some session homework took two to three weeks to complete) relating to topics such as extinction of negative behavior and positive interaction. Sessions were composed of written text, videos of interactions between a parent/child, and illustrations. Other features included downloadable material (e.g., reward system), multiple-choice questions with direct feedback, online diary keeping parents reports on their training and past week, and the possibility to pose questions to the research assistant and write comments to homework or to a monitored parenting discussion forum. The research assistants gave feedback (e.g., reinforcement on progress) through the website on work and distributed a new session each week.
Irvine et al. (2015)	(13.1)	At least four identified behaviors from a list of problematic behaviors (e.g., trouble at school)	230 (90, 140)	The program teaches adaptive parenting skills through scenario-based learning (problem-based learning).	WLC	I/2/2/1	6m	ECBI, PS, Parental Self Efficacy
				The computer program was used within community centers. It comprised two sessions in which the parents chose topics out of the available 9 (e.g., bedtime, fighting) and watched a short video (per topic) depicting a problematic situation. After the video played, parents were asked to choose which of the three parental responses was closest to what they would do. Only one choice was “correct” and led to a positive outcome. The program features also included on-screen Q&As that were designed to mimic a parent–therapist counseling session and to consider how behaviors affected the outcome. Parents were also encouraged to make up a personal action plan by selecting up to three parenting skills relevant to that problem situation to practice at home.
Kacir (2000)	12–18 (14.4)	58% of participants demonstrated some elevations (scores of 9 or greater) on the EXBI problems.	38 (19, 19)	The intervention was similar to Cefai et al. (2010)	WLC	I//2/1	3m	ECBI, PBQ
Morawska et al. (2014)	2–10 (6.1)	Parent concerns about the child's emotional or behavioral adjustment (ECBI problems mean > 15).	100 (45, 55)	The intervention was based on the Triple P program, which directs the parent to promote nurturing behaviors, a low conflict environment, and children's social and emotional well-being through positive parenting practices (Sanders 1999).	WLC	N/1/1/—	6m	ECBI, PS, PTC, CAPES confidence
				The program included seven online downloadable podcasts, ranging from 9 to 14 minutes, discussing parenting skills revolving around disruptive behaviors (e.g., dealing with aggression, sharing). The format of these podcasts was conversational, in which the presenter asked questions of the parenting expert relevant to the topics discussed. Podcasts were available for participating parents in 3 phases over 2 weeks (2 were available immediately, the next 3 after 5 days, and the final 2 an additional 5 days later). Parents were e-mailed when a new set of podcasts was available and were given an additional 2 weeks to implement the strategies. Podcasts remained available throughout the study.
Porzig-Drummond et al. (2015)	2–10 (5.7)	Elevated disruptive behavior based on ECBI (ECBI problems mean > 15)	62 (29, 33)	The intervention program was based on the 1-2-3 magic parenting program focusing on established behavioral, emotional, and cognitive principles, teaching the parents techniques, and providing guidance on ways to reduce children's disruptive behaviors (Phelan 2010).	WLC	N/1/1/3	6m	ECBI, PSI-SF
				The program consisted of 2 digital videos (DVD/online) viewed over 2 weeks, followed by 2 weeks when parents were asked to act. Each video included lecture components by the author, explaining the program's parenting strategies and their application at home and public settings. Features included numerous role-played video vignettes, demonstrating maladaptive parent–child interactions, and the more adaptive parenting techniques taught through the program. Total viewing time for both videos was 3 h 46 minutes. The first video addressed stop behaviors, such as straight thinking, and controlling obnoxious behavior. The second video addressed start behaviors, such as tactics for encouraging good behavior, and strategies for building self-esteem. Parents also received tip sheets, which summarized the main points of the program.
Sanders et al. (2012)	2–9 (4.7)	Elevated disruptive behavior based on ECBI (ECBI problems mean > 15)	116 (60, 56)	The intervention was based on the Triple P (Sanders 1999). The online program is an eight-module, interactive self-directed software that took an average of 8 weeks to complete. The intervention provided instruction in the use of core positive parenting skills presented in sequenced modules (e.g., managing misbehavior). The online program features included video-based modeling of parenting skills, diverse parent testimonials describing their experiences, personalized content, including goal setting, interactive exercises to prompt parental problem solving, downloadable worksheets and podcasts to review session content, and automated text messaging and e-mail prompts to increase the likelihood of program completion.	NT	I/1/1/2	6m	ECBI, PS, PTC

Dashes indicate that no data were reported and/or obtained.

Age is presented in years. Mean is presented in brackets.

The first value refers to the total number of parents in the study; the first number in brackets refers to the total number of parents in the treatment group and the second value in brackets refers to the number of parents in the control group.

The first grouping is a type of technology (I = interactive, N = noninteractive); the second grouping is a clinical symptom level (1 = clinical; 2 = not clinical); the third grouping is child age (1 = bellow mean age of 7, 2 = above mean age of 11.5); the fourth grouping is SES (1 = low, 2 = middle, 3 = high).

Only instruments that were used to calculate ESs are included.

CAPES, Child Adjustment and Parent Efficacy Scale (parenting confidence scale); CBCL, Child Behavior Checklist externalizing scale; ECBI, Eyberg Child Behavior Inventory; FAQ, frequently asked questions; Fo-up, follow-up; HSQ-M, Home Situations Questionnaire Modified version; MCQ, multiple choice questions; NT, no treatment; PBQ, Parent Behavior Questionnaire; PCIT, Parent–Child Interaction Therapy; PLOC, the Parental Locus of Control scale; PPI, The Parenting Practices Interview; PS, Parenting Scale; PSI-SF, Parenting Stress Index-Short Form; PSOC, Parenting Sense of Competence; PTC, Parenting Task Checklist; Triple p, positive parenting program; Q&A, questions and answers; SES, socioeconomic status; WLC, wait list control.

All interventions were based on theoretically driven evidence-based content, targeting parents' beliefs and behaviors around child rearing activities and discipline. Two of the three studies aimed for older nonclinical samples used the same intervention in different samples treated in different countries (Kacir and Gordon 2000; Cefai et al. 2010). These three studies were also based on the same theoretical basis of Parenting Adolescents Wisely (Gordon et al., unpublished work, 1996) and intervention design of scenario-based learning. Five interventions included interactive programs and two included noninteractive programs. Finally, one intervention included research assistant activity to remotely engage users with the intervention website (Enebrink et al. 2012). Six studies provided follow-up information: three in a clinical child sample at 6 months postintervention (Sanders et al. 2012; Morawska et al. 2014; Porzig-Drummond et al. 2015) and three in a nonclinical sample, two at 3 months postintervention (Kacir and Gordon 2000; Cefai et al. 2010), and one at 6 months postintervention (Irvine et al. 2015b).

Methodological quality

The quality assessments are summarized in Figure 2. In two studies (Cefai et al. 2010; Irvine et al. 2015a) it was unclear whether randomization was conducted with appropriate methods and whether the allocation was concealed from relevant staff. These studies, however, provided analysis of groups at baseline showing no significant differences in the variables measured, suggesting that the randomization procedure was adequate. In one study (Irvine et al. 2015a), a gap was found between the simplicity of the recruitment process, which was conducted remotely and included US $40 compensation for filling out the assessments, and the participation itself within the intervention group, which required participants to use the program within community centers. This gap resulted in low rates of participants beginning treatment within the intervention group. In one study (Kacir and Gordon 2000), the data at baseline pointed to an imbalance on several outcome measurements favoring the control group, which, in turn, yielded more conservative outcome ESs.

FIG. 2.

Risk of bias assessments using Cochrane Collaboration's tool.

Treatment effects: intervention versus control groups

Overall, compared to the control condition, digital-based parent training resulted in significantly greater improvement in child behavior (ES = 0.44, 95% confidence interval [CI]: 0.21–0.66, studies = 7, n = 718), parent behavior (ES = 0.41, 95% CI: 0.25–0.57, studies = 6, n = 632), and parental confidence (ES = 0.36, 95% CI: 0.12–0.59, studies = 4, n = 532) (Fig. 3). The funnel plots were examined and did not indicate a significant publication bias.

FIG. 3.

Forrest plots of standardized mean difference ESs (Cohen's d, positive favoring the intervention), 95% confidence interval lower and upper limits, and significance measurement. The random model ES is computed against the control studies and also an overall ES is computed. In relevant variables, ESs are also presented for clinical and nonclinical subgroups. ES, effect size.

The Q-statistic for heterogeneity showed a trend toward significance for child behavior [Q(6) = 11.99, p = 0.06, I ² = 49.98], being not significant for parent behavior [Q(5) = 2.73, p = 0.74] and for parental confidence [Q(3) = 5.21, p = 0.16].

We subgrouped the studies by clinical (n = 4) and nonclinical (n = 3) severity of the disruptive behavior problems and conducted an additional subgroup analysis for media type (interactive vs. noninteractive) with regard to all outcome measures (Table 2). Since all studies aimed at middle school children included only interactive features and consistently provided homogenous results (Q_w < 0.35), we compared the media type (interactive vs. noninteractive) only in studies that focused on young samples within the clinical range.

Table 2.

Meta-Analyses of Studies Examining the Effects of Technology-Based Parent Training for Children with Disruptive Behaviors

	k	Cohen's d effect size	95% CI	Z	P	Q_w	I²	Q_b
Child behavior
Overall	7	0.44	0.21–0.66	3.87	0.000	11.99^*	49.98	7.06^**
Clinical	4	0.61	0.40–0.82	5.67	0.000	4.75	36.9
Nonclinical	3	0.21	−0.01 to 0.42	1.89	0.058	0.18	0.00
Clinical subgroup analyses^a
Type								4.48^*
Interactive	2	0.82	0.54–1.11	5.60	0.000	0.04	0.00
Noninteractive	2	0.36	0.05–0.68	2.29	0.02	0.24	0.00
Parent behavior
Overall	6	0.41	0.25–0.57	5.05	0.000	2.73	0.00	1.37
Clinical	4	0.49	0.28–0.70	4.61	0.000	0.99	0.00
Nonclinical	2	0.30	0.05–0.55	2.38	0.02		0.00
Clinical subgroup analyses^a
Type								0.68
Interactive	2	0.57	0.29–0.85	3.97	0.000	0.00	0.00
Noninteractive	2	0.39	0.08–0.71	2.47	0.01	0.31	0.00
Parental confidence
Overall	4	0.36	0.12–0.59	2.96	0.003	5.21	42.44	1.04
Clinical	2	0.51	0.24–0.78	3.66	0.000	2.88^*	65.29
Nonclinical	2	0.23	0.01–0.46	2.04	0.04	0.00	0.00
Clinical subgroup analyses
Type								2.88^*
Interactive	1	0.73	0.36–1.11	3.82	0.000	—	—
Noninteractive	1	0.26	−0.14 to 0.66	1.29	0.20	—	—

The Q _w statistic estimates within group homogeneity. The Q _b statistic estimates whether the difference between the effect sizes in the subgroups is significant.

All subgroup analyses were conducted with random-effects analyses.

d, effect size; k, number of studies; CI, confidence interval.

p < 0.10; ^** p < 0.05; ^*** p < 0.01; ^**** p < 0.001.

Digital PT had a moderate ES advantage regarding child behavior in studies of young children with clinically significant disruptive behavior (ES = 0.61, 95% CI: 0.40–0.82) and a small ES with a trend toward significance for a middle school nonclinical sample (d = 0.21, 95% CI: −0.01 to 0.42). Analyses yielded similar results of higher ESs favoring studies of young children with clinically significant disruptive behavior for parent behavior and parental confidence, but the differences were not significant.

In a subgroup analysis of early-onset children with clinically significant disruptive behavior, a significant difference was found regarding child behavior between interactive and noninteractive programs (Q_b = 4.48, p < 0.01). Interactive programs had a large ES advantage (ES = 0.82, 95% CI: 0.54–1.11), whereas noninteractive programs had a small ES advantage (d = 0.36, 95% CI: 0.05–0.68) compared to control groups. While a similar trend was found for parental confidence (Q_b = 2.88, p < 0.1), this calculation was based on only one study per group (i.e., interactive vs. noninteractive).

Post-treatment follow-up effects: within intervention group

Follow-up effect analysis revealed no significant change during the nonintervention phase (postintervention follow-up in all studies with information) on child behavior (ES = 0.11, 95% CI: −0.03 to 0.25, studies = 6, n = 180), parental behavior (ES = 0.00, 95% CI: −0.23 to 0.22, studies = 4, n = 128), or parental confidence (ES = 0.14, 95% CI: −0.03 to 0.30, studies = 3, n = 111). That is, the effects found at the end of treatment were sustained during the follow-up period, that is, despite cessation of the intervention.

Discussion

The aim of this systematic review and meta-analysis was to summarize the reported effectiveness of DPT for children's disruptive behaviors to inform researchers and clinicians regarding this intervention's potential and to identify the main challenges in this field for future consideration. Out of the seven randomized studies meeting inclusion criteria, five were published after 2011 and three were published between 2014 and 2015, indicating recent progress made in this area. All interventions were guided by evidence-based theoretical frameworks, targeting parents' perceptions around discipline and child rearing behaviors and providing guidance regarding adequate and inadequate parenting behaviors.

Overall, we found that DPT resulted in positive improvements for the three outcome measurements with overall small to medium ESs of 0.44 for child behavior, 0.41 for parent behavior, and 0.35 for parental confidence in self-efficacy. We identified two groups of studies, however, one targeting young children within clinical range of disruptive behaviors and the other targeting older children mostly below the clinical range of disruptive behaviors, which differed significantly in their efficacy. The results point to a medium ES of 0.61 for child behavior in studies of young children within the clinical range of symptoms and a small ES of 0.21with a trend toward significance for child behavior in studies of older children not within the clinical range of targeted symptoms. Although not significant, the same trend of difference between these two study samples was apparent for parent behavior and confidence.

There are several explanations for this finding that should be addressed. First, there are three important baseline differences between these two study groups: targeted children's age, the reported range of clinical symptoms, and the SES of the samples. As has been shown in a previous comprehensive meta-analysis of parent training for disruptive behaviors (Lundahl et al. 2006), higher children's age, lower clinical symptom severity, and lower SES are all associated with significantly less improvement on child's behavior reports. Since digital PTs for older children targeted a nonclinical sample with lower SES on average, it was not feasible to distinguish between these overlapping characteristics. This implies that the smaller ESs found for this group might be due to the combination of these three baseline factors. Second, all interventions targeting older children were practically based on the same program design and theoretical framework, that is, Parenting Adolescents Wisely (Gordon et al., unpublished work, 1996). Therefore, one cannot rule out the possibility that another program's features would yield different outcomes. To further examine the potential of DPT for older children in improving children's behavior and other program outcomes, different samples in terms of SES and clinical range of symptoms, as well as different intervention programs need to be examined. These studies will enable researchers and service providers to better estimate the potential of digital PT to support families with older children and/or identify subgroups responding better to one or another approach.

Nonetheless, the ESs found overall for DPT for children resemble those reported for self-directed parent training in previous meta-analysis (Lundahl et al. 2006). This finding strengthens the use of digital programs as a delivery method of self-directed parental intervention that can preserve resources, increase access, and is widely scalable. The latter feature is highly relevant, as research suggests that antipsychotics are prescribed for youth with disruptive behaviors instead of or without attempting behavioral interventions, which have been found to be all too often inaccessible (Olfson et al. 2012, 2014). Moreover, the use of digital programs, mostly online, opens new possibilities not existing in previous self-directed designs (e.g., books), such as the applicability to examine parents' use of the program and self-reports embedded within the design to better manage and evaluate the usability and fidelity of the treatment.

Subsequently, our meta-analysis revealed that interactive programs resulted in a larger ES (0.82) for child's behavior compared to noninteractive programs (0.36) in the sample of younger children within the clinical range of symptoms. Since interactive programs as a delivery method offer much more tailored design in comparison to regular self-directed programs, this finding does not seem unreasonable. This finding points to the need to further investigate the effect of these two designs on the efficacy of parent training for disruptive behaviors, which in turn could help a great deal in the understanding of the way different delivery methods and designs affect programs' success.

Relating to studies' design, two considerations for future research should be noted. First, our systematic review revealed a lack of randomized trials comparing digital led and therapist led behavioral programs (individual or group behavioral interventions, but without a technological component), which is necessary to understand the therapeutic impact of digital programs. Another relevant yet missing comparator would be the usual care of youth with disruptive behavioral problems that all too often do not include formal evidence-based psychological interventions (Knapp et al. 2012; Rosato et al. 2012). Comparison with usual care would enable us to gauge the real-world clinical utility of digital-based parent training. Such studies should focus not only on efficacy but also on sustainability and cost effectiveness of the face-to-face and digitally based approaches. Second, while the reviewed programs' efficacy was evaluated using standardized methods, none of these studies provided data suggesting that the program resulted in earlier engagement with services, or engagement with care of a population that would not otherwise receive these services due to causes such as stigma, lack of access, or costs. Hence, following the establishment of these programs' efficacy, their promise in providing different pathways to care that enable better access and use of services should clearly be examined.

Several limitations of this meta-analysis require consideration. First, the number of studies included in our analysis was small, although it is above the median for reviews listed in the Cochrane Database of Systematic Reviews, which is six (Borenstein et al. 2009b). Moreover, the observed effects were consistent among the studies of the two identified subgroups (clinical, nonclinical), and therefore, the limitation of realizing the true impact of dispersion (Borenstein et al. 2009a) was less crucial. Nonetheless, more studies are needed to examine whether the calculated ES is stable across different samples and digital interventions. Second, the results are limited to parents' experience of their children, which limits our understanding of DPT effects on children's behavior in other settings. This limitation, however, is the result of the main outcome measures used within most parent training studies, since the primary focus of parent training is to manage child behaviors while the child is under parental guidance and supervision (Lundahl et al. 2006). Third, none of the studies targeting younger children focused on a sample with a low SES, a population which is of great importance in this field (Gortmaker et al. 1990; Loeber et al. 1995). Therefore, we cannot rule out the possibility that for families with low SES the studied interventions are not as effective. Future examinations of digital interactive parent training programs targeting early onset of disruptive behaviors in families with low SES are needed.

Conclusions

Despite these limitations, this study presents evidence that digital-based parent training for disruptive behaviors is an effective and promising intervention tool, with current data mainly supporting its use in young children experiencing a clinically significant range of symptoms.

Clinical Significance

As technology provides important opportunities to extend the reach of services, our hope is that this review will not only provide empirical information about the current state of the field but also information regarding further directions for research, enabling more families and youth to benefit from effective and evidence-based psychological interventions for disruptive behaviors of youth.

Footnotes

Disclosures

Dr. A.B. and Dr. A.P. have no financial interests to disclose. Dr. J.M.K. has been a consultant for Alkermes, Eli Lilly, EnVivo Pharmaceuticals (Forum), Forest, Genentech, H. Lundbeck. Intracellular Therapeutics, Janssen Pharmaceutica, Johnson and Johnson, Otsuka, Reviva, Roche, Sunovion, and Teva. He has received honoraria for lectures from Janssen, Genentech, Lundbeck, and Otsuka. Dr. J.M.K. is a Shareholder in MedAvante, Inc., and Vanguard Research Group. Dr. C.U.C. has been a consultant and/or advisor to or has received honoraria from AbbVie, Acadia, Actavis, Alkermes, Eli Lilly, Genentech, Gerson Lehrman Group, IntraCellular Therapies, Janssen/J&J, Lundbeck, MedAvante, Medscape, Otsuka, Pfizer, ProPhase, Reviva, Roche, Sunovion, Supernus, and Takeda. He has received grant support from Bristol-Myers Squibb, Otsuka, and Takeda.

References

Arnold

, O'Leary

, Wolff

, Acker

: The Parenting Scale: A measure of dysfunctional parenting in discipline situations. Psychol Assess, 5:137–144, 1993.

Baumel

, Muench

: Heuristic evaluation of Ehealth interventions: Establishing standards that relate to the therapeutic process perspective. JMIR Ment Health, 3:e5, 2016.

Borenstein

, Hedges

, Rothstein

: Meta-analysis: Fixed effect vs. random effects. http://www.metaanalysis.com/downloads/Meta-analysis_fixed_effect_vs_random_effects_sv.pdf.2007.

Borenstein

, Hedges

, Higgins

JPT

, Rothstein

(eds): When does it make sense to perform a meta‐analysis?. In: Introduction to Meta-Analysis. Hoboken (New Jersey), Wiley, 2009a, pp 357–364.

Borenstein

, Higgins

, Rothstein

:. Introduction to Meta-Analysis. Hoboken (New Jersey), Wiley, 2009b.

Breitenstein

, Gross

, Christophersen

: Digital delivery methods of parenting training interventions: A systematic review. Worldviews Evid Based Nurs, 11:168–176, 2014.

Cefai

, Smith

, Pushak

: Parenting wisely: Parent training via CD-ROM with an Australian sample. Child Fam Behav Ther, 32:17–33, 2010.

Chorpita

, Daleiden

, Ebesutani

, Young

, Becker

, Nakamura

, Phillips

, Ward

, Lynch

, Trent

: Evidence‐based treatments for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clin Psychol Sci Pract, 18:154–172, 2011.

Comer

, Chow

, Chan

, Cooper-Vince

, Wilson

: Psychosocial treatment efficacy for disruptive behavior problems in very young children: A meta-analytic examination. J Am Acad Child Adolesc Psychiatry, 52:26–36, 2013.

10.

Correll

, Manu

, Olshanskiy

, Napolitano

, Kane

, Malhotra

: Cardiometabolic risk of second-generation antipsychotic medications during first-time use in children and adolescents. JAMA, 302:1765–1773, 2009.

11.

DerSimonian

, Laird

: Meta-analysis in clinical trials. Control Clin Trials, 7:177–188, 1986.

12.

Egger

, Angold

: Common emotional and behavioral disorders in preschool children: Presentation, nosology, and epidemiology. J Child Psychol Psychiatry, 47:313–337, 2006.

13.

Enebrink

, Högström

, Forster

, Ghaderi

: Internet-based parent management training: A randomized controlled study. Behav Res Ther, 50:240–249, 2012.

14.

Eyberg

, Nelson

, Boggs

: Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. J Clin Child Adolesc Psychol, 37:215–237, 2008.

15.

Eyberg

, Robinson

: Conduct problem behavior: Standardization of a behavioral rating. J Clin Child Adolesc Psychol, 12:347–354, 1983.

16.

Galling

, Roldán

, Nielsen

, Gerhard

, Carbon

, Stubbs

, Vancampfort

, De Hert

, Olfson

: Type 2 diabetes mellitus in youth exposed to antipsychotics: A systematic review and meta-analysis. JAMA Psychiatry, 73:247–259, 2016.

17.

Gordon

, Stanar

: Lessons learned from the dissemination of Parenting Wisely, a parent training CD-ROM. Cogn Behav Pract, 10:312–323, 2003.

18.

Gortmaker

, Walker

, Weitzman

, Sobol

: Chronic conditions, socioeconomic risks, and behavioral problems in children and adolescents. Pediatrics, 85:267–276, 1990.

19.

Hedges

, Olkin

: Statistical Method for Meta-Analysis. London, Academic Press, 2014.

20.

Higgins

, Altman

, Gøtzsche

, Jüni

, Moher

, Oxman

, Savović

, Schulz

, Weeks

, Sterne

: The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ, 343:d5928, 2011.

21.

Higgins

, Thompson

: Quantifying heterogeneity in a meta-analysis. Stat Med, 21:1539–1558, 2002.

22.

Irvine

, Gelatt

, Hammond

, Seeley

: A randomized study of internet parent training accessed from community technology centers. Prev Sci, 16:597–608, 2015a.

23.

Irvine

, Russell

, Manocchia

, Mino

, Glassen

, Morgan

, Gau

, Birney

, Ary

: Mobile-Web app to self-manage low back pain: Randomized controlled trial. J Med Internet Res, 17:e1, 2015b.

24.

Johnston

, Mash

: A measure of parenting satisfaction and efficacy. J Clin Child Psychol, 18:167–175, 1989.

25.

Jones

, Forehand

, Cuellar

, Kincaid

, Parent

, Fenton

, Goodrum

: Harnessing innovative technologies to advance children's mental health: Behavioral parent training as an example. Clin Psychol Rev, 33:241–252, 2013.

26.

Kacir

, Gordon

: Parenting adolescents wisely: The effectiveness of an interactive videodisk parent training program in Appalachia. Child Fam Behav Ther, 21:1–22, 2000.

27.

Kazdin

, Blase

: Rebooting psychotherapy research and practice to reduce the burden of mental illness. Perspect Psychol Sci, 6:21–37, 2011.

28.

Kazdin

, Rabbitt

: Novel models for delivering mental health services and reducing the burdens of mental illness. Clin Psychol Sci, 2013. DOI: 10.1177/2167702612463566.

29.

Kling

, Forster

, Sundell

, Melin

: A randomized controlled effectiveness trial of parent management training with varying degrees of therapist support. Behav Ther, 41:530–542, 2010.

30.

Knapp

, Chait

, Pappadopulos

, Crystal

, Jensen

: Treatment of maladaptive aggression in youth: CERT guidelines I. Engagement, assessment, and management. Pediatrics, 129:e1562–e1576, 2012.

31.

Liberati

, Altman

, Tetzlaff

, Mulrow

, Gøtzsche

, Ioannidis

, Clarke

, Devereaux

, Kleijnen

, Moher

: The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Ann Intern Med, 151:W-65–W-94, 2009.

32.

Loeber

, Green

, Keenan

, Lahey

: Which boys will fare worse? Early predictors of the onset of conduct disorder in a six-year longitudinal study. J Am Acad Child Adolesc Psychiatry, 34:499–509, 1995.

33.

Lundahl

, Risser

, Lovejoy

: A meta-analysis of parent training: Moderators and follow-up effects. Clin Psychol Rev, 26:86–104, 2006.

34.

Maayan

, Correll

: Weight gain and metabolic risks associated with antipsychotic medications in children and adolescents. J Child Adolesc Psychopharmacol, 21:517–535, 2011.

35.

Merikangas

, Nakamura

, Kessler

: Epidemiology of mental disorders in children and adolescents. Dialogues Clin Neurosci, 11:7–20, 2009.

36.

Morawska

, Tometzki

, Sanders

: An evaluation of the efficacy of a triple P-positive parenting program podcast series. J Dev Behav Pediatr, 35:128–137, 2014.

37.

Olfson

, Blanco

, Liu

S-M

, Wang

, Correll

: National trends in the office-based treatment of children, adolescents, and adults with antipsychotics. Arch Gen Psychiatry, 69:1247–1256, 2012.

38.

Olfson

, Blanco

, Wang

, Laje

, Correll

: National trends in the mental health care of children, adolescents, and adults by office-based physicians. JAMA Psychiatry, 71:81–90, 2014.

39.

Phelan

: 1-2-3 Magic: Effective Discipline for Children 2–12. ParentMagic, Inc., Glen Ellyn (Illinois); (2010).

40.

Porzig-Drummond

, Stevenson

: Preliminary evaluation of a self-directed video-based 1-2-3 Magic parenting program: A randomized controlled trial. Behav Res Ther, 66:32–42, 2015.

41.

APA Publications and Communications Board Working Group on Journal Article Reporting Standards: Reporting standards for research in psychology: Why do we need them?. What might they be? Am Psychol, 63:839–851, 2008.

42.

Robinson

, Dickersin

: Development of a highly sensitive search strategy for the retrieval of reports of controlled trials using PubMed. Int J Epidemiol, 31:150–153, 2002.

43.

Rosato

, Correll

, Pappadopulos

, Chait

, Crystal

, Jensen

: Treatment of maladaptive aggression in youth: CERT guidelines II. Treatments and ongoing management. Pediatrics, 129:e1577–e1586, 2012.

44.

Sanders

. Triple P-Positive Parenting Program: Towards an empirically validated multilevel parenting and family support strategy for the prevention of behavior and emotional problems in children. Clin Child Fam Psychol Rev, 2:71–90, 1999.

45.

Sanders

, Baker

, Turner

: A randomized controlled trial evaluating the efficacy of Triple P Online with parents of children with early-onset conduct problems. Behav Res Ther, 50:675–684, 2012.

46.

Sanders

, Woolley

: The relationship between maternal self‐efficacy and parenting practices: Implications for parent training. Child Care Health Dev, 31:65–73, 2005.

47.

Segal

, Chen

, Gordon

, Kacir

, Gylys

: Development and evaluation of a parenting intervention program: Integration of scientific and practical approaches. Int J Hum Comput Interact, 15:453–467, 2003.

48.

Sourander

: Challenges in children's enrolment to psychosocial services. Eur Child Adolesc Psychiatry, 24:361–363, 2015.

49.

Webster-Stratton

, Kolpacoff

, Hollinsworth

: Self-administered videotape therapy for families with conduct-problem children: Comparison with two cost-effective treatments and a control group. J Consult Clin Psychol, 56:558, 1988.

50.

Wilson

, Lipsey

: The role of method in treatment effectiveness research: Evidence from meta-analysis. Psychol Methods, 6:413–429, 2001.

51.

Zisser

, Eyberg

: Parent-child interaction therapy and the treatment of disruptive behavior disorders. In: Evidence-Based Psychotherapies for Children and Adolescents. Weisz

, and Kazdin

, eds. New York, Guilford Press, 2010, pp. 179–193.