Structured Feedback Training for Time-Out: Efficacy and Efficiency in Comparison to a Didactic Method

Abstract

Although time-out has been demonstrated to be effective across multiple settings, little research exists on effective methods for training others to implement time-out. The present set of studies is an exploratory analysis of a structured feedback method for training time-out using repeated role-plays. The three studies examined (a) a between-subjects comparison to more a traditional didactic/video modeling method of time-out training, (b) a within-subjects comparison to traditional didactic/video modeling training for another skill, and (c) the impact of structured feedback training on in-home time-out implementation. Though findings are only preliminary and more research is needed, the structured feedback method appears across studies to be an efficient, effective method that demonstrates good maintenance of skill up to 3 months post training. Findings suggest, though do not confirm, a benefit of the structured feedback method over a more traditional didactic/video training model. Implications and further research on the method are discussed.

Keywords

time-out behavioral parent training child behavior problems training methodology

Time-out has been widely used as a strategy to decrease problem behaviors including noncompliance (e.g., Rortvedt & Miltenberger, 1994), disruption and aggression (e.g., Bostow & Bailey, 1969; Donaldson & Vollmer, 2011; Firestone, 1976), sibling conflict (Olson & Roberts, 1987), and tantrums (Marcus, Swanson, & Vollmer, 2001). It has been demonstrated to work effectively with children as young as 1-year-old (Mathews, Friman, Barone, Ross, & Christophersen, 1987), with toddlers (Larzelere, Schneider, Larson, & Pike, 1996), with older children diagnosed with attention-deficit/hyperactivity disorder (ADHD; Fabiano et al., 2004), and in psychiatric settings (Crespi, 1988).

In a review of the available literature on time-out, Warzak, Floress, Kellen, Kazmerski, and Chopko (2012) noted the importance of research on how to best train others to use time-out but also noted the paucity of such research. Most manualized treatment programs for behavior problems teach the use of time-out, each using varying methods (Barkley, 1997; McMahon & Forehand, 2003; McNeil & Hembree-Kigin, 2010; Patterson, Reid, Jones, & Conger, 1975; Sanders, Markie-Dadds, & Turner, 2001; Webster-Stratton & Reid, 2010). These varying methods include written instructions, didactic training, video modeling, discussion, and rehearsal and feedback (including bug-in-the-ear practice), with each program varying in methods.

To date, however, no published research has examined the effectiveness of any time-out training procedure in terms of effective implementation of parents in the home, much less a comparison of specific training methods. In fact, observation of parents using time-out in the home is rare in published research. Everett, Hupp, and Olmi (2010) reviewed the available research that included parent’s implementation of time-out (40 published studies). Though not specifically included in their review, we reviewed the same set of studies and found that only 24% included any in-home observation, with other research relying on in-clinic observations and parent report from home. Only one study (Rortvedt & Miltenberger, 1994) included any information on integrity of parent implementation of time-out. Although they found that parents implemented time-out effectively, they do not provide any information on the actual training procedure or proximity of training to observation.

Dropout and poor engagement in Behavioral Parent Training programs, one of the most common sources of time-out training, is an ongoing problem (Chacko et al., 2016). Given that time-out is one of the most effective treatments taught in these programs, poor ability to implement the time-out technique effectively could in fact be a driving force in high dropout rates. In their review of reasons for parental nonadherence, Allen and Warzak (2000) suggested several reasons why specific aspects of training may result in poor parental adherence, further emphasizing the importance of examining the efficacy of specific training methods.

One method for training of time-out with strong potential for success is to use behavioral rehearsal to achieve parental skill accuracy in implementing time-out. Although the concept of using behavioral rehearsal in the training of specific skills is not new, Beidas, Cross, and Dorsey (2014) outlined several advantages to the use of an analogue fidelity tool in which a role player assists in the training of the skill. They noted the benefits of such role-play training as both a training and assessment instrument. Marcus et al. (2001) described a specific method for using role-play training with parents. This method first provided a brief overview of the intervention plan, followed by a role-play with the parents to demonstrate the skill. Parents would then practice the skill while first receiving real-time feedback (immediate) and later feedback at the end of the session (delayed). To move to the next stage, 100% accuracy was required for a full session. They found this method to be effective in working with the parents of four children. Other studies have found similar rehearsal and feedback methods to be effective (Berard & Smith, 2008; Cobb, Leitenberg, & Burchard, 1982; Van Camp et al., 2008), but no comparisons of the training procedure to other methods have been explored.

In the absence of empirical findings on the efficacy of various time-out training procedures, we introduce a specific methodology for training parents and others to implement time-out effectively and examine the impact of training over time and in comparison to alternative methods. Following initial piloting to determine efficiency and efficacy, we conducted three evaluations of the training method. First, comparisons were made to a traditional didactic training (between subjects), followed by comparisons to training of an alternative skill using traditional didactic training (within subjects). Finally, a naturalistic pre–post home observation was conducted to assess generalization.

Method

A total of 20 self-referred parents were trained to use time-out across three studies (different participants for each study) to examine the efficacy of a structured feedback (SF) method of training. All responded to fliers sent through schools, referrals from family physicians, and word of mouth regarding a group parent training program. Children ranged in age from 2 to 12 years. Seventy-one percent of participants were female, 57% were Caucasian, and 25% Hispanic. Seven percent had not completed high school, 25% had completed high school, 46% had completed some college, and 21% had completed at least a bachelor’s degree. Just over half were married (54%) with the others being divorced, separated, or single. They were a fairly similar representation of the community in which the trainings occurred.

All training was provided by master’s level graduate students with at least 1-year experience in teaching behavioral strategies to parents, which included weekly supervision by a licensed psychologist using videotaped sessions and providing feedback on accuracy of implementation. All students had also taken a graduate course in behavioral interventions. All procedures for the experiments, as well as recruitment strategies, were approved by the university Institutional Review Board (IRB).

Structured Feedback Time-Out Training Procedure

The SF method of time-out training is based on the family training protocol presented by Marcus et al. (2001), in which immediate and delayed feedback during role-plays was used to teach specific skills.

Brief didactic instruction and modeling

During a 20-min instructional session, parents received (a) lecture-style instruction describing an 11-step time-out procedure, (b) video models of time-out showing correct and incorrect implementation from the Incredible Years Basic Parent Program (IY; Webster-Stratton & Reid, 2010), and (c) a question and answer discussion. Next, the therapist modeled correct time-out implementation in a role-play with the parent acting the role of their child. This served as a model to the parent for correct implementation of the time-out procedure, as well as a model to the therapist of the common child behaviors during time-out

Immediate feedback

The parent was then asked to practice the time-out procedure in a series of generic role-plays that were adapted to the specifics of the parent’s situation. A trained undergraduate student played the role of the child while the therapist provided immediate feedback to the parent throughout the session. The “child” role player acted out a specific, programmed set of behaviors that allowed evaluation of all 11 steps of the time-out protocol. Feedback consisted of praise for steps the parent demonstrated well and corrective feedback for steps that was missed or implemented incorrectly. These role-plays were repeated until the parent was able to implement the entire procedure without error. Participants were required to complete two role-play(s) at 100% before proceeding to delayed feedback.

Delayed feedback

The parent was then asked to implement the time-out procedure without the therapist in close proximity (behind a one-way mirror or at a distance in an adjacent room). Following completion of each role-play session, the therapist returned to the immediate area to provide the same kind of feedback described above. Delayed feedback sessions continued until two consecutive role-plays occurred without any errors.

Measurement, Procedural Integrity, and Interobserver Agreement

Parent accuracy of correct steps during role-play assessments was the main outcome measure across studies. For the third study, naturalistic home observations of time-out use were also conducted. In addition, parent-reported preference of time-out and frequency of time-out use were assessed via parent self-report.

Parent accuracy

Data were collected throughout the study on parent accuracy, defined as the percentage of correct steps of the time-out procedure completed by the parent. Parent accuracy was scored during each condition in role-play assessments. All role-play assessments were videotaped, and observers collected data both in vivo during sessions and from the videotapes following assessments. The steps comprising the time-out protocol that were used to assess accuracy were listed in an 11-step coding worksheet (Figure 1). These steps were scored as either correct or incorrect, and the number of steps correct was divided by the total number of steps, yielding a percentage of parent accuracy. Observers/coders were trained using repeated coding of sample videos and were required to meet a standard of at least 90% interobserver agreement with a master coder before coding any videos for the study. Coders were blind to condition and session number for each video coded. A second observer scored 33% of sessions for Study 1, 35% of sessions for Study 2, and 50% of sessions for Study 3. Interobserver agreement was 93%, 95%, and 92% across the three studies, respectively (see Jensen & Steiner, 2017 for more information on the use of role-play assessments).

Figure 1.

Parent accuracy measure and coding worksheet for time-out procedure.

Naturalistic observations

In Study 3, participants were asked to collect approximately 3 hr of naturalistic video both prior to and after completion of training. Data were collected on the number of opportunities the parent had to use a time-out procedure (i.e., the frequency of the parent-nominated child problem behavior) and parent accuracy in implementing time-out.

Parent-reported preference

Across all studies, parents were asked to fill out a four-question rating form asking them to indicate (a) their frequency of time-out use over the previous week, (b) their confidence in using time-out, (c) satisfaction with time-out, and (d) their intention to use time-out. The first question was a simple frequency report, and the last three were rated on a 5-point scale where 1 was strongly disagree and 5 was strongly agree. This measure was given before the baseline role-play, following formal instruction, after the training session, and before each follow-up session.

Specific Study Methods

Study 1

Twelve participants were randomly assigned by group to two methods of training for time-out: SF versus traditional didactic group training (IY; six in each). This provided between-subjects comparison for the same skill taught by two different methods. The baseline-intervention design without a reversal does not account for the impact of time on treatment, and thus does not fully allow for experimental control, but is used as an exploratory analysis comparing across the two methods. The traditional didactic group training consisted of video modeling, discussion, and role-play time-out training based on the IY Program (Webster-Stratton & Reid, 2010) and lasted approximately for 3 hr. SF lasted 1.5 hr or less. For the IY group, between one and three baseline role-play assessments as well as a single assessment at approximately 2 weeks post training were attempted. For the SF group, three baseline role-play assessments were conducted as well as 1-week, 1-month, and 3-month follow-up assessments. Because assessments were conducted during attendance at a weekly parent training class, participants who did not attend specific classes could not participate in all the baseline and/or follow-up sessions, and in those cases, an “X” is placed in the graph for the missing assessment.

Study 2

A within-subjects multiple baseline across skills methodology was then used to compare the SF training for time-out to the IY method for effective commands. Four subjects participated in this study. Effective commands were chosen as the comparison skill because similar to time-out, it requires teaching parents to use new skills while avoiding some previous habits. Three baseline role-play assessments as well as 1 and 2/3 weeks post training assessments were completed for each skill. Time-out training was provided during Week 5 (after the final baseline assessment), and effective commands were taught in Week 8, thus allowing the multiple baseline design but also providing a slightly longer follow-up assessment for time-out (5 vs. 2 weeks post training). As with Study 1, the multiple baseline design still does not allow full experimental control but is used as an exploration of the comparative effectiveness of the two methods.

Study 3

For the final study, four parents collected pre and post naturalistic observations before and after receiving SF training for implementing time-out. All sessions were conducted individually in participants’ homes for the purpose of training and implementing time-out in a naturalistic setting. Role-play assessment data were collected during training, similar to Studies 1 and 2 presented above but only naturalistic data are presented. The pre and post training observations were conducted to corroborate the efficacy of training in the naturalistic setting.

Results

In Study 1, SF participants demonstrated maintenance of high accuracy 1 week (M = 87%; range = 73%-100%) and 1 month following training (M = 87%; range = 73%-100%). Accuracy decreased at the 3-month follow-up for all participants (M = 75%; range = 64%-91%). IY participants demonstrated an increase in time-out accuracy post training; however, accuracy did not increase enough to reach therapeutic levels of 80% for any of the six participants (M = 63%; range = 50%-64%). Average improvement in accuracy from the last baseline data point to the 1-week follow-up was an increase of 59% for the SF group and only 21% for IY. The percentage of people above 80% at the 1-week follow-up was 80% for SF and 0% for IY. Although the two groups differed somewhat in baseline scores, with the SF group having lower baseline, the post treatment assessments differed meaningfully in the opposite direction, suggesting greater benefit from the SF training (see Figure 2).

Figure 2.

Between-subjects comparison for time-out training: SF versus IY.

In Study 2, baseline data suggested that participant accuracy for time-out was lower than that of effective commands, but both skills were low for most participants prior to providing training (time-out M = 25%; range = 18%-30%; effective commands M = 61%; range = 41%-82%). For SF time-out training, maintenance of parental accuracy remained high 1 week following training (M = 100%), 3 weeks following training (M = 91%; range = 82%-100%), and 5 weeks following training (M = 88%; range = 64%-100%). Immediately following didactic group training on effective commands, only one of four participants was able to increase parental accuracy above 80% (average: 68%; range: 59-82%), and accuracy remained lower 1 week following training (average: 69%; range: 59%-76%). Average improvement in accuracy from the last baseline data point to the 1-week follow-up was an increase of 55.7% for SF and 13.7% for IY (see Figure 3). Again, these findings suggest a benefit from the SF over the IY method.

Figure 3.

Within-subjects multiple baseline comparison of SF time-out training versus IY effective commands.

Parent Report

Participant’s ratings of time-out were fairly similar across studies. Prior to training, participants reported using time-out an average of 1.68 times/week and an average of 2.82 times/week post training. Overall ratings were high across assessments but demonstrated increases from pre to post training for confidence (3.71 to 4.55), satisfaction (3.65 to 4.62), and probability of use (4.14 to 4.69).

Naturalistic Observation

During 12 hr of pretraining naturalistic observations, only one parent implemented time-out, doing so three times. Accuracy across the time-outs implemented was 47%. Three of the four parents implemented time-out during the posttraining naturalistic video observations. Parent accuracy of time-out in posttraining observations was high, with all three participants averaging above 80% (M = 85%). The high accuracy for time-out implementation during the posttraining naturalistic observations further supports the efficacy of the training. The only participant who implemented time-out prior to training doubled accuracy (47% to 91%) post training. Furthermore, accuracy during the naturalistic observations was similar to that of the role-play assessments at both pretraining and posttraining. Interestingly, both parents who implemented more than one time-out during a single session decreased accuracy with each subsequent implementation (67%, 44%, and 30% for P027; 91% and 70% for P026). Figure 4 shows pre and post training naturalistic observation data on child behavior, percentage of target behaviors responded to with time-out, and time-out accuracy.

Figure 4.

Pre and post SF time-out training naturalistic observation data.

In addition to parent accuracy data, information on opportunities to use time-out (occurrence of identified negative child behaviors) was tracked. During preobservations, the targeted child behaviors occurred a total of 91 times. As noted, only one parent implemented three time-outs, making an overall 3% response rate across parents. Identified negative child behaviors occurred a total of 83 times during post observations, with only five resulting in a time-out (6% of opportunities).

Despite parents’ more effective use of time-out following training, children’s target behaviors did not change significantly, with the children of two participants making small improvements, one getting slightly worse, and one staying the same. This could have been impacted by the very low implementation of time-out in comparison to the number of targeted behaviors (6% of opportunities).

Discussion

The purpose of the present set of studies was to explore efficacy for a specific methodology for training time-out, with comparisons to an alternative method (didactic/video modeling). The SF method of training time-out used a brief 20-min didactic/video modeling group session, followed by providing structured feedback during individual role-play practice. Across all three studies, this method was suggested to be both effective (all participants reached 100% accuracy and naturalistic observations demonstrated high accuracy) and efficient (all but one participant completed training in less than 1 hr, 1.5 hr for the other participant). In addition, participants demonstrated strong maintenance of skill with most implementing the time-out procedure above the 80% criterion 1 month post training and averaging 75% accuracy at 3 months post training.

In comparison to the more traditional group discussion, video modeling, with some role-play method of training, participants who received the SF method of time-out training demonstrated a larger increase in treatment accuracy 1 week following training than those who received the traditional training (59% vs. 21%). Similarly, when the SF time-out training was compared within the same subjects receiving traditional group training on a different skill (effective commands), average improvement from baseline to 1-week follow-up was 47% for SF time-out training and 14% for traditional group IY training on effective commands. Although caution must be used in interpreting too strongly the results from these exploratory analyses, the findings certainly suggest the possible benefits of the SF training deserve further exploration.

In addition, several other findings merit discussion and further exploration. The first and most notable is the low use of time-out during the naturalistic observations in comparisons to the opportunities to implement time-out. Across pretraining observations, parents implemented time-out 3% of opportunities when the target behavior occurred. Even following treatment, this low implementation percentage only slightly increased to 6% of opportunities. Because no previous research has ever examined implementation percentage, it is unclear whether this low percentage is common or whether it might be specific to the current training methodology. The low percentage at least highlights the fact that knowledge of how to accurately implement a skill does not generalize to recognizing opportunities to implement that skill. Both, of course, are necessary for successful behavior change. The lack of relationship between skill acquisition and ability to recognize when to implement the skills is of importance in the overall intervention literature and deserves further investigation. Such research should focus on average implementation percentage of time-out in response to identified behavior issues pre and post training and should help clarify whether varying training techniques may lead to differential implementation. It should also address whether specifically teaching recognition of opportunities, which was not included in the present training, may improve overall implementation rate.

Another interesting trend in the naturalistic observation data was that when parents implemented multiple time-outs within the same observation, accuracy decreased with each successive implementation. Only two parents implemented multiple time-outs within the same observation, so the data are limited, but notable that in both cases accuracy decreased with each successive time-out. Allen and Warzak (2000) suggested that such phenomena of decreased treatment accuracy could be attributed to the lack of immediate change in child behavior following the time-out. Further research to determine whether this is a common occurrence when multiple time-outs are needed and what can be done to counteract the decreasing trend on accuracy would be helpful.

Comparing naturalistic observations and role-play assessments pre and post training allowed some comparison of similarity of parent performance during the role-play assessment and in the natural environment with their own child. For the one participant who implemented time-out pretraining, accuracy in the natural setting averaged 47% prior to training, almost identical to her baseline role-play (45%). Post training, three parents implemented time-out in the naturalistic observations. Accuracy across these natural setting time-outs was high, averaging 85%. Although not being equal to the average of 100% accuracy on the role-play assessment post training, this still lends support to the conclusion that the role-play assessment is accurately assessing the parents’ skill level.

A potential concern is the use of a 100% accuracy requirement for SF training and not the alternative training. Although this concern could suggest an unfair comparison, we believe that it represents a real and needed comparison. The requirement of 100% accuracy in the structure feedback is an important distinction from alternative methods and thus should be a part of the comparison to those methods. In addition, in each study, immediate post training results were not used, but alternatively, 1 week or later follow-ups were used to compare with other methods.

Another limitation is ambiguity in the coding sheet for time-out training, in that some descriptors (e.g., “very little activity”) were not further operationally defined. Coders were trained; however, using repeated practice and overall interobserver agreement was high suggesting this did not have a strong impact. An additional concern is that preference data were collected from a questionnaire created for the present study, and reliability and validity of the measure could not be determined. The measure is similar, however, to commonly used methods in other studies.

Studies that continue to compare additional outcome measures, such as naturalistic observation across participants that are trained using the SF method of time-out training and other methods, will help to further clarify the benefits of such training. The SF method has demonstrated success in teaching other skills as well (see Jensen & Steiner, 2017). Vollmer and colleagues (Marcus et al., 2001; Van Camp et al., 2008) have applied their training model upon which the SF method is based to multiple skills, though a comparison with alternative methods has not been evaluated. The current method is fairly experimenter-intensive, requiring a trained role player, an individual providing feedback, and one recording data for a single parent in training. The possibility of expanding to a group format in which parents take turns filling the roles of child role player, data accuracy tracker, and parent with a single experimenter providing feedback could also be pursued.

Conclusion

Although further research is needed to strengthen these results, preliminary findings suggest that the SF method of time-out training is an efficient and effective method that results in rapid improvement in accuracy of skill implementation (1 hr vs. 3 hr of training) with strong maintenance of skill over time, especially in comparison to more traditional didactic/video modeling methods. Further research is needed to resolve issues of generalizability, use with other skills, and to determine whether parents can participate more in the process to make the training less therapist intensive.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Scott A. Jensen is an associate professor and chair of the Department of Psychology at the University of the Pacific in Stockton California. His research focuses on parenting and child behavior problems, and especially interventions to help improve child behavior.

Sean Blumberg is a behavior consultant with A.G.E.S. Learning Solutions, Inc., in San Jose, CA. His research interests include parent training, behavioral skills training, and functional behavior assessment.

Megan Browning is a board certified behavior analyst in the greater Sacramento, California area. She works primarily with children and adults diagnosed with developmental disabilities and mental health disorders teaching communication, social and independent living skills.

References

Allen

K. D.

Warzak

W. J.

(2000). The problem of parental nonadherence in clinical behavior analysis: Effective treatment is not enough. Journal of Applied Behavior Analysis, 33, 373-391. doi:10.1901/jaba.2000.33-373

Barkley

R. A.

(1997). Defiant children: A clinician’s manual for assessment and parent training (2nd ed.). New York, NY: The Guilford Press.

Beidas

R. S.

Cross

Dorsey

(2014). Show me, don’t tell me: Behavioral rehearsal as a training and analogue fidelity tool. Cognitive and Behavioral Practice, 21, 1-11. doi:10.1016/j.cbpra.2013.04.002

Berard

K. P.

Smith

R. G.

(2008). Evaluating a positive parenting curriculum package: An analysis of the acquisition of key skills. Research on Social Work Practice, 18, 442-452. doi:10.1177/1049731508318656

Bostow

Bailey

(1969). Modification of severe disruptive and aggressive behavior using brief timeout and reinforcement procedures. Journal of Applied Behavior Analysis, 2, 31-37. doi:10.1901/jaba.1969.2-31

Chacko

Jensen

S. A.

Lowry

L. S.

Cornwell

Chimklis

Chan

. . . Pulgarin

(2016). Engagement in behavioral parent training: Review of the literature and implications for practice. Clinical Child and Family Psychology Review, 193, 204-215. doi:10.1007/s10567-016-0205-2

Cobb

E. J.

Leitenberg

Burchard

J. D.

(1982). Foster parents teaching foster parents: Communication and conflict-resolution skills training. Journal of Community Psychology, 10, 240-249. doi:10.1002/1520-6629(198207)10:3

Crespi

T. D.

(1988). Effectiveness of time-out: A comparison of psychiatric, correctional and day-treatment programs. Adolescence, 23, 805-811.

Donaldson

J. M.

Vollmer

T. R.

(2011). An evaluation and comparison of time-out procedures with and without release contingencies. Journal of Applied Behavior Analysis, 44, 693-705. doi:10.1901/jaba.2011.44-693

10.

Everett

G. E.

Hupp

S. A.

Olmi

D. J.

(2010). Time-out with parents: A descriptive analysis of 30 years of research. Education and Treatment of Children, 33, 235-259. doi:10.1353/etc.0.0091

11.

Fabiano

G. A.

Pelham

W. E.

Jr. Manos

M. J.

Gnagy

E. M.

Chronis

A. M.

Onyango

A. N.

. . . Swain

(2004). An evaluation of three time-out procedures for children with attention-deficit/hyperactivity disorder. Behavior Therapy, 35, 449-469. doi:10.1016/S0005-7894(04)80027-3

12.

Firestone

(1976). The effects and side effects of timeout on an aggressive nursery school child. Journal of Behavior Therapy and Experimental Psychiatry, 7, 79-81. doi:10.1016/0005-7916(76)90050-1

13.

Jensen

S. A.

Steiner

(2017). Role-play assessments as a measure of parenting skill. Behavior Analysis: Research and Practice. Advance online publication. doi.org/10.1037/bar0000083

14.

Larzelere

R. E.

Schneider

W. N.

Larson

D. B.

Pike

P. L.

(1996). The effects of discipline responses in delaying toddler misbehavior recurrences. Child & Family Behavior Therapy, 18, 35-57. doi:10.1300/J019v18n03_03

15.

Marcus

B. A.

Swanson

Vollmer

T. R.

(2001). Effects of parent training on parent and child behavior using procedures based on functional analyses. Behavioral Interventions, 16, 87-104. doi:10.1002/bin.87

16.

Mathews

J. R.

Friman

P. C.

Barone

V. J.

Ross

L. V.

Christophersen

E. R.

(1987). Decreasing dangerous infant behaviors through parent instruction. Journal of Applied Behavior Analysis, 20, 165-169. doi:10.1901/jaba.1987.20-165

17.

McMahon

R. J.

Forehand

R. L.

(2003). Helping the noncompliant child: Family based treatment for oppositional behavior (2nd ed.). New York, NY: The Guilford Press.

18.

McNeil

C. B.

Hembree-Kigin

T. L.

(2010). Parent-child interaction therapy (2nd ed.). New York, NY: Springer.

19.

Olson

R. L.

Roberts

M. W.

(1987). Alternative treatments for sibling aggression. Behavior Therapy, 18, 243-250. doi:10.1016/S0005-7894(87)80018-7

20.

Patterson

G. R.

Reid

J. B.

Jones

R. R.

Conger

R. E.

(1975). A social learning approach to family intervention: Vol. 1. Families with aggressive children. Eugene, OR: Castalia.

21.

Rortvedt

A. K.

Miltenberger

R. G.

(1994). Analysis of a high-probability instructional sequence and time-out in the treatment of child noncompliance. Journal of Applied Behavior Analysis, 27, 327-330. doi:10.1901/jaba.1994.27-327

22.

Sanders

M. R.

Markie-Dadds

Turner

K. M. T.

(2001). Practitioner’s manual for standard triple P. Milton, Australia: Families International.

23.

Van Camp

C. M.

Montgomery

J. L.

Vollmer

T. R.

Kosarek

J. A.

Happe

Burgos

Manzolillo

. (2008). Behavioral parent training in child welfare: Maintenance and booster training. Research on Social Work Practice, 18, 392-400. doi:10.1177/1049731508318658

24.

Warzak

W. J.

Floress

M. T.

Kellen

Kazmerski

J. S.

Chopko

(2012). Trends in time-out research: Are we focusing our efforts where our efforts are needed? The Behavior Therapist, 35(2), 30-33.

25.

Webster-Stratton

Reid

M. J.

(2010). The Incredible Years Parents, Teachers and Children Training Series: A multifaceted treatment approach for young children with conduct problems. In Weisz

Kazdin

(Eds.), Evidence-based psychotherapies for children and adolescents (2nd ed., pp. 194-210). New York, NY: The Guilford Press.