Do People With Severe Traumatic Brain Injury Benefit From Making Errors? A Randomized Controlled Trial of Error-Based and Errorless Learning

Abstract

Background. Errorless learning (ELL) and error-based learning (EBL) are commonly used approaches to rehabilitation for people with traumatic brain injury (TBI). However, it is unknown whether making errors is beneficial in the learning process to promote skills generalization after severe TBI. Objective. To compare the efficacy of ELL and EBL for improving skills generalization, self-awareness, behavioral competency, and psychosocial functioning after severe TBI. Method. A total of 54 adults (79% male; mean age = 38.0 years, SD = 13.4) with severe TBI were randomly allocated to ELL or EBL and received 8 × 1.5-hour therapy sessions that involved meal preparation and other goal-directed activities. The primary outcome was total errors on the Cooking Task (near-transfer). Secondary outcome measures included the Zoo Map Test (far-transfer), Awareness Questionnaire, Patient Competency Rating Scale, Sydney Psychosocial Reintegration Scale, and Care and Needs Scale. Results. Controlling for baseline performance and years of education, participants in the EBL group made significantly fewer errors at postintervention (mean = 36.25; 95% CI = 32.5-40.0) than ELL participants (mean = 42.57; 95% CI = 38.8-46.3). EBL participants also demonstrated greater self-awareness and behavioral competency at postintervention than ELL participants (P < .05). There were no significant differences on other secondary outcomes (P > .05), or at the 6-month follow-up assessment. Conclusion. EBL was found to be more effective than ELL for enhancing skills generalization on a task related to training and improving self-awareness and behavioral competency.

Keywords

rehabilitation executive function neuropsychology brain injuries

Introduction

People with severe traumatic brain injury (TBI) often lack self-awareness of impairments and have difficulty recognizing their own errors.^1,2 Damage to the prefrontal cortex and connections has been found to disrupt the metacognitive ability to accurately self-reflect and regulate one’s own behavior.³ Impairments of self-awareness and self-regulation reduce safety and independence and increase burden on family members to supervise daily tasks.⁴ People with metacognitive impairments are also less likely to transfer skills learned during rehabilitation to relevant daily situations.⁵ Lack of skills generalization is a major issue because interventions are typically delivered in a specific context and are cost and time limited.⁶ Therefore, metacognitive approaches targeting self-awareness and self-regulation are recommended following TBI.^6,7 However, there are contrasting views on whether it is more beneficial for these individuals to make errors or to avoid making errors when learning skills in rehabilitation.

Most cognitive rehabilitation trials focus on skill development in a particular domain or functional context (eg, sending a phone message). A prevailing view in the literature is that people with severe cognitive impairment learn new skills more successfully when errors are avoided during training—namely, errorless learning (ELL)—as compared to trial-and-error learning (ie, guessing with responses corrected by therapists).⁸ The efficacy of ELL for teaching information and skills to people with severe memory impairments is well supported.^8,9 However, ELL requires a high level of support from the therapist to prevent errors and facilitate error-free performance. Learning is “hyperspecific” (ie, discrete stimuli and skills are trained), and individuals are not provided with the opportunity to make errors, become aware of these, or self-correct. As a bottom-up approach, training effects of ELL are task specific, and skills are not expected to spontaneously generalize to novel situations beyond training.^10,11

Skills generalization requires people to apply strategies developed during training to situations outside of training and to vary their responses when task requirements differ. Toglia⁵ distinguished between near-transfer, the ability to apply strategies to activities that are similar to training, and far-transfer, the ability to apply strategies to activities that differ from training. In contrast to ELL, error-based learning (EBL) is a top-down approach that provides structured opportunities for people to make errors and learn to recognize and self-correct these. EBL incorporates metacognitive techniques, such as role reversal, video feedback, graded prompts, and pre-post task reflection, which systematically teach people to anticipate, check for and correct errors, and generate strategies.⁵ Pilot research indicates that EBL improves self-regulation skills on both training¹² and near-transfer tasks after TBI.^13,14 A previous randomized controlled trial (RCT) identified that use of multiple EBL techniques (ie, video plus verbal feedback) was more effective for reducing errors on the training task and improving self-awareness than verbal feedback or experiential feedback alone¹⁵; however, this trial did not examine skills generalization.

To achieve maximal benefits, cognitive rehabilitation needs to support people to flexibly apply strategies in daily situations.⁶ This RCT investigated whether EBL promotes greater skills generalization and self-awareness than ELL in order to clarify the benefits of teaching internal self-regulation skills to people with severe TBI. It was hypothesized that participants receiving EBL would demonstrate significantly greater skills generalization on self-regulation tasks at postintervention than those receiving ELL. Both near-transfer (primary outcome) and far-transfer indices of skills generalization were assessed. It was further hypothesized that EBL participants would demonstrate significantly greater self-awareness and behavioral competency at postintervention than ELL participants. A secondary aim was to compare the effects of EBL and ELL on psychosocial functioning (eg, work, relationships, independence, and mood) at postintervention and 6-month follow-up.

Methods

Ethics

Ethical approval was granted by Griffith University (PSY/55/13/HREC), University of Queensland (#2013000598), Metro South Hospital and Health Service (HREC/13/QPAH/096), and Royal Rehabilitation Centre (14SSA/03). All participants provided written informed consent to participate.

Trial Design

This article was prepared in accordance with the CONSORT statement.¹⁶ The study was an assessor-blinded RCT with 2 intervention groups and 3 phases (baseline/preintervention, postintervention, and 6-month follow-up). The trial was registered with the Australian New Zealand Clinical Trials Registry (ACTRN12613000585729), and the protocol was published.¹⁷

Participants

Participants were recruited from outpatient and community brain injury rehabilitation services in Brisbane over 3 years (July 2013 to 2016). Recruitment also occurred from community rehabilitation at a brain injury unit in Sydney over 15 months (July 2014 to October 2015). Participants were screened according to the following eligibility criteria: 18 to 70 years, diagnosis of severe TBI (as determined by posttraumatic amnesia [PTA] and Glasgow Coma Scale), deemed medically stable and out of PTA, lived within a 50-km radius of each metropolitan center, and demonstrated dysexecutive impairments (see the section on Measures) that warrant community support. Participants were excluded if they were unable to provide informed consent; had severe behavioral, motor, sensory-perceptual, language, or cognitive impairments that precluded participation; or displayed psychotic symptoms or severe mood symptoms not under effective management.

Sample size was estimated from a previous RCT⁴ and subsequent meta-analysis,¹⁸ which indicated that EBL was more effective for improving self-awareness (f = 0.25) and behavioral competency (f = 0.39) than another active intervention, with medium to large effect sizes. A conservative power analysis was conducted using an estimated medium effect size (f = 0.25), α of .05, and power of 0.8.¹⁹ A total sample of 123 participants was required to detect significant between-group differences, accounting for baseline scores.¹⁸

Measures

Assessment and Screening Measures

Participants were initially screened by treating clinicians, and a neuropsychological test battery was administered to examine cognitive status and confirm the presence of dysexecutive impairments. This battery included tests of attention, memory, and executive function (Digit Span, Hopkins Verbal Learning Test-Revised, Modified Stroop Test, Trail Making Test, Hayling Sentence Completion Test and Verbal Fluency). A global neuropsychological function (GNF) composite was calculated by averaging age-adjusted scores. Participants were classified as having “mild to moderate” (less than −2 SD) or “severe” deficits (greater than or equal to −2 SD) relative to normative data.

Primary Outcome

Skills generalization on a self-regulation task related to training (near-transfer) was assessed by total errors on the Cooking Task.¹ This task requires participants to independently bake a chocolate cake and prepare an omelet in their own kitchen. Standard instructions, utensils, ingredients, and scoring criteria are used. Five error types are summed to produce a total error score. Internal consistency (α = 0.74), interrater reliability (ICC = 0.93), and discriminant and convergent validity have been reported.^1,20

In the current study, interrater reliability was examined by 2 independent raters using a random selection (25%) of audiovisual recordings. Reliability was excellent for the total error score (ICC = 0.95).

Secondary Outcomes

The Zoo Map Test from the Behavioural Assessment of Dysexecutive Syndrome²¹ was used to examine skills generalization on a self-regulation task unrelated to training (far-transfer). In part 1 of Zoo Map, participants are required to navigate a route through a zoo on a map by following instructions that specify certain rules. In part 2, participants complete the route following a predetermined plan. Given its focus on self-regulation skills, the part 1 score (sequence score minus errors) was used in the analysis.

The Awareness Questionnaire (AQ)²² is a 17-item measure of self-awareness that requires respondents to compare the person’s postinjury and preinjury abilities (1 = much worse, 3 = about the same, 5 = much better) in sensory, physical, cognitive, and behavioral domains. Higher total scores reflect better functioning relative to preinjury levels. The AQ was completed by participants with TBI and their caregivers. More negative discrepancy scores (caregiver ratings minus participant ratings) reflect poorer self-awareness.

The Patient Competency Rating Scale (PCRS)²³ assesses the competency of the person with TBI across the domains of activities of daily living and interpersonal, cognitive, and emotional functioning. Caregivers rated the 30 items on a 5-point scale (1 = can’t do, to 5 = can do with ease), with higher total scores reflecting behavioral greater competency.

The Sydney Psychosocial Reintegration Scale (SPRS)²⁴ measures psychosocial reintegration following TBI. Form A assesses change since the injury and Form B assesses current competency. Form B was completed by caregivers to assess participants’ current level of functioning in 3 domains: occupational activities, interpersonal relationships, and independent living skills. The 4 items on each domain are rated on a 5-point scale (0 = extremely poor, to 4 = very good), with higher total scores reflecting better psychosocial functioning. The total score (0-48) was used in the present study to represent global psychosocial functioning.

The Care and Needs Scale (CANS)²⁵ is a validated measure²⁶ that assesses the extent to which individuals with TBI can be left alone in the community. Based on an interview with caregivers, a blind assessor identified weekly hours of assistance and rated participants on a 8-point scale (0 = Can live in the community, totally independently - does not need contact, to 7 = Cannot be left alone—needs nursing care, assistance, and/or surveillance 24 hours per day).

The Depression Anxiety and Stress Scales-21 (DASS-21)²⁷ is a 21-item self-report measure of mood symptoms that has been validated for use in the TBI population.²⁸ The scales were administered to monitor for changes in mood in response to the intervention.

Study Protocol

Randomization

Following study enrolment and the baseline assessment (conducted by blind assessor), participants were randomized to the EBL or ELL interventions using a predetermined computer-generated random sequence. Allocation was concealed using sequentially numbered and sealed opaque envelopes. The random assignment was stratified according to GNF (mild to moderate deficit vs severe deficit) and was conducted by a person independent of the research team.

Assessment

A blind assessor accessed hospital records for medical information and conducted the baseline assessment in participants’ own homes. Caregivers completed the PCRS, AQ, SPRS, and CANS in person or via telephone. Approximately 1 week after the final intervention session, a blind assessor conducted the postintervention assessment in participants’ homes. At 6-month postintervention, a follow-up assessment was conducted by a blind assessor via telephone with participants (AQ, PCRS, DASS-21) and caregivers (PCRS, SPRS, CANS).

Interventions

Over the course of the study, 4 occupational therapists were involved in delivering each intervention, with therapists’ years of experience working in brain injury rehabilitation (1-4 years) spread evenly across the 2 interventions. Both interventions entailed an 8-week home-based program (90-minute sessions per week) in which participants learnt to prepare a hot meal (a stir-fry) for the first 4 training sessions (1-4). For the last 4 training sessions (5-8), therapists developed a set of multiple tasks or a complex multistep activity that was related to participants’ goals and interests. Examples included performing household tasks, running errands in a shopping center, computer skills training, and locating resources in a local library or university. Manualized treatment protocols were followed for each intervention, and therapists were supervised by the lead investigators (TO and JF). The key principles, procedures, and techniques of EBL and ELL are summarized in Table 1.

Table 1.

Overview of Principles, Procedures, and Techniques of Error-Based Learning and Errorless Learning.^a

	Error-Based Learning	Errorless Learning
Principles	Top-down, metacognitive approach targeting self-awareness, self-monitoring, and self-regulation skills (error correction and strategy use)	Bottom-up, task-specific approach targeting error-free performance through observing and practicing only correct actions
Procedures	Therapists provide structured opportunities for participants to make errors and to learn to self-correct with graded prompts and feedback. They are taught to anticipate task difficulties, use the Stop, Check, and Notice (SCaN) strategy, and to reflect on their performance over sessions	Therapists prevent participants from making errors by modeling each step and providing physical and verbal guidance to ensure correct actions and habit formation over sessions
Techniques
Skill learning	Role reversal (session 1): Therapists make several errors across all steps of the activity; participants provide corrective feedback by checking steps of the recipe Pretask predictions^b: Participants preview the task and predict possible errors for each step and plan strategies	Modeling: Therapists verbalize and model correct performance of each action during all activity steps Therapists preview the activity with the participant and describe correct actions within each step
Skill practice	Participants complete the activity by following the instructions and using SCaN; therapists observe and verbally affirm correct actions and provide on-task prompting for error correction (see below)	Therapists initially model each action; participants copy the action and receive verbal affirmation; therapists anticipate errors and provide physical, nonverbal or verbal cues to guide participants’ error-free actions
Response to errors	Therapists delay responding to errors (up to 10 s) to allow participants to self-correct; if this does not occur, therapists provide a nonspecific prompt (eg, Can you stop and check where you are up to?), followed by a specific prompt if needed	If an error occurs, therapists immediately model and describe the correct action and participants repeat the action
Posttask review	Participants review all steps, reflect on their performance, and identify strategies and goals for improvement	Therapists review the activity steps and provide verbal affirmation of participants’ correct performance
Skill mastery	Therapists gradually fade specific and nonspecific prompts over sessions to support self-initiated checking, error correction, and strategy use	Therapists consistently provide verbal cues to ensure error-free performance but fade modeling and physical guidance as appropriate

Adapted from Ownsworth et al.¹⁷

At the start of session 4 of error-based learning, therapists showed participants an audiovisual recording of select activity steps recorded in session 3 to facilitate participants’ awareness of errors and strategy development for the final meal preparation session.

Therapist adherence to the treatment protocol was examined for a random sample (15%) of audiotaped sessions 1 to 8 for each intervention using a checklist based on Borelli’s²⁹ framework. Two examiners (1 per intervention) independent of the study were involved in rating therapists’ adherence to delivering components for each intervention. Both examiners were experienced in delivering cognitive rehabilitation to people with TBI and received training on rating therapist adherence.

Data Analysis

Data were screened for missing values, and assumptions relevant to the statistical analyses were checked. Attrition was managed on an intention-to-treat basis using the last observation carried forward method, which involves imputation of missing values based on the last data point available for a participant.³⁰ Participants were analyzed in the group to which they were allocated, irrespective of withdrawal from treatment. Independent t-tests and χ² tests were conducted to compare the intervention groups on demographic and clinical variables (namely, age, sex, education, chronicity, injury severity, referral setting, concurrent occupational therapy, and GNF) and outcome measures at baseline. Any demographic or clinical variables that significantly differed between the groups or variables significantly related to postintervention outcome (tested using t-tests or correlations) were treated as covariates in subsequent analyses. Analysis of covariance (ANCOVA) was conducted to compare EBL and ELL on the outcome measures, controlling for baseline scores and covariates. ANCOVA was also used to compare EBL and ELL on long-term outcomes at the 6-month follow-up, controlling for baseline scores and covariates. Partial η squared (η_p²; 0.01 = small, 0.06 = medium, 0.14 = large) and 95% CIs are reported.

Results

Descriptive Data and Baseline Comparability

Over the recruitment period, 81 individuals with severe TBI were referred to the study (see Figure 1). Of these, 16 did not meet inclusion criteria (9 did not display dysexecutive deficits, 5 were too severely impaired, and 2 had psychiatric disorders not under effective management), 7 declined participation, and 4 could not be contacted despite repeated attempts. A total of 54 participants aged 18 to 64 years (mean = 37.61; SD = 13.3) enrolled in the study. Of these, 50 (93%) completed the intervention and postintervention assessment. Most participants were male (80%), and the mean time since injury was 38.63 months (SD = 47.22). The demographic and clinical characteristics and baseline functioning of participants in both groups are summarized in Table 2. There were no significant between-group differences on any demographic or clinical variables (P > .05). However, the EBL group had higher global psychosocial functioning (SPRS: t = −2.01; P = .05) than the ELL group.

Figure 1.

Participant flow diagram.

Table 2.

Group Characteristics on Demographic, Clinical, and Baseline Variables.

Characteristic/Outcome	Error-Based Learning (n = 27), Mean (SD)/n (%)	Errorless Learning (n = 27), Mean (SD)/n (%)
Age (years)	37.37 (13.6)	37.86 (13.3)
Sex, males	20 (74.1)	23 (85.1)
Education (years)	12.81 (2.4)	12.33 (2.1)
Time since injury (months)	36.44 (45.8)	40.81 (49.3)
Duration of posttraumatic amnesia (days)	76.16 (60.5)	81.50 (42.4)
Glasgow Coma Scale	6.19 (3.9)	5.12 (2.9)
Referral setting
Hospital outpatient rehabilitation	6 (22.2)	8 (29.6)
Community rehabilitation service	21 (77.8)	19 (70.4)
Relationship status
Single	18 (66.7)	17 (63.0)
Married or in a relationship	9 (33.3)	10 (37.0)
Employment status
Part-time/Casual	1 (3.7)	1 (3.7)
Student	1 (3.7)	1 (3.7)
Volunteer	3 (11.1)	4 (14.8)
Unemployed	22 (81.5)	21 (77.8)
Cause of injury
Road traffic accident	11 (40.7)	16 (59.3)
Fall	10 (37.0)	9 (33.3)
Assault	3 (11.1)	2 (7.4)
Other (sport, work related)	3 (11.1)	—
Global neuropsychological composite	−2.33 (1.9)	−2.12 (1.6)
Mild moderate (<[−2 SD])	13 (48)	13 (48)
Severe (≥[−2 SD])	14 (52)	14 (52)
Receiving concurrent occupational therapy	6 (22.2)	6 (22.2)
Outcome measures
Cooking task (total errors)	49.96 (20.9)	45.41 (20.8)
Zoo Map part 1 (sequence minus errors)	−1.35 (3.2)	−1.73 (5.4)
AQ discrepancy	−11.52 (8.5)	−9.19 (7.6)
PCRS caregiver ratings	105.00 (14.6)	99.04 (21.4)
SPRS total score	28.89 (7.2)^a	24.19 (9.8)
CANS level	4.30 (1.4)	4.48 (1.4)
CANS hours	80.33 (60.8)	74.85 (58.8)
DASS Depression	9.70 (9.4)	14.81 (12.7)
Anxiety	4.74 (8.5)	6.30 (7.6)
Stress	11.93 (9.3)	14.30 (12.3)

Abbreviations: AQ, Awareness Questionnaire; CANS, Care and Needs Scale; DASS, Depression Anxiety and Stress Scales; PCRS, Patient Competency Rating Scale; SPRS, Sydney Psychosocial Reintegration Scale.

P < .05.

Effect of Intervention on the Primary Outcome

Preliminary correlation analyses identified that years of education was significantly correlated with Cooking Task errors at postintervention (r = 0.27; P < .05). As shown in Table 3, after controlling for education and baseline errors, there was a significant effect of intervention on Cooking Task errors (P < .05; η_p² = 0.10). The EBL participants made significantly fewer errors at postintervention (mean = 36.25; SE = 1.87) than ELL participants (mean = 42.57; SE = 1.87).

Table 3.

Between-Group Differences at Postintervention for the Error-Based Learning (EBL) and Errorless Learning (ELL) Groups.^a

Outcome	Estimated Marginal Means (SE)		Covariates		Intervention Effects		95% CI
Outcome	EBL	ELL	Variable	F	F	ηρ²	95% CI
Cooking task errors	36.25 (1.9)	42.57 (1.9)	Baseline	177.69***	5.67*	0.10	1.00, 11.66
Cooking task errors	36.25 (1.9)	42.57 (1.9)	Education	0.57	5.67*	0.10	1.00, 11.66
Zoo Map	1.75 (0.7)	0.66 (0.7)	Baseline	13.06**	1.20	0.02	−3.09, 0.91
Zoo Map	1.75 (0.7)	0.66 (0.7)	GNF	0.41	1.20	0.02	−3.09, 0.91
AQ discrepancy	−5.77 (1.5)	−10.12 (1.5)	Baseline	30.10***	4.13*	0.08	−8.64, −0.06
AQ discrepancy	−5.77 (1.5)	−10.12 (1.5)	Concurrent OT	4.24*	4.13*	0.08	−8.64, −0.06
PCRS (caregiver)	107.45 (1.4)	103.41 (1.4)	Baseline	236.12***	4.07*	0.08	−8.07, −0.02
PCRS (caregiver)	107.45 (1.4)	103.41 (1.4)	Concurrent OT	1.42	4.07*	0.08	−8.07, −0.02
SPRS total	28.96 (0.9)	29.08 (0.9)	Baseline	125.51***	0.01	0.00	−2.41, 2.65
			Age	5.72*
			GNF	0.32
CANS level	3.76 (0.2)	3.87 (0.2)	Baseline	111.27***	0.21	0.00	−0.39, 0.62
CANS level	3.76 (0.2)	3.87 (0.2)	GNF	0.43	0.21	0.00	−0.39, 0.62
CANS hours	61.84 (4.0)	71.86 (4.0)	Baseline	296.19***	3.15	0.06	−1.33, 21.37
CANS hours	61.84 (4.0)	71.86 (4.0)	GNF	.01	3.15	0.06	−1.33, 21.37
DASS-21 Depression	11.99 (1.6)	8.38 (1.6)	Baseline	31.51***	2.43	0.05	−8.25, 1.04
DASS-21 Depression	11.99 (1.6)	8.38 (1.6)	Chronicity	2.06	2.43	0.05	−8.25, 1.04
Anxiety	4.37 (1.0)	3.93 (1.0)	Baseline	38.45***	0.09	0.00	−3.35, 2.48
Stress	10.13 (1.5)	8.24 (1.5)	Baseline	27.44***	0.77	0.02	−6.25, 2.45
Stress	10.13 (1.5)	8.24 (1.5)	Chronicity	0.22	0.77	0.02	−6.25, 2.45

Abbreviations: AQ, Awareness Questionnaire; CANS, Carer Assessment of Needs Scale; DASS-21, Depression Anxiety Stress Scales–21; GNF, Global Neuropsychological Function; OT, Occupational Therapy; PCRS, Patient Competency Rating Scale; SPRS, Sydney Psychosocial Reintegration Scale.

*P < .05; **P < .01; ***P < .001.

Effect of Intervention on Secondary Outcomes

As reported in Table 3, after controlling for GNF (r = 0.28; P < .05) and baseline performance, there was no significant difference in Zoo Map part 1 scores between the intervention groups at postintervention (P > .05; η_p² = 0.02). However, there was a significant effect of intervention for self-awareness on the AQ. After controlling for baseline scores and concurrent occupational therapy (note: those receiving concurrent therapy had better self-awareness), EBL participants demonstrated better self-awareness than ELL participants (P < .05; η_p² = 0.08). Furthermore, after controlling for baseline scores and concurrent therapy, participants in the EBL group demonstrated significantly better behavioral competency on the PCRS than the ELL participants (P < .05; η_p² = 0.08).

After adjusting for age (r = 0.35; P = .01), GNF (r = 0.37; P < .01), and baseline functioning, there was no significant effect of intervention on the SPRS (P > .05). Similarly, after controlling for GNF (r = −0.30; P < .05) and baseline functioning, there was no significant effect of intervention on CANS level (P > .05). For CANS weekly hours, after adjusting for GNF (r = −0.41; P < .01) and baseline functioning, there was a nonsignificant trend for EBL participants to require fewer hours of weekly support than ELL participants (P = .08; η_p² = 0.06).

Effect of Intervention on Mood

As shown in Table 3, there were no significant between-group differences on the DASS-21 at postintervention after controlling for baseline scores and chronicity (depression and stress). However, within-group analyses for the ELL group revealed that mood symptoms were significantly lower at postintervention relative to preintervention for depression, anxiety, and stress (P < .05). There were no significant within-group differences in mood symptoms for the EBL group (P > .05).

Effects of Intervention at 6-Month Follow-up

A total of 41 participants (76%) were retained in the study at the 6-month follow-up. Participants who withdrew during the intervention or were lost to follow-up (n = 13) did not significantly differ from those retained on demographic, clinical, or baseline variables (P > .05). As shown in Table 4, there were no differences in self-awareness, behavioral competency, or psychosocial outcomes between the EBL and ELL groups at the 6-month follow-up (P > .05).

Table 4.

Between-Group Differences at the 6-Month Follow-up for the Error-Based Learning (EBL) and Errorless Learning (ELL) Groups.^a

Outcome	Estimated Marginal Means (SE)		Covariates		Intervention Group		95% CI
Outcome	EBL	ELL	Variable	F	F	ηρ²	95% CI
AQ discrepancy	−7.21 (1.3)	−6.31 (1.3)	Baseline	20.62***	0.23	.01	−4.62, 2.83
PCRS (caregiver)	107.94 (2.1)	106.10 (2.1)	Baseline	100.52***	0.38	.01	−7.81, 4.14
PCRS (caregiver)	107.94 (2.1)	106.10 (2.1)	Concurrent OT	2.41	0.38	.01	−7.81, 4.14
SPRS total	31.00 (1.0)	32.5 (1.0)	Baseline	99.26***	0.57	.01	−1.89, 4.18
SPRS total	31.00 (1.0)	32.5 (1.0)	Concurrent OT	8.41**	0.57	.01	−1.89, 4.18
CANS level	3.42 (0.2)	3.54 (0.2)	Baseline	87.50***	0.16	.00	−0.46, 0.69
CANS level	3.42 (0.2)	3.54 (0.2)	Concurrent OT	6.40*	0.16	.00	−0.46, 0.69
CANS hours	53.74 (5.5)	62.92 (5.5)	Baseline	122.90***	1.41	.03	−6.34, 24.69
CANS hours	53.74 (5.5)	62.92 (5.5)	GNF	0.03	1.41	.03	−6.34, 24.69
DASS-21 Depression	9.67 (1.6)	10.70 (1.6)	Baseline	35.31***	0.21	.00	−3.47, 5.51
DASS-21 Depression	9.67 (1.6)	10.70 (1.6)	Chronicity	5.25*	0.21	.00	−3.47, 5.51
Anxiety	1.95 (0.7)	3.61 (0.7)	Baseline	42.95***	2.85	.05	−0.32, 3.64
Stress	7.45 (1.1)	7.67 (1.1)	Baseline	46.76***	0.02	.00	−2.90, 3.35

*P < .05; **P < .01; ***P < .001.

Therapists’ Adherence to Intervention Protocol

An assessor for each intervention rated the therapists’ adherence (present, mostly present, or absent) to delivering core components of intervention according to the manualized treatment protocol (see Table 5). Therapists showed good adherence to the EBL intervention, with 82% to 97% of components identified as present across sessions (n = 33). Similarly, the ELL therapists’ adherence was generally good, with components present for 79% to 85% of sessions (n = 33).

Table 5.

Therapists’ Adherence to Core Components of Each Intervention.

Core Intervention Components	Error-Based Learning, n = 33 sessions (%)			Errorless Learning, n = 33 Sessions (%)
Core Intervention Components	Present	Mostly Present	Absent	Present	Mostly Present	Absent
Overview or review of intervention approach	82	18	0	79	9	12
Introducing the functional task	88	9	3	82	12	6
Teaching the functional task	88	12	0	85	15	0
Posttask review of performance	97	3	0	79	9	12

Discussion

EBL was found to promote greater skills generalization on a near-transfer task and was associated with greater gains in self-awareness and behavioral competency than ELL. However, EBL did not significantly differ from ELL in its effects on far-transfer or psychosocial outcomes at postintervention. Furthermore, there were no significant differences between the EBL and ELL groups at the 6-month follow-up on self-awareness, behavioral competency, or psychosocial functioning.

Generalization of skills is a major challenge in rehabilitation for people with severe TBI.⁶ To facilitate skills generalization, individuals in the EBL intervention were initially taught self-regulation skills (ie, anticipate, monitor, and self-correct errors and generate strategies) during the preparation of a meal and were supported to transfer these skills to another multitasking activity. According to Toglia’s⁵ activity transfer continuum, the near-transfer task of baking a chocolate cake and making an omelet is “somewhat similar” to the first training task (stir fry) because this was conducted in the same context (person’s own kitchen), and involved both similar (eg, use of stove top and microwave) and different task requirements (eg, use of the oven, ingredients). The findings indicate that EBL participants were able to transfer the self-regulation skills taught during training to a related task within the same functional context.

However, EBL was not found to promote skills generalization on a far-transfer task that required participants to use self-regulation skills to navigate a path on a map of a zoo. These task demands were quite distinct from training tasks. Hence, it is apparent that skills generalization is more likely to occur following EBL when there are similarities between the training task and untrained task, which assist individuals to apply strategies to new task demands.⁵ Despite the lack of skills generalization on the far-transfer task, there was evidence of more generalized benefits of EBL for improving self-awareness and behavioral competency.

These findings suggest that gains in self-awareness are more likely to occur in rehabilitation when individuals have structured opportunities to make errors, reflect on their functional significance, and engage in problem solving to recognize and correct their own errors. However, given that the EBL and ELL conditions differed in several respects (see Table 1), further research is needed to identity the active ingredients of EBL. Improvements in self-awareness after EBL were not associated with increased emotional distress or other adverse reactions, which reinforces previous findings that a supportive therapeutic approach buffers the emotional impact of self-awareness training.¹⁵ Interestingly, the ELL approach of providing a high level of guidance to avoid errors was associated with improved emotional well-being at postintervention relative to preintervention levels. A possible explanation for this finding is that during ELL training, participants’ performance was consistently reinforced without them being exposed to difficulties that signify their impairments. Improvements in mood have also been found after therapist-led ELL memory training for people with Alzheimer’s disease.³¹

The observed improvements in self-awareness after EBL are consistent with the findings of a previous RCT,¹⁵ which indicated that multimodal feedback on a meal preparation task was effective for updating self-knowledge of TBI-related impairments. Improvements in behavioral competency were also demonstrated after an 8-session metacognitive and occupation-based intervention in an earlier study.³² However, broader gains in psychosocial functioning were not evident in comparison to other intervention approaches. In line with these past findings, there were no benefits of EBL over ELL for improving psychosocial functioning or long-term outcomes at the 6-month follow-up. It is possible that an 8-week EBL intervention is not sufficiently intensive for achieving broader gains in independence, and social and occupational function relative to another active intervention. Similarly, a trial comparing goal management training approaches for facilitating functional skills found task-specific training effects but no differential effects of intervention on broader cognitive function and quality of life.³³

Clinical Implications

The present findings broadly support that errors play an important role in learning of skills in rehabilitation. However, as well as allowing participants to make errors and reflect on these, EBL provided more opportunities for them to retrieve sequences of action from long-term memory than ELL. In the education literature, retrieval practice or the opportunity to test one’s recall of information or task steps has been found to enhance consolidation of learning, self-monitoring and self-regulation, and skills transfer.^34-36 Hence, further research is needed to determine the specific benefits of making errors during training as compared with retrieval practice for people with TBI.

The key finding that EBL facilitated skills generalization on a near-transfer task, but not on a far-transfer task, highlights the need for intervention goals to inform the nature of training activities. Although ELL may be appropriate for training skills that are relatively fixed or task specific (eg, using a mobile phone alarm function), EBL is likely to be more effective for training skills with varying task demands within the same functional context, for which there is greater reliance on self-monitoring and problem-solving skills. However, if far-transfer of skills is an intervention goal, a graded and successive activity plan involving a continuum of training activities that progressively vary in the similarity of task parameters and context (eg, preparing the same meal in a different kitchen or completing a different meal in the same kitchen) is likely to be needed.⁵ Sessions are also recommended to support maintenance of gains in self-awareness and self-regulation skills over time.

Booster sessions are commonly recommended after interventions for people with TBI and have been found to support maintenance of treatment gains in the context of psychotherapy.³⁷ However, Chiaravalloti et al³⁸ found that monthly booster sessions were not effective for sustaining postintervention improvements in learning and memory in participants with moderate to severe TBI. Further research is needed to determine the optimal intensity and content of booster sessions following cognitive rehabilitation interventions, including EBL.

Limitations and Future Research

A main limitation of this study relates to the smaller-than-planned sample size, which may have lowered statistical power for some analyses (eg, CANS hours). A relatively large number of outcomes (n = 10) was examined at postintervention without adjustment to the significance level. This was deemed appropriate because the Cooking Task total errors was identified as the primary outcome prior to commencing the study¹⁷ and to balance between the risk of type I and type II errors given the modest sample size. Importantly, the between-group differences on the Cooking Task, AQ, and PCRS were all in the medium effect size range.

As a further limitation, participants were recruited on a nonconsecutive basis from outpatient and community rehabilitation services, which may have affected the representativeness of the sample. Time since injury was highly variable (4-204 months), although this was not associated with postintervention outcomes. Therefore, some caution is needed in generalizing these findings to the broader TBI population.

The main focus of this study was on skills generalization (near- and far-transfer effects) after the intervention; consequently, the maintenance of gains on skills generalization indices over time was not examined. The durability of gains in self-regulation and self-awareness and the factors influencing maintenance of these outcomes are important to address in future research. More generally, it would also be valuable to identify the neuropsychological characteristics of individuals most likely to benefit from metacognitive approaches such as EBL. Further research on interventions designed to promote skills generalization on far-transfer tasks is needed, guided by Toglia’s⁵ graded activity continuum.

Footnotes

Acknowledgements

The authors would like to acknowledge the expertise of the following people involved in the study: Louise Lee, Stephanie Fletcher, Tereza Stillerova, Tenelle Hodson, Sarah de Charentenay, Gwen Liddle, Ruth Swanton, Beth Nadwie, Freyr Patterson, Stephanie Jones, Lauren Spriggens, and Travis Wearne. We are also very grateful to the clinicians at the Acquired Brain Injury Outreach Service, Brain Injury Rehabilitation Unit of the Princess Alexandra Hospital, and the Royal Rehabilitation Centre for facilitating participant recruitment. There are no industrial links or affiliations of authors involved in this study and no conflicts of interest.

Authors’ Note

Trial registration: Australian New Zealand Clinical Trials Registry Identifier Number ACTRN12613000585729.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This trial was funded by a National Health and Medical Research Council Project Grant (APP1043677). This funding body was otherwise not involved in the study.

References

Chevignard

Taillefer

Picq

Poncet

Noulhiane Pradat-Diehl

Ecological assessment of the dysexecutive syndrome using execution of a cooking task. Neuropsychol Rehabil. 2008;18:461-485.

Ownsworth

Fleming

The relative importance of metacognitive skills, emotional status and executive functioning in psychosocial adjustment following acquired brain injury. J Head Trauma Rehabil. 2005;20:315-332.

Ham

Bonnelle

Hellyer

et al . The neural basis of impaired self-awareness after traumatic brain injury. Brain. 2014;137(pt 2):586-597.

Ownsworth

Fleming

Strong

Radel

Chan

Clare

Awareness typologies, long-term emotional adjustment and psychosocial outcomes following acquired brain injury. Neuropsychol Rehabil. 2007;17:129-150.

Toglia

JP.

The dynamic interactional model of cognition in cognitive rehabilitation. In: Katz

, ed. Cognition, Occupation and Participation Across the Life Span: Neuroscience, Neurorehabilitation and Models of Intervention in Occupational Therapy. 3rd ed. Bethesda, MD: AOTA Press; 2011:161-201.

Cicerone

Langenbahn

Braden

et al . Evidence-based cognitive rehabilitation: updated review of the literature from 2003 through 2008. Arch Phys Med Rehabil. 2011;92:519-530.

Tate

Kennedy

Ponsford

et al . INCOG recommendations for management of cognition following traumatic brain injury, part III: executive function and self-awareness. J Head Trauma Rehabil. 2014;29:338-352.

Ehlhardt

Sohlberg

Kennedy

et al . Evidence-based practice guidelines for instructing individuals with neurogenic memory impairments: what have we learned in the past 20 years? Neuropsychol Rehabil. 2008;18:300-342.

Velikonja

Tate

Ponsford

McIntyre

Janzen

Bayley

;INCOG Expert Panel. INCOG recommendations for management of cognitive following traumatic brain injury, part V: memory. J Head Trauma Rehabil. 2014;29:369-386.

10.

Clare

Jones

RS.

Errorless learning in the rehabilitation of memory impairment. Neuropsychol Rev. 2008;18:1-23.

11.

Evans

Wilson

Schuri

et al . A comparison of “errorless” and “trial-and-error” learning methods for teaching individuals with acquired memory deficits. Neuropsychol Rehabil. 2000;10:67-101.

12.

Ownsworth

Fleming

Desbois

Strong

Kuipers

PA.

Metacognitive contextual intervention to enhance error awareness and functional performance following traumatic brain injury: a single case experimental design. J Int Neuropsychol Soc. 2006;12:54-63.

13.

Ownsworth

Quinn

Fleming

Kendall

Shum

Error self-regulation following traumatic brain injury: a single case study evaluation of metacognitive skills training and behavioural practice interventions. Neuropsychol Rehabil. 2010;20:59-80.

14.

Toglia

Johnson

Goverover

Dain

A multicontext approach to promoting transfer of strategy use and self-regulation after brain injury: an exploratory study. Brain Inj. 2010;24:664-677.

15.

Schmidt

Fleming

Ownsworth

Lannin

Video feedback on functional task performance improves self-awareness after traumatic brain injury: a randomized controlled trial. Neurorehabil Neural Repair. 2013;27:316-324.

16.

Schulz

Altman

Moher

;CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomized trials. Ann Intern Med. 2010;152:726-732.

17.

Ownsworth

Fleming

Tate

et al . Comparison of error-based and errorless learning for people with severe traumatic brain injury: study protocol for a randomized control trial. Trials. 2013;14:369.

18.

Schmidt

Lannin

Fleming

Ownsworth

Feedback interventions for impaired self-awareness following brain injury: a systematic review. J Rehabil Med. 2011;43:673-680.

19.

Faul

Erdfelder

Lang

Buchner

G*Power 3: a flexible statistical power analysis program for the social, behavioral and biomedical sciences. Behav Res Methods. 2007;39:175-191.

20.

Poncet

Swaine

Taillefer

Lamoureux

Pradat-Diehl

Chevignard

Reliability of the cooking task in adults with acquired brain injury. Neuropsychol Rehabil. 2015;25:298-317.

21.

Wilson

Alderman

Burgess

Emslie

Evans

JJ.

Behavioural Assessment of the Dysexecutive Syndrome (BADS). Bury St Edmunds, UK: Thames Valley Test; 1996.

22.

The Center for Outcome Measurement in Brain Injury. Introduction to the awareness questionnaire (AQ). http://www.tbims.org/combi/aq. Accessed May 30, 2017.

23.

Prigatano

Altman

IM.

Impaired awareness of behavioral limitations after traumatic brain injury. Arch Phys Med Rehabil. 1990;71:1058-1064.

24.

Tate

Simpson

Soo

Lane-Brown

AT.

Participation after acquired brain injury: clinical and psychometric considerations of the Sydney Psychosocial Reintegration Scale (SPRS). J Rehabil Med. 2011;43:609-618.

25.

Tate

RL.

Assessing support needs for people with traumatic brain injury: the Care and Needs Scale (CANS). Brain Inj. 2004;18:445-460.

26.

Soo

Tate

Aird

et al . Validity and responsiveness of the Care and Needs Scale (CANS) for assessing support needs following traumatic brain injury. Arch Phys Med Rehabil. 2010;91:905-912.

27.

Lovibond

PF.

Manual for the Depression Anxiety Stress Scales. 2nd ed. Sydney, Australia: Psychology Foundation of Australia; 1995.

28.

Dahm

Wong

Ponsford

Validity of the Depression Anxiety Stress Scales in assessing depression and anxiety following traumatic brain injury. J Affect Disord. 2013;151:392-396.

29.

Borrelli

Sepinwall

Ernst

et al . A new tool to assess treatment fidelity and evaluation of treatment fidelity across ten years of health behavior research. J Consult Clin Psychol. 2005;73:852-860.

30.

Gupta

SK.

Intention-to-treat concept: a review. Perspect Clin Res. 2011;2:109-112.

31.

Lee

Yip

CCK

ECS

Man

DWK

. Evaluation of a computer-assisted errorless learning-based memory training program for patients with early Alzheimer’s disease in Hong Kong: a pilot study. Clin Interv Aging. 2013;8:623-633.

32.

Ownsworth

Fleming

Shum

Kuipers

Strong

Comparison of individual, group and combined intervention formats in a randomised controlled trial (RCT) for facilitating goal attainment and improving psychosocial function following acquired brain injury. J Rehabil Med. 2008;40:81-88.

33.

Bertens

Kessels

Fiorenzato

Boelen

Fasotti

Do old errors always lead to new truths? A randomized controlled trial of errorless learning goal management training in brain injured patients. J Int Neuropsychol Soc. 2015;21:639-649.

34.

Dunlosky

Rawson

Marsh

Nathan

Willingham

DT.

Improving students’ learning with effective learning techniques: promising directions from cognitive and educational psychology. Psychol Sci Public Interest. 2013;14:4-58.

35.

Fernandez

Jamet

Extending the testing effect to self-regulated learning. Metacogn Learn. 2017;12:131-156.

36.

Roediger

III Butler

AC.

The critical role of retrieval practice in long-term retention. Trends Cogn Sci. 2011;15:20-27.

37.

Ponsford

Lee

Wong

et al . Efficacy of motivational interviewing and cognitive behavioral therapy for anxiety and depression symptoms following traumatic brain injury. Psychol Med. 2016;46:1079-1090.

38.

Chiaravalloti

Sandry

Moore

DeLuca

An RCT to treat learning impairment in traumatic brain injury: the TBI-MEM Trial. Neurorehabil Neural Repair. 2016;30:539-550.