Improving self-regulation and executive functioning skills in primary school children in a remote Australian Aboriginal community: A pilot study of the Alert Program®

Abstract

Self-regulation and executive functioning impairments are common in children with fetal alcohol spectrum disorder. Given the high rates of fetal alcohol spectrum disorder identified amongst children living in the remote Fitzroy Valley region of Western Australia, the Alert Program® was identified as a culturally safe intervention for use in local primary schools. Researchers collaborated with Aboriginal Elders, community members, and staff from a Fitzroy Valley primary school to trial the Alert Program®. Teachers were trained to deliver eight Alert Program® lessons to children in class. Self-regulation and executive functioning were measured using teacher and parent/caregiver questionnaires three times. Data were analyzed using repeated-measures ANOVA. Teacher-led delivery of the Alert Program® was feasible in a region with high fetal alcohol spectrum disorder rates. As measured by teacher and parent/caregiver ratings, this curriculum may improve the self-regulation and executive functioning of children for some outcomes and provide sustained effects for some children. This community partnered pilot research, evaluated a school-based program to reduce the behavioral impact of fetal alcohol spectrum disorder, and informed design of a larger trial across eight Aboriginal community schools.

Keywords

Fetal alcohol spectrum disorder Aboriginal students intervention primary school self-regulation executive functioning remote areas

Introduction

Executive functioning and self-regulation skills are integral to success in daily life and fundamental in the school context (Diamond, 2016). At school, executive functioning and self-regulatory abilities enable children to concentrate on learning, ignore distractions, organize themselves, follow rules and instructions, remember things, and monitor and change behavior in response to feedback. When these skills are impaired, children can struggle with learning and exhibit externalizing behavior problems which can negatively impact academic and social outcomes (Blair & Raver, 2015; Bridgett, Oddi, Laake, Murdock, & Bachmann, 2013).

Executive functions are a set of interrelated cognitive processes that contribute to goal-directed behavior (Best & Miller, 2010). These processes include working memory, inhibitory control, cognitive flexibility, planning, organization, monitoring, and initiation (Bridgett et al., 2013; Diamond, 2016). Self-regulation involves the ability to change or maintain an appropriate level of arousal to meet situational demands (Diamond, 2016). Executive functioning and self-regulation abilities are similar as both involve inhibition and the use of planning to achieve goals (Bridgett et al., 2013; Diamond, 2016). Without these skills, disruptive and problematic behaviors can manifest in children in a range of environments, including at home and in school (Eisenberg, Spinrad, & Eggum, 2010; Schonfeld, Paley, Frankel, & O'Connor, 2006). Executive functioning and self-regulation impairments are commonly observed in children with fetal alcohol spectrum disorder (FASD) (Kodituwakku, 2009; Kodituwakku, Handmaker, Cutler, Weathersby, & Handmaker, 1995). FASD is a term referring to a set of lifelong neurodevelopmental conditions associated with prenatal exposure to alcohol (Bower et al., 2017). Further, children’s exposure to adverse childhood experiences, including early life trauma (ELT), poverty, and stress can impact the development of these skills (Evans & Kim, 2013; Shonkoff & Garner, 2012).

In 2009, Aboriginal community leaders from the remote Fitzroy Valley region of Western Australia courageously sought to address the locally identified concern that FASD and ELT were impacting local children and families (Latimer et al., 2010). The culturally rich Fitzroy Valley is home to approximately 3500 people from the Bunuba, Walmajarri, Gooniyandi, Wangkatjunga, and Nyikina language groups who live in 45 distinct Aboriginal communities (Morphy, 2010). Whilst FASD is an issue affecting both Indigenous and non-Indigenous communities, from a range of socioeconomic, educational, and cultural backgrounds (Australian Medical Association, 2016), Fitzroy Valley leaders were particularly concerned about the impact of FASD and ELT on the preservation of Aboriginal cultural traditions, stories, and languages (Fitzpatrick, Oscar, et al., 2017; Gooda, 2010).

Subsequently, local Aboriginal leaders established a “Circle of Friends”¹ with whom they collaborated to develop and implement the Marulu² Strategy—a community-led initiative designed to prevent and diagnose FASD and support individuals and families affected by FASD and ELT (Gooda, 2010). The Lililwan³ Study was part of the Marulu Strategy and focused on diagnosis. It was initiated by the local Aboriginal community and was the first Australian study of the prevalence of FASD identifying a prevalence rate of 194 per 1000 children (Fitzpatrick, Latimer, et al., 2017). Thirty percent of all children in this study were identified with possible executive functioning problems, using brief screening measures (Fitzpatrick, Latimer, et al., 2017). This included children with a FASD diagnosis but also 21% of children without a FASD diagnosis. Data from the 2005 Western Australian Aboriginal Child Health Survey (WAACHS) also reported many children in this region at risk of clinically significant emotional or behavioral problems (22%) and conduct disorders (38%). During the WAACHS survey, 21% of children’s carers also reported their family experiencing seven or more life stressors, for example, family break-up, financial problems, or chronic illness (Zubrick et al., 2005b). Lililwan Study and WAACHS data suggest that both FASD and adverse childhood experiences may contribute to the executive functioning and self-regulation difficulties experienced by Fitzroy Valley children (Evans & Kim, 2013; Fitzpatrick, Latimer, et al., 2017; Zubrick et al., 2005a).

During the Lililwan Study, parents/carers, community Elders, and school staff identified an urgent need for locally relevant and culturally safe strategies to support Fitzroy Valley children improve their behavior, learning, and attention at school. Self-regulation and executive functioning skills are also important in this context to support the intergenerational transmission of language and culture (Latimer et al., 2010). Unfortunately, in the Fitzroy Valley, it is not possible to offer regular therapy from specialists such as occupational therapists to support children with FASD, due to limited resources, and vast distances clinicians must travel to reach children and their families (Wilkinson & Beattie, 2006). An alternative way to support children’s self-regulation and executive functioning development is via a classroom-based program delivered by trained teachers and school support staff, such as a curriculum version of the Alert Program®. This would enable staff to embed Alert Program® concepts in the usual school day, providing opportunities for children and staff to practice program language and strategies. Importantly, it was anticipated that classroom delivery would enable all children, not just those with a FASD diagnosis, to learn about and develop strategies to improve self-regulation. Additionally, school staff who live in the communities can further share program language and strategies with parents/caregivers through their strong community relationships.

The Alert Program® was identified as a promising intervention following positive feedback from Fitzroy Valley-based clinicians and school staff who had used the program in a limited manner in local classrooms (Wagner et al., 2016; Weston & Thomas, 2014). Additional discussions with key local stakeholders and review of FASD intervention literature (Bertrand, 2009) further supported the choice to adapt and trial the Alert Program® for use in Fitzroy Valley classrooms. The Alert Program®, developed in the United States of America, supports children to understand the concepts of arousal/alertness and self-regulation through the analogy of a car engine. Levels of alertness (explained to children as their “engine level”) are described as being in a “high,” “low,” or “just right” state (Williams & Shellenberger, 1996b). Children, and their caregivers, are taught sensorimotor strategies they can use to self-regulate by changing or maintaining an optimal level of arousal/alertness to suit the task at hand.

There are currently no studies assessing the effectiveness of a classroom-based Alert Program® for improving executive functioning and self-regulation when delivered by teachers in Australia or within remote community schools. Three previous international studies reported significant improvements in self-regulation and executive functioning of children with FASD aged between 6 and 12 years after receiving the Alert Program® in clinic settings (Nash et al., 2014; Soh et al., 2015; Wells, Chasnoff, Schmidt, Telford, & Schwartz, 2012). None of these programs were conducted in Indigenous settings. Outcomes were measured using parent rated and child measures that included the Behavior Rating Inventory of Executive Function (BRIEF) (Nash et al., 2014; Soh et al., 2015; Wells et al., 2012) and Roberts Apperception Test for Children (Wells et al., 2012). A small number of studies have evaluated the use of the Alert Program® in classrooms by teachers. While these studies did not focus on Indigenous children or those with FASD, the authors concluded that the Alert Program® had promoted improvements to children’s self-regulation (Barnes et al., 2008; Mac Cobb, Fitzgerald, & Lanigan-O’Keeffe, 2014).

This pilot study aimed to provide preliminary findings on effectiveness—but also, importantly, to refine the design of a larger self-controlled cluster randomized trial in partnership with Fitzroy Valley communities and associated primary schools (Wagner et al., 2018). In this pilot study, it was hypothesized that children’s self-regulatory behaviors and executive functioning skills, rated by teachers and parents/caregivers, would improve following the implementation of the Alert Program® curriculum by trained teachers in their classrooms. Results and key lessons from this pilot study are described in this paper. In-depth discussion and learnings from the year-long formative process, of which the pilot study was a part, are described in a separate paper by Wagner et al. (2016).

Materials and methods

Community collaboration

“Brokerage” is a key mechanism that strengthens organizations’ and individuals’ ability to develop culturally secure ways of working with Aboriginal communities. It involves the development of two-way communication, respect, trust, and community protocols in consultation with Aboriginal community leaders and Elders (Coffin, 2007; Coffin & Green, 2017). Therefore, as part of project planning, relationships between researchers and community leaders involved in the earlier community led Marulu FASD/ELT Strategy and Lililwan FASD prevalence study were re-established. The current study represents a direct response to the high FASD prevalence documented in the Fitzroy Valley and concern raised by the impact of ELT. The chief executive officer of the local Aboriginal health organization partnered with researchers to provide cultural expertise in developing the research protocol and was appointed as a project investigator (MC). Additionally, five of the original Lililwan prevalence study collaborators (JL, JF, HCO), including Aboriginal leaders from local education (EW) and community organizations (MC), were study investigators and contributed to the preliminary project planning and ongoing decision making.

Setting

The pilot study was conducted in a remote Aboriginal community in the Fitzroy Valley region of WA. The community comprises 135 residents, mostly Australian Aboriginal people (Morphy, 2010). The closest town, Fitzroy Crossing, is located 2600 kilometers from the WA capital city of Perth. Following input from local stakeholders, the study coordinator (BW), who had lived and worked in the Fitzroy Valley as an educator for over five years, met with several Aboriginal Elders from the community to discuss the Alert Program® study. Elders agreed that their community would be a suitable location to conduct the initial formative work and pilot study. Following the advice of Elders, school support, and ethical approvals, a barbecue was held at the local school to discuss the study with local families. The local community primary school had 42 children with a teacher in each of three multi-year classes; kindergarten to year 3 (3.5–9.5 years of age), year 3 to 5 (7.5–11.5 years of age), and year 5 to 7 (9.5–13.5 years of age). The kindergarten to year 3 class received support from an education assistant.

Community researcher role

Aboriginal Elders not only guide community decision making, but also develop and mentor others in their community (Kickett-Tucker & Hansen, 2017). Therefore, based on the recommendation of local Elders, a lead Aboriginal community researcher (SC) was recruited to work in a two-way partnership with the study coordinator to foster culturally secure research processes (Mc Loughlin, Hadgraft, Atkinson, & Marley, 2014). The lead community researcher provided vital expertise in cultural protocols, local languages, and relationship facilitation for non-Aboriginal research team members. Further, the lead community researcher sought feedback from local families about the suitability of research protocols and practices. Two additional community researchers were later recruited, and, following two-way capacity building, community researchers also conducted participant recruitment and consent, data collection and entry, and dissemination of results.

Recruitment and consent

Following the consultation and recruitment phase, Aboriginal community researchers and the study coordinator undertook home visits over a two-week period to further discuss the study with families and to seek written consent for their child’s involvement. The community researchers verbally translated study documents into Kimberley Kriol or the local Aboriginal language, as English is a second or third language for many residents.

Inclusion criteria

All children enrolled in years 1 to 5 who had attended 20% or more school days in the six months prior to data collection were eligible for the pilot study. This ensured that teachers had sufficient knowledge of children’s behaviors to complete the study questionnaires. School attendance in this region varies greatly, impacted by a range of complex social, cultural, health, and locational factors (Department of Education Services, 2012).

Study design

This pilot study used a single group pre-, post-, and follow-up intervention design. All children received the intervention in their regular classroom.

Ethical approvals

Ethics approval was provided by the University of WA Human Research Ethics Committee (RA/4/1/7234), the WA Aboriginal Health Ethics Committee (601), and the WA Country Health Service (2015:04). Additional approvals were provided by the Kimberley Aboriginal Health Planning Forum Research Sub-Committee and the WA Department of Education.

Alert Program® intervention

Suitable training for school staff, including teachers and Aboriginal and Islander Education Officers, in the classroom intervention was developed by the study coordinator and a local occupational therapist by adapting original Alert Program® resources with the support of the program designers. Both had experience working in Fitzroy Valley schools and had completed the Alert Program® Online Course. During the pilot study, school staff trialed the proposed training, lesson plans, and resources. Staff then provided in-depth feedback which helped researchers refine the intervention that would later be used in the main trial.

The formative process and intervention have been described by Wagner et al. (2016). Briefly, teachers delivered eight, one-hour Alert Program® lessons to children during class for eight weeks from July to September 2015. Lessons focused on Stage One and Two of the original three-stage Alert Program®. During lessons one to three, children learnt to recognize engine levels and use the associated program language. Children then experimented with sensorimotor tools to maintain or modify engine levels during weeks 4–8. Following the final lesson, the curriculum guide provided ideas to help students independently regulate their engine levels which relate to Alert Program® Stage Three (Williams & Shellenberger, 1996a). All children attending school received Alert Program® lessons regardless of their study enrollment. The research team provided teachers with all equipment and resources needed to deliver the program.

School staff training involved a combination of face-to-face delivery by the study coordinator and video modules from the Alert Program® Online Course. Prior to program delivery, staff attended a half-day training session covering the study’s purpose, information about FASD, definitions of self-regulation and executive functioning, and an overview of the Alert Program®. Following training session one, but prior to Alert Program® lesson delivery, teachers received individual instruction on how to use the curriculum guide. Between weeks 1 and 3 of program delivery, teachers independently viewed the relevant Alert Program® Online Course modules. Between weeks 3 and 4 of lesson delivery, the study coordinator facilitated a further group training session, discussed teacher experiences, clarified aspects of lesson delivery, and teachers viewed additional Alert Program® modules (Wagner et al., 2016).

Outcome measures

Pilot study outcome measures were chosen following consultation with project investigators, local Aboriginal leaders, and local clinicians. Considerable time was spent determining the optimal outcome measures with consideration of the time demands on families and teachers balanced with the need for valid and reliable outcome measures (Wagner et al., 2016).

Teacher Sutter–Eyberg Student Behavior Inventory-Revised (SESBI-R) & Parent Eyberg Child Behavior Inventory (ECBI)

The 38-item teacher-rated SESBI-R and the 36-item parent-rated ECBI are measures of disruptive behavior (Eyberg & Pincus, 1999) and were used as proxy measures of self-regulation. Teachers and parents/caregivers rated the frequency of disruptive behaviors on a seven-point “Intensity” scale ranging from never [“1”] to always [“7”]. Summed Intensity scores greater than 150 on the teacher questionnaire, and 130 on the parent form are considered clinically significant requiring further investigation. The SESBI-R and ECBI Intensity scales have high internal consistency (α = .95 and .98, respectively) and reliability (r = .87 and .80; (Eyberg & Pincus, 1999)).

Behavior Rating Inventory of Executive Function 2 (BRIEF2)—Teacher and Parent Screening Forms

In this pilot study, executive functioning was measured at pre-and post-intervention using the teacher- and parent-rated 86-item BRIEF questionnaire. Whilst local stakeholders were involved in this choice of measure, subsequent feedback from participants indicated that the full BRIEF was onerous, and some items not contextually appropriate (Wagner et al., 2016). Given that a 12-item BRIEF2 Screening Form became available during the pilot study, it was agreed by stakeholders and researchers to use this questionnaire at follow-up. Raw scores on both teacher and parent BRIEF2 Screening Forms are highly correlated to the full BRIEF2 Global Executive Composite scores in clinical and standardized samples (r ≥ .93; (Gioia, Isquith, Guy, & Kenworthy, 2015). Thus, the 12 BRIEF2 Screening Form questions were extracted for analysis from the full BRIEF administered at pre-test and post-test, and the term “BRIEF2 Screening Form” is used to refer to data derived in this pilot study. Frequency of child behaviors are rated by parents as occurring never [“1”], sometimes [“2”], or often [“3”]. The teacher and parent BRIEF2 Screening Forms incorporate emotional, cognitive, and behavioral regulation items (Gioia et al., 2015). Clinical relevance of summed total scores (“normal,” “potentially clinically significant,” or “clinically significant”) is determined by age and gender. Internal consistency and test–retest reliability of the teacher (α = .91, r = .87) and parent (α = .91, r = .87) BRIEF2 Screening Forms are high (Gioia et al., 2015).

Additional feedback from Aboriginal community researchers and local educators indicated that two of the 12 questions on the parent BRIEF2 Screening Form were not relevant to the remote community settings in which children lived (Q6: Does not plan ahead for school assignments, Q7: Written work is poorly organized; (Wagner et al., 2016)). After communication with authors of the BRIEF2, these two questions were removed from the parent Screening Form at follow-up, and the remaining three cognitive regulation items were pro-rated to derive scores for questions six and seven (see above).

Data collection

Teacher and parent/caregiver questionnaires were collected during three 2-week periods. As illustrated in Figure 1, the first data collection period occurred just prior to delivery of the intervention (pre), the second occurred immediately following the intervention (post), and the final data were collected at eight weeks (follow-up). There was an unavoidable overlap between data collection and the commencement and conclusion of the intervention (24, 35, and 26% of children with pre-intervention teacher data, pre-intervention parent/caregiver data, and post-intervention parent/caregiver data, respectively). Researchers recognized the need for flexible data collection processes to enable sufficient time for questionnaires to be completed with parents/caregivers (Mc Loughlin et al., 2014). Many families travel to and from surrounding communities for cultural reasons; examples include “sorry business”⁴ (Carlson & Frazer, 2015) or personal business such as shopping and medical appointments. Consequently, parents/caregivers were not always available during research team visits. Extending the data collection period into the intervention phase also provided teachers sufficient time to complete all questionnaires for their children while managing the intensity of teaching responsibilities in these remote communities.

Figure 1.

Alert Program® pilot study timeline.

Parents/caregivers completed questionnaires for their child during home visits conducted by an Aboriginal community researcher, accompanied by a research assistant or the study coordinator. As detailed elsewhere, verbal and written translations of questionnaire items into local Aboriginal languages were provided to parents/caregivers by community researchers as needed (Wagner et al., 2016). Teachers independently completed children’s questionnaires with parent/caregiver’s consent.

Data analysis

Data were analyzed using repeated measures analysis of variance (ANOVA) in SPSS version 25. Changes in teacher and parent/caregiver ratings of children were assessed at the .05 level of significance from: (1) immediately before the Alert Program® began (pre-intervention) to immediately after the Alert Program® was completed (post-intervention); (2) post-intervention to eight weeks after the Alert Program® was completed (follow-up); and (3) pre-intervention to follow-up. Prior to analysis, box-plots were inspected for outliers and removed if caused by data entry or measurement errors. Normality was assessed using Shapiro Wilk’s test. Due to the robustness of repeated measures ANOVA (Blanca, Alarcon, Arnau, Bono, & Bendayan, 2017), non-normally distributed data (p < .05) were retained for analysis. The assumption of sphericity was assessed using Mauchley’s test of sphericity. Where sphericity was violated (p < .05), a Greenhouse–Geisser correction was applied. ANOVA post-hoc analyses using Bonferroni’s adjustment were used to compare mean differences of assessment scores between each time-point. Effect sizes were defined by partial eta square cut offs of small = .02; medium = .13; large = .26 (Cohen, 1988). Due to the overlap between data collection and deployment of the intervention, sensitivity analyses were also performed. Participants for whom pre-intervention data collection took place on the day of or after the first Alert Program® lesson were removed and analysis re-run per procedures above. Primary results are reported unless specified as referring to sensitivity analyses. Confounders were not controlled for in either primary or sensitivity analyses due to small sample size.

Results

Participants

Of 27 eligible children, 25 were enrolled in the study (93%) following written consent from parents/caregivers. Children’s demographics are reported in Table 1. The mean age at consent was 8.8 years old (SD = 1.35), ranging from 6.2 to 11.9 years.

Table 1.

Demographics of children enrolled in the Alert Program® pilot study.

	Total	Percentage
Demographics	N	%
Sex
Male	12	48
Female	13	52
Indigenous status
Indigenous	23	92
Non-Indigenous	2	8
School year level
Year 1	3	12
Year 2	4	16
Year 3	6	24
Year 4	8	32
Year 5	4	16

Of the 25 children enrolled in the pilot study, two left the community following pre-intervention data collection. Other reasons for loss to follow-up included families declining to complete questionnaires at one or more time-points or being away from the community for cultural, medical, or other personal reasons during data collection visits. The number of participants with data for each outcome measure and time-point are stated in Table 2.

Table 2.

Questionnaire completion figures and final number of participants included in data analysis.

Questionnaire completion	SESBI-R	T-BRIEF	ECBI	P-BRIEF
Pre-	25 (100%)	25 (100%)	20 (80%	(80%)
Post-	23 (92%)	23 (92%)	15 (60%)	16 (64%)
Follow-up	23 (92%)	23 (92%)	15 (60%)	15 (60%)
Primary analysis	23 (92%)	23 (92%)	11 (44%)	12 (48%)
Sensitivity analysis	17 (68%)	17 (68%)	10 (40%)	11 (44%)

Outcomes

Mean scores and standard deviations for each assessment are shown in Table 3. Post-hoc ANOVA analysis results are presented in Table 4.

Table 3.

Change in children’s mean assessment scores across time-points.

Measures	Time-point
	Pre-	Post-	8-weeks follow-up
	M^a (SD)	M (SD)	M (SD)
Teacher
SESBI-R intensity raw score (n = 23)	147.78 (45.65)	134.87 (33.40)	119.83 (32.59)
Teacher BRIEF2 Screening Form raw score (n = 23)	25.35 (4.40)	21.83 (4.46)	23.22 (4.41)
Parent/caregiver
ECBI intensity raw score (n = 11)	133.91 (34.68)	108.55 (27.79)	100.55 (41.24)
Parent BRIEF2 Screening Form raw score (n = 12)	23 (6.12)	18.75 (4.33)	21.83 (6.73)

Decrease in mean scores indicates improvement.

Table 4.

ANOVA post-hoc values for child assessments.

Measures	MD^a	SE	p	95% CI
Teacher
SESBI-R Intensity raw score
Pre- to post	12.91	5.29	.069	–.79, 26.61
Post-to follow-up	15.04	3.77	.002	5.29, 24.80
Pre- to follow-up	27.96	5.87	<.001	12.70, 43.15
Teacher BRIEF2 screening form raw score
Pre- to post	3.52	.87	.002	1.28, 5.77
Post-to follow-up	–1.39	1.00	.538	–3.99, 1.21
Pre- to follow-up	2.13	.85	.062	–.08, 4.34
Parent/caregiver
ECBI Intensity raw score
Pre- to post	25.36	8.63	.044	.61, 50.12
Post-to follow-up	8.00	12.22	1.00	–27.07, 43.07
Pre- to follow-up	33.36	8.08	.006	10.16, 56.57
Parent BRIEF2 Screening Form raw score
Pre- to post	4.25	1.05	.006	1.28, 7.22
Post-to follow-up	−3.08	1.18	.073	–6.42, 0.25
Pre- to follow-up	1.17	1.42	1.00	–2.85, 5.18

Higher mean differences indicate greater improvement.

Teacher Sutter–Eyberg Child Behavior Inventory (SESBI-R)

SESBI-R Intensity scores decreased significantly over time with a large effect size following the Alert Program® intervention (F[1.64, 36.11] = 15.351, p < .001). Sensitivity analysis excluding children with data collection overlapping the intervention period confirmed robustness of primary analysis results (n = 17). This reduction in mean Intensity scores indicates a decrease in the frequency of children’s disruptive behavior as rated by teachers. Post-hoc primary and sensitivity analyses showed mean differences in children’s Intensity scores reduced significantly from: (1) pre-intervention to follow-up and (2) post-intervention to follow-up. Scores decreased between (3) pre- and post-intervention; however, the difference was not statistically significant (Table 4).

Of the 23 children with data for all three time-points, four children moved from above to below the SESBI-R clinical cut-off score post-intervention. This improvement was maintained by three of these children at follow-up. Additionally, three children moved from above to below the clinical cut-off score at post-intervention. Of the remaining 17 children, 15 did not move above or below the clinical cut-off score at any time-point.

Teacher Behavior Rating Inventory of Executive Function 2 (BRIEF2) Screening Form

Teacher ratings following the Alert Program® intervention showed a significant reduction in total raw scores as measured using the BRIEF2 Screening Form (F(2, 44) = 7.594, p < .001) with a large effect size. This reduction in mean scores indicates improvements to children’s executive functioning behaviors as rated by teachers. The sensitivity analysis also indicated a significant impact of the Alert Program® over time with a medium effect size (n = 17). Post-hoc primary testing revealed a statistically significant decrease in children’s mean total scores from: (1) pre- to post-intervention. There were no statistically significant changes from: (2) post-intervention to follow-up; or (3) pre-intervention to follow-up (Table 4). Unlike post-hoc primary analysis, post-hoc sensitivity analysis showed no significant changes at any time-point.

Of the 23 children who had data across all three time-points, six children moved to a less clinically elevated category on the teacher BRIEF2 Screening Form with three children maintaining the improvement at follow-up. Seventeen children did not move between the three BRIEF2 Screening Form clinically relevant categories.

Parent Eyberg Child Behavior Inventory (ECBI)

There was a significant reduction over the Alert Program® intervention period in children’s ECBI Intensity scores with a large effect size (F[2, 20] = 6.298, p = .008). This reduction in mean Intensity scores indicates improvements to children’s disruptive behavior as rated by parents/caregivers. Sensitivity analysis results echoed primary analysis findings (n = 10). Primary post-hoc testing revealed significant improvements to children’s ECBI Intensity scores from (1) pre- to post-intervention and (2) pre-intervention to follow-up. Whilst mean scores decreased from (3) post-intervention to follow-up, this finding was not statistically significant (Table 4). Post-hoc sensitivity analysis results revealed only significant improvements to children’s ECBI Intensity scores from pre-intervention to follow-up. The mean difference in pre-intervention to post-intervention scores following post-hoc sensitivity analysis was greater than post-hoc primary analysis; however, this change was not significant.

Of the 11 children who had data for all three time-points, six children moved from above to below the ECBI Intensity clinical cut-off score post-intervention. This improvement was maintained by four children at follow-up. Five children did not move above or below the clinical cut-off score.

Parent Behavior Rating Inventory of Executive Function (BRIEF2) screening form

Within subjects analysis showed that parent/caregivers ratings of children using the total raw scores of the BRIEF2 Screening Form decreased significantly over time following the Alert Program® intervention with a large effect size (F[2, 22] = 6.377, p = .007). Sensitivity analysis results also indicated significant improvements (n = 11). These decreases in scores indicate improvements to children’s executive functioning behaviors as rated by parents/caregivers. Both primary and sensitivity analysis post-hoc testing indicated that children scores decreased significantly from: (1) pre- to post-intervention, but not from (2) post-intervention to follow-up; or (3) pre-intervention to follow-up (Table 4).

Of the 12 children who had data for all three time-points, six children did not move between the three parent BRIEF2 Screening Form clinically relevant categories. Of the remaining six children, four improved at one or more time-points although only one maintained the improvement at follow-up.

Discussion

Previous Alert Program® research has shown improvements in behavior regulation and executive functioning among children with FASD in one-on-one and group therapy (Nash et al., 2014; Soh et al., 2015; Wells et al., 2012). However, it was unknown whether a curriculum-based approach to deliver the Alert Program® in remote Aboriginal community schools, where high rates of FASD are known to exist, would be feasible and effective. Following a year-long formative process conducted in collaboration with a locally based Aboriginal cultural health organization, community researchers, Elders, Aboriginal and Islander Education Officers and teachers, this pilot study found that teacher-led delivery of an Alert Program® curriculum is feasible in remote schools (Wagner et al., 2016). This curriculum may improve the self-regulation behaviors and executive functioning skills of children in a region where high rates of FASD (Fitzpatrick, Latimer, et al., 2017) and adverse childhood experiences (Zubrick et al., 2005b) may have contributed to impairment.

The frequency of disruptive behaviors (a proxy measure of self-regulation), as rated by teachers and parents/caregivers on the SESBI-R and ECBI Intensity scales, decreased at each successive time-point from pre-intervention to eight weeks follow-up. Teachers were encouraged to embed Alert Program® language and concepts in their teaching following the intervention period, and this may enable children to self-regulate at school outside the discrete one-hour lessons. For those for whom data were available, the reduction in disruptive behaviors reported by parents/caregivers suggests that children were also better able to self-regulate at home following the intervention. Primary analyses indicated that children’s executive functioning skills, as rated by teachers and parents/caregivers on the BRIEF2 Screening Forms, also improved significantly from pre- to post-intervention. Interestingly, unlike SESBI-R and ECBI results, BRIEF2 scores did not change significantly from pre-intervention to follow-up. It is important to note that post-hoc sensitivity analysis revealed no significant changes to children’s scores, as rated by teachers at any time-point.

Findings from this pilot study and concurrent formative work have informed study protocols for a larger Fitzroy Valley Alert Program® cluster randomized trial (Wagner et al., 2018). The strengths of the current pilot study include the high level of community and stakeholder input that followed re-establishment of partnerships from previous community-led studies (Lililwan FASD prevalence study and Marulu FASD Strategy). Local Aboriginal community researcher capacity was developed, with Aboriginal community researchers trained in the project and questionnaire methods. Non-Aboriginal researchers received guidance from Aboriginal Elders and community researchers to improve their cultural responsiveness. To enhance translation of findings and community capacity, Alert Program® curriculum guides were provided for teachers at the end of the project, enabling them to continue teaching the program to new incoming students. Whole-of-class program delivery was a strength, as it maximized the exposure of students to the intervention language and strategies in a resource-limited environment.

While results regarding improvement in disruptive behavior (and likely enhanced self-regulation) are encouraging, this pilot study had limitations. Blinding of researchers, teachers, and parents/caregivers to intervention delivery and timing was not possible due to the single group design. This may have biased teacher and/or parent results. This means that study results should be interpreted with caution. Also, pre- and post-intervention data collection overlapped with the first and/or last Alert Program® lesson for some children, although this was dealt with interpretation of findings via sensitivity analysis. Despite the high participation rate, the sample size was small, limiting control for confounders (age, gender, FASD diagnosis, intervention dose). Several of these limitations will be addressed in the subsequent self-controlled cluster randomized trial. The trial will be larger, involving eight schools with approximately 300 children, allowing mixed effect modelling of outcomes adjusted for other confounders.

Loss to follow-up is a common challenge when conducting research in remote community settings (Hunter & Smith, 2002), partly contributed to by the mobile nature of the population and burdens of data collection on participants. Despite consultation with local stakeholders regarding the choice of outcomes, completing multiple questionnaires at three time-points for each child enrolled in the study was a challenge for teachers and caregivers alike. Parents/caregivers also required community researchers to translate questionnaires into Kimberley Kriol. This is because English is a second (or in many cases third) language for most residents in this region, further adding to time demands. Community and stakeholder feedback has informed the revised assessment battery selected for the main trial.

There may have been variability in the way teachers continued to embed Alert Program® concepts within the classroom after the intervention concluded. This variability is inherent to Stage Three of the program as children become more independent in their use of program language and strategies. Extending formal Alert Program® lessons beyond an eight-week curriculum was not feasible as lessons needed to fit within one 10-week school term. The pilot study timeline was necessary, as it provided a week-long buffer for teacher training, child assessments, and post-intervention assessments to take place both before and after the intervention period. Whilst a research limitation, the varied teacher implementation of Stage Three reflects real-world program delivery as children master self-awareness of their arousal state, program language, and use of self-regulatory strategies (Williams & Shellenberger, 1996a), and teachers continue to support their students.

Conclusions

This research clearly aligns with community-led strategies to overcome the impact of FASD and ELT and to further develop research capacity amongst Aboriginal community members. Of great benefit, Aboriginal community researchers developed non-Aboriginal researchers’ capacity to work in culturally secure ways by sharing their cultural, language, and contextual expertise. This pilot project informed research protocols and processes for a larger self-controlled cluster randomized trial and provides early results that suggest that the Alert Program® may be useful in improving children’s self-regulation and, possibly, executive functioning abilities. Whilst limitations of the pilot study suggest caution when interpreting study results, this research does provide evidence that a classroom-based Alert Program® intervention is feasible in remote Australian Aboriginal community schools when local Aboriginal community members are valued and involved as research partners.

Footnotes

Acknowledgements

The authors thank the students, families, school staff and community who were involved in and supported this pilot study. We acknowledge members of the Alert Program® Study team who contributed to this work; Avinna Trzesinski, Professor Karen Edmond, Glenn Pearson, Professor Branko Celler, Isabelle Palmer and Katrina Shaw. Additionally, we thank Doctor Therese Shaw, Emma Adams and the Telethon Kids Institute Biostatistics team for their statistical advice. Thanks to Kristen Campbell, Alison Walker, Andrew Waters and Kato Matthews (Kimberley Population Health Unit); Jo Fox, Adriaan Wolvaardt, Bev Vickers (Department of Education, Western Australia); Julie Munro (occupational therapy consultant) and Sue Thomas (Marninwarntikura Women’s Resource Centre, Marulu FASD Unit) for their support and project advice. We thank our international colleagues for their input: Mary Sue Williams, Sherry Shellenberger, Molly McEwen (Therapy Works Inc.); Tammy Favreau and Mary Markesteyn (Lois Riel School Division, Canada). The authors also acknowledge Nindilingarri Cultural Health Service, Marninwarntikura Fitzroy Women’s Resource Centre and WA Health for their project support.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: National Health and Medical Research Council project grant (1086145); an Australian Government Research Training Program Scholarship (BW); National Health and Medical Research Council Research Fellowship (GNT 1119339; DC).

Notes

References

Australian Medical Association. (2016). Fetal alcohol spectrum disorder position statement. Kingston: Australian Medical Association. Retrieved from https://ama.com.au/position-statement/fetal-alcohol-spectrum-disorder-fasd-2016

Barnes

K. J.

Vogel

K. A.

Beck

A. J.

Schoenfeld

H. B.

Owen

S. V.

(2008). Self-regulation strategies of children with emotional disturbance. Physical & Occupational Therapy in Pediatrics, 28, 369–387.

Bertrand

(2009). Interventions for children with fetal alcohol spectrum disorders (FASDs): Overview of findings for five innovative research projects. Research in Developmental Disabilities, 30, 986–1006.

Best

J. R.

Miller

P. H.

(2010). A developmental perspective on executive function. Child Development, 81, 1641–1660.

Blair

Raver

C. C.

(2015). School readiness and self-regulation: A developmental psychobiological approach. Annual Review of Psychology, 66, 711–731.

Blanca

M. J.

Alarcon

Arnau

Bono

Bendayan

(2017). Non-normal data: Is ANOVA still a valid option? Psicothema, 29, 552.

Bower

Elliott

E. J.

Zimmet

Doorey

Wilkins

Russell

, … Watkins

(2017). Australian guide to the diagnosis of foetal alcohol spectrum disorder: a summary. Journal of Paediatrics and Child Health, 53, 1021–1023.

Bridgett

D. J.

Oddi

K. B.

Laake

L. M.

Murdock

K. W.

Bachmann

M. N.

(2013). Integrating and differentiating aspects of self-regulation: Effortful control, executive functioning, and links to negative affectivity. Emotion, 13, 47–63.

Carlson

Frazer

(2015). It’s like going to a cemetery and lighting a candle: Aboriginal Australians, sorry business and social media. AlterNative: An International Journal of Indigenous Peoples, 11, 211–224.

10.

Coffin

(2007). Rising to the challenge in Aboriginal health by creating cultural security. Aboriginal and Islander Health Worker Journal, 31, 22–24.

11.

Coffin

Green

(2017). Decolonising Australian community development tools. In Kickett-Tucker

, Bessarab

, Coffin

Wright

(Eds.), Mia Mia Aboriginal community development: Fostering cultural security. Cambridge: Cambridge University Press.

12.

Cohen

(1988). Statistical power analysis for the behavioral sciences ED. 2nd ed. Pennsylvania: Lawrence Erlbaum Associates.

13.

Department of Education Services (2012). Report on the rural and remote education advisory council regional meeting in Broome. Retrieved from http://rreac.des.wa.gov.au/SiteCollectionDocuments/Report%20Regional%20Meeting%20Broome%20-%20Rural%20and%20Remote%20Education%20Advisory%20Council.pdf

14.

Diamond

(2016). Why improving and assessing executive functions in early life is critical. In Griffin

J.P

. McCardle

. Freund (Eds.), Executive function in pre-school aged children: Integrating measurement, neurodevelopment, and translational research. (pp. 11–43). Washington, DC: American Psychological Association.

15.

Eisenberg

Spinrad

T. L.

Eggum

N. D.

(2010). Emotion-related self-regulation and its relation to children’s maladjustment. Annual Review of Clinical Psychology, 6, 495–525.

16.

Evans

G. W.

Kim

(2013). Childhood poverty, chronic stress, self-regulation, and coping. Child Development Perspectives, 7, 43–48.

17.

Eyberg

Pincus

(1999). Eyberg child behavior inventory and Sutter-Eyberg student behavior inventory revised: Professional manual. Lutz, FL: Psychological Assessment Resources Inc.

18.

Fitzpatrick

J. P.

Latimer

Olson

H. C.

Carter

Oscar

Lucas

B. R.

, … Elliott

E. J.

(2017) Prevalence and profile of neurodevelopment and fetal alcohol spectrum disorder (FASD) amongst Australian Aboriginal children living in remote communities. Research in Developmental Disabilities, 65, 114–126.

19.

Fitzpatrick

J. P.

Oscar

Carter

Elliott

E. J.

Latimer

Wright

Boulton

(2017) The Marulu Strategy 2008–2012: Overcoming Fetal Alcohol Spectrum Disorder (FASD) in the Fitzroy Valley. Australian and New Zealand Journal of Public Health, 41, 467–473.

20.

Gioia

G. A.

Isquith

P. K.

Guy

S. C.

Kenworthy

(2015). BRIEF2: Behavior Rating Inventory of Executive Function professional manual. Lutz, FL: Psychological Assessment Resources Inc.

21.

Gooda

(2010). 2010 Aboriginal and Torres Strait Islander Social Justice Commissioner Social Justice Report. Retrieved from www.humanrights.gov.au/publications/social-justice-report-2010

22.

Hunter

B. H.

Smith

D. E.

(2002). Surveying mobile populations: Lessons from recent longitudinal surveys of Indigenous Australians. Australian Economic Review, 35, 261–275.

23.

Kickett-Tucker

Hansen

(2017). Ngalang moort: family as the building block of community development. In Kickett-Tucker

, Bessarab

, Coffin

Wright

(Eds.), Mia Mia Aboriginal community development: Fostering cultural security. Cambridge: Cambridge University Press.

24.

Kodituwakku

P. W.

(2009). Neurocognitive profile in children with fetal alcohol spectrum disorders. Developmental Disabilities Research Reviews, 15(3), 218–224.

25.

Kodituwakku

P. W.

Handmaker

N. S.

Cutler

S. K.

Weathersby

E. K.

Handmaker

S. D.

(1995). Specific impairments in self-regulation in children exposed to alcohol prenatally. Alcoholism: Clinical and Experimental Research, 19, 1558–1564.

26.

Latimer

Elliott

E. J.

Carter

Oscar

Fitzpatrick

J. P.

Ferreira

Kefford

(2010). The George Institute for International Health, Sydney. Marulu, the Lililwan project, fetal alcohol spectrum disorders (FASD) prevalence study in the Fitzroy Valley: A community consultation.

27.

Mac Cobb

Fitzgerald

Lanigan-O’Keeffe

(2014). The Alert Program for self-management of behaviour in second level schools: Results of phase 1 of a pilot study. Emotional & Behavioural Difficulties, 19, 410–425.

28.

Mc Loughlin

Hadgraft

N. T.

Atkinson

Marley

J. V.

(2014). Aboriginal health research in the remote Kimberley: An exploration of perceptions, attitudes and concerns of stakeholders. BMC Health Services Research, 14, 517.

29.

Morphy

(2010). Population, people and place: The Fitzroy Valley population project. In CAEPR WORKING PAPER No. 70/2010. Canberra: The Australian National University.

30.

Nash

Stevens

Greenbaum

Weiner

Koren

Rovet

(2014). Improving executive functioning in children with fetal alcohol spectrum disorders. Child Neuropsychology, 21, 191–209.

31.

Schonfeld

A. M.

Paley

Frankel

O'Connor

M. J.

(2006). Executive functioning predicts social skills following prenatal alcohol exposure. Child Neuropsychology, 12, 439–452.

32.

Shonkoff

J. P.

Garner

A. S.

(2012). The lifelong effects of early childhood adversity and toxic stress. Pediatrics, 123, e233–e246.

33.

Soh

D. W.

Skocic

Nash

Stevens

Turner

G. R.

Rovet

(2015). Self-regulation therapy increases frontal gray matter in children with fetal alcohol spectrum disorder: Evaluation by voxel-based morphometry. Frontiers in Human Neuroscience 9: 1–12.

34.

Wagner

Fitzpatrick

Symons

Jirikowic

Cross

Latimer

(2016). The development of a culturally appropriate school based intervention for Australian Aboriginal children living in remote communities: A formative evaluation of the Alert Program® intervention. Australian Occupational Therapy Journal, 64(3), 1–10.

35.

Wagner

Fitzpatrick

J. P.

Mazzucchelli

T. G.

Symons

Carmichael Olson

Jirikowic

, … Latimer

(2018). Study protocol for a self-controlled cluster randomised trial of the Alert Program to improve self-regulation and executive function in Australian Aboriginal children with fetal alcohol spectrum disorder. BMJ Open, 8, 1–11.

36.

Wells

A. M.

Chasnoff

I. J.

Schmidt

C. A.

Telford

Schwartz

L. D.

(2012). Neurocognitive habilitation therapy for children with fetal alcohol spectrum disorders: An adaptation of the Alert Program®. American Journal of Occupational Therapy, 66, 24–34.

37.

Weston

Thomas

(2014). Understanding and addressing the needs of children and young people living with fetal alcohol spectrum disorders (FASD): A resource for teachers. Abbotsford: National Curriculum Services on behalf of Kimberley Success Zones.

38.

Wilkinson

Beattie

(2006). Enhancing allied health services to rural and remote Indigenous communities: Phase 1. Retrieved from http://www.wacountry.health.wa.gov.au/fileadmin/sections/allied_health/WACHS_R_EnhancingAHSToRuralRemoteCommunities.pdf

39.

Williams

M. S.

Shellenberger

(1996a). “How does your engine run?” A leader’s guide to the Alert Program for self-regulation. Albuquerque, NM: TherapyWorks, Inc.

40.

Williams

M. S.

Shellenberger

(1996b). An introduction to “How does your engine run?”: The Alert Program for self-regulation. Albuquerque, NM: Therapy Works Inc.

41.

Zubrick

Silburn

Lawrence

Mitrou

Dalby

Blair

, …, Lix

(2005a). The social and emotional wellbeing of children and young people: Malarabah (Derby) ATSIC region. Summary of findings from volume two of the Western Australian Aboriginal Child Health Survey. Retrieved from www.telethonkids.org.au/our-research/aboriginal-health/waachs/waachs-regional-profiles/

42.

Zubrick

Silburn

Lawrence

Mitrou

Dalby

Blair

,… Lix

(2005b). Western Australian Aboriginal child health survey: The emotional and behavioural health of Aboriginal children and young people. Vol.2. Perth: Curtin University of Technology and Telethon Institute for Child Health Research.