Cross-Site Process Evaluation Results for the Early Childhood Education Center Setting: CORD Study

Abstract

Background:

The Childhood Obesity Research Demonstration project aimed to deliver evidence-based obesity prevention interventions to at-risk families at three demonstration sites. The interventions were delivered in multiple settings, including early childhood education centers (ECECs), public schools, and primary care clinics. An evaluation center conducted cross-site process, impact, and sustainability evaluations. Results of the cross-site process evaluation for the ECECs will be described.

Methods:

Reach (proportion of the target population who participated), dose delivered (materials and interventions that were distributed), and fidelity (proportion of planned intervention components delivered) were assessed at two levels (researcher-to-provider and provider-to-family levels). Standardized data forms were completed by research team members at each demonstration site with assistance from the evaluation center.

Results:

The Childhood Obesity Research Demonstration project reached 5174 children and 390 teachers in 58 ECECs. The centers delivered an average of 3.9 hours of training to teachers. A total of 1382 different types of materials were distributed to providers, and from 1.3 to 4.3 hours of technical support were delivered to centers monthly. For fidelity at the researcher-to-provider level, 49.5% (n = 370) of eligible teachers completed all training sessions. Considerable variations across demonstration sites in reach, dose delivered, and fidelity across were observed.

Conclusion:

The Childhood Obesity Research Demonstration project reached large numbers of children, families, teachers, and ECECs. Maintaining intervention fidelity while reaching large numbers of at-risk individuals proved to be a challenge.

Introduction

The Childhood Obesity Research Demonstration (CORD) project was a cooperative agreement funded by the Centers for Disease Control and Prevention (CDC) to deliver evidence-based interventions to at-risk families based on the Obesity Chronic Care Model, which suggests that sustainable weight control could be achieved through multisector environmental and policy changes.¹ The primary and secondary prevention interventions supported efforts to improve child diet, physical activity, and screen time through environment, systems, and policy changes.

Different interventions were coordinated and delivered by demonstration project teams from San Diego State University (CA-CORD),² Massachusetts State Department of Health (MA-CORD),³ and the University of Texas Health Science Center (UTHealth) School of Public Health (TX-CORD).⁴ The evaluation center (EC-CORD) was located at the University of Houston and led cross-site impact, sustainability, and process evaluations.⁵ An advisory workgroup from the CDC participated in all phases of the project.

CORD programming was delivered in the health care, school, community, and early childhood education center (ECEC) sectors. Results reported thus far show that the CORD programs were successful in reducing body–mass index (BMI) percentiles among preschoolers from some of the most at-risk segments of the population.^6–8 Compared with standard care, one intervention community from MA-CORD found preschoolers recruited from the Women, Infant, and Children (WIC) Supplemental Food Program and pediatric clinics with significantly reduced BMI z-score units (0.08–0.16 units/year), while there was no change in the other intervention community or the community assigned to standard care.^8,9 In TX-CORD, preschoolers in the intervention community had a significantly greater reduction in the BMI z-score (0.26 units) compared with the control community.⁶ Process evaluations help to describe the extent to which interventions were delivered as intended and reached the target audience.^10,11 Reports of this nature may help others design and evaluate similar types of interventions. This article describes the cross-site process evaluation results for the ECEC setting of the CORD project.

Methods

At all sites, ECECs were eligible to participate in the CORD project if they were a licensed child care facility in the targeted community. There were additional criteria that varied by site. For example, CA-CORD excluded ECECs that catered to a special needs population. Across all sites, healthy children, ages two through 12, who lived in the targeted communities were eligible to participate in the CORD project. Some sites had additional restrictions based on income.

At the CA-CORD site, a program, activities, and supplies were delivered to ECECs.² The Sports, Play, and Active Recreation for Kids (SPARK)^© program (including SPARK 101 and the Summer Institute), an evidence-based physical activity curriculum for trained teachers to deliver in preschools, was implemented.¹² Additionally, CA-CORD provided teachers and staff with training and resources to assist teachers in infusing their curriculum with health behavior lessons. CA-CORD also trained ECEC staff to measure and provide parent feedback on the BMI of their students. Last, CA-CORD provided ECECs with physical activity equipment, water dispensers, posters, and a cooking kit.

At the MA-CORD site, the Nutrition and Physical Activity Self-Assessment for Child Care (NAPSACC) program and I am Moving, I am Learning (IMIL) were the evidence-based programs delivered in MA ECECs.¹³ NAPSACC is a self-assessment and goal planning intervention for ECEC administrators and teachers to complete to assess policies, practices, and environmental qualities related to nutrition and physical activity. After self-assessment and goal setting concluded, two additional days of teacher and staff training on nutrition and physical activity were offered. IMIL aims to increase moderate to vigorous physical activity and healthy dietary habits by training staff and teachers to integrate more fun nutrition and physical activities in the classroom.¹⁴

At the TX-CORD, the Coordinated Approach to Child Health (CATCH) Early Childhood (EC) intervention was delivered in ECECs.⁴ In this evidence-based program, ECEC teachers were trained to deliver the intervention to children and families. CATCH EC includes a classroom nutrition and gardening curriculum, developmentally appropriate activities to increase time spent in moderate to vigorous physical activity, and bilingual parent tip sheets. A coordination kit was developed as part of the CORD program to align activities across the program. The CATCH EC components include parent handouts, physical activities, interactive nutrition and gardening lessons, and a coordination kit to coordinate the program activities across sectors (e.g., food service and classroom) of the ECEC.

The cross-site process evaluation plan has been described in detail elsewhere,¹⁵ but will be briefly summarized here. The goal was to describe intervention implementation across all demonstration sites in the first year of the project. However, demonstration projects implemented different interventions. In the initial phase, team members from the EC-CORD, demonstration sites, and CDC collaboratively developed the process evaluation plan. Only those constructs that could be assessed in a standardized manner across the sites in year 1 were included in the plan.

The process evaluation plan, based on Steckler and Linnan's model, included assessment of reach (proportion of eligible members of the target population who participated), dose delivered (materials and activities that were delivered), and fidelity (proportion of CORD components delivered as planned) at two levels (researcher-to-provider and provider-to-family levels).¹⁶ Activities that were delivered by the research team to providers (e.g., teachers) were considered to occur at the researcher-to-provider level, and those activities delivered by providers to children and families (e.g., nutrition lessons delivered by teachers in ECEC classes) were considered to occur at the provider-to-family level.

To summarize findings across all sites, it was critical that the data collection procedures be standardized. Research staff from the demonstration sites collected the data. Data were collected in accordance with ethical standards established by the Helsinki Declaration of 1975 (revised in 2008). Institutional review boards from each demonstration site (Massachusetts Department of Public Health Institutional Review Board, Institutional Review Board for Baylor College of Medicine and Affiliated Hospitals, and San Diego State University Institutional Review Board) and the EC (University of Houston Institutional Review Board) approved the protocols for this study. All human subjects provided assent and/or consent before enrollment. The EC-CORD created data reporting forms and assisted the sites in completing them as needed. Data were collected from the sites during their first intervention delivery wave between October 2011 and September 2014. Data collection captured start-up activities, such as training and material development. Data collection and program implementation dates were flexible due to differing schedules at each site.

The final process evaluation data collected from each site represented reach, dose delivered, and fidelity at the researcher-to-provider and provider-to-family levels.¹⁵ Reach at the researcher-to-provider level was the total number of ECECs that participated in the CORD project, the percentage of eligible ECECs that participated in the CORD project, the number of teachers who completed any CORD training for programs delivered in each ECEC, and the percentage of eligible teachers who completed any training for programs delivered in each ECEC. Reach at the provider-to-family level was the total number of children enrolled in ECECs that received CORD programming. The dose delivered variable at the researcher-to-provider level was assessed by the weighted average (weighted by number of ECECs per demonstration site) of hours of training provided by researchers to teachers for each program delivered in the ECECs, the number of materials distributed to providers, and the average number of hours of additional support given to each ECEC by the researchers (e.g., answering questions by email or phone). Dose delivered at the provider-to-family level was not assessed because there were no data (on this) collected consistently across all sites. Last, fidelity at the researcher-to-provider level was the percentage of eligible teachers who completed all required training sessions for each program. Fidelity at the provider-to-family level was a subjective rating on a scale from 0% to 100% indicating the extent to which researchers felt their CORD programs were delivered as intended.

Investigators from the demonstration sites described barriers and facilitators to intervention delivery experienced in the ECE setting at both the researcher-to-provider and provider-to-family levels. These barriers and facilitators were identified by personal observation and experience of the investigators.

Results

Process evaluation results are presented below at the researcher-to-provider and provider-to-family levels for reach, dose delivered, and fidelity (Tables 1 and 2).

Table 1.

Obesity Prevention Program Reach and Dose Delivered in Early Childhood Education Centers at the Researcher-to-Provider (R = >P) and Provider-to-Family (P = >F) Levels of the Childhood Obesity Research Demonstration Project

Variable	Definition	MA-CORD	CA-CORD	TX-CORD
Reach: P = >F	Number of children	1456	1302	2416
Reach: R = >P	Number of ECECs (% eligible)	9 (75%)	13 (100%)	26 (60.5%)
DD: R = >P	Number of CORD materials provided to ECECs in year 1	9	317	1056
DD: R = >P	Number of hours of extra support provided to ECECs in year 1	Not available	41.2	1.4

CORD, Childhood Obesity Research Demonstration; DD, dose delivered; ECECs, early childhood education centers.

Table 2.

Reach, Dose Delivered, and Fidelity at the Researcher-to-Provider (R = >P) and Provider-to-Family (P = >F) Levels of Obesity Prevention Programs Delivered in Early Childhood Education Centers of the Childhood Obesity Research Demonstration Project

		Interventions delivered in MA ECECs		Interventions delivered in CA ECECs		Interventions delivered in TX ECECs
		NAPSACC	IMIL	SPARK^a	Healthy Curriculum training	CATCH EC
		Self-assessment and goal planning activity for ECEC administrators and teachers plus 2 days of training on best practice recommendations for ECECs	Physical activity and nutrition curriculum delivered by trained teachers	Physical activity curriculum delivered by trained teachers	Health behavior lessons for trained teachers to supplement their regular instruction	Physical activity and nutrition curriculum delivered by trained teachers
Reach: R = >P	% of eligible staff who received at least one training	86.7% (196^b/226)	37.9% (86^b/226)	59.1% (68/115)	100% (13/≥13^c)	83.2% (139/167)
Reach: R = >P	% of eligible staff who received at least one training	86.7% (196/226)		59.1% (68/115)		83.2% (139/167)
DD: P = >F	Not collected	N/A	N/A	N/A	N/A	N/A
DD: R = >P	Number of hours of training offered by each ECEC per program	2.2	3.1	1	1	3.7
	Number of hours of training offered by each ECEC	5.3		2		3.7
Fidelity: R = >P	% of eligible staff who received complete training	86.7% (196/226)	37.2% (84/226)	18.3% (21/115)	69.2% (9/≥13)	35.9% (60/167)
	% of eligible staff who received full training on at least one program	62.0%^d		43.8%^e		35.9%
Fidelity: P = >F	Subjective rating (0–100)	N/A^f	0	38.5	32.7	47.9
Fidelity: P = >F	Subjective rating (0–100)	0		35.6		47.9

Includes SPARK, SPARK 101, and SPARK Summer Institute.

The ECEC teachers in MA had training sessions on IMIL and NAPSACC. Some teachers received training on both and some received training on only one.

At least one teacher per ECEC was expected to receive the Healthy Curriculum training.

86.7% (for NAPSACC) +37.2% (for IMIL)/2 = 62.0%.

18.3% (for SPARK) +69.2% (for Healthy Curriculum) = 43.8%.

Not tracked.

CATCH EC, Coordinated Approach to Child Health Early Childhood; IMIL, I am Moving, I am Learning; NAPSACC, Nutrition and Physical Activity Self-Assessment for Child Care; SPARK, Sports, Play, and Active Recreation for Kids.

Reach

Researcher-to-provider level

Forty-eight ECECs participated in the CORD project, representing 72.7% of all eligible ECECs in each demonstration site community (Table 1). The number of ECECs participating in the CORD project at each demonstration site ranged from 9 to 26, reflecting the relative sizes of the respective municipalities in which the demonstration projects occurred. The percent of eligible ECECs participating in CORD at each demonstration site ranged from 60.5% to 100%. The number of eligible teachers who were reached from each site ranged from at least 115 to 226 (Table 2). Across all sites and ECECs, at least 403 (196 staff from MA +68 staff from CA +139 staff from TX) teachers of 508 eligible teachers (79.3%) completed some level of training on at least one CORD program (Table 2).

Provider-to-family level

Child enrollment at participating ECECs was 5174 and this ranged from 1302 to 2416 across the sites (Table 1).

Dose Delivered

Researcher-to-provider level

A total of 171.0 hours of training was provided to teachers to deliver five different programs by all 48 ECECs (data not shown). This represents a weighted average of 3.5 hours of training delivered by each ECEC, which varied between demonstration sites (2.0–5.3 hours per ECEC). The number of hours of training delivered by each ECEC varied by program as well (1–3.7 hours per program per ECEC). A total of 1382 unique materials were given to ECECs and teachers. The number of materials provided to ECECs varied substantially from 9 to 1056 as well as the number of hours of additional support provided to ECECs per month (1.4–41.2 hours, Table 1).

Provider-to-family level

Not applicable; data were not collected consistently across all sites.

Fidelity

Researcher-to-provider level

On average, 49.5% of eligible teachers were completely trained on each of the CORD programs (i.e., NAPSACC, IMIL, SPARK, Healthy Curriculum, and CATCH EC) across sites (Table 2). In addition, an average of 43.1% of eligible teachers completed all training requirements across demonstration sites, but this ranged from 35.9% to 62.0%.

Provider-to-family level

The weighted average fidelity rating of CORD programs delivered in ECECs was 35.6% (Table 2). The fidelity ratings for each of the five programs delivered in ECECs ranged from zero (program not delivered) to 47.9%. Fidelity was not tracked for one of the programs (NAPSACC MA).

Facilitators and Barriers

Investigators identified important facilitators or barriers to reaching the target population and delivering the intended programs with fidelity. The sites were better able to achieve their goals when there were existing federal and state programs that they could leverage. For example, in CA-CORD, the public health department provided teacher training. MA-CORD relied on the MA Children at Play (MCAP) collaboration between the MA Department of Public Health, MA Department of Early Education and Care, and the MA Department of Elementary and Secondary Education to support the ECECs as they implemented CORD programs. Specifically, MCAP provided MA-CORD ECECs with referrals for mentors, workshops, resources, and structured activities to facilitate program implementation. While these partnerships helped to facilitate the goals of the CORD project with fewer resources, there were also challenges posed by this approach. Relying on staff from external agencies reduced accountability and commitment to CORD in some cases. For example, in MA-CORD, MCAP staffing issues and communication challenges among mentors led to reduced fidelity. When community partnerships were synergistic, there was greater success. For example, all sites included Head Start Centers and these federally funded ECECs are required to meet nutrition and physical activity performance standards to maintain accreditation and funding. Because CORD participation helped Head Start Centers meet these performance standards, motivation to participate and adherence among Head Start Centers were high.

Unforeseen events posed unique challenges to some of the sites. Inclement weather affected teacher attendance at training sessions, and the CORD program start date occurred in the summer/early fall when ECEC staff turnover is greatest.^9,18 The federal government sequestration of 2012–2014 posed a significant barrier to implementation because many of the eligible ECECs were Head Start Centers, which are federally funded ECEC programs.¹⁹ During sequestration, Head Start Centers in Texas experienced disruptions in enrollment and programming to comply with reduced budgets.

In addition to contextual factors of the outer context, the CORD investigators found modifiable strategies that enhanced fidelity and reach. Effective teacher training was necessary for adherence and fidelity. TX-CORD found that teachers significantly improved their knowledge of CATCH from pre- to post-training sessions.¹⁹ CA-CORD found that teacher training sessions offered during work hours were better attended. CA-CORD also found better adherence for activities that carried a low teacher burden. Strong and frequent communication was critical for promoting adherence and fidelity. In CA-CORD, conducting frequent in-person meetings with ECECs was a good strategy for maintaining open communication. TX-CORD developed a coordination kit that was seminal in aligning the programs within centers across classrooms. Prior experience with CORD programs (e.g., SPARK) by ECECs in CORD was another facilitator of fidelity. Some of the ECECs had long-standing relationships with the investigative teams from each site as these ECECs were part of earlier studies to develop and test the programs used in CORD. For all sites, lack of universal institutional leadership and staff buy-in and staff turnover were significant barriers to adherence and fidelity. Strong and frequent communication and effective training may have reduced the impact of these barriers.

Discussion

Variations in intervention implementation can greatly influence effectiveness. For example, in a large study of the NAPSACC program, a significant treatment effect on nutrition practices was shown only among ECECs that had high fidelity to the protocol, but not the whole sample.²¹ The results of the CORD process evaluation for the ECEC setting described how CORD programs were delivered in ECECs and some of the strengths, weaknesses, facilitators, and barriers associated with delivering diverse interventions across multiple ECECs.

The most striking strength of CORD was the ability of each demonstration site to reach relatively large numbers of at-risk children, their families, and ECECs in catchment areas compared with most studies of EC obesity prevention programs. Most eligible ECECs participated in the CORD project, suggesting that there may be general interest in adopting health policies and delivering health behavior interventions among ECEC administrators. Public health initiatives to extend child obesity prevention strategies into ECECs may be leading to increased awareness among ECEC administrators and staff of their unique role in promoting health behaviors.²⁰ This was particularly evident among Head Start Centers, which have nutrition and physical activity performance standards that they must meet. CORD participation helped them to meet these standards, thereby increasing interest in participating in the CORD program.

All sites trained teachers to deliver at least one evidence-based nutrition and physical activity intervention. The number of hours of teacher training delivered by each ECEC and the number of hours of support provided to ECECs each month varied somewhat across sites. The number of CORD educational materials provided to ECECs and percent of eligible staff who received training varied substantially across demonstration sites. Variations in training and support provided to ECECs are likely due to differences in the interventions delivered in the ECECs, qualities of the ECECs, and goals of the demonstration sites. Furthermore, core competencies regarding nutrition and physical activity for ECEC teachers differ by state.^22–24 Teachers of ECECs in states that require teachers to meet several nutrition and physical activity core competencies may need less training and resources than ECEC teachers in states with few or no nutrition and physical activity core competency requirements. In addition, primary obesity prevention performance standards for ECECs do not generally recommend minimum dosages of teacher training, intervention programming, or support resources required to improve health behaviors among preschoolers.²⁵ Without national recommendations, there will be great variability in how primary obesity prevention programs are delivered in ECECs.

The challenges of delivering these interventions in ECECs to so many children were reflected in variable fidelity at both the researcher-to-provider and provider-to-family levels within and between sites. Fidelity of ECEC interventions in CORD ranged from low to moderate across demonstration sites. On average, 43.6% of the eligible teachers received full training on the CORD interventions. These results are comparable with other large-scale, ECEC intervention studies. A large, multicountry, ECEC nutrition intervention study reported low to moderate fidelity based on teacher surveys.²⁶ In a physical activity intervention delivered across 10 ECECs, fidelity was reported to be moderate (67%) based on observations.²⁷ Those studies that had high fidelity reached few ECECs, teachers, and children; delivered one standard intervention across all ECECs; and provided greater oversight of teacher facilitators.^28,29

Facilitators of CORD activities included ECEC stakeholder buy-in, ability to leverage existing resources, use of activities that produced low burden on staff and teachers, strong teacher training, and frequent and good communication between ECECs and investigative teams. These are consistent with frequently cited facilitators of health promotion programs in ECECs.^30–32 The barriers to delivering CORD activities as planned were similar to those experienced among other obesity prevention programs delivered in ECECs and these included staff time constraints, staff turnover, and lack of buy-in from the leadership, institution, and/or staff.^31,32 Other barriers experienced by CORD were unique to time (e.g., government sequestration) and region (e.g., inclement weather).

Strengths of this process evaluation study include a large sample, standardized data collection procedures and measures, and an examination of process variables at both the researcher-to-provider and provider-to-family levels. However, results of this process evaluation are limited by the fact that the types of interventions and numbers of interventions differed across sites. The constructs and definitions we evaluated may be a bit unconventional from traditional implementation science (e.g., do not follow the RE-AIM framework), but this process evaluation was restricted to those process measures that were able to be collected at all the sites in the first year of the CORD project.

Conclusions

The CORD project successfully delivered evidence-based nutrition and physical activity programs to many children in ECECs in three different regions of the United States, suggesting interest among ECECs in nutrition and physical activity programs. Adherence to intervention protocols was supported by teacher training, open and frequent communication between investigators and ECEC staff, and contextual factors of the macroenvironment (e.g., state policies, programs, and social networks). Breakdowns in communication, lack of buy-in, and situational factors reduced fidelity to protocols and dose received. These results demonstrate the need for implementation science to understand factors such as readiness to implement programs in ECEC settings and the predictors of implementation success. National recommendations coupled with work on whether the recommendations are being implemented and strategies to support implementation, including accountability and training, are needed. This study may serve as a model for how to use an external evaluation center to conduct process evaluations of multisite, multisector obesity prevention interventions in ECECs.

Disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the CDC.

Footnotes

Funding Information

This research was supported, in part, by cooperative agreement RFA-DP-11-007 (U18DP003350) from the Centers for Disease Control and Prevention.

Author Disclosure Statement

T.L. and D.T. have no competing financial interests. T.O'C. is a consultant on a U01 from NIMHD: U01 MD010667, PI R Lee, “Partnering for PA in Early Childhood: Sustainability via Active Garden Education (SAGE).” T.O'C. has received sponsored travel to conferences by the AAP (American Academy of Pediatrics) and NCCOR (National Collaborative on Childhood Obesity Research).

D.A. and C.K. have no competing financial interests. D.P.O'C. is a consultant for Nimbic, Inc., Fondren Orthopedic Research Institute, and Snorelax. His work for these companies is completely unrelated to this study.

CA team: S.-F.L. and A.B.-V. have no competing financial interests; MA team: R.E.B. has no competing financial interests; and TX team: S.S. and D.M.H. have no competing financial interests.

References

Foltz

, Belay

, Dooyema

, et al. Childhood Obesity Research Demonstration (CORD): The cross-site overview and opportunities for interventions addressing obesity community-wide. Child Obes, 2015; 11:4–10.

Ayala

, Ibarra

, Binggeli-Vallarta

, et al. Our Choice/Nuestra Opcion: The Imperial County, California, Childhood Obesity Research Demonstration study (CA-CORD). Child Obes, 2015; 11:37–47.

Taveras

, Blaine

, Davison

, et al. Design of the Massachusetts Childhood Obesity Research Demonstration (MA-CORD) study. Child Obes, 2015; 11:11–22.

Hoelscher

, Butte

, Barlow

, et al. Incorporating primary and secondary prevention approaches to address childhood obesity prevention and treatment in a low-income, ethnically diverse population: Study design and demographic data from the Texas Childhood Obesity Research Demonstration (TX CORD) study. Child Obes, 2015; 11:71–91.

Layne

, Parker

, Soltero

, et al. Are physical activity studies in Hispanics meeting reporting guidelines for continuous monitoring technology? A systematic review. BMC Public Health, 2015; 15:917.

Sharma

, Vandewater

, Chuang

, et al. Impact of the Coordinated Approach to Child Health Early Childhood program for obesity prevention among preschool children: The Texas Childhood Obesity Research Demonstration study. Child Obes, 2019; 15:1–13.

Nelson

, Colchamiro

, Perkins

, et al. Racial/ethnic differences in the effectiveness of a multisector childhood obesity prevention intervention. Am J Public Health, 2018; 108:1200–1206.

Taveras

, Perkins

, Anand

, et al. Clinical effectiveness of the Massachusetts Childhood Obesity Research Demonstration initiative among low-income children. Obesity (Silver Spring), 2017; 25:1159–1166.

Woo Baidal

, Nelson

, Perkins

, et al. Childhood obesity prevention in the women, infants, and children program: Outcomes of the MA-CORD study. Obesity (Silver Spring), 2017; 25:1167–1174.

10.

Pettigrew

, Borys

, du Plessis

, et al. Process evaluation outcomes from a global child obesity prevention intervention. BMC Public Health, 2014; 14:757.

11.

Lin

, Binggeli-Vallarta

, Cervantes

, et al. Process evaluation of an early care and education intervention: The California Childhood Obesity Research Demonstration study (CA-CORD). Health Promot Pract, 2020; 21:298–307.

12.

Sallis

, McKenzie

, Alcaraz

, et al. The effects of a 2-year physical education program (SPARK) on physical activity and fitness in elementary school students. Sports, Play and Active Recreation for Kids. Am J Public Health, 1997; 87:1328–1334.

13.

Davison

, Falbe

, Taveras

, et al. Evaluation overview for the Massachusetts Childhood Obesity Research Demonstration (MA-CORD) project. Child Obes, 2015; 11:23–36.

14.

Fox

, Hallgren

, Boller

, et al. Administration for children and families efforts to meet children's physical activity and nutritional needs: Findings from the I Am Moving, I Am Learning Implementation Evaluation. In: Services UDoHaH (ed). Washington, D.C., US Administration for Children and Families, Office of Planning, Research, and Evaluation, 2010. https://www.acf.hhs.gov/sites/default/files/opre/implement_moving_learning.pdf (Last accessed May 27, 2020).

15.

Joseph

, Stevens

, Ledoux

, et al. Rationale, design, and methods for process evaluation in the Childhood Obesity Research Demonstration project. J Nutr Educ Behav, 2015; 47:560–565.

16.

Process Evaluation for Public Health Interventions and Research. Jossey-Bass: San Francisco, CA, 2002.

17.

Chuang

, Brunner

, Moody

, et al. Factors affecting implementation of the California Childhood Obesity Research Demonstration (CA-CORD) project, 2013. Prev Chronic Dis, 2016; 13:E147.

18.

Ganter

, Aftosmes-Tobio

, Chuang

, et al. Lessons learned by community stakeholders in the Massachusetts Childhood Obesity Research Demonstration (MA-CORD) project, 2013–2014. Prev Chronic Dis, 2017; 14:E08.

19.

Sharma

, Chuang

, Byrd-Williams

, et al. Using process evaluation for implementation success of preschool-based programs for obesity prevention: The TX Childhood Obesity Research Demonstration study. J School Health, 2019; 89:382–392.

20.

Ward

, Benjamin

, Ammerman

, et al. Nutrition and physical activity in child care: Results from an environmental intervention. Am J Prev Med, 2008; 35:352–356.

21.

(IOM) IoM. Early childhood obesity prevention policies. In: Medicine Io (ed). National Academies Press: Washington

, 2011. Institute of

Medicine

. 2011. Early Childhood Obesity Prevention The National Academiss Press. https://doi.org/10.17226/13124 (https://www.nap.edu/catalog/13124/early-childhood-obesity-prevention (Last accessed May 27, 2020).

22.

McClean

, Bridges

California early childhood educator competencies. In: Division CD (ed). California Department of Education: Sacramento, CA, 2011. https://www.cde.ca.gov/sp/cd/re/documents/ececompet (Last acceessed May 27, 2020).

23.

Committee

CC.

Core Competencies for Early Education and Care and Out of School Time Educators. Massachusetts Department of Early Education and Care: Boston, MA, 2010.

24.

Adams

, Brown

, Cardenas-Hagan

, et al. Texas Core Competencies for Early Childhood Practitioners and Administrators. Texas Early Childhood Professional Development System: Houston, TX, 2013.

25.

American Academy of Pediatrics

APHA

, and National Resource Center for Health and Safety in Child Care and Early

Education

. Preventing Childhood Obesity in Early Care and Education: Selected Standards from Caring for Our Children: National Health and Safety Performance Standards; Guidelines for Early Care and Education

Programs

. 2012; 3rd ed. https://nrckids.org/CFOC/Childhood_Obesity (Last accessed May 27, 2020).

26.

Pinket

, Van Lippevelde

, De Bourdeaudhuij

, et al. Effect and process evaluation of a cluster randomized control trial on water intake and beverage consumption in preschoolers from six European Countries: The ToyBox-Study. PLoS One, 2016; 11:e0152928.

27.

Alhassan

, Whitt-Glover

. Intervention fidelity in a teacher-led program to promote physical activity in preschool-age children. Prev Med, 2014; 69 Suppl 1:S34–S36.

28.

Barber

, Jackson

, Hewitt

, et al. Assessing the feasibility of evaluating and delivering a physical activity intervention for pre-school children: A pilot randomised controlled trial. Pilot Feasibility Stud, 2016; 2:12.

29.

Kennedy

, Schenkelberg

, Moyer

, et al. Process evaluation of a preschool physical activity intervention using web-based delivery. Eval Program Plann, 2017; 60:24–36.

30.

Bell

, Davies

, Finch

, et al. An implementation intervention to encourage healthy eating in centre-based child-care services: Impact of the Good for Kids Good for Life programme. Public Health Nutr, 2015; 18:1610–1619.

31.

Finch

, Wolfenden

, Falkiner

, et al. Impact of a population based intervention to increase the adoption of multiple physical activity practices in centre based childcare services: A quasi experimental, effectiveness study. Int J Behav Nutr Phys Act, 2012; 9:101.

32.

Wolfenden

, Jones

, Williams

, et al. Strategies to improve the implementation of healthy eating, physical activity and obesity prevention policies, practices or programmes within childcare services. Cochrane Database Syst Rev, 2016; 10:CD011779.