The Implementation and Five-Year Evolution of a Childhood Healthy Weight Program: Making a Health Care

Abstract

Background:

Sustained implementation of moderate to high-intensity interventions to treat childhood obesity meets many barriers. This report uses the Centers for Disease Control and Prevention's (CDC's) Replicating Effective Programs framework to describe and evaluate the implementation of a 5-year health care-community collaborative program.

Methods:

Interviews with program leadership provided information on setting, organizational culture, program creation and adaptation, and costs. Administrative data were used for number of sessions and their characteristics; referrals; and 2018–2019 participant enrollment, attendance, completion numbers, and completer outcomes.

Results:

Preconditions for this program were high childhood obesity prevalence, and the complementary strengths of the health care organization (primary care treatment referral stream, population health orientation, alternative Medicaid funding) and the community organization (accessible space and time, staffing model, and organization mission). Preimplementation steps included collaborative design of a curriculum and allocation of administrative tasks. Implementation led to simultaneous deployment in as many as 17 community locations, with sessions offered free to families weekday evenings or weekends, delivered in English or Spanish. From 2018 to 2019, 2746 children were referred from nearly 300 providers, 832 (30.3%) enrolled, and 553 (66.3%) attended at least once, with 392 (70.8% of attenders and 47.1% of enrolled) completing the program. Outcomes in completers included improvement in %BMI_p95 [−2.34 (standard deviation, SD 4.19)] and Progressive Aerobic Cardiovascular Endurance Run (PACER) laps [2.46 (SD 4.74)], p < 0.0001 for both. Evolution, including in referral process, Spanish program material and delivery, and range of ages, occurred continuously rather than at discrete intervals. Major system disruptions also affected the implementation. Maintenance of the program relied on the health care organization's administrative team and the collaboration with the community organization.

Conclusion:

This program's collaboration across organizations and ongoing adaptation were necessary to build and sustain a program with broad reach and positive health outcomes. The lessons learned may be helpful for other programs.

Background

The prevalence of childhood obesity continues to be a public health and clinical challenge.¹ Systematic reviews find evidence that moderate- to high-intensity comprehensive, behavior-based interventions result in improvement in weight status.² Despite the literature supporting this approach, such programs are difficult to implement. They are not a natural fit with health care settings, which are set up to see patients, even those with chronic conditions, during brief, infrequent visits. Reimbursement is poor or uncertain.^3–5 Health care visits may require long travel times, and visits are generally scheduled at times that conflict with school and parents' employment. Moreover, enrollment rates for eligible children are often low and attrition is high.^6–8

Thus, although programs with documented positive effects exist, the dissemination and implementation challenges prevent treatment from having an impact on the population. The difficulty of translating evidence-based interventions into practice is well known.^9,10 To improve this process, the science of dissemination and implementation examines the components needed to move an effective approach into practice and sustain it.¹¹

Multiple models and frameworks are used, but all share an emphasis on the contextual effects of rolling out an intervention or practice. The contextual characteristics of a target organization includes its structure, size, resources, and culture, and the context outside the organization can include circumstances or trends that push the organization toward and away from a new practice. Use of a framework to examine a program implementation provides a systematic process to better understand what worked and why. Implementation frameworks have been developed to guide prospectively the process of program development and rollout. However, implementation frameworks can be applied retrospectively.¹²

The Replicating Effective Programs (REP) framework was developed by the Centers for Disease Control and Prevention (CDC) specifically for health services interventions rolled out in a community-based setting.¹³ This framework describes four phases.

Precondition: Implementation prerequisites are a health condition that needs treatment and an effective intervention appropriate for the target settings. This step includes creation of a package for program training, delivery, and assessment.

Preimplementation: The program undergoes customization for the specific setting. A working group of stakeholders collaboratively plan logistics, training, and pilot testing of program.

Implementation: This phase requires partnership with community organizations. Sites require training and technical assistance. Evaluation should include process (e.g., fidelity, reach) and participant outcomes.

Maintenance and evolution: In contrast to a structured research program or a one-time clinical intervention, a successful program undergoes change and evolution in response to new contexts. Organizational financial changes often occur, and further changes may be needed if a program moves to another setting.

This framework has been used prospectively to develop several adult diabetes intervention studies in clinical settings, but not weight management interventions in adults or children.^14–16

The aim of this report is to describe and evaluate the implementation of Get Up & Go, a weight management intervention for children, established over 5 years ago in Dallas, Texas. The program creators and managers did not follow an implementation framework; however, retrospective application of the CDC REP framework, selected because it targets health programs in the community, is useful to understand the successes and challenges in the establishment of Get Up & Go.

Methods

The four phases of the CDC REP were used to frame a retrospective examination of the initial development, early implementation, and sustained delivery of Get Up & Go. Table 1 aligns the phases and their recommended processes with the Get Up & Go implementation.

Table 1.

Application of the Centers for Disease Control and Prevention's Replicating Effective Programs Framework to the Get Up and Go Program for Childhood Healthy Weight

Replicating Effective Programs phases		Get Up and Go application
Precondition
Health condition needing treatment	+	Childhood obesity, which requires multicomponent, moderate-to-high intensity, behavior-based treatment
Effective program	−	Based on existing local program and following USPSTF recommendations but not rigorously evaluated before implementation
Identification of needs and barriers	+	Identification of CH's referral base and resource commitment and YMCA's infrastructure for health-focused classes at times and locations convenient for families. Congruent mission to improve child health at population level
Drafting of package	+	Plan created collaboratively to implement program
Preimplementation
Community working group to refine package	+	Curriculum of local program revised and standardized by working group of content and media specialists from both organizations. Process and training content established
Pilot test package	−	Package tested with rather than before orientation step
Orientation (logistics plan and kick-off)	+	Initial class at single site before expanding sites. Subsequent creation of curricula for different ages
Implementation
Site training	+	CH oversaw class leader training
Technical assistance	+	Division of responsibility for technical assistance: in addition to training, CH managed referrals, participant scheduling, class location. YMCA hired and supervised class leaders
Evaluations	+/−
Formative		(−) No explicit evaluation of formative process at time of initial implementation
Fidelity		(−) No fidelity assessment
Participant outcomes		(+) Participant outcome assessment (body mass index metric, endurance, and reported behavior) Reach and engagement tracking (referrals, initiation, attendance, and completion)
Return on investment		(−) Accounting of costs but no return on investment analysis
Feedback and refinement	+	Ongoing evolution of program: e.g., class and session number, family material, curricula for different age ranges, Spanish-language accommodation, provider referral options
Maintenance and evolutions
Organizational, financial changes	+	Expansion of referral sources after organizational change in primary care association. Adaptation to financial changes
National dissemination	−	Program remains within the two partner organizations
Recustomize for new sites	−	Future: potential for customization for dissemination to new sites

CH, Children's Health.

Information about the preconditions and preimplementation came from telephone interviews with three stakeholders involved in the early history of the program, including the primary champions from the health care and community organization and one of the curriculum developers.

Each interviewee described the perceived need for a community intervention at the time the program was established; the institutional experiences that determined the goals and structure of the program; the cultures and priorities of the partner institutions; and the forces external to the institutions that positively and negatively influenced the establishment of the program. They were asked about preimplementation planning, including the source of the program content and the adaptations that were made. Also elicited were the administrative structure that the partner organizations established to deliver the program, the selection of outcome measures, and the processes created for delivering training, scheduling sessions, enrolling families, and monitoring outcomes. The interviewees described modifications to the process and program content and why they were needed.

The interviews were not transcribed, but the interviewer (S.E.B.) created written summaries, which were then reviewed and amended by a second person (V.N.) who listened to all interviews as they occurred. Each interviewed person also confirmed that the report reflected his/her statements. Coding and identification of recurrent themes were not done because a single person described the perspective of each element or process.

Information about the implementation and maintenance came from a current Get Up & Go leader and included the administrative infrastructure, allocation of cost, and recent significant organizational changes and their impact on the program. The interview was summarized by S.E.B. and reviewed by V.N. and the interviewed person, as described above.

Quantitative descriptions of implementation came from program data, including program availability (number of locations and number of sessions offered by age and language), program engagement, and participant characteristics and outcomes. For engagement, we examined the numbers of children referred, families who committed to participation, families who attended at least one session, and children who met criteria for program completion, which was six sessions after the orientation session in this 10-week program.

Because of limited resources to compile data from all years, participant description and outcome were restricted to 2018–2019, which was the most recent 2-year period at the time of data compilation. Participant characteristics included age, sex, obesity class, and language. Families were assigned to the Spanish language category if they enrolled in a session delivered in Spanish or if they reported Spanish as preferred language when sessions were delivered simultaneously in English and Spanish. Participants were not asked to report race, ethnicity, income, or insurance; such questions were deemed unnecessary for program delivery and potentially intrusive.

The participant measures included weight on a digital scale, height on a portable stadiometer, for calculation of body mass index (BMI). Children performed the Progressive Aerobic Cardiovascular Endurance Run (PACER) test,^17,18 which is scored by the number of laps successfully completed within the allotted time. A standard questionnaire about child's eating and activity behavior, Behavior Assessment Questionnaire (BAQ), was completed by the parent. Each of the BAQ's 10 items has 5 responses, scaled from 0 to 100, with higher values indicating healthier behavior, and the mean value of the 10 item responses is reported. A report of its reliability and validity is under review.

Measures were taken at orientation (baseline) and at last session. Children could join as late as the third week (second education session), and measures were performed at their initial session. Some families absent from the last session completed the behavior questionnaire and weight and height measures at other times. Children could enroll more than once when space was available.

Using SAS 9.4, BMI z-scores, percentiles, and percent of the 95th percentile (BMI_p95) were generated as recommended by the CDC for 2000 growth charts (www.cdc.gov/nccdphp/dnpao/growthcharts/resources/sas.htm). Descriptive analyses of the categorical and continuous data were performed using frequency, proportions, means, and standard deviations (SDs). The demographic and anthropometric characteristics of the program completers and noncompleters were compared with chi-square test, Fisher's exact test, two-sample t-test, and Wilcoxon rank-sum test, as appropriate to the variable's level of measurement and distribution. For completers, the baseline and completion %BMI_p95, PACER, and behavior questionnaire scores were compared by paired t-test. Linear regression model was used to explore the association between clinical outcomes (%BMI_p95 change and PACER change) and various characteristics (sex, age, language, number of classes, associated adult attenders, and associated children attenders).

The Human Studies committee at University of Texas Southwestern Medical School determined that this report did not require IRB oversight.

Results

Precondition

In 2003, the state of Texas required schools to screen for type 2 diabetes in children by identification of acanthosis nigricans (Texas HB 2721). The pediatric endocrinology practice at Children's Health (CH), Dallas, received many referrals from this screening. Few children had diabetes, but most had high BMI and metabolic risks, which would benefit from weight management.

CH received a foundation grant to support the development of a clinical program to address childhood obesity. This program comprised (1) a multidisciplinary clinic to evaluate children individually and (2) group classes offered to the clinic patients. The multidisciplinary clinic quickly faced high referral volumes and long wait times. The third-party payor process, which prohibited no-charge encounters, led to high out-of-pocket costs for families with commercial insurance and was unaffordable for families without insurance. The group curriculum, free to families, consisted of 12 weekly lessons for children 6–11 years of age, accompanied by a parent and other family members. The groups were scheduled in the early evening at the medical center and ran episodically, when enough clinic families were ready to begin. Records of participation in this program do not exist, but interviewed staff recall that only a small minority of clinic families chose to enroll. The program did not fulfill the vision of effective and accessible care.

When the Affordable Care Act passed in 2010, CH leaders anticipated that Medicaid reimbursement would move to a model of full capitation rather than fee-for-service, creating an incentive to address childhood population health. To prepare, CH created an alliance of area pediatric practices and also founded a group of medical homes for Medicaid-eligible children, eventually as many as 20. Recognizing the interest in and value of accessible obesity support for this constituency. CH used available funds from the Texas 1115 Waiver, a program that grants a state greater flexibility in use of Medicaid funds than outlined by federal statute, to create population health programs, including one to address childhood obesity.

In parallel, the Dallas area YMCA had foundation funding by 2009 to offer a turn-key program for childhood weight management. The 10-week programs were delivered free to families at three YMCA sites. Although participants liked the program and had good outcomes, recruitment was difficult; many families did not want to commit to the twice weekly classes, and the YMCA lacked a connection to health care institutions that would provide a steady flow of referrals. When the foundation funding ended, the YMCA sought alternative approaches to deliver community-based childhood weight management.

The complementary strengths and barriers of CH and the YMCA led to the creation of the Get Up & Go program in 2013. CH had a curriculum, a high demand from patients, and financial resources for pediatric population health, but lacked delivery sites close to the patients. The YMCA had community settings with space for group activities and experienced staff but lacked a curriculum, funding, and a stream of referrals.

Preimplementation

Preparation for implementation included revision of the curriculum, establishment of outcome measures, and logistical planning for delivery. CH and YMCA staff collaboratively completed this work over a 6-month period. The original CH curriculum provided nutrition education, a healthy snack, and goal setting during each session, and weekly between-session staff phone check-in. One parent or guardian was required to participate, and all other family members were welcome. Measures (weight, height, PACER, questionnaires) were performed at initial class and last class. Changes included expansion of BMI eligibility to ≥85th %ile, a switch from MyPyramid to MyPlate material, and messaging focused on healthy lifestyle rather than weight control. YMCA staff created physical activity lessons that were fun and focused on family participation.

The new Get Up & Go curriculum targeted children 6–11 years of age initially, with later curricula created for 12–14 years, 15–18 years, and 2–5 years. A written curriculum to guide session leaders started as a flip chart and then was put into a handbook given to each participant family. Media specialists contributed graphic design elements and ensured a sixth-grade literacy level. The team selected relevant outcome measures that would be feasible for YMCA staff to perform. The YMCA was responsible for employing two program leaders per session and scheduling referred children. CH staff led 1-day training sessions and selected neighborhoods to offer programs, based on residences of interested families.

Implementation Process

The first pilot session launched at a Dallas YMCA recreation center in the spring of 2014, after 6 months of preimplementation work. As referrals steadily increased, new sites opened. In 2016, the YMCA was delivering the program at 17 sites, with most sites offering sessions four times a year.

Accessibility was a priority. Program location and number varied from year-to-year, in response to demand. Referrals were accepted from all primary care providers, regardless of affiliation with CH. Programs were free to all enrolled families, and families had a free 3-month family YMCA membership, with a 3-month extension when a family met program completion criteria.

CH staff delivered refresher training to program leaders annually. Program leaders unable to attend could watch a recording, and then answer a question as verification of completion. The ∼40 program leaders came from backgrounds in nutrition, physical therapy, and public health as well as wellness coaches, teachers, WIC employees, and medical students. About half spoke Spanish.

The program made multiple adaptations. Within the first year, the program length was reduced to 10 classes, in response to family reports that the schedule was too long. A lesson on bullying was added in 2017. Programs for 2–5 years and 15–18 years were rolled out in 2016, and then discontinued in 2019 because of low demand.

Language accommodation changed. Sessions delivered in Spanish were available immediately although Spanish material was developed in 2015. Spanish-speaking families often waited 6 to 12 months for a session taught in Spanish close to their home. A Spanish session delayed the next session in English, and long waits in both groups reduced actual attendance. To alleviate these delays, the YMCA in 2019 ensured that one of the two session leaders in most locations was bilingual. Spanish-speaking families could then join English-speaking families in the next available session. CH provided an on-site interpreter for the occasional family who spoke a language other than English or Spanish.

The referral and scheduling process changed. Initially, primary care providers faxed referrals to the YMCA; however, the medical screening for Get Up & Go differed from the YMCA's usual youth enrollment process. In 2015, the referrals were sent to the CH Get Up & Go office, which contacted families to describe the program and assess their readiness-to-change with a 22-item survey; ensured that primary care providers cleared children to participate in physical activity; and scheduled children to the appropriate session near their home. The CH Get Up & Go office communicated with referring offices about enrollment and outcomes. In 2016, providers who had the CH electronic health record (EHR) could place a referral within the EHR, as they did for referrals to medical specialists. The readiness-to-change survey was discontinued in 2017 because screenings did not correlate with attendance or program completion.

Outcome tracking changed. Program leaders at each program site recorded baseline and outcome measures (weight, height, PACER score, and questionnaire responses) on data collection sheets to send to the Get Up & Go office, where staff stored data, provided individual information to each child's primary care provider (PCP), and aggregated outcomes. In 2016, the information was moved from spread sheets to the CH EHR. Staff assigned a medical record number to non-CH patients to record demographics and baseline and end-of-session data. The program reported weight outcome as categorical change in BMI z score. The CDC now describes the limitations of BMI z score in children with severe obesity.¹⁹ Because a high proportion of children in the Get Up & Go program have severe obesity, this report uses change in percent of the 95th percentile BMI.

Implementation Quantitative Outcomes

The sessions offered between 2014 and 2019, including number of sites, session age range, language, and time-of-day, are shown in Table 2. Table 3 shows the annual number of referrals and referring providers between 2016 (the first full year that CH managed referrals and scheduling) and 2019. Most providers referred a small number of patients each year, and a few providers (<10%) referred large numbers. In 2018–2019, CH received referrals for 2746 children. After up to 3 contact attempts, 834 (30.3%) were enrolled and assigned a session. Of the 834 assigned a session, 553 (66.3%) attended at least one session. Those who enrolled but never attended did not differ significantly from attenders in age [mean years (SD) 9.81 (2.31) vs. 9.55 (2.14)], sex (male 50.5% vs. 52.9%), and language (English 64.8% vs. 65.5%).

Table 2.

Number of Sites (YMCA Centers), Number of Sessions Offered, and Characteristics of Sessions, Including Target Age Range, Language, and When Scheduled, between 2014 and 2019

	2014	2015	2016	2017	2018	2019
Site number	7	11	17	17	14	13
Session number, total	9	20	73	71	46	43
Session number, by age group
2–5 years	0	0	2	0	0	N/A
6–11 years	9	18	62	55	43	34
12–14 years	0	2	9	15	3	9
15–18 years	0	0	0	1	0	N/A
Session number, by language
English	9	19	65	63	27	17
Spanish	0	1	8	8	19	N/A
Bilingual	N/A	N/A	N/A	N/A	N/A	26
Session number, by time scheduled
Weekday evening	N/A	14	58	55	32	31
Weekend day	N/A	6	15	16	14	12

“Session” refers to the curriculum offered over 10 weeks.

Table 3.

Referrals 2016–2019: Number of Providers Making Referrals and Provider Count by Referral Volume

	2016	2017	2018	2019
Referral count	350	2524	1231	1515
Unique providers	113	297	246	296
Number of providers by referral volume category
>100	0	2	1	1
20–100	2	27	7	12
10–19	3	21	20	12
5–9	14	43	19	37
<5	94	204	199	234

Among the 553 children who attended at least one session in 2018–2019, 392 (70.8%) completed the program. The baseline characteristics of completers and noncompleters are shown in Table 4. The data are for unique patients; when children enrolled more than once, data come from their first completed session or, when multiple enrollments without completion, data come from initial program start. Completers did not differ from noncompleters in mean age, sex, language, or assigned session curriculum. Both groups had similar baseline BMIz and %BMI_p95. Over half (57%) of all participants had class 2 or 3 obesity, with slightly higher percentage (66%) among noncompleters. Baseline PACER and behavior questionnaire scores did not differ. Noncompleters on average attended fewer than two classes. Among all who started the program, the median number of accompanying adults was 1 and accompanying children older than 4 years was also 1, but the ranges of associated attenders were higher among completers.

Table 4.

Baseline Characteristics and Measures of Unique Participants, by Program Completion Status

	Did not complete N = 161	Completed N = 392	p
Age years: mean (SD)	9.54 (0.17)	9.54 (0.11)	0.97^a
Class curriculum: n (%)			0.40^b
2–5 years	4 (2.5)	7 (1.8)
6–11 years	137 (85.1%)	322 (82.1)
12–14 years	17 (10.6)	59 (15.1)
15–18 years	3 (1.9)	4 (1.0)
Sex: n (%)			0.88^c
Male	86 (53.4)	206 (52.7)
Female	75 (46.6)	185 (47.3)
Language^d: n (%)			0.096^c
English	115 (71.4)	247 (63.0)
Spanish	42 (26.1)	123 (31.4)
Other	4 (2.5)	22 (5.6)
BMIz: mean (SD)	2.25 (0.45)	2.24 (0.45)	0.81^a
%BMI_p95: mean (SD)	129.0 (25.3)	131.2 (58.3)	0.54^a
BMI category: n (%)			0.03^c
Not overweight (BMI <85th %ile)	3 (2.0)	7 (1.8)
Overweight (BMI 85th–94.9th %ile)	12 (8.2)	23 (5.9)
Class 1 (BMI 95th–119.9% BMI_p95)	35 (23.8)	139 (35.9)
Class 2 (120–139.9% BMI_p95)	64 (43.5)	121 (31.3)
Class 3 (≥140% BMI_p95)	33 (22.5)	97 (25.1)
PACER: mean (SD)	11.41 (6.6)	11.92 (7.0)	0.45^a
Behavior questionnaire responses: mean (SD)	48.25 (13.0)	48.5 (14.5)	0.89^a
Total classes attended: mean (SD)	1.87 (1.69)	8.01 (0.98)	<0.0001^a
Associated attenders: median (range)
Total number	2 (1–5)	2 (1–10)	<0.0001^e
Number adults	1 (1–3)	1 (1–4)	0.0001^e
Number children ≥4 years	1 (0–4)	1 (0–7)	0.003^e

Two-sample t-test.

Fisher exact.

Chi-square.

Language variable constructed from language of class assignment and reported language preference, as described in methods

Wilcoxon rank-sum.

PACER, Progressive Aerobic Cardiovascular Endurance Run; SD, standard deviation.

Changes in outcome measures among the 392 completers are shown in Table 5. %BMI_p95, PACER, and behavior questionnaire scores all improved significantly. Simple regression examined associations between each outcome measure and sex, age, age group, language, number of classes attended, associated adult attenders, and associated children attenders. No association was found for %BMI_p95 change and PACER change. Improvement in behavior questionnaire was associated with (1) lower baseline BMI categories (p = 0.03) and (2) attending more classes (p = 0.03). When baseline BMI categories and class attendance were included in multivariable analysis to examine association with questionnaire improvement, neither was significant (Regression data are not shown.).

Table 5.

Change in Measures between Completion and Baseline among Completers

	Change (completion minus baseline) Mean (SD)	IQ 25th–75th	p
%BMI_p95n = 349	−2.34 (4.19)	−4.57 to 0.12	<0.0001^a
PACER lapsn = 346	2.46 (4.74)	0 to 5	<0.0001^a
Behavior Assessment Questionnaire(Potential score range 0–100)n = 360	17.0 (15.5)	7.5 to 27.5	<0.0001^a

Paired t-test.

Maintenance

Maintenance of this program required CH's financial support of the YMCA activities as well as support for the CH administrative work. The YMCA costs included instructor wages for training and delivery, site use fees, national YMCA overhead, administration effort, and the purchase of healthy snacks. CH administrative costs included 2.0 FTE for management of referrals, calls, and enrollment, 1.0 FTE dietitian time, and 0.25 FTE for program director. Additional costs were mileage for CH staff, interpretation for families not speaking English or Spanish, and program handbook printing. Including all financial support, each 10-week program cost approximately $7000 to deliver. CH administration costs accounted for about 40% of the delivery costs.

In addition to the modifications to content and process throughout this program's existence, the program had to adapt to two major events. First, CH sold its group of medical home practices in mid-2018. A new EHR accompanied the ownership change, no longer connecting directly to the Get Up & Go office. When referrals from PCPs decreased, the opportunity to refer was extended to health care professionals other than PCPs. As a result, specialists who address weight management, such as dietitians, lipid specialists, and fatty liver specialists, could refer directly rather than request referral from the patient's PCP. Referral also expanded to school nurses. In several local school districts, nurses who perform the state-mandated screen for acanthosis nigricans in the presence of high BMI could send a referral to Get Up & Go, with a letter informing parents of the referral. The Get Up & Go office called families to offer enrollment.

The second major challenge was the ongoing financial support of the program. CH turned to its development office, which was able to secure a major gift in 2019 with a 2-year commitment, and the potential for renewal in the future.

Discussion

This report uses the CDC's REPs framework to describe the implementation and maintenance of a family-based, community-located pediatric weight management program. Key findings include the importance of the complementary priorities and strengths of the major partners; the continuous rather than discrete revision that took place to allow the program to survive and grow; the trade-off between program availability versus customization for age or language; and the challenge of maintaining a program that does not live comfortably within the current health care reimbursement system. Table 1 summarizes the application of the REP framework to Get Up & Go.

This program's implementation history and its outcomes may be helpful to organizations that want to deliver pediatric weight management programs. Improvement in major outcomes of %BMI_p95 and PACER are comparable to other programs.^20–22 The broad utility of this program is indicated by the lack of outcome variation with characteristics such as age, sex, and Spanish language (Racial and ethnic identity were not collected.). An important process measure in pediatric weight management programs is attrition, which can be very high⁶ but was low at 29.2% in this program.

The preconditions under the CDC REP framework for this intervention included an important, prevalent health condition,¹ an effective treatment,² and the development of a delivery package. The most important variation from recommended preconditions was the lack of a controlled study of this specific program, a gap reflecting the priority of the stakeholders to create a rapid, practical response to the obesity epidemic. However, the program curriculum adhered to the USPSTF recommendations of providing nutrition, physical activity, and behavior change education and involvement of parents.² The program was shorter than the 26 or more hours that have characterized effective research studies; program length was guided by site feasibility and participant feedback. Although a well-studied program is recommended under this framework, adaptation of standard programs can be part of package development.²³

The preimplementation step included not only the pilot delivery of the program but also the collaboration between the community and health care organizations. The strengths of the partner organizations were complementary. Dissemination and implementation literature identifies alignment between organizers and community sites as a component of success not only in the establishment of a program but also when weathering changes, such as shifts in key staff, organizational priorities, or relevant polices.^24,25 The shared priorities and effective working relationship are more important than quantifiable knowledge and physical resources.²⁶

The strengths of the implementation phase of the program included the training and technical assistance, the participant outcomes, and the feedback and refinement, while the weaknesses were an absence of initial formative assessment and any ongoing fidelity assessment. The “technical assistance” was delivered through a robust administrative infrastructure, which managed referrals, class location and scheduling, and also oversaw training. This infrastructure, which was a substantial part of delivery cost, supported the program's sustained delivery. The four different evaluation steps in the REP framework—formative process, fidelity assessment, participant outcome assessment, and cost with return-on-investment analyses—were partially met. An explicit formative evaluation did not occur at initial implementation and could have identified problems for early correction, although the ongoing revision may have compensated for this gap. The program never assessed fidelity of delivery, and therefore, consistent delivery of program by various leaders is unknown.

The program did a particularly good job of standardized assessment of participant outcomes, allowing demonstration of consistent positive effects in the BMI metric, the reported health behaviors, and the endurance test. Additional strengths, not listed as steps in REP, include monitoring reach (the participation rate of eligible people and their representativeness) and engagement of those who were referred. These are common measures of implementation and would be valuable additions to the REP framework.²⁷ Delivery costs were calculated but a return-on-investment analysis was not possible because the program was delivered outside the health care payment system. The near-continuous refinement of the program delivery, including Spanish-language accommodation, curriculum length, and provider referral options, prioritized the needs of the family participants as well as the referring providers.

The CDC REP distinguishes the stage of implementation from the state of maintenance and evolution; however, content and administrative modification occurred continuously as noted above. Major changes in the health care organization, when the expected health care policy to capitate spending did not materialize, led to uncertainty about long-term financial support of the program. The need for consistent support is a common threat to program implementation, especially pediatric weight management programs.^24,28,29 However, the sustained 5-year support by CH fostered a well-functioning administrative process between the two partners, a broad referral base, and a track record of good attendance and outcomes that made the program attractive to foundation support.

An important limitation is the lack of an initial randomized controlled trial. However, the components of the Get Up & Go program align with USPSTF-recommended approaches, and its effectiveness has been consistent. With additional resources, data from 2014 to 2017 years could have been compiled and examined for changes over time. Had less time passed since the initial program development, rigorous qualitative methodology could have explored a broader range of perspectives within the stakeholder organizations.

Conclusion

The implementation steps of the Get Up & Go program, reviewed here by retrospective application of the CDC REP framework resulted in an effective, accessible program for children with overweight and obesity and their families. The program's success resulted from commitment and complementary collaboration between health care and community organizations, and the continuous modification in response to ongoing needs. The evolution to fewer program variations (two age groups rather than four, and bilingual delivery) to maximize ongoing availability may be a useful model for others. A robust, ongoing program delivery requires a stable financial stream that finances the sessions as well as the administrative infrastructure to maintain recruitment, delivery, and data. A consistently offered and highly accessible program has the best chance of well-subscribed sessions, which mean that the per-family cost is low.

Footnotes

Acknowledgments

The authors thank the following people for the information they provided and the careful review of this report: LeAnn Kridelbaugh, MD; Maren Wolff, PhD, RD, LD; Cheryl McCarver; Christina Green-Spillers, BSN, MPH; Tracey Burns, Anne Crowther Cummins, MPH, MCHES; and Victoria Nelson.

Authors' Contributions

S.E.B.: conception, acquisition and interpretation of data, drafting and revising work, final approval, and accountability. A.L.: acquisition and interpretation of data, revising work, final approval, and accountability. A.R.: acquisition and interpretation of data, revising work, final approval, and accountability. R.H.: analysis and interpretation of data, revising work, final approval, and accountability. J.S.Y.: interpretation of data, revising work, final approval, and accountability. S.E.M.: conception and interpretation of data, revising work, final approval, and accountability.

Funding Information

No funding was received for this article.

Author Disclosure Statement

None of the authors reports competing or personal financial interests. No research support or employment or any other competing interests that could be affected by publication are reported.

References

Skinner

, Ravanbakht

, Skelton

, et al. Prevalence of obesity and severe obesity in US children, 1999–2016. Pediatrics, 2018; 141:1–17.

Force

USPST

, Grossman

, Bibbins-Domingo

, et al. Screening for obesity in children and adolescents: US preventive services task force recommendation statement. JAMA, 2017; 317:2417–2426.

Eneli

, Norwood

, Hampl

, et al. Perspectives on obesity programs at children's hospitals: Insights from senior program administrators. Pediatrics, 2011; 128 Suppl 2:S86–S90.

Slusser

, Staten

, Stephens

, et al. Payment for obesity services: Examples and recommendations for stage 3 comprehensive multidisciplinary intervention programs for children and adolescents. Pediatrics, 2011; 128 Suppl 2:S78–S85.

Wilfley

, Staiano

, Altman

, et al. Improving access and systems of care for evidence-based childhood obesity treatment: Conference key findings and next steps. Obesity, 2017; 25:16–29.

Skelton

, Beech

. Attrition in paediatric weight management: A review of the literature and new directions. Obes Rev, 2011; 12:e273–281.

Alff

, Markert

, Zschaler

, et al. Reasons for (non)participating in a telephone-based intervention program for families with overweight children. PLoS One, 2012; 7:e34580.

Barlow

, Butte

, Hoelscher

, et al. Strategies to recruit a diverse low-income population to child weight management programs from primary care practices. Prev Chronic Dis, 2017; 14:E138.

Morris

, Wooding

, Grant

. The answer is 17 years, what is the question: Understanding time lags in translational research. J R Soc Med, 2011; 104:510–520.

10.

Neta

, Brownson

, Chambers

. Opportunities for epidemiologists in implementation science: A primer. Am J Epidemiol, 2018; 187:899–910.

11.

Bauer

, Damschroder

, Hagedorn

, et al. An introduction to implementation science for the non-specialist. BMC Psychol, 2015; 3:32.

12.

Tabak

, Khoong

, Chambers

, Brownson

. Bridging research and practice: Models for dissemination and implementation research. Am J Prev Med, 2012; 43:337–350.

13.

Kilbourne

, Neumann

, Pincus

, et al. Implementing evidence-based interventions in health care: Application of the Replicating Effective Programs framework. Implement Sci, 2007; 2:42.

14.

Wallace

, Perkhounkova

, Tseng

, Schillinger

. Influence of patient characteristics on assessment of diabetes self-management support. Nurs Res, 2013; 62:106–114.

15.

Ramsay

, Janevic

, Hogan

, et al. Using the Replicating Effective Programs framework to adapt a heart health intervention. Health Promot Pract, 2019; 20:760–769.

16.

Kwan

, Rementer

, Richie

, et al. Adapting diabetes shared medical appointments to fit context for practice-based research (PBR). J Am Board Fam Med, 2020; 33:716–727.

17.

Leger

, Mercier

, Gadoury

, Lambert

. The multistage 20 metre shuttle run test for aerobic fitness. J Sports Sci, 1988; 6:93–101.

18.

Cureton

KJP

, Plowman

, Mahar

. Aerobic Capacity Assessments. In: S Plowman, SA Meredith, ed. Fitnessgram/Activitygram Reference Guide.The Cooper Institute: Dallas, TX, 2013.

19.

Freedman

, Butte

, Taveras

, et al. BMI z-Scores are a poor indicator of adiposity among 2- to 19-year-olds with very high BMIs, NHANES 1999–2000 to 2013–2014. Obesity, 2017; 25:739–746.

20.

Butte

, Hoelscher

, Barlow

, et al. Efficacy of a community-versus primary care-centered program for childhood obesity: TX CORD RCT. Obesity, 2017; 25:1584–1593.

21.

D'Agostino

, Patel

, Hansen

, et al. Effect of participation in a park-based afterschool program on cardiovascular disease risk among severely obese youth. Public Health, 2018; 159:137–143.

22.

Braun

, Kay

, Cheung

, et al. Impact of an elementary school-based intervention on physical activity time and aerobic capacity, Georgia, 2013–2014. Public Health Rep, 2017; 132(2_,suppl):24S–32S.

23.

Hill

, Zoellner

, You

, et al. Participatory development and pilot testing of iChoose: An adaptation of an evidence-based paediatric weight management program for community implementation. BMC Public Health, 2019; 19:122.

24.

Heerman

, Schludnt

, Harris

, et al. Scale-out of a community-based behavioral intervention for childhood obesity: Pilot implementation evaluation. BMC Public Health, 2018; 18:498.

25.

Damschroder

, Aron

, Keith

, et al. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implement Sci, 2009; 4:50.

26.

Teeters

, Heerman

, Schlundt

, et al. Community readiness assessment for obesity research: Pilot implementation of the Healthier Families programme. Health Res Policy Syst, 2018; 16:2.

27.

Glasgow

, Vogt

, Boles

. Evaluating the public health impact of health promotion interventions: The RE-AIM framework. Am J Public Health, 1999; 89:1322–1327.

28.

Vidgen

, Love

, Wutzke

, et al. A description of health care system factors in the implementation of universal weight management services for children with overweight or obesity: Case studies from Queensland and New South Wales, Australia. Implement Sci, 2018; 13:109.

29.

Biddinger

. 2013. survey findings of children's hospitals obesity services. 2014. Available at www.childrenshospitals.net/obesitysurvey (Last accessed January 13, 2021).

The Implementation and Five-Year Evolution of a Childhood Healthy Weight Program: Making a Health Care–Community Partnership Work