A Process Evaluation of an Efficacious Family-Based Intervention to Promote Healthy Eating

Abstract

Entre Familia: Reflejos de Salud was a successful family-based randomized controlled trial designed to improve dietary behaviors and intake among U.S. Latino families, specifically fruit and vegetable intake. The novel intervention design merged a community health worker (promotora) model with an entertainment-education component. This process evaluation examined intervention implementation and assessed relationships between implementation factors and dietary change. Participants included 180 mothers randomized to an intervention condition. Process evaluation measures were obtained from participant interviews and promotora notes and included fidelity, dose delivered (i.e., minutes of promotora in-person contact with families, number of promotora home visits), and dose received (i.e., participant use of and satisfaction with intervention materials). Outcome variables included changes in vegetable intake and the use of behavioral strategies to increase dietary fiber and decrease dietary fat intake. Participant satisfaction was high, and fidelity was achieved; 87.5% of families received the planned number of promotora home visits. In the multivariable model, satisfaction with intervention materials predicted more frequent use of strategies to increase dietary fiber (p ≤ .01). Trends suggested that keeping families in the prescribed intervention timeline and obtaining support from other social network members through sharing of program materials may improve changes. Study findings elucidate the relationship between specific intervention processes and dietary changes.

Keywords

diet family health Latino nutrition process evaluation

Research shows substantial disparities in dietary intake between U.S. Latinos and other racial/ethnic groups (Centers for Disease Control and Prevention, 2010). While Latino adults consume more fruits than other groups, they consume significantly fewer vegetables (Centers for Disease Control and Prevention, 2010). For example, results from the 2009 Behavioral Risk Factor Surveillance Survey found that Hispanic adults had the highest prevalence of daily fruit consumption (37.2% consumed 2+ servings/day) and the lowest prevalence of vegetable consumption (19.7% consumed 3+ servings/day) compared to other racial/ethnic groups (Centers for Disease Control and Prevention, 2010). Considering that in some age and gender groups more Latinos are obese than any other racial/ethnic group, it is critical to develop and evaluate interventions to improve dietary intake (Pleis, War, & Lucas, 2010).

Entre Familia: Reflejos de Salud (Entre Familia) was a family-based intervention designed to improve the dietary behaviors and intake of predominantly Mexican-origin families. Entre Familia employed a community health worker (promotor(a)) model and incorporated an “edutainment” (i.e., entertainment education) component. Outcome evaluation of this randomized controlled trial determined that mothers in the intervention versus control condition showed significant increases in vegetable intake, and in the use of behavioral strategies to increase dietary fiber intake and decrease dietary fat intake 4 months postbaseline (Ayala, Ibarra, Horton et al., 2015). Additionally, children in the intervention versus control condition increased the variety of fruits and vegetables consumed and reduced intake of fast food (Horton et al., 2013). The outcome results of Entre Familia add to the existing literature supporting use of promotora models to promote health behavior change in Mexican immigrant/Mexican American communities (Ayala, Vaz, Earp, Elder, & Cherrington, 2010; Elder et al., 2005; Keller, 2008; Perez-Escamilla, Hromi-Fiedler, Vega-Lopez, Bermudez-Millan, & Segura-Perez, 2009).

Complementing efficacy studies are studies examining the process of intervention implementation (Linnan & Steckler, 2002). Process evaluation enables researchers to assess implementation fidelity, or the degree to which the intervention was delivered as planned (Linnan & Steckler, 2002), and it highlights systemic and contextual factors to consider during implementation of future interventions to maximize fidelity. Process evaluation also can identify mediators of behavior change related to implementation, or factors that explain the behavior change process (Crowley, Coffman, Feinberg, Greenberg, & Spoth, 2013; Saunders, Evans, & Joshi, 2005). Most public health interventions, including Entre Familia, apply multiple behavior change techniques to multiple behavioral determinants (Moore et al., 2014). Process evaluation enables researchers to examine effects of specific techniques on behavior change, information that is not included in standard efficacy studies. Finally, process evaluation studies can prevent Type III error, which occurs when researchers erroneously attribute behavior change to the intervention (Basch, Sliepcevich, Gold, Duncan, & Kolbe, 1985; Schwartz & Carpenter, 1999). In short, process evaluation studies provide valuable information to understand why an intervention was effective, and how best to design future interventions.

The benefits of process evaluation studies are clear, yet process evaluation data are often insufficient to draw meaningful conclusions or guide researchers seeking to replicate an intervention (Moore et al., 2014; Oakley et al., 2006). Moreover, process evaluation studies of interventions with unique elements such as promotoras using edutainment to deliver a family-based intervention can contribute to the field of implementation science. To fill a gap in research, this study had three objectives. The first was to provide a comprehensive description of the process evaluation measures used in Entre Familia. These measures included both standard process-related variables such as dose delivered, as well as new measures unique to this intervention. For instance, we assessed the participants’ perceived similarities to the promotora assigned to them. We hypothesized that participants could be influenced as much by the characteristics of the promotora as they were by the intervention, based on research regarding patient–practitioner relationships (Baldwin, Wampold, & Imel, 2007; Elkin, 1999). Although this phenomenon has not been examined empirically in promotora interventions, numerous qualitative studies have noted the importance of participant satisfaction and camaraderie with the promotora as a possible mechanism of program success (Bill, Hock-Long, Mesure, Bryer, & Zambrano, 2009; Deitrick et al., 2010; Reinschmidt et al., 2006). The second objective was to assess intervention fidelity. The third objective was to determine which process evaluation measures predicted dietary behavior and intake changes in the mothers, the primary study participants.

Method

Participants and Setting

Entre Familia was conducted in Imperial County, California, along the U.S.–Mexico border. Data show that 77% of residents are Latino and predominantly Mexican-origin (U.S. Census Bureau, 2014). Compared to California state averages, Imperial County residents are more likely to live below the poverty level (23% vs. 14%) and are less likely to have a college degree (13% vs. 30%; U.S. Census Bureau, 2014). Also, adult obesity rates in Imperial County exceed national and state averages (41.7% in Imperial County vs. 24.8% in California and 34.9% in the United States; California Health Interview Survey, 2014; Ogden, Carroll, Kit, & Flegal, 2014).

Participating families were a convenience sample recruited through staff presentations at local venues including schools, flyers and ads, and letters mailed to families of pediatric patients of Clinicas de Salud del Pueblo, Inc., a federally qualified health center that provides primary care services to Imperial and Riverside County residents. Primary study participants were 361 Latino mothers with children (7-13 years old). Only mothers assigned to the intervention condition (n = 180) are included in the current analyses.

Intervention and Procedure

Entre Familia was a family-based intervention; efficacy was evaluated using a randomized controlled design with a delayed-treatment control condition. The intervention goals were to increase fruit and vegetable intake and improve dietary behaviors that increase fiber and decrease fat intake. The primary intervention participants were the mothers and secondarily one child per family, but all family members were encouraged to participate in intervention activities. The intervention consisted of 11 structured, promotora-led home visits delivered over 4 months; the first eight took place in the families’ homes during consecutive weeks and the last three alternated between home and telephone visits tapering from biweekly to one in the last month. Contact with participants was tapered off to promote independence and decrease reliance on the promotora.

Three intervention components were developed for Entre Familia. First, researchers developed a nine-part, Spanish-language, sitcom-style DVD series. The series followed a typical Mexican family as they tried to change their eating habits amid everyday challenges. Each episode included a 12-minute storyline. Second, researchers developed a family manual to supplement the DVDs containing the objectives of the visit, the DVD series’ key points, and behavior change tools consistent with behavior change theory (Bandura, 1989; Michie et al., 2013), including goal setting and self-monitoring forms assigned as homework. Third, an interpersonal intervention was developed similar to previous efforts (Ayala et al., 2010). During home visits, promotoras used the DVD series and manual to facilitate the delivery of informational, emotional, and instrumental social support. The promotoras received 80 hours of training and a training manual to use as a reference throughout the intervention. Additional details on the development and implementation of Entre Familia, including the rationale for each intervention component, are published elsewhere (see Ayala et al., 2012; Ayala, Ibarra, Horton et al., 2015; Horton et al., 2013).

The promotora home visits followed a standard protocol. The promotora reviewed the previous week’s homework, watched one DVD episode, delivered a mini-presentation, and facilitated family activities including helping the family set a behavior change goal and reviewing the next homework assignment. Home visits were expected to last 1.5 to 2 hours, which adds up to a minimum of 16.5 hours of planned in-person contact across the 11 home visits. During brief telephone calls on intervening weeks, promotoras problem-solved barriers to meeting the goals of the week. All procedures received institutional review board approval at San Diego State University.

Data Sources

Process evaluation data were obtained from promotora notes and participant interviews. Outcome data, including dietary intake, were obtained during participant interviews.

Promotora Home Visit Forms

Each promotora documented all contact with their assigned families, including duration of each visit (i.e., dose delivered) and details about the visit (i.e., intervention delivery, distractions during visits). Notes were recorded on a hardcopy form created by Entre Familia researchers. The forms captured the date and start and end time of each home visit, the number and relationship of family members in attendance, and space to write notes about what happened at the visit. The forms also included a series of checkboxes that allowed the promotoras to record which activities were completed at the visit and how the family member responded to the visit. Promotoras were instructed to complete the form immediately following each visit. To ensure forms were completed in a timely manner, promotoras met each week with a supervisor, during which time the forms were reviewed and checked for completeness.

Participant Interviews

Trained bilingual, bicultural Evaluation Assistants conducted structured, in-person interviews. Mothers completed the interview at baseline and immediately following the intervention conclusion (4 months postbaseline). The interviews included questions regarding demographics, dietary behaviors, and process evaluation measures including dose received (see Measures section). Items not previously available in Spanish (see Ayala et al., 2005; Elder et al., 2010) were translated and evaluated for conceptual equivalence by a translator and trained staff (Sperber, Devellis, & Boehlecke, 1994). The interview questions were identical at each time point, except for the process evaluation items that were only included in the follow-up interview.

Measures

Process evaluation measures included dose delivered, dose received, participant rating of the promotora, and fidelity. Dose delivered was derived from the promotora home visit forms, and dose received and participant rating of the promotora were derived from the participant interviews. Fidelity was determined using both promotora home visit forms and participant interviews.

Dose Delivered

Dose delivered was defined as the amount of in-person contact the promotoras had with the families (Linnan & Steckler, 2002). For this study, dose delivered was measured with three variables from the promotora home visit forms: number of home visits (computed via a basic count), total minutes of in-person contact, and number of days of participation in the home visit portion of the intervention. The promotora logged the start and end time at each visit; total minutes of in-person contact were calculated by summing the number of minutes at each home visit. Number of days of participation in the home visits was calculated as the days between the first and final home visits. These three variables were important to examine to determine whether behavior change occurred as a function of the time promotoras spent with families (i.e., minutes of contact), or the length of the relationship with the promotora (i.e., days of participation).

Dose Received

Dose received was defined as the extent to which families were exposed to the intervention and interacted with intervention materials (Linnan & Steckler, 2002). Three intervention components were measured: family participation, participant satisfaction with the DVD series, and use of the family manual. Two family participation variables were derived to measure exposure to the intervention. Family involvement, which was the sole dose received variable derived from the promotora home visit forms, was measured as the median number of family members present during each promotora home visit, divided by the total number of family members living in the household to account for family size. The median of these percentages was calculated across all visits to determine the attendance rate based on family size. The goal of this intervention was to involve as many household family members as possible, to create family system changes. As such, home visits were scheduled to maximize attendance by all household family members (e.g., late afternoons, evenings, and weekends). However, given differing work schedules, the promotoras were instructed to maximize attendance by, at minimum, the mother and the target child. The family workbook was left with the family after each home visit to maximize the opportunity for exposed household family members to share the information with other household family members.

The second family participation variable was from the interview question, “On a scale from 1 to 10, how much effort do you feel your family put into the program?” Participant satisfaction with the DVD series was measured with a 7-item scale. Mothers rated several components of the series including trustworthiness and helpfulness of the information. Response options ranged from 1 (not at all) to 5 (completely). A mean score was calculated from all items with higher scores indicating greater satisfaction (α = .78). Three interview questions measured use of the family manual. The first and second questions referred to the family’s use of the manual outside of promotora visits: “Did you ever read the manual on your own when the promotora was not with you?” and “About how much time did you spend reading the family manual each week?” Response options for the first question were “yes” and “no,” and the second asked mothers to report the number of minutes. The third item asked, “Did you share the program materials with anyone (e.g., friends, family, neighbors)?” with response options “yes” and “no.”

Promotora Traits

A measure of mothers’ similarity to the promotoras was also included based on previous qualitative research showing the more similar mothers perceived the promotora to be, the more secure and confident they felt about the program (Reinschmidt et al., 2006). Mothers were asked to compare themselves to their promotora on four dimensions: socioeconomic status, social status, background, and in general. Response options included “very different to you,” “a little different to you,” “a little similar to you,” and “very similar to you.” A mean score was calculated from the four items with higher scores indicating a higher degree of perceived similarity (α = .82).

Fidelity

Fidelity was defined as the degree to which the Entre Familia intervention components were delivered as planned (Linnan & Steckler, 2002) and was derived by comparing the dose delivered and dose received to the original intervention plan (see Intervention Description).

Dependent Variables

Three dietary behaviors were examined as dependent variables: daily vegetable intake, and the use of behavioral strategies to increase dietary fiber and decrease dietary fat intake. These variables were selected because each was previously shown to have improved among intervention versus control mothers in the Entre Familia intervention (Ayala, Ibarra, Horton, et al., 2015).

Vegetable intake was measured with the National Cancer Institute’s Fruit and Vegetable Screener (Applied Research Center, National Cancer Institute, National Institutes of Health, 2000; Thompson et al., 2000), which assesses the quantity and frequency of fruits and vegetable consumed in the past month. For the current analysis, only vegetable intake items were used and responses for frequency and quantity items were multiplied to determine the average number of servings consumed daily. Intake was dichotomized as “less than ½ cup change” or “greater than or equal to ½ cup increase” based on the differences observed between baseline and the 4-month postintervention (postbaseline) measures.

Behavioral strategies to increase dietary fiber were measured with an 11-item scale (Elder, Ayala, Slymen, Arredondo, & Campbell, 2009) assessing the frequency with which mothers used specific fiber-related behaviors including adding vegetables to a dish. Response options ranged from 1 (never) to 4 (always). A mean score was calculated from all items with higher scores indicating more frequent use (α = .71).

Behavioral strategies to decrease dietary fat were measured with a 19-item scale (Elder et al., 2009). Scale items assessed how often mothers used strategies to reduce fat intake, including replacing dark poultry meat with white meat. Response options ranged from 1 (never) to 4 (always). A mean score was calculated from all items with higher scores signifying more frequent use of the strategies (α = .76).

Sociodemographic Characteristics

Sociodemographic characteristics were collected for both mothers and children. Characteristics of the mother included mother’s age (years), country of birth (United States or outside the United States), education (less than high school vs. high school or more), marital status (married/cohabitating vs. unmarried), and measured body mass index (BMI; kg/m²). Additionally, enrollment in a public food assistance programs such as WIC or EBT (yes or no) was used as a proxy for socioeconomic status. Child characteristics included age (years) and measured BMI z-score.

Analysis

Descriptive statistics were calculated for all variables to examine normality. Bivariate regression analyses were used to identify independent correlations between all sociodemographic, process evaluation variables and each dependent variable. Separate regression analyses were performed for each of the three dependent variables; logistic regression was used to examine predictors of a change from baseline to 4 months postbaseline in daily vegetable intake greater than or equal to ½ cup, and linear regression was used to examine predictors of change in behavioral strategies to increase fiber and decrease fat. Analyses were conducted using IBM SPSS Statistics version 19.0 with all significance tests using an alpha level of .05.

Results

Participant Characteristics

Descriptive characteristics of participating mothers and their children are shown in Table 1. The mothers’ mean age was 38 years (SD = 8.1), and the child’s mean age was 10 years (SD = 1.8). About half of the mothers (49%) completed high school or higher, and most (96%) were married. With respect to dietary changes 4 months postbaseline, nearly half (44%) of the 159 mothers with vegetable intake data reported an increase of at least ½ cup per day. Approximately 68% increased the use of behavioral strategies to increase fiber intake and 70% reported increasing the use of strategies to decrease fat intake (data not shown).

Table 1.

Demographic and Intervention Characteristics of 180 Mexican-Origin Mothers Participating in a Family-Based Healthy Eating Intervention.

Characteristic	Mean (SD), median (range), or n (%)
Demographics—Mother
Age (years)	38.4 (8.1)
Foreign-born	141 (78%)
High school education or more	89 (49%)
Married	173 (96%)
Public food assistance (yes)	96 (54%)
Body mass index	31.0 (7.2)
Demographics—Child
Age (years)	10.0 (1.8)
Body mass index z-score	−0.2 (range = −1.4 to 7.4)
Intervention factors
Similarity to promotora^a	2.9 (0.7)
Dose delivered
Number of promotora home visits	11 (range = 0-11)
Minutes of promotora in-person contact	918.3 (287.8)
Days to complete intervention	99 (range = 1-173)
Dose received
Number of family members in attendance at each home visit	2 (range = 1-4)
Percentage of family in attendance at each home visit	40% (range = 13% to 93%)
Effort put into the program (0-10)	8.3 (1.5)
Satisfaction with DVD series^b	4.3 (0.5)
Use of manual outside of home visits (yes)	139 (89%)
Minutes per week use manual outside of home visits	28.8 (33.2)
Shared manual with others (yes)	106 (68%)

Response options ranged from 1 (very different to you) to 4 (very similar to you). ^bResponse options ranged from 1 (not at all satisfied) to 5 (completely satisfied).

Process Evaluation Measures

Dose Delivered

The median number of promotora visits per family was 11 (Table 1), with 87.5% of families receiving 11 visits (data not shown). On average, families completed the intervention in 98.5 days (SD = 28.7; Table 1); however, nearly half (44%) took longer than the planned 3 months to complete the promotora home visits. The mean number of minutes of in-person promotora contact was 918.3 (SD = 287.8), or just under 16 hours.

Dose Received

The median number of family members attending the home visits was 2, which translated to 40% of household family members (Table 1). Mothers reported high satisfaction with the DVD series, giving it a mean score of 4.3 out of 5. The majority (89%) of mothers reported using the family manual outside of the home visits for an average of 28.8 min/week (SD = 33). Approximately two thirds of mothers (67.5%) reported sharing the manual with others outside their immediate family and reported putting a great deal of effort into the program, with an average score of 8.3 out of 10 (SD = 1.5).

Promotora Ratings

Mothers felt the promotora’s background and socioeconomic status was fairly similar to their own, with a mean similarity rating of 2.9 out of 4 (SD = 0.7).

Fidelity

Comparisons of dose delivered to the original intervention protocol indicated the intervention was implemented with fidelity. Nearly 90% of families received the 11 planned visits (Table 1), and promotoras had approximately 16 hours of in-person contact with mothers.

Predictors of Behavior Change

Bivariate Analyses

Table 2 depicts bivariate regression analyses examining predictors of change in vegetable consumption (<½ serving change or ≥½ serving change), increase in use of strategies to increase dietary fiber, and increase in use of strategies to decrease dietary fat. Enrollment in public food assistance programs was the sole variable significantly predictive of increased vegetable intake (odds ratio [OR] = 2.13, p ≤ .05). More specifically, participants who were enrolled in food assistance programs were over two times as likely to increase their vegetable consumption by ½ serving or more than participants who were not enrolled in food assistance programs.

Table 2.

Bivariate Regression Analysis Examining Predictors of Change in Three Dietary Behaviors: Vegetable Intake, Use of Strategies to Increase Fiber, and Use of Strategies to Decrease Fat Intake.

	Change in vegetable intake^a		Change in strategies to increase fiber^b		Change in strategies to decrease fat^b
Characteristic	OR	(95% CI)	Standardized β	(95% CI)	Standardized β	(95% CI)
Demographics—Mother
Age	0.98	(0.94, 1.02)	0.03	(−0.01, 0.01)	−0.01	(−0.01, 0.01)
Foreign-born	0.99	(0.46, 2.14)	−0.09	(−0.34, 0.09)	−0.04	(−0.30, 0.16)
Education (less than high school)^c	1.00
High school or more	1.39	(0.74, 2.61)	0.06	(−0.11, 0.25)	0.00	(−0.19, 0.19)
Married	0.49	(0.09, 2.63)	−0.05	(−0.56, 0.30)	−0.07	(−0.71, 0.26)
Public food assistance (no)	1.00
Yes	2.13	(1.12, 4.04)**	−0.00	(−0.18, 0.18)	0.04	(−0.14, 0.24)
Body mass index	0.99	(0.95, 1.04)	0.03	(−0.01, 0.02)	−0.00	(−0.01, 0.01)
Demographics—Child
Age	0.98	(0.83, 1.17)	0.03	(−0.04, 0.06)	−0.09	(−0.08, 0.02)
Body mass index z-score	1.07	(0.79, 1.45)	0.08	(−0.04, 0.13)	−0.04	(−0.12, 0.07)
Intervention factors
Similarity to promotora	1.22	(0.75, 1.99)	−0.05	(−0.18, 0.09)	−0.12	(−0.22, 0.03)
Dose delivered
Number of promotora home visits	1.06	(0.90, 1.25)	−0.01	(−0.05, 0.04)	−0.31	(−0.11, −0.04)***
Minutes of promotora in-person contact	1.00	(0.99, 1.00)	−0.01	(−0.00, 0.00)	−0.28	(−0.00, −0.00)***
Days to complete intervention	0.99	(0.98, 1.01)	0.002	(−0.00, 0.00)	−0.22	(−0.01, −0.00)**
Dose received
% of family in attendance at home visit	0.31	(0.04, 2.45)	0.00	(−0.57, 0.59)	0.00	(−0.58, 0.59)
Effort put into the program	0.86	(0.70, 1.06)	0.09	(−0.02, 0.09)	0.09	(−0.02, 0.08)
Satisfaction with DVD series	1.00	(0.53, 1.88)	0.23	(0.09, 0.44)**	0.09	(−0.07, 0.26)
Use of manual outside home visits (no)	1.00
Yes	1.71	(0.61, 4.80)	−0.00	(−0.28, 0.28)	0.07	(−0.14, 0.37)
Minutes/week used manual outside of home visits	0.99	(0.98, 1.01)	−0.01	(−0.00, 0.00)	0.16	(0.00, 0.01)*
Shared manual with others (no)	1.00
Yes	0.97	(0.50, 1.90)	0.15	(−0.01, 0.37)^{^}	0.09	(−0.08, 0.27)

Note. OR = odds ratio; CI = confidence interval.

Logistic regression (<½ serving change [referent] vs. ≥½ serving change). ^bLinear regression. ^cReference group shown in parentheses.

p < .10. *p < .05. **p < .01. ***p < .001.

In terms of behavioral strategies, mothers who reported greater satisfaction with the DVD series reported more frequent use of behavioral strategies to increase fiber intake (standardized β = .23, p ≤ .01). Numerous process indicators were significantly predictive of increases in the use of strategies to decrease fat intake at the bivariate level (Table 2). Total number of promotora visits, minutes of promotora in-person contact, and intervention length were significant predictors, but the relationship was inverse of what was expected. More home visits (standardized β = −0.31, p ≤ .001), more minutes of promotora contact (standardized β = −0.28, p ≤ .001) and greater intervention length (standardized β = −0.22, p ≤ .01) were associated with the use of fewer behavioral strategies to decrease fat intake. Conversely, more frequent weekly use of the family manual was associated with greater use of strategies to decrease fat intake (standardized β = .16, p ≤ .05).

Multivariable Analyses

The results of the multivariable analyses are shown in Table 3. Given our conceptual approach to program evaluation, all process evaluation variables examined at the bivariate level were included in the multivariable analyses with two exceptions. Total number of promotora visits was excluded from the multivariable analyses due to strong correlations with total minutes of contact (r = .87, p ≤ .001) and intervention length (r = .79, p ≤ .001). No other multicollinearity issues were found. Additionally, the dichotomous variable assessing use of the family manual outside promotora visits (yes or no) was excluded in favor of minutes/week used the manual due to a lack of variance in the former (i.e., 88.5% of mothers used the manual on their own time). Multivariable analyses controlled for mother’s education, BMI, and food assistance status.

Table 3.

Multivariable Regression Analysis Examining Predictors of Change in Three Dietary Behaviors: Vegetable Intake, Use of Strategies to Increase Fiber, and Use of Strategies to Decrease Fat Intake^a.

	Change in vegetable intake^b		Change in strategies to increase fiber^c		Change in strategies to decrease fat^c
Intervention factor	OR	(95% CI)	Standardized β	(95% CI)	Standardized β	(95% CI)
Intervention factors
Similarity to promotora	1.11	(0.64, 1.91)	−0.07	(−0.20, 0.08)	−0.11	(−0.22, 0.04)
Dose delivered
Minutes of promotora in-person contact	1.00	(0.99, 1.00)	−0.02	(0.00, 0.00)	−0.03	(−0.00, 0.00)
Days to complete intervention	0.99	(0.98, 1.01)	−0.01	(0.00, 0.00)	−0.15	(−0.01, 0.00)^{^}
Dose received
% of family in attendance at home visit	0.50	(0.05, 5.41)	−0.02	(−0.72, 0.55)	0.07	(−0.37, 0.81)
Effort put into the program	0.88	(0.70, 1.10)	0.01	(−0.06, 0.06)	0.04	(−0.04, 0.07)
Satisfaction with DVD series	1.01	(0.51, 1.99)	0.24	(0.09, 0.45)**	0.11	(−0.05, 0.29)
Minutes/week used manual outside of home visits	0.99	(0.98, 1.01)	−0.05	(−0.00, 0.00)	0.13	(−0.00, 0.00)
Shared manual with others (no)	1.00
Yes	0.88	(0.41, 1.87)	0.15	(−0.02, 0.39)^{^}	0.07	(−0.11, 0.27)

Note. OR = odds ratio; CI = confidence interval; BMI = body mass index.

Analyses controlled for mother’s education, BMI, and enrollment in food assistance programs. ^bLogistic regression (<½ serving change [referent] vs. ≥½ serving change). ^cLinear regression.

p < .10. *p < .05. **p < .01. ***p < .001.

No process-related variables were significantly related to changes in servings of vegetable consumption after adjustment. Only education was significantly related to vegetable intake; mothers with greater than a high school education were nearly two times as likely to increase their vegetable consumption by at least ½ serving (OR = 2.12, p ≤ .05; data not shown). In the multivariable model examining predictors of use of strategies to increase dietary fiber, only mothers’ satisfaction with the DVD series remained significant (standardized β = 0.24, p ≤ .001; adjusted R² = .02). Finally, no variables were significantly related to use of behavioral strategies to reduce dietary fat intake (adjusted R² = .02), after adjustment.

Discussion

This study examined the implementation of a successful intervention targeting the dietary behaviors and intake of Mexican-origin mothers. Results indicated that the intervention was delivered with fidelity, and mothers were satisfied with the intervention. Additionally, this evaluation examined dose–response relationships between intervention dose and dietary behavior changes, an approach that is typically not included in traditional outcome and process evaluations (Blue & Black, 2005). These results are useful for understanding the ways in which the intervention was successful and the specific components that led to dietary changes among mothers.

As Carroll et al. (2007) point out, without appropriate measurement of intervention fidelity researchers cannot expect to understand how or why an intervention was successful.

The results of this process evaluation indicate Entre Familia was implemented with fidelity. Dose delivered was excellent; promotoras spent an average of 918 minutes (~16 hours) across a mean of over 10 visits with participating families compared to the planned 990 minutes across 11 visits, though the intervention length was slightly longer than planned. Dose received was also exceptional, both in terms of mothers’ exposure to the intervention and satisfaction with the program. A majority of mothers reported using the manual outside of the promotora home visits and sharing the program materials with others; these forms of generalization are important in the behavior change process and thus may be instrumental for sustaining such changes (Bandura, 1986). Additionally, participants were very satisfied with the program components and reported putting forth almost the maximum amount of effort toward the program. These metrics of dose received connote a high degree of participant responsiveness to, or engagement in, the intervention because they indicate participants’ independent interest in the intervention content. Although participant engagement is often not reported in standard process evaluations, it is an important predictor of program outcomes (Berkel, Mauricio, Schoenfelder, & Sandler, 2011).

Examination of process evaluation measures as predictors of changes in three specific dietary behaviors revealed several interesting associations. Having more than a high school education was associated with greater increases in vegetable consumption (data not shown). While the Entre Familia materials and curriculum were specifically developed for use in populations with low literacy levels, this finding suggests continued efforts must be made to reach participants with limited education. Use of and satisfaction with program materials, indicators of dose received, predicted use of strategies to increase dietary fiber. Participant satisfaction with the DVD series significantly predicted increased use of strategies to increase fiber intake. Though the use of similar edutainment tools are not uncommon (Brown, 2012), it is a novel strategy for dietary interventions and this result indicates it can be effective for improving dietary behaviors. Additionally, a trend was observed such that sharing the family manual with other friends and relatives was associated with more frequent use of strategies to increase fiber. It is possible that in sharing the manual with individuals in their social network, mothers experienced increased social support to change their dietary behaviors related to fiber intake. Additionally, sharing the manual could indicate that mothers were particularly responsive to the program content, and therefore were more motivated to make behavior changes.

Although relationships were observed between dose received and behavior change, no indicators of dose delivered significantly predicted behavioral outcomes in the multivariable analyses. The only relationship that approached significance was days to complete the intervention; however, counterintuitively, longer intervention length predicted less frequent use of strategies to decrease dietary fat consumption. The former findings support the importance of participant engagement in dietary interventions; while the quantity of intervention exposure is certainly important, participant responsiveness to the intervention content and design is equally, if not more important in the behavior change process (Berkel et al., 2011). The results of this study indicate that Entre Familia successfully engaged participants in the intervention, which contributed to the efficacy of the program. The trends observed on intervention length are consistent with a previous study, which found that intervention length predicted relapse to baseline levels of dietary fat (Baquero et al., 2009). It is possible that mothers became somewhat dependent on the promotoras, making change harder once contact ended. Additionally, extended intervention length may reflect postponements and/or rescheduling of planned promotora visits, suggesting that these families may have been less engaged than those who completed it as planned.

Limitations

Several limitations should be addressed. First, Entre Familia was implemented in a unique region with a predominantly Mexican-origin sample. Thus, the results of this study and the techniques applied may not generalize to other Latino subgroups. Future research should examine the potential generalizability of this approach to dietary change. Second, the participants completed in-person interviews with Evaluation Assistants and extant research points to the potential for Latino participants to respond in a socially desirable way. The Evaluation Assistants were trained to minimize biases during the interview process but it remains a potential limitation. Third, dose delivered measures were based on promotoras’ hand-written notes and are subject to error. Nevertheless, promotoras underwent extensive training (Ayala et al., 2012) to ensure appropriate record keeping procedures were followed. Promotora notes were summarized, submitted electronically, and cross-referenced with hard copy notes. Few discrepancies were found and were addressed as needed.

Conclusions

Entre Familia: Reflejos de Salud was a successful intervention that employed a novel combination of intervention strategies to promote healthy eating. Findings from this process evaluation indicate that Type III error did not occur and this type of intervention can be implemented with fidelity. Taken together, the outcome evaluation results (Ayala, Ibarra, Horton et al., 2015; Horton et al., 2013) and these process evaluation results support the need to translate these types of evidence-based intervention into clinical practice and delivered by promotoras. Efforts are now underway to test the potential to sustain a modified version of this intervention as part of an Obesity Care Model implemented by the federally qualified health center and assess its cost effectiveness (Ayala, Ibarra, Bingelli-Vallarta et al., 2015).

Although this study did not identify significant relationships between many of the process measures and behavior change, the results add to the literature regarding appropriate process measures of intervention delivery regardless of target behavior or population. Where and when possible, interventions should include stringent process evaluation measures and report the results. Over time, this information will improve our understanding of the relationship between implementation fidelity and program outcomes, and ultimately advance the field.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

Applied Research Program, National Cancer Institute, National Institutes of Health. (2000). Percent energy from fat screener: Overview. Retrieved from www.riskfactor.cancer.gov/diet/screeners/fat

Ayala

G. X.

Ibarra

Arredondo

Horton

Hernandez

Parada

. . . Elder

J. P.

(2012). Promoting healthy eating by strengthening family relations: Design and implementation of the Entre Familia: Reflejos de Salud intervention. In Elk

Landrine

(Eds.), Cancer disparities: Causes and evidence-based solutions (pp. 237-251). New York, NY: Springer.

Ayala

G. X.

Ibarra

Binggeli-Vallarta

Moody

McKenzie

T. L.

Angulo

. . . Elder

J. P.

(2015). Our Choice/Nuestra Opción: The Imperial County, California Childhood Obesity Research Demonstration Study (CA-CORD). Childhood Obesity, 11, 37-47.

Ayala

G. X.

Ibarra

Horton

Arredondo

E. M.

Slymen

D. J.

Engelberg

. . . Elder

J. P.

(2015). Evidence supporting a promotora-delivered entertainment education intervention for improving mothers’ dietary intake: The Entre Familia: Reflejos de Salud study. Journal of Health Communication, 20, 165-176.

Ayala

G. X.

Mueller

Lopez-Madurga

Campbell

N. R.

Elder

J. P.

(2005). Restaurant and food shopping selections among Latino women in Southern California. Journal of the American Dietetic Association, 105, 38-45. doi:10.1016/j.jada.2004.10.023

Ayala

G. X.

Vaz

Earp

Elder

J. P.

Cherrington

(2010). Outcome effectiveness of the lay health advisor model among Latinos in the United States: An examination by role. Health Education Research, 25, 815-840. doi:10.1093/her/cyq035

Baldwin

S. A.

Wampold

B. E.

Imel

Z. E.

(2007). Untangling the alliance-outcome correlation: Exploring the relative importance of therapist and patient variability in the alliance. Journal of Consulting and Clinical Psychology, 76, 842-852. doi:10.1037/0022-006X.75.6.842

Bandura

(1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall.

Bandura

(1989). Social cognitive theory. In Vasta

(Ed.), Annals of child development: Six theories of child development: Revised formulations and current issues (pp. 1-60). Greenwich, CT: JAI Press.

10.

Baquero

Ayala

G. X.

Arredondo

E. M.

Campbell

N. R.

Slymen

D. J.

Gallo

Elder

J. P.

(2009). Secretos de la Buena Vida: Processes of dietary change via a tailored nutrition communication intervention for Latinas. Health Education Research, 24, 855-866. doi:10.1093/her/cyp022

11.

Basch

C. E.

Sliepcevich

E. M.

Gold

R. S.

Duncan

D. F.

Kolbe

L. J.

(1985). Avoiding type III errors in health education program evaluation: A case study. Health Education & Behavior, 12, 315-331. doi:10.1177/109019818501200311

12.

Berkel

Mauricio

A. M.

Schoenfelder

Sandler

I. N.

(2011). Putting the pieces together: An integrated model of program implementation. Prevention Science, 12, 23-33.

13.

Bill

D. E.

Hock-Long

Mesure

Bryer

Zambrano

(2009). Healthy Start Programa Madrina: A promotora home visiting outreach and education program to improve perinatal health among Latina pregnant women. The Health Educator, 41(2), 68-76.

14.

Blue

C. L.

Black

D. R.

(2005). Synthesis of intervention research to modify physical activity and dietary behaviors. Research and Theory for Nursing Practice, 19, 25-61.

15.

Brown

W. J.

(2012). Promoting health through entertainment-education media. In Obregon

Vaisbord

(Eds.), The handbook of global health communication (pp. 121-143). Oxford, England: Blackwell.

16.

California Health Interview Survey. (2014). 2011-2012 Health profiles: Adults. Retrieved from http://healthpolicy.ucla.edu/health-profiles/adults/Documents/2011-2012/Counties/Imperial.pdf

17.

Carroll

Patterson

Wood

Booth

Rick

Balain

(2007). A conceptual framework for implementation fidelity. Implementation Science, 2, 40. doi:10.1186/1748-5908-2-40

18.

Centers for Disease Control and Prevention. (2010). State-specific trends in fruit and vegetable consumption among adults in the United States, 2000-2009. Morbidity and Mortality Monthly Report, 59(35), 1125-1130. Retrieved from www.cdc.gov/mmwr/pdf/wk/mm5935.pdf

19.

Crowley

D. M.

Coffman

D. L.

Feinberg

M. E.

Greenberg

M. T.

Spoth

R. L.

(2013). Evaluating the impact of implementation factors on family-based prevention programming: Methods for strengthening causal inference. Prevention Science, 15, 246-255. doi:10.1007/s11121-012-0352-8

20.

Deitrick

L. M.

Paxton

H. D.

Riviera

Gertner

E. J.

Biery

Letcher

A. S.

. . .Salaz-Lopez

(2010). Understanding the role of the promotora in a Latino diabetes education program. Qualitative Health Research, 20, 386-399. doi:10.1177/1049732309354281

21.

Elder

J. P.

Arredondo

E. M.

Campbell

N. R.

Baquero

Duerksen

Ayala

G. X.

. . . McKenzie

(2010). Individual, family, and community environmental correlates of obesity in Latino elementary school children. Journal of School Health, 80, 20-30. doi:10.1111/j.1746-1561.2009.00465.x

22.

Elder

J. P.

Ayala

G. X.

Campbell

N. R.

Slymen

Lopez-Madurga

E. T.

Engleberg

Baquero

(2005). Interpersonal and print nutrition communication for a Spanish-dominant Latino population: Secretos de la Buena Vida. Health Psychology, 24(1), 49-57.

23.

Elder

J. P.

Ayala

G. X.

Slymen

D. J.

Arredondo

E. M.

Campbell

N. R.

(2009). Evaluating psychosocial and behavioral mechanisms of change in a tailored communication intervention. Health Education & Behavior, 36, 366-380. doi:10.1177/1090198107308373

24.

Elkin

(1999). A major dilemma in psychotherapy outcome research: Disentangling therapists from therapies. Clinical Psychology: Science and Practice, 6, 10-32.

25.

Horton

L. A.

Parada

Slymen

D. J.

Arredondo

Ibarra

Ayala

G. X.

(2013). Targeting children’s dietary behaviors in a family intervention: “Entre familia: Reflejos de Salud.” Salud Publica de Mexico, 55, S397-S405.

26.

Keller

C. S.

(2008). Camina por salud: Walking in Mexican-American women. Applied Nursing Research, 21, 110-113. doi:10.1016/j.apnr.2006>.12.003

27.

Linnan

Steckler

(Eds.). (2002). Process evaluation for public health interventions and research. San Francisco, CA: Jossey-Bass.

28.

Michie

Richardson

Johnston

Abraham

Francis

Hardeman

. . . Wood

C. E.

(2013). The behavior change taxonomy (v1) of 93 hierarchically clustered techniques: Building an international consensus for the reporting of behavior change interventions. Annals of Behavioral Medicine, 46, 81-95. doi:10.1007/s12160-013-9486-6

29.

Moore

Audrey

Barker

Bond

Bonnell

Cooper

. . . Baird

(2014). Process evaluation in complex public health intervention studies: The need for guidance. Journal of Epidemiology and Community Health, 68, 101-102. doi:10.1136/jech-2013-202869

30.

Oakley

Strange

Bonell

Allen

Stephenson

RIPPLE Study Team. (2006). Process evaluation in randomized controlled trials of complex interventions. British Medical Journal, 332, 413-416. doi:10.1136/bmj.332.7538.413

31.

Ogden

C. L.

Carroll

M. D.

Kit

B. K.

Flegal

K. M.

(2014). Prevalence of childhood and adult obesity in the United States, 2011-2012. Journal of the American Medical Association, 311, 806-814.

32.

Perez-Escamilla

Hromi-Fiedler

Vega-Lopez

Bermudez-Millan

Segura-Perez

(2009). Impact of peer nutrition education on dietary behaviors and health outcomes among Latinos: A systematic literature review. Journal of Nutrition Education and Behavior, 40, 208-225. doi:10.1016/j.jneb.2008.03.011

33.

Pleis

J. R.

War

B. W.

Lucas

J. W.

(2010, December). Summary health statistics for U.S. adults: National Health Interview Survey, 2009. Vital Health Statistics, Series 10, (249), 1-207.

34.

Reinschmidt

K. M.

Hunter

J. B.

Lourdes Fernandez

Lacy-Martinez

C. R.

Guernsey

Zapien

Meister

(2006). Understanding the success of promotoras in increasing chronic disease screening. Journal of Health Care for the Poor and Underserved, 17, 256-264.

35.

Saunders

R. P.

Evans

M. H.

Joshi

(2005). Developing a process-evaluation plan for assessing health promotion program implementation: A how-to guide. Health Promotion Practice, 6, 134-147. doi:10.1177/1524839904273387

36.

Sperber

A. D.

Devellis

R. F.

Boehlecke

(1994). Cross-cultural translation: Methodology and validation. Journal of Cross-Cultural Psychology, 25, 501-524.

37.

Thompson

F. E.

Kipnis

Subar

A. F.

Krebs-Smith

S. M.

Kahle

L. L.

Midthune

. . . Schatzkin

(2000). Evaluation of 2 brief instruments and a food-frequency questionnaire to estimate daily number of servings of fruits and vegetables. American Journal of Clinical Nutrition, 71, 1503-1510.

38.

U.S. Census Bureau. (2014). State & County Quickfacts: Imperial County, CA. Retrieved from http://quickfacts.census.gov/qfd/states/06/06025.html