Randomized Trial of Healthy Families Arizona

Home visitation research remains on the forefront, as policy makers look to evidence for programs they believe are worthy of investment. This interest is fueled by the potential wide ranging and significant outcomes associated with home visitation. Home visitation programs expect outcomes across many domains: child maltreatment rates, positive and nurturing parenting behaviors, prenatal and child health, child development and school readiness, and maternal educational and employment success. The infusion of significant dollars from the Maternal, Infant, and Early Childhood Home Visitation Program; the Children’s Bureau; and various state sources accentuates the need for new research and lessons learned that can influence future implementation and build more effective practice models for home visitation.

There are many home visitation programs being implemented across the country. Programs deemed evidence based (see Avellar, Paulsell, Sama-Miller, & Grosso, 2014) are being the most carefully reviewed for ongoing evidence. A total of 19 programs meet Department of Health and Human Services (DHHS) criteria for evidence including Child FIRST, Durham Connects/Family Connects, Early Head Start-Home Visiting, Early Intervention Program for Adolescent Mothers, Early Start (New Zealand), Family Check-Up, Family Spirit, Healthy Families America (HFA), Healthy Steps, Home Instruction for Parents of Preschool Youngsters, Maternal Early Childhood Sustained Home Visiting Program, Minding the Baby, Nurse Family Partnership, Oklahoma’s Community-Based Family Resource and Support Program, Parents as Teachers, Play and Learning Strategies, Infant, and SafeCare Augmented.

Within this group of evidence-based programs, one of the most widely adopted programs is HFA. This program, like the other home visitation programs, still has limited evidence of program effectiveness (Avellar et al., 2014).

Home visitation services include a broad set of goals for families. Most home visitation studies focus on four primary areas that include child safety and health, parenting practices, maternal health and self-sufficiency, and mental health and coping. There are five recent experimental studies that have examined the effectiveness of the Healthy Families program model using a variety of outcome measures. A series of studies in New York found positive results showing reductions in several types of abusive and neglectful parenting practices (DuMont et al., 2008) and positive results in reducing the risk of a low birth weight baby (Lee et al., 2009). A further analysis (DuMont et al., 2010) found more positive outcomes on measures of reduction of harsh discipline for first-time, prenatally enrolled mothers when compared with the control participants. One study (Barlow et al., 2006), less recent but often not cited, is of a Healthy Families program that examined program outcomes with an American Indian population. This is one of the few studies looking at program impacts within a minority population. This small randomized trial was conducted with one Apache and three Navajo communities where paraprofessionals delivered the program prenatally. Program participants showed positive impacts on measures of parent knowledge and maternal involvement when compared with a control group. A Massachusetts study (Easterbrooks et al., 2012) found mothers enrolled in the Healthy Families program reported less parenting stress than control mothers. There were no program differences between the two groups on measures of self-reported child maltreatment, and in one analysis, there was a significant difference from the control group of reported cases of child maltreatment in favor of the Healthy Families group.

Another recent study (LeCroy & Krysik, 2011) found positive results on reductions of harsh discipline similar to the New York study (DuMont et al., 2008). The Arizona study also found positive results in comparison to the control condition on use of safety practices, parenting attitudes (e.g., inappropriate expectations), reading to children, use of resources, reduced alcohol use, and greater maternal education and training. A recently published study (Green, Tarte, Harrison, Nygren, & Sanders, 2014) reported results from a randomized trial that found positive outcomes showing Healthy Families mothers read more frequently to their children, provided more developmentally supportive activities, and had less parenting stress than the control group.

The areas of focus that continue to direct research attention in home visitation include quality of service delivery and implementation (Azzi-Lessing, 2013); timing of service initiation (prenatal or postnatal enrollment) and limitation of services to first-time moms or multiparous mothers (Huntington & Galano, 2013); and families with high-risk factors such as depression, violence, and substance abuse (Ammerman, Putnam, Bosse, Teeters, & Van Ginkel, 2010).

Depression has been strongly linked to poor parenting and associated with child maltreatment (see, e.g., Ammerman et al., 2010; Conron, Beardslee, Koenen, Buka, & Gortmaker, 2009; Duggan, Caldera, Rodriguez, Burrell, & Crowne, 2007; Easterbrooks et al., 2013). The research question most often examined was, what impact does maternal depression have on program outcomes? Ammerman, Putnam, Bosse, Teeters, and Van Ginkel (2010) have focused research on depressed mothers, and in their initial study, they found that depressed mothers were less likely to benefit from home visitation services. Other researchers have also found nondepressed mothers obtained better outcomes (Easterbrooks et al., 2012; Green et al., 2014; Mitchell-Herzfeld, Izzo, Greene, Lee, & Lowenfels, 2005). Stevens, Ammerman, Putnam, and Van Ginkel (2002) report that maternal depression was present for 30% of the Healthy Families participants. Also, Duggan, Berlin, Cassidy, Burrell, and Tandon (2000) report that program outcomes for depressed mothers can be negatively impacted by attachment style.

Research continues to stress the importance of program implementation and fidelity. Research studies have found that families that receive more home visits (higher program dosage) are more effective (Howard & Brooks-Gunn, 2009). More recently studies (see Kahn & Moore, 2010) found that program intensity measured by the number of home visits within a set time period had the greatest impact. In particular, programs that achieved more than four home visits per month over a 1-year period had more positive outcomes than did programs with fewer visits (Kahn & Moore, 2010). Nievar, Van Egern, and Pollard (2010) found that two visits per month was a critical threshold for showing impact. Green, Tarte, Harrison, Nygren, and Sanders (2014, p. 6) summarize the issue: “the quality of program implementation, and in particular the dosage, frequency, and content of home visits is a near-universal challenge for home visiting programs and associated research.” The recent report by Boller et al. (2014), Making Replication Work, examined program implementation across the sites funded by the Evidence-Based Home Visiting initiative to prevent child maltreatment. The study found that all home visitation programs struggled to maintain caseloads and deliver service at the intended intensity. In addition, they found that fidelity of implementation was partly a function of “model factors” and that high-risk families were more likely to leave the program early. A study by Allen (2007) concluded that the relationship between the home visitor and the parent may impact the intensity of services that the parents receive.

In spite of new findings, reviewers continue to characterize home visitation programs as having uncertain impact, for example, “despite their growing popularity, there is considerable uncertainty regarding the efficacy of home visitation programs to produce meaningful and lasting outcomes for the children and families they serve” (Azzi-Lessing, 2013, p. 377). Furthermore, because home visitation is receiving significant fiscal support, ongoing evidence of program effectiveness is an important research priority. Therefore, it is critical that ongoing documentation of outcomes—beyond child maltreatment—be presented and published. The present study adds to this existing literature by conducting a randomized control trial of a well-established HFA program in a state that included an extensive quality assurance program, used a well-developed curriculum, and has obtained statewide accreditation. Furthermore, the present study is an important contribution because it extends the examination of potential outcomes by including several additional measures to test broader aspects of the program’s impact.

Although most of the literature on Healthy Families evaluations has been quantitative outcome studies, there has been an increasing interest in augmenting outcome studies with qualitative methods (McCall & Green, 2004). In a review of critical issues in home visitation, Azzi-Lessing (2011, p. 394) states, “qualitative evaluation methods should be employed to capture the experiences and responses of families served by home visitation programs.” Many studies have focused on either a quantitative approach or a qualitative approach. The current study includes a rigorous experimental study and adds an innovative qualitative method to the analysis.

Method

Participants

Participant families were recruited from local hospitals. Prior to enrollment, participants were explained the nature of the research study and told that with their consent to participate, they would be randomly assigned to either an intervention group that included home visits or a child development group that included parent and child assessments. Participants were included in the study on the basis of risk assessment. A hospital screen involved identification of risk factors present including such factors as low income or being a single parent. Two or more risk factors indicate a positive screen. A second-tier risk screen was conducted using the Parent Survey based on the Kempe Family Stress Checklist (Prevent Child Abuse America, 2000), which is a semistructured interview that asks about stress, parents’ childhood history, potential for violence, stressors, parents’ expectations, and other areas. Participants who scored 25 or higher on this measure were eligible for the services. The interview is completed and scored, and a risk threshold must be met to continue to meet criteria for program inclusion. These are the same screening and assessment procedures that are used for eligibility into the Healthy Families program. The study took place in Arizona from 2010 to 2014 and covered mostly urban and some rural settings. The study was primarily located in Tucson, a medium size urban city, but also included services in smaller outlying towns and cities in southern Arizona.

Randomization

This study employed a randomized, intent-to-treat design that examines short-term outcomes from program participation. Families were randomized into the treatment or control group by research staff using random number procedures and group status was blind to data collection staff. By design, more families were assigned to the control group than the intervention group. This was done because enrollment into the treatment group depended on available slots from the program but enrollment in the control group did not, therefore randomization switched to a 2:1 randomization in order to increase the power of the experiment (Dumville, Hahn, Miles, & Torgerson, 2006). Those families assigned to the Healthy Families treatment group received the normal course of services available from the Healthy Families Arizona program and those assigned to the “Child Development Group” received assessment information about their child’s developmental progress and referrals as needed. The latter group constituted the control group. This design was deemed the minimal level of information that would still be valuable enough for participants to agree to participate in the study. To increase the likelihood of participation in the study and to reduce differential attrition from the control group, a minimal level of services were provided. Offering a minimal level of intervention was also important for the program staff who were uncomfortable with families receiving no services when they were at risk and in need of assistance. The study protocol obtained institutional review board approval before the commencement of the study. Multiple outcome measures were selected to assess the program’s impact across four theoretical domains of interest including safety and resources, parenting attitudes and practices, health and maternal outcomes, and mental health and coping. Some measures were administered at baseline and 6 months, parenting measures were administered only at the 6-month follow-up period.

Participant recruitment took place in several hospitals in the maternity ward. A total of 2,426 individuals constituted the recruitment population, of these 1,430 were not eligible (see consort diagram), 120 were eligible but not participating, and 631 were not available to randomize into the study.

A total of 245 participants were invited into the study and randomized. As noted in Figure 1, participants were not recruited if they were out of state, not present when recruitment staff visited, were discharged from the hospital, or expressed no interest in participating in the study. A total of 245 families met study criteria for inclusion and signed informed consent agreements. The total sample included in this study for analysis was 245 families randomized at baseline (147 control and 98 experimental); of this total, 100% completed baseline interviews. At the 6-month assessment; 79 (81%) were retained in the Healthy Families condition and 120 (82%) were retained in the control condition. Participants received US$20.00 incentives at baseline and at the 6-month follow-up assessment.

OPEN IN VIEWER

Figure 1.

Consort diagram for participant flow.

Results

Table 1 presents information on the background characteristics of the participants in the experimental and control groups. As the table shows the groups had similar baseline equivalence and there were no significant differences between the groups. Both groups reported a high school graduation or a GED (58% both groups), emotional abuse was reported by 22.7% of the treatment group and 25.7% of the control group, and past involvement with child protective services (CPS) included 12.3% (treatment group) and 14.1% (control). These indicators suggest baseline equivalence and show that the program was targeting a group of families that could be considered at risk for child abuse and neglect.

OPEN IN VIEWER

Table 1.

Baseline Comparisons of the Healthy Families and Control Groups.

Characteristics	Experimental Condition
	Healthy Families (n = 98)	Control Group (n = 147)	Significance
Demographic characteristics
Hispanic	74.2%	74.8%	NS
White	7.2%	15.0%	NS
Average number children prior to birth	1.4	1.2	NS
Mother’s average age in years	26.9 (6.8)	25.8 (5.9)	NS
Prenatal and birth characteristics
Average birth weight	7.1 (1.5)	7.2 (1.3)	NS
Average number of prenatal visits	12.8 (5.3)	13.1 (5.6)	NS
Children prior to current birth	1.4 (1.5)	1.2 (1.2)	NS
Smoked during pregnancy	6.2%	7.4%	NS
Used alcohol during pregnancy	5.2%	5.4%	NS
Received prenatal care	97.9%	98.6%	NS
Income related factors
Health Insurance with AHCCCS	71.6%	62.2%	NS
Mother employed	21.6%	28.4%	NS
Own a car	44.3%	50.7%	NS
History of childhood maltreatment
Neglected by caretakers	20.6%	16.9%	NS
Emotional abuse	22.7%	25.7%	NS
Sexual abuse	12.4%	12.8%	NS
Involvement with CPS as a parent	12.3%	14.1%	NS

Note. CPS = child protective services; NS = not significant; AHCCCS is the state of Arizona Medicaid program.

Table 2 presents key outcome measures across four different domains including safety and resources, parenting attitudes and practices, mental health and coping, and maternal outcomes. In each of the major domains, there was a significant outcome for the Healthy Families Arizona experimental group in contrast to the control group. In the safety and resources domain, outcomes at 6 months showed the Healthy Families experimental group had implemented more safety practices in the home (p = .05, d = .24), used more resources to meet family needs (p = .10, d = .24), and scored higher on mobilizing resources (p = .007, d = .43).

OPEN IN VIEWER

Table 2.

Comparison of Outcome Measures by Group.

	Healthy Families Group		Control Group
	Baseline	6 months	Baseline	6 months
Outcome	M (SD)	M (SD)	M (SD)	M (SD)	p	d
Safety and resources
Safety practices		23.6 (1.8)		22.88 (1.90)	.003*	.44
Use of resources	3.4 (1.6)	4.1 (1.7)	2.2 (1.2)	3.0 (1.9)	.10*	.24
Mobilizing resources			22.4 (5.4)	24.6 (2.1)	.007*	.43
Parenting attitudes and practices
Mother’s reading		4.08 (.96)		3.64 (1.28)	.01*	.38
Home environment		42.8 (5.1)		39.9 (5.1)	.003*	.47
Role satisfactiona		25.7 (4.5)		26.9 (3.9)	.06*	.33
Parent/child behavior		46.0 (4.0)		44.9 (4.7)	.13	.24**
Parent efficacy		26.2 (3.7)		25.8 (3.7)	.47	.11
Regular routines		1.8 (.7)		1.6 (.8)	.02*	.36
Reduced chaotic household		1.2 (.5)		1.4 (.7)	.04*	.29
Health and maternal outcomes
Breastfeeding		89%		78%	.04*	.29
Immunizations		13.7 (3.7)		13.7 (3.6)	.79	.04
Well-baby checks		4.0 (1.9)		4.1 (1.9)	.70	.06
Contraception use		76%		66%	.14	.21**
Subsequent pregnancy		1 participant		3 participants
Job training or employment		49%		52%	.54	.09
Substance abuse treatment		1 participant		1 participant
Mental health and coping
Mental Health Index		19.0 (1.7)		24.7 (16.7)	.02*	.35
Depression		39.8 (5.2)		39.8 (5.5)	.85	.03
Social support		21.6 (4.0)		20.6 (4.0)	.26	.17
Problem-solving		24.6 (3.9)		23.8 (3.9)	.20	.20**
Personal Care		19.2 (3.7)		18.7 (3.6)	.38	.14

^aDifference favored the control group.

*p < .10. **Meaningful effect size of d ≥ .20.

With regard to parenting attitudes and practices, four outcomes favored the Healthy Families condition including quality of the home environment (p = .003, d = .47), regular routines (p = .02, d = 36), reduced chaotic household (p = .04, d = .29), and reading to the child (p = .03; d = .31). Positive parent/child behavior was not significant but obtained a meaningful effect size (p = .13, d = .24). Two measures showed no difference between the two groups (role satisfaction and parent efficacy). Role satisfaction favored the control group over the Healthy Families experimental condition (p = .06, d = .32).

Results for the health and maternal outcomes domain found Healthy Families participants had higher rates of breast feeding (p = .04, d = .29) and greater contraception use (p = .14, d = .21). There were no differences between the two groups on immunizations, well-baby checks, subsequent pregnancy, job training, or employment, and substance abuse treatment.

Mental health and coping measures found two positive outcomes for the Healthy Families participants in comparison to the control condition, the Mental Health Index (p = .02, d = .35) was higher in the Healthy Families group. Problem-solving did not show a significant difference but obtained an effect size that favored the Healthy Families group (p = .20, d = .20). There were no significant between-group differences on depression, social support, or personal care.

An examination of program dosage revealed several significant findings. Dosage was measured as the total number of home visits received. Families who received more home visits showed significantly better improvement on the following outcomes: social support (t = 2.27, p = .03), depression (t = 2.29, p = .03), role satisfaction (t = 2.07, p = .04), problem-solving (t = 1.79, p = .08), breast-feeding (t = 1.71, p = .09), and Mental Health Index (t = 2.72, p = .01). Trends showing a relationship between program dosage and outcomes were evident for parental efficacy (t = 1.49, p = .14).

Qualitative Results

Results revealed many significant differences in linguistic categories between the treatment and control group (see Table 3). In particular, at the 6-month comparisons, 11 comparisons showed significance, all of which favored the Healthy Families treatment group. The Healthy Families group expressed more positive emotions (p = .02, d = .37), less negative emotions (p = .08, d = .29), less sadness (p = .01, d = .42), showed more feeling expressions (p = .002, d = .50), had enhanced cognitive mechanisms (p = .007, d = .44), and greater insight (p = .05, d = .33).

OPEN IN VIEWER

Table 3.

Means, Standard Deviations, Significance, and Effect sizes for Linguistic Dimensions for the Treatment and Control Groups: Linguistic Analysis of Treatment and Control Group at 6 months.

Linguistic Dimension	Treatment	Control	p (d)
	6 months	6 months
	n = 75	n = 75
	M (SD)	M (SD)
Past	0.94 (1.6)	1.65 (1.4)	.10* (−.27)
Present	17.0 (5.8)	14.6 (5.5)	.008* (.34)
Future	0.18 (0.77)	.20 (0.67)	.81 (−.02)
First person	3.7 (3.5)	3.3 (4.2)	.60 (.00)
Affective processes	17.4 (8.4)	15.0 (11.4)	.15 (.24)**
Positive valenced	15.3 (8.3)	12.0 (9.2)	.02* (.37)
Negative valenced	1.9 (2.4)	2.8 (3.6)	.08* (−.29)
Anxiety	0.20 (0.74)	0.53 (2.9)	.35 (−.15)
Anger	0.39 (1.1)	0.47 (1.4)	.70 (−.06)
Sad	0.78 (1.3)	1.5 (2.0)	.01* (−.42)
Perceptual processes	4.2 (3.6)	2.9 (4.1)	.04* (.33)
Feeling expression	1.6 (1.8)	0.77 (1.5)	.002* (.50)
Cognitive mechanism	16.4 (6.5)	13.4 (7.2)	.007* (.44)
Insight	3.2 (3.1)	2.2 (3.0)	.05* (.33)
Cause	2.1 (2.3)	1.3 (1.8)	.01* (.39)
Certainty	1.4 (2.5)	0.82 (1.6)	.08* (.27)

*significant α, p < .10. **meaningful effect size of d > .20.

Discussion

This study found positive short-term effects for the Healthy Families group across a variety of domains. The program evaluated is accredited and has been in operation for over 23 years; therefore, implementation of the program is likely to be more consistent and effective than in the past or compared to other programs. The broader range of outcome indicators and the qualitative assessment have also contributed to the findings, as past studies have been limited to a more narrow range of measures.

The strongest program effects were in the areas of safety, parenting practices, the home environment, and mobilization of resources. The safety practices are an important outcome indicator, because unintentional injuries are a leading cause of child death (Deal, Gomby, Zippiroli, & Behrman, 2000). The safety scale had low reliability but when we examined individual items we also found significant differences between the Healthy Families and control group.

Knowing how to use resources may be an important skill for families that face multiple difficulties, as Krysik, LeCroy, and Ashford (2008) note, participation in home visitation may, “provide a positive experience for families that will promote future involvement with other social service programs when families need help in the future” (p. 59).

Outcomes were also observed in the parenting attitudes and behavior domain. The scales of parenting competence, parent/child behavior, and overall home environment all showed positive program effects. The home environment had the largest effect size and is considered a good indicator of the program’s effort to increase the quality of the environment is a manner that promotes positive child development.

There were important findings in the health domain. Particularly important was the finding that Healthy Families mothers reported greater breast feeding than control mothers. This could add significantly to the long-term health impact for the infants. Studies have consistently found that breast milk can improve a baby’s health, lower childhood obesity, decrease asthma, and promote better brain development (Horta & Victora, 2013). Furthermore, no breastfeeding may put women at higher risk for breast and ovarian cancer, diabetes, cardiovascular disease, and other health conditions (Harmon, 2010). Although the results were positive, they were quite modest and programs should target breast-feeding, given the significant benefits. There was a modest impact on use of contraception. This outcome can reap important health benefits, as birth spacing is strongly related to positive health status and child abuse and neglect (Conde-Agudelo, Rosas-Bermudez, Castano, & Norton, 2012; Zuravin, 1988).

The two measures of more routine health status—immunizations, and well-baby visits did not show any program effects. The outcomes for immunizations may be different at the 12-month period than at 6 months when the families are still more strongly connected with health care. The well-baby visits show that both the Healthy Families and the control group are seeking similar health care for their children.

With regard to mental health, the study found a strong effect on the Mental Health Index which measures both psychological distress and psychological well-being. This was the most comprehensive measure of mental health used in the study. Three additional measures in the mental health domain did not show a difference between the Healthy Families and control group: depression, social support, and personal care. However, when we examined the relationship between dosage and outcomes, both the depression scale and the social support scale showed an effect for the Healthy Families group. Personal care and social support are similar concepts, and although home visitation programs provide additional support from the home visitor, the parent may continue to feel a sense of loneliness and have less focus on personal care as a result of her new role as a mother.

McCall and Green (2004) noted the need to augment experimental evaluations with other qualitative methods in the evaluation of home visiting programs. Azzi-Lessing (2011) reviewed home visiting programs and recommended that “qualitative evaluation methods should be employed to capture the experiences and responses of families” (p. 394). The present study adds a new qualitative element to the experimental design that provided robust differences between the intervention and control groups. Furthermore, the results suggest that the program may work in ways not previously expected since intervention families developed a different language or narrative in describing their approach to parenting.

The Growing Great Kids curriculum, used with the majority of the parents in the program, focuses on “cues and communication” teaching parents to accentuate the positive, calls attention to positive process in the relationship, and teaches the steps of building a more positive family environment suggesting words and strategies to accomplish this. As parents construct a different story of who they are as parents, it is likely that their language and corresponding psychological processes change. The research on linguistic analysis has found when people change their perspective in how they talk they tend to function in a more healthy way. For example, the more people use first person pronouns compared to other pronouns, the better their health was (Moore & Brody, 2009; Pennebaker, 2011a; Pennebaker et al., 1997). Rude, Gortner, and Pennebaker (2004) found linguistic differences between depressed and nondepressed individuals, whereby depressed individuals used more negative emotion words and fewer positive emotion words. The qualitative linguistic analysis from this study adds a new dimension in understanding the impact of home visitation services and coupled with the impacts found on the self-report measures adds additional evidence that the program is having an impact on parents.

This study is limited due to the small number of participants. Another consideration in the interpretation of the results is that the control group did not receive services, but they did get ongoing child development assessments, referral information as needed, and had a consistent, caring research assistant who asked them about their child and family life—all actions which diminished the difference between the active treatment and the control condition. The results presented in this study are limited to short-term follow-up period and we do not know if these results will persist at longer follow-up time points. The linguistic analysis is a new qualitative method and it will be important to determine whether other researchers can replicate our findings in this regard.

Research on home visitation continues to evolve. Prevention research of this nature is complex due to measurement difficulties, control group limitations, engagement and retention issues, and the diverse recipients of the services. Continued research must grapple with how to best design studies and measure outcomes on widely diverse programs that service a diverse group of participants. Examining a broader range of potential outcomes may add new understandings to the potential program impact of home visitation programs.