The Relationship Between Family Education and Support Services and Parent and Child Outcomes Over Time

Research Support for FES

The research base supporting FES can be described as slim but encouraging. In the most comprehensive review of the research on FES to date, Hoagwood and colleagues (2010) described family support services as being

directed at meeting the needs of parents or caregivers of children with mental health needs with the explicit purpose of helping parents/caregivers (a) clarify their own needs or concerns; (b) reduce their sense of isolation, stress, or self-blame; (c) provide education or information; (d) teach skills; and (e) empower and activate them, so that they can more effectively address the needs of their families. (p. 3)

FES is commonly delivered through clinician-led, peer-to-peer, or professional–parent team mechanisms. In their review of the family support literature, Hoagwood and colleagues (2010) identified 50 studies that met criteria for inclusion and had evaluation data. Clinician-led programs (n = 33) were the most numerous, followed by family-led (n = 11) and team-led (n = 6) programs. Although there were differences in the rigor of methodology between the three types of programs, the contents of the components were similar with some differentiation of emphasis across the three types. In terms of outcomes, the authors concluded that there was encouraging initial evidence of the value of FES in reducing child symptoms and improving child functioning. Furthermore, some benefits to the parents and caregivers were found, including a reduction of stress, improved mental health and well-being, increased self-efficacy, perceived social supports, and increased treatment engagement.

In an earlier review, Robbins et al. (2008) identified 31 peer-to-peer FES programs; however, only two programs were evaluated using a rigorous design and had a focus on children’s mental health. Results from these studies indicated that receipt of peer-to-peer support was related to increased parent empowerment, increased access to children’s mental health services (Elliot, Koroloff, Koren, & Friesen, 1998), and increased perceived support by parents (Ireys & Sakwa, 2006).

Two later studies completed after these reviews are noteworthy. In the first of these studies, Burnett-Zeigler and Lyons (2010) found that families reporting higher levels of strain at baseline accessed more mental health services for their children and had increased retention of services. Their results were counter to findings from other studies (Angold et al., 1998; Kazdin, Holland, & Crowley, 1997) in which families reporting high strain had low engagement and retention in outpatient mental health services. However, Burnett-Zeigler and Lyons (2010) pointed out that families received support that was not available to families in the other studies, which may have reduced strain and facilitated engagement in services.

In the second of these studies, Kutash, Duchnowski, Green, and Ferron (2011) used a randomized controlled trial to implement and evaluate a peer-to-peer support program for parents of children with SED. They found that, compared with highly strained parents in the comparison group, highly strained parents who participated in the peer support program experienced decreased strain and need for support. In addition, the parents experienced increased empowerment, mental health services efficacy, and hopefulness. Compared with children of parents in the comparison group, children of parents in the peer support group experienced a decrease in emotional impairment and an increase in academic achievement over one school year. These children also had fewer out-of-school suspensions and received more mental health services than children of parents in the comparison group.

Although the results of studies in the current empirical base are encouraging, they are limited in scope and lack rigorous support (Hoagwood et al., 2010), which has limited the potential of FES to improve outcomes for children in need. Because of significant weaknesses and gaps that exist in the extant literature, there is confusion over basic issues such as the core function of FES and the parameters that may mediate or moderate its implementation and effectiveness.

Increased Attention to FES

Efforts to increase the provision of FES to families have received a major boost from the implementation of the Substance Abuse and Mental Health Services Administration’s Comprehensive Mental Health Initiative (CMHI). The CMHI is the largest federally funded service program in children’s mental health and promotes the implementation of services that value families and are family driven. In a study of 39 communities funded between 1997 and 2000 (Gyamfi et al., 2010), FES was examined in 2,853 families who were enrolled in their community CMHI program. Results indicated that about one third of the families received FES for an average of 14.5 times over a 6-month period. FES included a broad range of services, such as parent training and support, behavior management, and participation in support groups. Families who received FES had higher caregiver strain at intake than families who did not receive FES, and 65% of families who received FES expressed satisfaction with the services.

With increasing budget deficits at the federal and state level, program administrators are interested in the potential of FES to augment workforce needs and contribute to improved outcomes for children and families, especially those program models that use peers to provide support. Although the research base is thin and the cost-effectiveness of peer-to-peer FES has not been established, 22 states have enabled the service to be billable under their state Medicaid plans (Hoagwood et al., 2010). Consequently, the field needs solid information on the diverse aspects of this potentially important service.

The Current Study

The aim of the current study is to contribute to the knowledge base examining FES. To this end, we examine the incremental changes associated with receipt of FES (i.e., parent training and support services that include behavior management classes and support groups) in five targeted outcomes: child behavioral and emotional problems, overall child functioning, school functioning, caregiver strain, and receipt of services. A secondary analysis was conducted using CMHI data (Manteuffel, Stephens, Sondheimer, & Fisher, 2008) to investigate the longitudinal impact of FES on child and family outcomes controlling for baseline characteristics of the study sample. Although the literature indicates that researchers have begun to examine the general pattern of the effects of FES on outcomes, in the current study we propose a different approach; we describe the relationship between the longitudinal trajectories of FES and outcomes using latent variables to model these trajectories. Specifically, we examined the receipt of FES at the time services are delivered and the effects of FES on parent and child outcomes at intervals of 6, 12, and 18 months of participation in the CMHI.

Data Source and Participants

The data used for the analyses come from children and families enrolled in the CMHI. Children and families eligible for inclusion in the current study received CMHI services from the 67 communities that participated in the longitudinal outcome study of the national evaluation of the CMHI. These communities represent three phases of initial program funding: Phase II communities received initial funding between 1997 and 1998, Phase III communities received initial finding between 1999 and 2000, and Phase IV communities received funding between 2002 and 2004. Rolling recruitment of participants occurred within each community during this time period.

Within each community, data collectors conducted structured interviews with caregivers, most of whom were the parents (biological, adoptive, foster) or the grandparents of the children enrolled in services. Data were collected at the child’s intake into services (baseline) and again at Time 1 (6 months), Time 2 (12 months), and Time 3 (18 months); for each interview, parents/caregivers reflected on their experiences over the previous 6 months. For a detailed description of the national evaluation protocol and data collection procedures, see Center for Mental Health Services, 2001, 2004; Holden, Friedman, and Santiago, 2001; and Manteuffel, Stephens, and Santiago, 2002.

Study participants were selected from all children and families involved in the longitudinal outcome study of the national evaluation of the CMHI (N = 14,407). Of these children and their families, 64% (n = 9,272) met three criteria for inclusion in the current study: (a) The child was between 6 and 18 years old, (b) valid data on demographic characteristics at baseline (i.e., child age, gender, race, annual household income) were available, and (c) valid data on all outcome measures at baseline were available. Children included in the sample were predominantly male (67%) and had an average age of 12.3 years (SD = 3.0). Nearly half of the children were White (46%) and lived in a household with an average annual income less than US$15,000 (45.3%). The most frequent referrals to the CMHI for children in the sample were through the mental health (31%) and school (27%) systems (see Table 1).

OPEN IN VIEWER

Table 1.

Characteristics of the Study Sample at Baseline Compared to Individuals Not Included in the Current Study.

Variables	Study sample (n = 9,272) a % or M (SD)	Individuals not included in the study sample b % or M (SD)
Child gender
Male	66.9%	66.2%
Female	33.1%	33.8%
Child age	12.3 (3.0)	11.1(4.9)*
Child race
White only	46.9%	43.8%*
African American only	27.0%	21.4%*
Hispanic	16.4%	14.7%*
Other (including multiracial)	9.7%	20.1%*
Household income <15K	45.3%	49.2%*
Enrolled in services between 1997 and 1998 (Phase II sites)	31.8%	50.2%*
Enrolled in services between 1999 and 2000 (Phase III sites)	29.2%	27.6%*
Enrolled in services between 2002 and 2004 (Phase IV sites)	38.9%	22.2%*
Child functioning c	70.1%	60.9%*
Behavioral and emotional problems d	70.1 (9.7)	68.2 (10.6)*
School functioning e	6.9 (2.9)	7.5 (3.0)*
Caregiver strain f	8.9 (2.6)	8.3 (2.7)*

a

n for race = 9,053. ^bn for age = 4,937; gender = 4,976; race = 4,504; income = 3,580; enrollment = 5,135; child functioning = 2,927; caregiver strain = 2,855; behavioral and emotional problems = 2,595; school functioning = 1,759. ^cPercentage of children with impaired functioning as indicated by a score of 100 or more on the Child and Adolescent Functional Assessment Scale, or 15 or more on the Columbia Impairment Scale. ^dMean scores on the Total Problems T-score on Child Behavior Checklist; range = 50 to 100; higher scores indicate more behavior problems; scores between 60 and 63 are in the borderline clinical range; scores 64 and over are in the clinical range. ^eMean standard score on the School Functioning subscale of the Behavioral and Emotional Rating Scale; range = 1 to 17, with an average between 8 and 12; higher scores indicate greater strengths. ^fMean global score on Caregiver Strain Questionnaire; range = 3 to 15; higher scores indicate more strain.

*

Difference significant at the p < .05 level.

To determine whether the participants for the current study differed from the sample from which they were drawn, the two samples were compared on key variables. This comparison revealed that participating children were older, less likely to live in a household with an annual income less than US$15,000, and had more behavioral and emotional problems, greater functional impairment, and fewer educational strengths than those not included in the study. Furthermore, parents/caregivers included in the current study reported higher levels of caregiver strain than those not included in the study. Study participants did not differ from the sample from which they were drawn with regard to gender (see Table 1).

During the course of the study, a number of participants did not provide data on one or more outcome measures. More specifically, out of the 9,272 participants initially included in the current study, 38.0%, 50.1%, and 62.5% had at least one outcome measure missing at 6, 12, and 18 months, respectively. A comparison of current study participants with complete outcome data and those without revealed that children in communities funded in Phase III (OR = 0.51, p < .01) and those children whose parents had higher caregiver strain at intake (OR = 0.78, p < .05) were less likely to be missing outcome data, whereas the opposite was true for youth in low-income households (OR = 1.33, p < .05). No other significant differences were found between participants with and without complete outcome data. Missing data were accounted for in the analyses by including a set of covariates (child age, child gender, household income, baseline caregiver strain, baseline child behavioral and emotional problems, baseline child functioning, baseline school functioning, funding phase, and time since intake) that may plausibly predict differences in missing patterns.

Design and Instrumentation

Analysis

To examine the relationship between receipt of FES, and parent and child outcomes over time, data were examined using shared parameter modeling. The reader is referred to Vonesh, Greene, and Schluchter (2006) and Rademaker et al. (2003) for more detailed information about shared parameter modeling. Generally, however, shared parameter modeling entails factoring the joint distribution of repeated measurements and event times, assuming these two outcome measures are conditionally independent given a set of shared random effects (i.e., shared parameters). The model used in this study includes a latent growth model for FES that is linked to a latent growth model for a chosen outcome Y (e.g., CBCL). The latent trajectories are defined by intercepts and slopes, which are based on the observed variables at each point in time. The latent outcome trajectories are assumed to vary across participants as a function of measured variables (i.e., those exogenous variables included in the model) and unmeasured variables (i.e., relevant variables not included in the model). Thus, by controlling for the latent outcome trajectory when modeling the outcome, we are able to control for measured and unmeasured variables, which should give a better estimate of the direct effects of FES on the outcome.

The model was estimated using maximum likelihood techniques and is formally specified by the set of parameters shown in Figure 1. The parameters of this model include (a) the average FES growth trajectory and the average Y growth trajectory (i.e., µ _if = the referent FES intercept, µ _sf = the referent FES slope, µ _iy = the referent Y intercept, and µ _sy = the referent Y slope), (b) the random variance in the trajectories (σ² _if = random variance in the FES intercepts, σ² _{i y} = random variance in the Y intercepts, σ² _sy = random variance in the Y slopes, and σ²_isy = covariance between Y intercept errors and Y slope errors), (c) the association between the FES and Y trajectory (λ _s ), (d) the association between the exogenous variables and the latent growth trajectories (β_if1 through β_if9 = the effects of the nine exogenous variables on the FES intercepts, β_sf1 through β_sf9 = the effects of the nine exogenous variables on the FES slopes, β_iy1 through β_iy9 = the effects of the nine exogenous variables on the Y intercepts, and β_sy1 through β_sy9 = the effects of the nine exogenous variables on the Y slopes), and (e) the direct effects of FES on Y (β₆ = effect of FES in the previous 6 months, β₁₂ = effect of FES between 6 and 12 months ago, and β₁₈ = effect of FES between 12 and 18 months ago).

OPEN IN VIEWER

Figure 1.

Overview of the Shared Parameter Model.

There are nine exogenous variables included in the analyses (child gender, child age, income < US$15,000, baseline CGSQ, baseline CBCL, baseline BERS, baseline CAFAS/CIS, Phase IV, Phase IV × baseline CAFAS/CIS), all of which were measured prior to intervention (see Figure 1). Note that in each model, only eight of the nine exogenous variables were used to predict the Y trajectories, as the baseline measure of Y was not used to predict the Y trajectory. Instead, the baseline measure was used as the first of the four points in the outcome trajectory. Owing to a change in interview procedures regarding service receipt (participants from Phase II/III sites reported how many times they received a service; participants from Phase IV sites reported how many days they received a service), observations are dummy coded to reflect whether or not they occurred in Phase IV, and Phase IV is included as an exogenous variable in the analyses. Notably absent from the model is a site variable. Because variation across sites may not be completely accounted for by the set of observed covariates included in the model, the inclusion of a site-level random intercept would have resulted in an improved representation of the process under study, had it been feasible to estimate. However, there is a limit on the number of relevant aspects that can be represented in a statistical model, which can be estimated.

Participant Characteristics

Results indicate that the characteristics of youth and parents in the current sample are consistent with the population of youth with SED. The majority of youths scored in the clinical range on the CBCL and in overall functioning (using either the CAFAS or CIS) for all three administrations of the instruments. Similarly, impairment in youth school functioning was reflected in their mean BERS-2 scores. Parents of youth also revealed moderate levels of stress with mean CGSQ scores falling between 7 and 8 on a scale with the highest level of strain being a score of 15 (see Table 2).

OPEN IN VIEWER

Table 2.

Participant Characteristics and Service Receipt Over Time.

	Time 1	Time 2	Time 3
Variables	6-Month InterviewM (SD)	12-Month InterviewM (SD)	18-Month InterviewM (SD)
Child functioning a	59.3%	55.0%	51.9%
	n = 6,418	n = 5,243	n = 4,005
Behavioral and emotional problems b	67.4 (10.8)	65.9 (11.0)	64.8 (11.6)
	n = 6,437	n = 5,276	n = 4,022
School functioning c	7.5 (3.0)	7.6 (3.0)	7.7 (3.0)
	n = 6,215	n = 5,066	n = 3,847
Caregiver strain d	8.1 (2.7)	7.7 (2.6)	7.5 (2.7)
	n = 6,333	n = 5,180	n = 3,989
Number of services received in last 6 months	4.8 (3.0)	4.0 (3.1)	3.4 (3.1)
	n = 6,444	n = 5,307	n = 4,110
Services received in previous 6 months e
Individual therapy	69.1%	59.3%	51.3%
Case management	67.0%	56.1%	46.9%
Assessment or evaluation	55.8%	41.2%	35.3%
Medication treatment-monitoring	52.6%	48.2%	44.1%
Family therapy	33.6%	26.5%	22.1%
Recreational activities	28.9%	24.7%	21.5%
Group therapy	25.9%	23.4%	20.0%
Family education and support	25.8%	21.3%	17.8%
	n = 6,255	n = 5,094	n = 3,849

a

Percentage of children who scored above 100 on the Child and Adolescent Functional Assessment Scale or above 15 on the Columbia Impairment Scale, indicating impaired functioning. ^bTotal Problems T-score on Child Behavior Checklist; range = 50 to 100; higher scores indicate more behavior problems; scores between 60 and 63 are in the borderline clinical range; scores 64 and over are in the clinical range. ^cMean standard score on the School Functioning subscale of the Behavioral and Emotional Rating Scale; range = 1 to 17, with an average between 8 and 12; higher scores indicate greater strengths. ^dGlobal score on Caregiver Strain Questionnaire; range = 3 to 15; higher scores indicate greater strain. ^eServices listed on this table are those that participants most frequently endorsed as received at each time point.

Receipt of Services

About a quarter of families received FES services during each 6-month interval, with fewer families using this type of service over time. On average, families also reported using an additional 4 to 5 services during each 6-month interval, with fewer services being used over time. Of the 22 services listed on the multisector services contact questionnaire (not including FES), participants in the current sample most often received individual therapy, case management services, assessment or evaluation services, medication treatment-monitoring services, family therapy services, recreational activities, and group therapy (see Table 2).

Effect of FES on Child Behavioral and Emotional Problems

Results suggest a positive association between CBCL scores and higher initial FES levels (β_if5 = .17, p < .001) such that families with children who had more behavioral and emotional problems at baseline were more likely to initially receive FES services. Regarding the effects of FES on child behavioral and emotional problems, results indicate that β₁₂ (the effect of FES received between 6 and 12 months ago) was statistically significant and estimated to be −.13, which suggests, after controlling for measured and unmeasured variables, those who received FES had children with fewer behavior problems 6 to 12 months after receiving these services. In addition, β₁₈ (the effect of FES received between 12 and 18 months ago) was statistically significant and estimated to be −.12, which suggests that the positive impact of FES on child behavior continues to be apparent 12 to 18 months after receiving these services. Parameter estimates for the direct effects of FES on CBCL, as well as the remaining study outcomes, are presented in Table 3.

OPEN IN VIEWER

Table 3.

Parameter Estimates for the Direct Effects of FES on Outcomes.

	Direct effect at 6 months	Direct effect at 12 months	Direct effect at 18 months
Variables	β6 (SE)	β12 (SE)	β18 (SE)
Child functioning a	−.01 (.06)	−.11 (.06)	−.15 (.08)
Behavioral and emotional problems b	−.02 (.03)	−.13 (.03)**	−.12 (.04)**
School functioning c	.02 (.03)	.00 (.03)	−.03 (.04)
Caregiver strain d	.02 (.03)	−.06 (.03)*	−.04 (.03)
Services received in the last 6 months	.17 (.02)**	.01 (.02)	—

Note. FES = family education and support. β₁₈ is not estimated for the variable, Services Received in the Last 6 Months, as receipt of FES was not assessed prior to the first measure of this variable.

a

Measured by the Child and Adolescent Functional Assessment Scale or the Columbia Impairment Scale. ^bMeasured by the Total Problems T-score on Child Behavior Checklist. ^cMeasured by the School Functioning subscale of the Behavioral and Emotional Rating Scale. ^dMeasured by the Global score on Caregiver Strain Questionnaire.

*

p < .05. **p < .01.

Effect of FES on Child Functioning

Results indicate that child functioning at baseline was not associated with initial receipt of FES services, nor were the direct effects of FES on child functioning statistically significant (β₆ = −.01, p = .92; β₁₂ = −.11, p = .076; and β₁₈ = −.15, p = .057). It is therefore not possible to conclude with confidence that there were direct effects on child functioning. However, estimates of these effects increased with longer lapses of time (β₆ < β₁₂ < β₁₈ ) and the effect estimates were negative, which suggests less impairment in child functioning with receipt of FES.

Effect of FES on School Functioning

Results suggest a positive association between BERS-2 School subscale scores at baseline and initial receipt of FES services (β_if6 = .17, p < .01), such that families with children who had better school functioning at baseline were more likely to initially receive FES services. However, there is no evidence for direct effects of FES on school functioning (β₆ = .02, p = .54; β₁₂ = .00, p = .97; β₁₈ = −.03, p = .39).

Effect of FES on Caregiver Strain

Results for caregiver strain indicate a positive association between caregiver strain at baseline and receipt of FES (β_if4 = .08, p < .01) such that parents with higher caregiver strain at baseline were more likely to receive FES services. Regarding the effects of FES on follow-up measures of caregiver strain, β₁₂ is statistically significant and estimated to be −.06 (p = .048), which suggests that after controlling for measured and unmeasured variables, those who received FES had lower caregiver strain 6 to 12 months after receipt of FES. The direct effects of FES are not statistically significant 0 to 6 months after receiving FES (β₆ = .02, p = .40) or 12 to 18 months after receiving FES (β₁₈ = −.04, p = .20).

Effect of FES on Other Services Received

To model the number of other types of services received, the general model was altered in two ways. First, the outcome was treated as a count variable. Second, because there were no measures of FES prior to the first measure of number of services, the effect of receiving FES in the previous 12 to 18 months could not be estimated. Results from this altered model suggested that there was a positive direct effect of FES on number of other services received, such that those who received FES within the previous 6 months were more likely to report seeking additional services during the same time period (β₆ = .17, p < .001). The direct effect of FES on other services received was not statistically significant 6 to 12 months after receiving FES (β₁₂ = .01, p = .64).

Summary of Results

Overall, results from the current study suggest that higher caregiver strain, more behavioral and emotional problems, and better school functioning at baseline were all individually associated with a greater likelihood of initially receiving FES. Regarding the effects of FES on outcomes, fewer child behavioral and emotional problems and lower caregiver strain were observed 6 to 18 months after receiving FES. Finally, results suggest that families who received FES were more likely to seek additional services during the same time period in which FES was received.

Limitations

As in any study using a secondary analysis of extant data, the current study was limited by the variables and measures originally chosen to be studied. There were instances where we would have preferred other measures, such as in the case of school functioning. A second limitation is that we used the shared parameter model, a method of analysis that is not extensively used in children’s mental health services research. Although there are some advantages to this method, it should be noted that the data are observational in nature and the model rests on distributional assumptions, some of which cannot be tested. Thus, the effect inferences are more limited than those that could be obtained in a randomized experiment. In addition, as higher caregiver strain and more impaired child functioning were observed at baseline for families who initially received FES, a third limitation of this study is nonequivalence of groups and the possibility of regression to the mean.

The method used to determine whether FES was received by a family (i.e., parents selected a “yes” or “no” option for receipt of FES on a self-report instrument) is an additional limitation of this study, as it did not allow us to explore who delivered the service (e.g., peer or professional) or the components of the service (e.g., emotional support, education, information, or instrumental support). A related limitation is the lack of a robust definition of FES. Although we used the definition put forth by Hoagwood et al. (2010) as a guide, we agree with these authors that the lack of a rigorous functional definition of FES inhibits the potential of this service. Finally, because of the methods used for measuring FES receipt, and the lack of a strong conceptual framework and definition of FES components and how they relate to other constructs (e.g., caregiver strain) and services, we were unable to adequately identify mediators and moderators of FES in the current study.

Future Steps

Although our findings suggest some positive impact of FES on children who have SED and their families, the effect of FES appears to be complex. The cycle of family stress, seeking help, improvement in child and family outcomes, and then a new stressor event that restarts the cycle will require adequate measures and methodologies to increase our understanding of the role of FES. For example, long-term longitudinal studies that go beyond 12 months will be needed to adequately track the cycle of family stress and its response to FES. In addition, research is also needed to examine site-level variables that might influence the potential of FES to impact outcomes as well as the effects of FES in combination with other services received.

We agree with Hoagwood and her colleagues in that much more work is still required to develop a functional logic model that will serve as a foundation for rigorous research on FES. This is a crucial next step in crafting a comprehensive research agenda in FES. Although the Double ABCX model (McCubbin & Patterson, 1983) and the Unified Theory of Behavior (Jaccard et al., 2002) each offer a good theoretical starting point to guide the research, a theoretical foundation more specific to FES is needed. With more specificity available about the nature and components of FES, future research could examine alternative model specifications that explore different exogenous variables, and potential mediating and moderating effects on child and family outcomes.

Although FES is growing in availability within the children’s mental health field, it has been the advocates and leaders in public policy, not researchers, who have led the way in developing and facilitating FES services implementation in communities across the country. It is important that researchers become involved in examining these services so that the discussion on refining and improving these services for a most needy group of children and their families are informed by empirical investigations.