Response to the Barth Commentary (2012)

Development of PFR

PFR (Kelly, Zuckerman, Sandoval, & Buehlman, 2008) evolved as training program for service providers who worked with our most vulnerable families with infants and toddlers to infuse infant mental health principals into their practice. Like many community interventions, PFR did not follow the four-phase progression of biomedical clinical trials (testing safety, identifying benefits, efficacy, and effectiveness). PFR was never marketed “to promote secure attachments” by its developers, as Barth suggests. It was promoted as a type of intervention practice that could be used to support families at risk. PFR instruction includes a 10-visit protocol that trainees implement with families while participating in regular reflective consultation with a PFR trainer. PFR trainers, in turn, receive regular reflective consultation with PFR developers. The essential knowledge base includes the theoretical foundation of attachment theory perspectives on social and emotional development. Essential skills are the consultation strategies of positive comments, positive comments with instructive feedback, and reflective comments and questions, particularly when employed in a video feedback session with the parent. The consultation strategies can be used with families throughout the course of intervention at various levels and intensity. It is not expected that all providers, once they have acquired the knowledge base and skills, would apply PFR only in this 10 home-visit format. It is expected, however, that once trained, agencies would commit to a plan for ongoing PFR reflective consultation. Lessons learned from practical applications of PFR, training experience with providers, and subsequent demand for training resulted in program adaptations and training innovations that continue. The 3-day PFR orientation workshop, manual, handouts, and other materials for providers are available through NCAST Programs (http://www.ncast.org/); more intensive training via distance learning or on-site mentoring are conducted by staff at the Barnard Center for Infant Mental Health.

A Framework for Program Evaluation

PFR was an actively disseminated training program before it was the subject of a randomized clinical trial. Unlike some programs with proven efficacy, PFR had capacity as a self-sustaining program to get the intervention out into the real world. Prior to the initiation of the study, PFR was delivering two fully manualized mentored training models throughout the northwest. One model was a 40-hr “Train the Learner” model and the other an 80-hr “Train the Trainer” model. A third, 60-hr model was used to train providers for the present study. The models differed by the number of training families—two, three, or four—with which a trainee was mentored. In effect, the research team followed the recommendation of Rotheram-Borus and Duan (2003) to “find a program with a vehicle” or means for dissemination and “enhance the program’s efficacy.” The critical importance of this dissemination factor has been emphasized in implementation research (Glasgow, Vogt, & Boles, 1999; Kessler et al., 2012; Mason, Haggerty, Fleming, Oats, & Thompson, 2012; Self-Brown, Whitaker, Berliner, & Kolko, 2012). The appeal of PFR for our research program was its use of essential elements that could be applied in various settings and by individuals in various roles, including in child welfare. By the time we completed the trial examined in Spieker et al. (2012), PFR developers had piloted and revised a distance learning training model, with mentorship via video conferencing, making wider dissemination to child welfare agencies more feasible and affordable.

Barth acknowledges that many “child welfare practices do not have research evidence of effectiveness but are used because they appear, on the face of it, to be valuable.” This type of evidence reflects professional experience, wisdom, and values, and should be integrated with the best available scientific research (Buysse & Wesley, 2006). This has been the case with PFR, which started in the field. Several child welfare agencies in our state sensed the value of PFR as a staff training program and sought PFR at a Train the Trainer level. Their efforts were supported by a state contract that included PFR training and subsequent ongoing reflective consultation for the agency trainers with PFR developers. The contract ended 2 years ago, but the child welfare agencies continue to find funds to pay for regular reflective consultation with PFR developers. One agency, in consultation with PFR developers, has been using a 10-session, baby–mother group format that PFR developers had piloted with low-income grandmothers caring for grandchildren and mothers in transitional housing, in groups for women recovering from drug/alcohol abuse recently reunified with their children. For this agency, the research evidence came after uptake and is likely to be confirming, but not essential, to their continued implementation of PFR. The majority of the trainers in this agency are at the AA or BA level. In our experience, effective use of PFR is dependent on a provider’s capacity to be reflective, not necessarily on their level of education.

Additional practice-based evidence that PFR is beneficial is found in retention of staff and staff observed changes in the relationship quality between parents and their young children. Staff retention is an important issue in implementing evidenced-based interventions, and ongoing consultation has been shown to positively affect staff retention (Aarons, Fettes, Sommerfeld, & Palinkas, 2012). However, there is also a reasonable expectation that practices will have research evidence for effectiveness, and it will be increasingly the case that child welfare offices will not be able to disperse funds unless the program they desire to implement is not only evidence based, but accessible, feasible, and sustainable. Prior and ongoing dissemination of PFR has shown that it is accessible, feasible, and sustainable. Our program of research is evaluating its effectiveness in child welfare.

A Real-World Evaluation

We appreciate that Dr. Barth calls the study “highly ambitious” and is aware of how challenging it is to conduct intervention research with children and families in the child welfare system. We note that two aspects of the study, in particular, enhanced its relevancy for child welfare but also made testing efficacy of PFR more challenging. First, this was a community-based participatory study that sought and responded to community input into the service delivery model. Although the original design called for implementation with foster parents, we quickly learned that community partners also wanted to address the needs of children experiencing placement moves to other caregivers such as relatives and birth parents, which reflects real-world conditions for children in child welfare. We responded to this need by serving a wider population. This increased heterogeneity in the sample and complexity in the service model. In addition, to respond to the needs of children we made a commitment to follow a child to a new caregiver. If a child experienced a placement disruption during the intervention, we provided the intervention to the newly constituted dyad. By being responsive to the lived experiences of young children in the child welfare system, we limited our ability to test PFR with any one caregiver type. The challenges experienced by our team represent real-world experiences of infants and toddlers. A pure efficacy study would have been designed to avoid these issues altogether. Embracing the complexity of the lived lives of infants and toddlers in child welfare, however, is informative and enabled us to expand our program of research. We are exploring adaptations to PFR for evaluation in new studies. Plans include adding to the parental emotion self-regulation component; using PFR with recently reunified birth parents; using PFR in child welfare parental visitation; and using PFR with birth and foster caregivers to plan child welfare transitions that support child well-being. Both individual and group-based implementation models of PFR will be evaluated.

A second ambitious aspect of this study was that we utilized community-based providers to deliver the intervention rather than university-based clinicians. This decision meant a trade-off between sacrificing control and gaining the ability to test the intervention under real-world conditions. Reflective consultation and monitoring of fidelity to PFR were conducted by members of the research team, but the community providers typically carried a full client caseload, and devoted only about a quarter of their time to using PFR with study families.

Barth notes that caregiver sensitivity improved in the PFR condition. There were, however, no significant differences between experimental conditions in toddler attachment security, a null finding that Barth considers from three perspectives. Because of the challenges involved in conducting research in child welfare, especially the inevitable diminished sample size at later follow-ups due to placement moves within and out of the child welfare system, we argue that is important not to overinterpret null results for attachment and other child outcomes. Eventually, we will be able to consider these findings along with findings from other trials of PFR, as we expand and deepen the program of research involving PFR with different child welfare groups and various types of providers. In addition to the current study, this body of work includes small, quasi-experimental studies, which permitted the developers to adapt the training to providers working in different contexts, including child care, early intervention, and child welfare (Kelly, 2007; Kelly, Buehlman, & Caldwell, 2000; Kelly, Zuckerman, & Rosenblatt, 2008). We are also conducting an ongoing comparative effectiveness trial, Training Social Work Providers: Intervention for Maltreating Families of Infants and Toddlers, of 260 families of infants and toddlers under Child Protective Services (CPS) supervision (1R01HD061362-01A2, M. Oxford, PI). Other planned PFR evaluations in specific child welfare contexts were described above.

Attachment Theory Is Foundational to PFR

Barth comments on the “lack of parsimony in the conceptual foundation” for PFR and is puzzled why the research is framed as “an attachment intervention.” We think that this is an understandable misattribution about PFR. Attachment theory provides the rationale for keeping the focus of PFR’s consultation strategies on the parent–child relationship. Attachment theory asserts that both the parent and the child have basic social emotional needs to feel safe, effective, and loved. Finally, attachment theory helps operationalize parental sensitivity as an important determinant of relationship quality. Although sensitivity is a more proximal outcome of PFR than child security, the conceptual foundation for both is attachment theory, and sensitivity is one of a constellation of positive parenting qualities that are predictive of long-term child outcomes (Landry, Smith, & Swank, 2006), including resilience (Southwick & Charney, 2012). We agree with Barth’s point that “measures of attachment are not necessary to understanding children’s behavior or showing improved parenting.” However, children in out-of-home placement are at risk of insecure and disrupted attachments due to maltreatment and multiple moves (Zeanah, et al., 2004). We believe that attachment theory, which provides core principles for infant mental health, is a critical knowledge base for all who work with children in the child welfare system, including their caregivers. It is essential for service providers to understand that children in child welfare are vulnerable because they have experienced relationship disruptions, an assertion at the core of attachment theory (Mennen & O’Keefe, 2005; Zeanah, Shauffer, & Dozier, M, 2011).

Fidelity to PFR, in both research and training, involves adequately applying and adhering to the consultation strategies that help caregivers first feel safe, and then build on their confidence, competence, ability to reflect on their own and their children’s feelings and needs, see the world from their children’s point of view, and promote sensitivity and mutual delight. That these consultation strategies promote a therapeutic alliance has been widely validated (c.f. Norcross, 2002). Training in the PFR consultation strategies helps providers interact with caregivers in ways that convey respect, nonjudgment, and compassion. Using these consultation strategies enhances the effectiveness of any provider and is an important component of cultural competence, which has been shown to increase family engagement in home-based services (Damashek, Bard, & Hecht, 2012). PFR training could be a useful complement to other programs, such as SafeCare, particularly with families who are especially challenging to engage (Chaffin, Hecht, Bard, Silovsky, & Beasley, 2012).

In conclusion, we greatly appreciate the opportunity to grapple with the questions posed by Barth and reflect on the challenges inherent in child welfare research. For any program to move forward and truly support parents and their children, partnerships between stakeholders in the community and experts in research will be essential. We must continue to implement and improve on programs that show a good fit to community needs. As researchers, we gain insights by sharing our struggles, by participating in exchanges like this one made possible by Child Maltreatment, and by anchoring our evaluations in the real-world conditions in which we expect them to be effective.