Reflections on the Use of Evidence in Child Welfare: A Commentary on Barth et al.

Abstract

To create a child welfare system that meets the needs of abused and neglected children, it is essential to critically evaluate core claims and assumptions about how the current child welfare system operates and its impacts on children and families. Yet, misleading and false claims are frequently repeated in academic, professional, and media outlets. Barth and colleagues (2021) sought to name and evaluate some of these claims, reasserting the central role of research evidence in policy and practice debates. In this commentary, I argue that a commitment to “consider the evidence” may do little to improve the quality of policies or practices in the child welfare system. Rather, I assert that because research findings are commonly mischaracterized and overgeneralized, we are left with widespread disagreement about what to conclude from existing research and how certain those conclusions are.

Keywords

child welfare system reform evidence-based practice

Barth et al. seek to correct the record on a variety of false and misleading claims made against the child welfare system (CWS). This effort is essential, as the misleading or false assertions to which they respond are commonly repeated by policymakers, the media, and—most unfortunately—professors and researchers. Such claims are thus positioned to undermine effective policies and practices, and divert resources away from victims of child maltreatment. This commentary will highlight two concerns about the use of evidence in policy and practice.

A Note of Caution on the Use of Evidence

It seems clear that a research-driven CWS would be far better for children than what currently exists. Yet, what would this look like? Some of the authors’ own examples of the how research evidence is or could be used to improve child welfare practice are as much cautionary tales as success stories.

As one example, the authors discuss how kinship care was previously underutilized due to widespread fears that relatives of maltreating parents would themselves be disproportionately likely to maltreat children. They argue that research showed that these fears were overblown, allowing for the proliferation of kinship care. Yet, the picture is far more complicated than that. Kinship care—in which relatives who were able to pass a home study and background checks serve as children’s foster parents—is generally safe and stable, with mixed data on wellbeing outcomes (e.g., Font, 2014; Helton, Boutwell, & DiBernardo, 2017; Ryan et al., 2010; Winokur, Holtan, & Batchelder, 2014). Yet, that evidence is often construed to mean that kinship care is inherently and universally better for children than any other form of care, resulting in widespread willingness to place children with relatives who do not pass background checks and who receive limited training, support, or oversight (American Bar Association, 2015; Font & Gershoff, 2020; U.S. Children’s Bureau, 2011). It has also led to decisions that remove children from stable and well-suited non-relative foster or pre-adoptive homes to live with relatives who have no personal relationship with the child (Font & Gershoff, 2020). In other words, the evidence base for kinship care supports the engagement of children’s pre-existing safe and supportive attachment figures as placement resources. Yet, that evidence has been used to support policies and practices that extend far beyond what the evidence can speak to.

Drawing on evidence requires careful consideration of its nature (to whom the evidence applies, to what set of outcomes, and under what circumstances) and its strength (single study vs. totality of evidence; quality of study design). Unfortunately, researchers often need to quickly and concisely summarize a complicated topic, and the potential for public confusion is high. The brief discussion by Barth et al. of the possible role of material supports in reducing child maltreatment provides an example of this problem. Barth et al. assert (following evidence) that impoverished families are more likely to engage in child maltreatment and as such are disproportionately likely to be involved with CWS; they then promote material supports as a promising strategy for reducing child maltreatment and CWS involvement. This provides an easy opportunity for confusion. For example, it is important to differentiate between preventing and responding to child maltreatment, especially in an article that is mostly focused on the CWS. In the same way that the flu vaccine sometimes prevents the flu but is not a viable treatment for flu-infected individuals, there is little research evidence to suggest that material supports are positioned to fix the circumstances of families whose children have already been maltreated. Similarly, distinctions between causal and correlative studies, between unconditional and work-based supports, and among the various forms and degrees of CWS involvement are elided—these distinctions generally point to a more tentative and conditional conclusion about the potential effects of material supports (e.g., Berger et al., 2017; Klevens et al., 2017; Raissian, 2015; Raissian & Bullinger, 2017). All of these issues are clearly beyond the scope of the authors’ focus; my point is only that social research is extremely complicated to synthesize and thus brief characterizations are highly prone to misinterpretation and overgeneralization by both lay and scholarly readers.

Is Evidence Ignored, Misrepresented, or Contested?

On any number of issues where it would be sound advice to “consider the evidence” or “follow the science,” there are (at least) two opposing sides that both claim scientific support. Indeed, the self-described “abolitionist” movement cites research studies as support for at least some of their claims. Are they considering the evidence? A crucial point here is that data and evidence are not the same thing. Evidence requires “meaning-making” or translation of the results of a study into a general conclusion. The nature and certainty of that conclusion depends on the rigor, scope, and quality of the study design. This is where consensus-building goes awry.

For example, the “abolition movement” cites data indicating that most CWS cases involve neglect allegations against families in poverty as evidence that CWS conflates neglect with poverty (Dettlaff et al., 2020; UpEnd Movement, 2021). Perhaps obviously, data on who/what cannot answer a why question (Pearl & Mackenzie, 2020) and there are numerous alternative explanations for those same data (as Barth et al. explain). Nevertheless, this example suggests that the problem is not merely a failure to use data or cite research; it is a failure to appreciate what a particular study or set of data can and cannot tell us.

In sum, agencies, practitioners, the media, and policymakers are poorly equipped to use research responsibly. While universities can and should do a better job teaching students how to consume information and reconcile opposing claims, researchers also bear the burden of drawing clear distinctions between research results (what a particular model of a particular set of data show) and the meaning we derive from those results. We should also commit to using precise but accessible language, especially when communicating with a non-researcher audience. It is not enough to say that we should “consider the evidence” when there is too much confusion about what constitutes evidence as applied to a particular question or hypothesis. Most urgently, researchers should reject the model of advocacy “research”—whereby researchers gather data and build models with goal of producing “results” that can be claimed as “evidence” of a predetermined conclusion. But that alone will not resolve the confusion that arises when thoughtful and careful research gets reduced to a splashy soundbite or packaged into a brief summary for a general audience.

Barth et al. are commended for elevating the conversation on a range of difficult and contested topics. Now the scientific community must forge a path forward that emphasizes clear and cautious communication with the public. CWS reforms based on the misapplication or overgeneralization of research results—or on low-quality or advocacy-based research—may be no better than reforms designed without considering research at all.

Footnotes

ORCID iD

Sarah A. Font

References

American Bar Association (2015). New model family foster home licensing standards: An overview. https://www.americanbar.org/groups/public_interest/child_law/resources/child_law_practiceonline/child_law_practice/vol-34/february-2015/new-model-family-foster-home-licensing-standards--an-overview1/.

Berger

L. M.

Font

S. A.

Slack

K. S.

Waldfogel

(2017). Income and child maltreatment in unmarried families: Evidence from the earned income tax credit. Review of Economics of the Household, 15(4), 1345-1372. https://doi.org/10.1007/s11150-016-9346-9.

Dettlaff

A. J.

Weber

Pendleton

Boyd

Bettencourt

Burton

(2020). It is not a broken system, it is a system that needs to be broken: The upEND movement to abolish the child welfare system. Journal of Public Child Welfare, 14(5), 500–517. https://doi.org/10.1080/15548732.2020.1814542.

Font

S. A.

(2014). Kinship and nonrelative foster care: The effect of placement type on child well-being. Child Development, 85(5), 2074–2090. https://doi.org/10.1111/cdev.12241.

Font

S. A.

Gershoff

(2020). Foster care and “best interests of the child”: Integrating research, policy, and practice. Springer. https://doi.org/10.1007/978-3-030-41146-6.

Font

Gershoff

E. T.

(2020). Foster care: How we can, and should, do more for maltreated children. Social Policy Report, 33(3), 1–40. https://srcd.onlinelibrary.wiley.com/doi/epdf/10.1002/sop2.10.

Helton

J. J.

Boutwell

B. B.

DiBernardo

(2017). The relative safety of paternal, maternal, and traditional foster care placements. Child Abuse & Neglect, 70, 1–10. https://doi.org/10.1016/j.chiabu.2017.05.006.

Klevens

Schmidt

Luo

Ports

K. A.

Lee

R. D.

(2017). Effect of the earned income tax credit on hospital admissions for pediatric abusive head trauma, 1995-2013. Public Health Reports, 132(4), 505–511. https://doi.org/10.1177/0033354917710905.

Pearl

Mackenzie

(2020). The book of why: The new science of cause and effect (Reprint edition). Basic Books.

10.

Raissian

K. M.

(2015). Does unemployment affect child abuse rates? Evidence from New York State. Child Abuse & Neglect, 48, 1–12. https://doi.org/10.1016/j.chiabu.2015.06.008.

11.

Raissian

K. M.

Bullinger

L. R.

(2017). Money matters: Does the minimum wage affect child maltreatment rates? Economic Causes and Consequences of Child Maltreatment, 72, 60–70. https://doi.org/10.1016/j.childyouth.2016.09.033.

12.

Ryan

J. P.

Hong

J. S.

Herz

Hernandez

P. M.

(2010). Kinship foster care and the risk of juvenile delinquency. Children and Youth Services Review, 32(12), 1823–1830. https://doi.org/10.1016/j.childyouth.2010.08.003.

13.

UpEnd Movement (2021). How we endUP: A future without family policing. http://upendmovement.org/wp-content/uploads/2021/06/How-We-endUP-6.18.21.pdf.

14.

U.S. Children’s Bureau (2011). Report to Congress on states’ use of waivers of non-safety licensing standards for relative foster family homes (pp. 1–14). U.S. Department of Health and Human Services, Administration on Children and Families, Administration on Children, Youth and Families.

15.

Winokur

Holtan

Batchelder

K. E.

(2014). Kinship care for the safety, permanency, and well-being of children removed from the home for maltreatment. Campbell Systematic Reviews, 2, CD006546. https://doi.org/10.4073/csr.2014.2.