Abstract
Hughes, Rycus, Saunders-Adams, Hughes, and Hughes’s article represents an important effort to critically examine the foundation of thought and empirical evidence associated with the rise in prominence of differential response (DR) within child welfare systems throughout the United States. The insights and criticisms offered are an important contribution to the professional literature. The arguments and perspective put forward can best be understood within the context of continuing conflict and debate (within child welfare) stemming from the countervailing influences of the principle of least intrusion and demands to ensure child safety. Although effective arguments are put forth that support many of the conclusions and interpretations, there are a few comments that require further discussion and reflection. More specifically, it is unclear if there is sufficient evidence to support their claim that children in DR alternative tracks may or are at a more heightened safety risk.
Keywords
I want to thank the editor of Research on Social Work Practice for the opportunity to provide feedback as part of a broader critical response to the Hughes, Rycus, Saunders-Adams, Hughes, and Hughes’s (2013) article on Issues in Differential Response. Such endeavors and the proposed format of this issue, I think, are very important elements of critical discussion and debate regarding timely and (sometimes) controversial practice issues and intervention models. Hughes et al. and the institutions they represent (the Institute for Human Services within the North American Resource Center for Child Welfare) have made important contributions in research and to the professional knowledge within child welfare. This article represents an effort to critically examine the foundation of thought and empirical evidence (thus far) supporting the perceived value and utility of differential response (DR) and its derivatives or parallel forms (e.g., alternative response, family assessment response, or multiple track response) within child protective services (CPS). The comments and criticisms presented by the authors, on the whole, are appropriately targeted and thought provoking. However, there are a few comments and conclusions that require further discussion and reflection, especially in light of methodological issues and considerations (with the studies they critique).
Value, Utility, and Support for DR
The authors have provided a fairly effective overview of evolution of thoughts, legislative events, and practices (since the enactment of the Child Abuse Prevention and Treatment Act of 1974) that became the foundation or justification (in part) for DR models. 1 Particular note is made of the prevailing support (which evolved over time) for the integration of family-centered and strengths-based practices into child welfare practice. There is little to disagree with in terms of their summary of the historical time line of thoughts, practices, and countervailing influences that shaped an apparent justification for DR and the authors’ concerns about possible “ … serious unintended consequences … particularly related to children’s safety” (Hughes, Rycus, Saunders-Adams, Hughes, & Hughes, 2013, p. 495).
Hughes et al. (2013) highlight a number of trends and issues denoted by other authors (Child Welfare Information Gateway, 2008; Connolly, 2005; Lohrbach, 2011; National Quality Improvement Center on Differential Response in Child Protective Services, [NQIC-DR] 2011; Schene, 2005; Waldfogel, 2009; Yuan, 2005) . Within their review, the authors have identified one of the prevailing contradictions and conflicts constraining any child welfare system: “The confluence in both law and policy of these compelling practice principles reflected one of the most prevalent and difficult inherent tensions in CPS practice: that is, how to promote engagement, empowerment, and support of families without compromising the essential need and capacity to use more authoritative interventions to protect children at high risk of serious harm when less authoritative methods could not” (Hughes et al., 2013, p. 494).
Although the authors have highlighted some of the legislative and practice trends (guided, in part, by professional principles and the research literature, albeit limited in scope and rigor) that have shaped this conflict, there is an additional underlying conflict or philosophical/legal debate that has shaped some of these trends and provided a context for why there may be variation in DR models across states (in the United States), provinces (in Canada), and countries. This embodies the rights of individuals and families contrasted against the “best interests” and rights of a child/children. As the ramifications of the Child Abuse Prevention and Treatment Act of 1974 (in the United States) were manifested, there were a number of legal scholars (Schoeman, 1980; Wald, 1975, 1976) that argued for minimal and restricted state intervention (among other child welfare system reforms) where allegations of child abuse were made (and/or substantiated). These arguments, in part, were supported or spurred on by the writings and recommendations made in the works of Goldstein, Freud, & Solnit (GFS, 1973, 1979). The reader is encouraged to read Davis (1988), Moorhead (1996), Schuerman (1997), and Taub (1983) for perspectives on the influence of GFS (1973, 1979) on child welfare policy and practice. Schuerman (1997) provides a particularly relevant discussion of the influence of select principles of their (GFS) writings within the context of the rise of influence and subsequent “blacklash” against family preservation initiatives.
Although debate (legal and otherwise) has continued regarding the role and influence of state intervention in child welfare matters (in keeping with evidence and information associated with practice trends denoted by the authors), it appears that most state (and provincial, in Canada) laws since the 1980s have evolved to consistently embrace the idea of investigating and engaging families in the “least detrimental” or “least intrusive” or “least restrictive” manner possible as long as such does not compromise the safety of the child/children. The five questions posed by the authors, I assert reflect thoughts regarding whether DR effectively embodies the entirety of the principle of “least intrusion” or (at times) respects the first tenant of the above principle (being a least detrimental means of engaging and supporting a family), while compromising the second tenet of the principle (child safety). These questions are transposed against a broader issue of whether or not DR—like select family preservation initiatives (see Schuerman, 1997)—is a practice trend supported by value-based perspectives, logical arguments, and select but limited research that does not meet the standards of empirically supported practice.
This conflict is manifested in the dual roles that child welfare systems and their agents (e.g., administrators, workers, advocates, etc.) have historically assumed. DR has been seen as a possible solution or process aimed at minimizing the conflict between these roles by stratifying response based upon initial assessments (albeit with varied rigor but typically reflective of legislative mandated protocols) of risk and screening procedures, subsequently engaging those that may need services (from a support and prevention standpoint) but do not meet statutory guidelines that would warrant a full abuse/maltreatment investigation. The intent seems “good” but the authors argue that good intentions and some of the protocols extending from DR models (reflective of some of the core elements embraced by the National Quality Improvement Center on Differential Response in Child Protective Services (NQIC-DR) may compromise the safety of select children (see Findings #3). With respect to discussion surrounding Finding #3, the authors do provide an informative itemization and overview of a number of issues and realities associated with initial screening decisions and assessments of maltreatment. These include (but are not limited to) variations across states in terms of populations and case or maltreatment types mandated for investigations (or different tracks), inconsistencies in the use of valid risk assessment tools and decision-making processes, and variations in the rigor of studies that have led to “overreaching” (Hughes et al., 2013, p. 504) claims regarding the impact of DR models on child safety.
Concern is raised regarding a “… defining characteristic of the alternative DR track is that caseworkers neither substantiate allegations of child maltreatment, nor do they confirm the perpetrator” (Hughes et al., 2013, p. 501). An argument is put forth, in essence, that children could be at risk—given restrictions placed on the caseworker providing service to a DR alternative family—due to the potential fallibility of the screening process (based on limited information) that assigned the case and restrictions placed on these caseworkers to adhere to a model of practice that (although reinforcing strength-based ideas) purposively restricts the collection of information that could complete or allow a thorough maltreatment investigation and subsequently better protect children. Although I agree with their assessment of the many potential failings of screening processes and the need for “skilled clinicians with well-developed interviewing skills” (Hughes et al., 2013, p. 501), it is unclear if there is sufficient evidence to support their claim (or heightened safety risk in alternative tracks), in part given their own assertions and cited references. Although their concerns about a potential biased sample of lower risk families randomly assigned to different DR groups in select studies (see Hughes et al., 2013, p. 502) may be warranted, Shusterman, Hollinshead, Fluke, and Yuan’s (2005) findings using National Child Abuse and Neglect Data System (NCANDS) data (appropriately cited by the authors) might suggest otherwise (although more rigorous study is needed). Even if select investigatory tasks are “discouraged” by alternative track DR models, this is not tantamount to or evidence that caseworkers typically (or at a level imposing statistical risk) ignore evidence or engage in behaviors that would “… prevent a thorough assessment of risk and safety from occurring in alternative tracks” (Hughes et al., 2013, p. 503) if a child’s safety was thought to be a concern. Such assertions should be treated as conjectures (supported by the logical arguments and legitimate concerns of the authors) that need to be tested. A review of the impact of mandatory reporting laws across states with DR models (or versions thereof) upon the likelihood of investigations on cases referred (at any time) for the alternative DR track versus the investigation track (using rigorous and sound methods and appropriate statistical models) would help answer these questions.
The authors choose their words carefully and present a compelling argument. I agree with most of their assessments and conclusions; however, there appears to be somewhat of a contradiction when they (I think accurately) conclude (with respect to Finding #3) “Based on currently available data, we simply cannot know whether being served in an alternative track is undermining children’s safety” (Hughes et al., 2013, p. 504), yet stated (in the prior sentence) “… it is not possible to conclude that a DR model exists that can assure that children’s safety is not compromised in alternative tracks” (Hughes et al., 2013, p. 504). This last sentence seems a little “overreaching” (to use the authors reference) as, alternatively, it could be asserted that it is not possible (with the evidence thus far) to conclude that a DR model exists where a child’s safety is compromised in alternative tracks. Thus far, there is no evidence suggesting that one track has a more measured (and sustained) impact (across a variety of child welfare systems) on minimizing the likelihood of a future occurrence of maltreatment than another.
Methodological Rigor and Empirical Support for DR
The questions posed by Hughes et al. (2013) are poignant and important for any review of a practice that has received considerable attention and prominence within the professional literature and been incorporated (in some shape and form) into established protocols for processing allegations of maltreatment across a number of states (or regions and counties within select states). There is an attempt to be comprehensive in their selection and review of key studies and formal evaluations of DR within a variety of contexts. The methods outlined for selecting literature and research to review and the key informants to interview are encompassing and appropriate in efforts to gain a representative sample of information needed to answer the proposed questions. The open-ended interview questions (Appendix C) are detailed and thorough, thus making inquiries that should generate meaningful content to be assessed for the purposes of this study. These efforts are meant to enhance the external validity of the generated findings and interpretations. More compelling are the authors’ efforts to engage in a critical review of the internal validity and interpretations and conclusions of key research studies (Center for Child and Family Policy, Duke University, 2009; Conley & DuerrBerrick, 2010; Institute of Applied Research, 1997, 2000, 2004a, 2004b, 2006, 2009, 2010; Ruppel, Huang, & Haulenbeek, 2011; Shusterman, Hollinshead, Fluke, & Yuan, 2005) more broadly commented upon in Appendix A. These types of reviews are extremely important elements of critical inquiry and help supplement existing literature reviews (NQIC-DR, 2011; Waldfogel, 2009) that may highlight the strengths, limitations, and variations in DR models (and their impact) evaluated within the literature but not necessarily with the same level of attention (which is needed) to the impact of methodological and design issues on the internal and external validity of study findings. Toward this end, this article is an important contribution to the professional literature and hopefully will further fruitful discussion of the validity, value, and utility of using DR across a variety of child welfare systems.
As a whole, I appreciate the efforts of the article and I am in agreement with the majority of observations and conclusions drawn from their review of the literature and feedback from key informants. However, I am not necessarily surprised by the findings, given that some of their findings appear to corroborate select observations or conclusions of others (Child Welfare Information Gateway, 2008; Lohrbach, 2011; Morley & Kaplan, 2011; NQIC-DR, 2011). Further, modifications or clarification of content and methods used by Hughes et al. (2013) need to be made so the reader can have a more involved understanding of the representative nature of the conclusions/interpretations they have made.
The Need for States as Units of Analyses
The authors have identified that they utilized, in part, resources provided by the American Humane Association, the NQIC-DR, and other sources (including federal and state documents and website information) to target specific key informants within each state. In total, representatives from 28 states (of the 35 identified to have some involvement with DR programs) were interviewed as key informants. The authors list in Appendix C the names of the 20 states from which data obtained from key informants was used in their analyses. In addition, Appendix C highlights the names of eight states where representatives were interviewed but data was not considered, it is presumed because they “…were considering DR but had not yet implemented it” (Hughes et al., 2013, p. 496). The authors do not identify the informants or the state they represent in their summary of findings. This was done in order to “… maintain [their] confidentiality …” to ensure they received “ … honest, detailed, and accurate information” (Hughes et al., 2013, p. 496). This was done even though “most” (no percentage is given) “did not feel that confidentiality was a concern” (Hughes et al., 2013, p. 496).Given the critical nature of the study, it appears that the authors wanted to guard against reactive or socially desirable responses. This is an acceptable and desired practice in such studies, although (as “most” of the respondents identified) the questions and their responses may not have been as controversial, reactive prone, or risk based for the potential respondent. Regardless, this protocol is understandable and hopefully had the intended effect.
However, it is unclear why the authors could not identify the specific states from which the key informants came from (without identifying or linking specific responses to each state and/or key informant) in their summary of many of their findings. This is important because there have been a number of writings and summaries/classification of DR practices and their variations across individual states for which findings in this study could be contrasted against (at an aggregate level). For example, the Children’s Bureau (Child Welfare Information Gateway, 2008) itemized the extent to which DR/alternative response models (they identify six common characteristics of these models) are reflected in the policies, practices, and statutes of 35 states. A contrast of select findings from Hughes et al.(2013) with those summarized by Child Welfare Information Gateway (2008; especially related to form and function of DR models) would be of value.
Further, the NQIC-DR has developed a classification (which has been modified slightly over the past few years) of DR across state systems (in terms of its level of implementation)that identifies each state's status as either having statewide implementation, regional/local pilot site implementation, planning/considering implementation, discontinued implementation, tribal model (in select locations within five states), screen out response models, and no DR (or no data on DR; Gilmore,2010; Morley & Kaplan, 2011). This mapping of states by these classifications is updated (most recently in October of 2012) on their website. Please note that although a particular state may be classified as having implemented DR, there has been acknowledgment within the literature that DR has been implemented with considerable variation (Child Welfare Information Gateway, 2008; Lohrbach, 2011; Morley & Kaplan, 2011; NQIC-DR, 2011; Shusterman et al., 2005; Williams-Mbengue, Ramirez-Fry, & Crane, 2010) for which some of the Hughes et al. (2013) findings clearly corroborate.
When the states from which key informant data were obtained is contrasted against the most recent NQIC-DR classification data, 11study states report statewide implementation of DR (Hawaii, Kentucky, Louisiana, Maine, Massachusetts, Minnesota, Missouri, North Carolina, Oklahoma, Vermont, Virginia), 4 states have regional/county implementation of DR (Nevada, New York, Ohio, and Wisconsin), 2 screen out responses (California and New Jersey), 1 is considering DR (Florida), and 2 have discontinued DR (Alaska and West Virginia). Given variation in the magnitude to which DR is implemented among different states and potential variations in how these practices are manifested and how such may change over time (Child Welfare Information Gateway, 2008; Morley & Kaplan, 2011; NQIC-DR, 2011) it would be of value to have weighted the responses of key informants and other state data in accordance with the current level of implementation of DR (as classified by NQIC-DR) and elements of DR manifestation that were evident 4 years ago (Child Welfare Information Gateway, 2008).
Without violating confidentiality, the authors could stratify responses based upon the level of current DR implementation status or cross-reference or cross-classified response to select questions, especially those associated with Finding #1. Although the authors indicate that they did not quantify any interview responses as “Data gleaned from the interviews was qualitative in nature and was compiled using only thematic analysis” (Hughes et al., 2013, p. 496), some quantification (or stratification) of responses and comparison/contrast with earlier findings would add to the significance (or lack thereof)of select interpretations. Without reference to state identifiers, the authors frequently use (exceptions exist) qualitative descriptors such as “most,” “some,” “many,” and so on, when describing and discussing findings generated from key informant interviews and/or their review of the literature. These descriptors do not provide a full or detailed/true gauge of the magnitude or incidence of the assessed or summarized condition/event at a national level. An identification of the numbers represented by these descriptors and assurances (using individual states as the unit of analysis) that there are no duplicate counts of the same systems (as some of the cited literature comments on the same states) are needed.
Tempered Conclusions
The authors have presented sufficient information and evidence (from their reviews) that reinforce many of the findings presented. With respect to Finding #1, I find little to disagree with. Some of the observations/conclusions are supported by other critiques or summaries of the literature, a number of which were cited and reviewed by the authors. The development and implementation of DR models has expanded and diversified over time reinforcing the authors’ conclusions and concerns that there is no consistent, standard, or uniformed implementation of a model. Although I agree with many of the concluding statements related to Finding #1, some of the conclusions could be tempered. True, there is no consistent, universally applied DR model across all states or regions where DR is asserted to be implemented. Many variations of DR exist, in part, because of some of the same influences the authors thoroughly summarize (and cite) that contributed to variations in how families are assigned to different tracks. Many of these variations are a by-product of variations in expectations for DR that are encapsulated with state legislation (National Conference of State Legislatures, 2012; Williams-Mbengue et al., 2010) upon track assignment, and possible service options and protocols. DR is an idea whose form and function may vary from state to state given variations in expectations by policy makers possibly influenced by advocates of DR and research findings that are limited and/or lack the rigor (and replication) that would allow DR models to be deemed “evidence-based.” The authors’ targeted and poignant review of methodological problems in DR research (see Finding #2 and information in Appendix A) reinforce this last point. In fact, the greatest contribution of this article is the critical appraisal of methods and itemization of inherent threats to internal and external validity of findings generated from cited studies. Here, the assessment and discussion presented by the authors provides the reader with a foundation for critiquing the research literature/studies cited and other syntheses of DR research and evaluation studies. These observations or conclusions of the authors lead them to state that “the claims presented in the research reports frequently overreached what might have legitimately been concluded” (Hughes et al., 2013, p. 500). This statement is augmented by their supplemental discussion in Appendix B (Science or Promotion?) and itemization of select DR promotional claims and concerns with each (Hughes et al., 2013, table 2). This discussion is welcomed and these considerations are very important for any researcher or practitioner interested in familiarizing themselves with the DR literature (or any research or practice trends for that matter). The examples provided appear to corroborate the authors’ statements; however, a more quantitative (and rigorous) content analyses of reviewed articles and the incidence and prevalence of these “inflated” or “promotional claims” contrasted against any discussion of strengths and limitations by cited authors (and other studies) is needed.
Although the authors have supported their claim that there are no consistent DR models, there have been attempts to provide “definitional clarity of program components and interventions” (Hughes et al., 2013, p. 507) that could allow “… large-scale and ongoing research possible to determine what outcomes can be expected from these interventions, and … enable researchers to compare outcomes both within and across research studies.” DR models have evolved over time and have been differentially defined or implemented (as the authors’ review and critique have aptly highlighted) across different child welfare systems (whether county, state, or regionally defined). However, efforts to identify commonalities across models led to the identification of six common characteristics (that they are assessment focused, individualized, family centered, community oriented, selective, and flexible) of DR systems in a publication of the U.S. Department of Health and Human Services in 2008. Of course, how each of these elements is manifested can vary substantially across systems and subsequently reinforce concerns denoted by the authors.
The NQIC-DR has highlighted a typology of eight core elements of DR that are based, in part, on the components identified by Merkel-Holguin, Kaplan, and Kwak (2006). Although the typology presented by the NQIC-DR does not necessarily reflect the views of the Children’s Bureau, the work of the NQIC-DR is funded via a cooperative agreement with the Children’s Bureau of the U.S. Department of Health and Human Services. This work includes a current study by NQIC-DR to use randomized control trials at three sites in Colorado, Illinois, and Ohio that will have comparable core DR elements. This study/evaluation reportedly includes a uniform process evaluation, child and family outcomes, and cost study components intended to allow meaningful cross-site comparisons. This is a valued venture; however, more specific method and design details and a summary of findings (forthcoming in 2013) are needed to ascertain whether this effort meets the standards of methodological rigor and scientific skepticism Hughes et al. (2013) rightfully proclaim are necessary to advance knowledge on the effectiveness of DR. Ideally, future evaluations of DR will benefit from many of the insights and criticisms put forth by Hughes et al.; all in the effort of advancing the professional knowledge base for the benefit of the children and families.
Footnotes
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.
