A Standardized Matrix for Mental Health Diversion Program Reporting

Abstract

A matrix is proposed to characterize mental health diversion programs and standardize the reporting of program context, outputs, outcomes, and community settings. Data collection for program reporting is challenging because individual programs report on what is relevant to local conditions and may omit or overlook important contextual or structural factors that are key to programmatic success or failure. This limits generalizability and comparability. Using a standard matrix reporting framework clearly lists the constructs of the program, context, and community systems. Two diversion programs are reported demonstrating the use of the matrix. Although different jurisdictions have a wide spectrum of agencies and resources available to support diversion, and may approach the concept differently, standardizing and streamlining reporting will assist with evaluation of diversion and the creation of sustainable programs.

Keywords

diversion program evaluation reporting

Introduction

Diversion programs of justice-involved individuals with mental health issues exist at multiple levels within the criminal justice system. Programs are aimed to shift the justice-involved individual from punishment into mental health treatment (Lange et al., 2011). The philosophy of diversion suggests mental health treatment in jail is less effective than in a community-based option (Ryan et al., 2010). Whereas jail provides a controlled environment for a medication regimen, incarceration separates individuals from support networks and jobs, which exacerbates stigma and alienation from the community, thereby creating a revolving door of reincarceration and recidivism that diversion programs seek to mitigate:

We measure what we value, and for that reason any metric we select is likely to tell us as much about those who operate the criminal justice system as it is to tell us about the effectiveness of the system itself. (Klingele, 2019)

There are a myriad of different diversion programs for justice-involved individuals with mental illness all adapted to their local conditions using good practices (Perla et al., 2013). There is persistent tension between “top-down” versus “bottom-up” approaches to best practices in mental health diversion. However, “bottom-up,” local adaptation, carries an inherent risk that little of the program could be generalizable if some sort of standardization was not applied (Øvretveit et al., 2011; Sorenson & Sørensen, 2001). It is important for diversion programs to function within their context and community setting so local needs are not subsumed or local innovation stifled in the push to standardize reporting, but adaptation to local conditions does not preclude standardized definitions. The standard report matrix is not the end goal or a “top-down” requirement, rather standard definitions will make the entire system more measurable, streamline evidence for best practices (Øvretveit et al., 2011) and provide sufficient evidence for implementation in another jurisdiction (Bardach & Patashnik, 2019).

Standardization of definition when discussing, evaluating, and reporting diversion programs provides the ability to characterize programs across jurisdictional settings and allows better understanding of the fit between program, organizational context, and ecological settings. A standardization of program reporting enables jurisdictions to better manage the fit between their resources, needs, and diversion programs, including generating consistent feedback on how the programs are delivered to diverse clients and the resulting outputs and outcomes. More than 3,000 (LEAD and CIT) diversion programs (Paccone, 2020), ~300 pretrial diversion programs, and >150 pre-plea diversionary adult court programs (“No Entry a National Survey of Criminal Justice Diversion Programs and Initiatives,” 2013) have been implemented across the United States, with the goal of reducing the number of individuals with mental illness in the criminal justice system. But there are many more types of diversion programs; thus, 3,000 does not capture other types of programs with diversionary functions for justice-involved individuals with mental health issues (e.g., mental health courts, pretrial diversion initiatives). Comparing programs can be difficult because different programs rarely report the same output and/or outcome measures nor do they describe the setting context and broader ecological system similarly. Issues with diversion program evaluation are not new; concerns date back to the 1970s (Agopian, 1977; Blomberg, 1983). This article focuses on mental health diversion but may be broadly applicable to other diversion programs. Currently, diversion program evaluation uses criminal justice system outputs (e.g., length of incarceration, recidivism, time between arrests, failure to appear) rather than functional individual outcomes (e.g., vocational and social functioning) or clinical outcomes (e.g., symptom management and resolution) (Remington, 2010; Rempel et al., 2018).

Reporting on both process outputs and individual-level outcomes as part of diversion program evaluation is necessary to understand the chronic effect these programs have on individuals. Evaluation of individual-level outcomes is especially important when the individual being diverted has mental illness or substance use disorder (SUD) because they are more likely to be re-incarcerated or have their probation or parole revoked (Lange et al., 2011). Diversion for justice-involved individuals with mental illness program differences can be substantial and affect evaluation of the program’s effect on a multitude of criminal justice outputs (e.g., recidivism, new charges, failure to appear, lengths of stay) and complicates comparisons across jurisdictions. Depending on a justice-involved individual with mental illness’s point of contact with the criminal justice system, either through arrest or through the emergency room, very different types of diversion could be offered (Draine et al., 2005). Mental illness outcomes are rarely reported on in criminal justice diversion programs because the programs are short-term acute interventions to keep individuals out of jail, with main outputs reported as shortened jail length of stay, reduction in new arrests or technical rule violations, and failure to appear to court. Conversely, in health care, mental health interventions typically involve continuous community-based treatment, with main outcomes reported as stabilization of symptoms and compliance with medication and treatment. The use of health care by diverted individuals is difficult to quantify, and mental illness and SUDs may be comorbid with physical issues requiring treatment. Evaluation difficulties are compounded because definitions and methods are not shared between criminal justice and health care agencies conducting diversion, making unified analysis of diversion extremely challenging (Rogers et al., 2019).

Comparing Diversion Programs

Prior work suggests standard definitions for diversion program context (Potter, 2010), content and outputs/outcomes (Sirotich, 2009), and the limits of generalizability (Bardach, 2004) necessary for comparison. Lack of definitions for demographics and individual outcomes limit evaluation (Sirotich, 2009). Juvenile justice diversion programs (excludes mental health) do a better job of standardizing measures for program comparison. Juvenile justice diversion programs provide reporting definitions to compare programs including the following components: (a) method of entry, (b) jurisdiction, (c) structure (e.g., extent of control), (d) enrollment (by offense category), (e) type of services, and (f) program sanctions (Development_Services_Group_Inc., 2017). However, juvenile justice diversion programs are very different than adult programs, and a 2012 meta-analysis showed the major program type to reduce recidivism was family-based programming (Schwalbe et al., 2012), which would not be appropriate in adult situations. The criminal justice system diverts adults and juveniles differently; adults are prosecuted for crimes and punished, whereas juveniles are prosecuted for delinquent acts and rehabilitated. The focus of diversion is different; juveniles are taught skills to change current behaviors enabling them to grow into responsible young adults, while adults are held accountable for their actions with interventions focusing on being productive citizens. In addition, the meta-analysis also suggested that there is a high degree of variability in diversion programs again pointing to the need for the field to strengthen standardized program reporting so jurisdictional comparisons can be made (Schwalbe et al., 2012). Diversion conceptualized as a public health issue provides a system-level understanding of the totality of resources to treat, maintain, and minimize recidivism for individuals with chronic mental illness (L. Gittner & Dennis, 2021).

The sequential intercept model (SIM) defines processes to divert individuals with mental illness and SUDs (The Sequential Intercept Model, 2020). Unfortunately, SIM does not provide definitions necessary for evaluation or comparisons across programs (Data Collection Across the Sequential Intercept Model, 2019). Context of the setting often interacts with programs and needs to be defined during evaluation; interactions between the program, context, and ecological system likely affect the transferability of a program from one jurisdiction to another and also make an assessment of comparability more difficult (Rychetnik et al., 2002). Standardized comparison of diversion program components would greatly improve the ability to evaluate and compare costs, successes, and failures. However, diversion program structure is dependent on program implementation via local governments by the presence and participation of supporting health care and criminal justice organizations, local policies, and local politics. The multiplicity of diversion programs across the United States cannot be standardized in form because agencies, resources, and objectives will always differ by locality, yet a standardized matrix stands to provide insight into how diversion programs differently approach and measure outputs and outcomes.

Diversion programs, as currently evaluated, typically capture only criminal justice system measures rather than also encompassing health care system measures (L. Gittner & Dennis, 2021). Potter (2010) used a “public health lens” to compare diversion and jail outcomes but found the comparison challenging because of the lack of shared definitions between different agencies (Potter, 2010). A public health lens prioritizes prevention over treatment and the community over an individual; thus, multiple contexts need to be defined and measured to explain a program. In addition, evaluation of mental health diversion programs using health care concepts leads to the conclusion that the programs are not scientifically rigorous and too diverse to classify (Sirotich, 2009). A systematic matrix for adult mental health diversion program evaluation and reporting can provide a methodical and rigorous appraisal that is embedded not only within the program but also within the complex inter-organizational (context) and community politics (ecological system) that determine diversion program policies and practices. Solutions are complex for criminal justice problems such as recidivism of individuals with mental illness. The problems require diversion programs to “cut across sectors to take account of the broader social, cultural, economic, political and physical environments which shape people’s experiences” (Morgan & Ziglio, 2007). Previous definitions from the public health context (Potter, 2010; Rychetnik et al., 2002) accounting for the broader context can be built upon to develop a matrix to compare diversion programs.

Dynamic Sustainability framework

The Dynamic Sustainability framework adapted in Figure 1, models the factors that contribute to the sustainability of a program in the real world over time. It considers three dimensions of sustainability: broader system (e.g., structure, climate, culture, resources), setting (e.g., context the program is delivered within), and individual program (e.g., eligibility, use, and delivery mode) (Chambers et al., 2013) and the change that occurs over time. The proposed matrix provides definitions for this framework anchoring the ultimate benefit (e.g., functional outcomes) in terms of a program’s ability to be implemented in a real-world setting. Thus, the proposed matrix provides definitional context using the framework to categorize a diversion program (intervention type, actors, outputs, outcomes, delivery mode) within a context (resources, culture, training), and the community system (policies, regulations, population characteristics).

Figure 1.

Dynamic Sustainability Framework for Diversion Programs. Modified from Chambers, et al., 2013 to include diversion constructs (Chambers, Glasgow and Stange, 2013).

Although diversion programs differ in nature, the common goal is reduction in organizational burden for correctional and legal agencies. The definition of diversion most often mentioned is the suspension of formal criminal proceedings against an individual with outcomes listed as reduced incarceration time, transfer into a non-criminal justice program, and program completion. All of these outcome factors are important, however, the relationship between the program with its context within the broader community system is just as important as outcomes. Thus, we propose a standard reporting matrix of evidence based on public health program evaluation standards when reporting diversion program evaluations, outputs, and outcomes. The three previous meta-analysis of mental health court adult diversion programs all point to the heterogeneity among participants, treatment types, evaluation methodology, and design leaving uncertainty regarding to whom, what conditions, and which diversion program designs actually provide positive mental health outcomes (Fox et al., 2021; Lowder et al., 2018; Sarteschi et al., 2011). The following matrix is proposed to assist program reporting, so a standardized system is used regardless of where the diversion program is administered; standard reporting will advance both research and policy.

Materials and Methods

Proceedure

A comprehensive literature review of adult mental health diversion programs from 2000 to 2022 was performed using EBSCO academic search using the key phrases: mental health diversion, mental health treatment diversion, adult diversion, mental health court diversion, pre-arrest diversion, and criminal justice mental health diversion. There were 400 results and once duplicates, editorials, commentaries, and citations were removed, there were 109 unique articles; the articles were sorted by date and intervention type. The articles were scanned for descriptors of program types, setting context, and ecological system definitions and components. The constructs were grouped according to the Dynamic Sustainability Framework as a definition in one of the following elements: (a) program, (b) setting context, and (c) community system.

Definitions

The Matrix displayed in Table 1 was developed to compare diversion programs. The Dynamic Sustainability Framework is an implementation model and, as adapted to diversion programs, uses three major elements to frame and define a program. The main elements are the actual program type, the setting in which the diversion program is delivered, and the broader ecological system within which the diversion program setting exists and operates. A stability construct is included within each element to capture program drift and voltage drop that could explain outcome variability and non-jurisdictional transferability. This is a measure of how well a program maintains its original design and intent over time. Program drift occurs when a program changes in ways that are not intended, such as when staff members make changes to the program during implementation. Voltage drop occurs when a program’s effectiveness is reduced due to factors such as staff turnover, budget cuts, or changes in the target population.

Table 1.

Proposed Matrix for Diversion Program Reporting.

Program	Variables
Engagement	Pre-arrest Pre-booking Post-booking (pretrial) Pre-sentencing Post-sentencing (post-trial)
Treatment venue	Within detention center Without incarceration In patient Community out patient
Length of service delivery	Active days, hours, months Passive days, hours, months
Treatment type	Mental health/behavioral health treatment, Competency restoration, Substance use/alcohol treatment, Case management, Vocational educational services, Supportive housing
Eligibility	Mental health/co-occurring Substance Use Disorder (SUD) Dx, First offense, Crime classification (misdemeanor/felony), Crime classification (drug, property, person)
Budget	Per agency costs Total program costs (i.e., costs including internal agency costs when multiple agencies participate, costs estimated for all agencies.)
Staffing	Qualifications Full Time Equivalent (FTE)
Outcome measures	Stated success measures
Stability	Years program has been in operation
Context
Socioeconomic status	Median community income;Education is percent of individuals >25 years with bachelor’s degreeEmployment is percent individuals >16 years participating in labor forcePercent of population below us defined poverty level
Racial demographics	Percent White, Black, and other races; and percent Hispanic ethnicity in population
Homelessness	Most recent point-in-time count
Politics	Political orientation defined by the political party of the presidential candidate with the most votes classified as democrat, independent, or republican
Oversight/ primary agency	Categorize the primary agency:Local jail,Police force,Sheriff’s office,Detention center,Mental health agency,Hospital emergency room,Non-profit agency
Number of Collaborating Agencies	Type and number of collaborating agencies
Stability	Funding governmentFunding non-profitFunding privateCloseness of the collaboration
Community system
Region	Northeast,Midwest,South,West
Rurality	Urban,Suburban,Rural
Size	City—large, midsize, smallSuburban—large, midsize, smallTown—fringe, remoteRural—fringe, distant, remote
Population stability	In migration rateOut migration rate

Data for the Matrix Exemplar

Two community mental health diversion programs were compared. The programs are VetStar in Lubbock County Texas and the Miami Model Eleventh Judicial Circuit Criminal Mental Health Project. The two programs were purposively selected to show how programs appearing disparate may have commonalities once standard reporting is used. The programs examined are not intended to be representative of U.S. diversion programs, but rather they represent two different types of programs having available data to make a comparison using the matrix. Data regarding community characteristics were from the U.S. Census Bureau (US_Census_Bureau, 2019) American Community Survey 1-year estimates. Retrieved from Census Reporter Profile page for Lubbock, TX http://censusreporter.org/profiles/16000US4845000-lubbock-tx/ and Miami-Dade County, FL http://censusreporter.org/profiles/05000US12086-miami-dade-county-fl/; The Department of Housing and Urban Development Point in Time Count for Lubbock and Miami-Dade (The 2022 Annual Homelessness Assessment Report [AHAR] to Congress Point-in-Time Estimates of Homelessness, 2023) political orientation for Lubbock at (https://www.votelubbock.org/election-information/historical-election-results/) and Miami-Dade at (https://enr.electionsfl.org/DAD/2779/Summary/). Data for the program jurisdiction and attributes were obtained from: Lubbock County Detention Center website, the final P & I Guide for FY17 JMHCP Lubbock Category 1 Collaborative County grant project MO-BX-0031, and VetStar Stakeholder reports (VetStar, 2017, 2018); for Miami-Dade Department of Corrections and Rehabilitation website, Florida Department Of Corrections 2016–2017 Annual Report, and GOB Project 193—Mental Health Diversion Facility Service Capacity and Fiscal Impact Estimates June 9, 2016 Report (GOB Project 193—Mental Health Diversion Facility Service Capacity and Fiscal Impact Estimates June 9, 2016, 2016).

Results and Discussion

Matrix Components

Definition and delineation of the program external context and factors within each program allows: (a) insight regarding the potential to translate the program to other sites, (b) understanding of mechanisms and/or connections needed for implementation, and (c) exact definitions of outcomes. By using a matrix, connections between the components can further understanding about what makes a program successful, or not, and may provide insight into previously unobserved macro-contextual factors.

Program Explained

Comparing pre- versus post-booking diversion programs is almost impossible because the programs serve different sub-populations (Draine et al., 2005). Mental health courts also involved in diversion programs vary dramatically and are typically developed at the local level by communities in response to specific local issues (Frisman et al., 2017; Hiday & Moloney, 2014; Steadman et al., 2005). In addition, there are informal diversion programs that rely on first responders for referrals to community-based treatment, potentially using the threat of arrest for the individual to accept diversion to treatment (L. Gittner & Dennis, 2021; Kopak & Frost, 2017). Thus, clear description of the programs is needed (Potter, 2010). To fully define a program, the following constructs should be reported: (a) Engagement, (b) Treatment Venue, (c) Length of Service, (d) Treatment Type(s), (e) Eligibility, (f) Budget, (g) Staffing, (h) Outcome Measures, and (i) Stability. Table 2 displays the constructs needed for complete program description reporting.

Table 2.

Program Variable Definitions Explained.

Context	Variable definition
Community Socioeconomic StatusSocial determinants of health are the conditions in which people “are born, grow, live, work, and age” shaped by the distribution of money, power, and resources (e.g. education, employment for adults, distribution of wealth, and characteristics of housing and other built structures) (Compton & Shim, 2015). Social determinants affect health but also affect opportunity, potential criminal activity, and access to appropriate care such as diversion.	Median community income; education is percent of individuals >25 years with bachelor’s degree (US_Census_Bureau, 2019), employment is percent individuals >16 years participating in labor force (US_Census_Bureau, 2019); percent of population below US defined poverty level (US_Census_Bureau, 2019)
Racial DemographicsThere is a complex relationship between race ethnicity and health (Singh et al., 2017). Community racial/ethnic composition are based on US census categories (US_Census_Bureau, 2019)	Percent White, Black, and other races; and percent Hispanic ethnicity in population (US_Census_Bureau, 2019)
HomelessnessHomelessness is another major diversion program outcome influencer, with homeless individuals having an increased risk of justice-involvement (Greenberg & Rosenheck, 2008)	Most recent Point-in-time count (The 2022 Annual Homelessness Assessment Report (AHAR) to congress point-in-time estimates of homelessness 2023)
PoliticsCrime is a partisan political issue impacting community support, policies, and resources for diversion programs (Tonry, 2014). Crime politics support punishment over treatment (Campbell, 2011), resulting in less support and buy-in for diversion programs	Political orientation defined by the political party of the winning presidential candidate classified as democrat, independent, or republican
Oversight/Primary AgencyProgram jurisdiction and design can be associated with participant attrition; higher attrition occurs in mental health provider run programs versus lower attrition in criminal justice programs (Crisanti et al., 2014). Jurisdiction can vary between multiple municipal police departments and/or county sheriff’s offices, multiple courts (municipal, magistrate, county), mental health authorities, and/or non-profit agencies	Categorize the primary agency responsible for diversion:Local jail,Police force,Sheriff’s office,Detention center,Mental health agency,Hospital emergency room,Non-profit agency
Number of Collaborating AgenciesDiversion of justice-involved persons with mental illness may be seen politically as not being “tough on crime” and therefore may limit collaboration among all agencies necessary to make diversion successful (Tonry, 2014)	Type and number of collaborating agencies
StabilityCo-location of staff from different agencies to share expertise and interagency continuity of care, shared treatment plans and client records, and shared case review meetings create more stable programs (Lee et al., 2013)	Funding—governmentFunding—Non-profitFunding—privateCloseness of the collaboration

Context Explained

From evaluation of public health programs, we have learned context matters because community factors do affect program output and outcomes (Morgan & Ziglio, 2007). Just as in a public health program, descriptions of a diversion program’s context should include (a) the setting and environmental influences (e.g., history, geography, politics, social and economic conditions) and (b) the processes involved (e.g., between related or competing organizations within which the program operates) (Milstein & Wetterhall, 1999). To fully define a context of the program setting, the following constructs should be reported: (a) Socioeconomic Status (SES), (b) Racial Demographics, (c) Homelessness, (d) Politics, (e) Oversight/Primary Agency, (f) Collaborating Agencies, and (g) Stability. Table 3 displays the constructs needed for thorough context reporting.

Table 3.

Context Variable Definitions Explained.

Context variables	Definition
Community Socioeconomic StatusSocial determinants of health are the conditions in which people “are born, grow, live, work, and age” shaped by the distribution of money, power, and resources (e.g. education, employment for adults, distribution of wealth, and characteristics of housing and other built structures) (Compton & Shim, 2015). Social determinants affect health but also affect opportunity, potential criminal activity, and access to appropriate care such as diversion	Median community income; education is percent of individuals >25 years with bachelor’s degree (US_Census_Bureau, 2019), employment is percent individuals >16 years participating in labor force (US_Census_Bureau, 2019); percent of population below US defined poverty level (US_Census_Bureau, 2019)
Racial DemographicsThere is a complex relationship between race ethnicity and health (Singh et al., 2017). Community racial/ethnic composition are based on US census categories (US_Census_Bureau, 2019)	Percent White, Black, and other races; and percent Hispanic ethnicity in population (US_Census_Bureau, 2019)
HomelessnessHomelessness is another major diversion program outcome influencer, with homeless individuals having an increased risk of justice-involvement (Greenberg & Rosenheck, 2008)	Most recent Point-in-time count (The 2022 Annual Homelessness Assessment Report (AHAR) to Congress Point-in-time estimates of homelessness, 2023)
PoliticsCrime is a partisan political issue impacting community support, policies, and resources for diversion programs (Tonry, 2014). Crime politics support punishment over treatment (Campbell, 2011), resulting in less support and buy-in for diversion programs	Political orientation defined by the political party of the winning presidential candidate classified as democrat, independent, or republican
Oversight/ Primary AgencyProgram jurisdiction and design can be associated with participant attrition; higher attrition occurs in mental health provider run programs versus lower attrition in criminal justice programs (Crisanti et al., 2014). Jurisdiction can vary between multiple municipal police departments and/or county sheriff’s offices, multiple courts (municipal, magistrate, county), mental health authorities, and/or non-profit agencies	Categorize the primary agency responsible for diversion:local jail,police force,sheriff’s office,detention center,mental health agency,hospital emergency room,non-profit agency
Number of Collaborating AgenciesDiversion of justice-involved persons with mental illness may be seen politically as not being “tough on crime” and therefore may limit collaboration among all agencies necessary to make diversion successful (Tonry, 2014)	Type and number of collaborating agencies
StabilityCo-location of staff from different agencies to share expertise and interagency continuity of care, shared treatment plans and client records, and shared case review meetings create more stable programs (Lee et al., 2013)	Funding—governmentFunding—non-profitFunding—privateCloseness of the collaboration

Community System Explained

The politics of local diversion programs are divided by party and race interacting to produce very different viewpoints on issues related to crime (Eckhouse, 2019) and subsequently affecting reporting funding decisions and sustainability of the programs. Understanding program influences (e.g., history, geography, politics, social and economic conditions, efforts of related or competing programs) is essential for a context-sensitive evaluation and aid with interpreting findings accurately and assessing generalizability (Milstein & Wetterhall, 1999). To fully define the community system, the program within the following constructs should be reported: (a) Region, (b) Rurality, (c) Size, and (d) Population Stability. Table 4 displays the constructs needed for describing the community the program is contained within.

Table 4.

Community System Variable Definitions Explained.

Community system	Variable definition
RegionRegional differences produce variation in local criminal justice practices, resources, attitudes, and policies toward diversion programs (Lattimore et al., 2020) creating significant differences in implementation and outcomes (Pickett, 2019; Potter, 2010). Region is defined according to the US Census tract regions: Northeast, Midwest, South, and West further broken down to capture sub-regional difference (US_Census_Bureau, 2020)	Northeast,Midwest,South,West
RuralityRurality is defined using the U.S. Census bureau classification. An urban classification includes urbanized areas and clusters, while rural classification includes all “territory not included within an urban area” (US_Census_Bureau, 2020)	Urban,Suburban,Rural
SizeSize is defined by the National Center for Education Statistics to classify population density (Geverdt, 2019). Size entails a great number of factors including community perspective and resources (Ritchie, 2013). Large urban communities have numerous support agencies and are different than small rural communities with few agencies outside of the criminal justice system and often lack specialized mental health facilities	City—large, midsize, smallSuburban—large, midsize, smallTown—fringe, remoteRural—fringe, distant, remote
Population StabilityCensus population migration in versus out using the most recent census flow mapper (U.S. Census Bureau, Net County-to-County Migration Flow [Year Estimate], 2023)	In migration rateOut migration rate

Using the Matrix

Reporting the program, context, and community system using shared definitions makes it easier to decide if a program implemented in one city can be transferred to another setting. Diversion programs are designed to address specific problems in a specific community; it is important to understand if context and community systems are similar. The matrix is designed to provide the information needed to assess the similarity and applicability of programs in different jurisdictions.

Exemplar

The proposed matrix model was used to compare Lubbock County Justice-Involved Veteran Diversion (K. B. Gittner et al., 2022) to Miami-Dade County SMI Diversion (GOB Project 193—Mental Health Diversion Facility Service Capacity and Fiscal Impact Estimates June 9, 2016, 2016) in Table 5. The matrix identifies similarities and differences when comparing programs and may provide insight into the potential to successfully implement a program across jurisdictions (e.g., external validity; clientele, context, and system similarities; understand nuances; define policy; Bardach & Patashnik, 2019).

Table 5.

Using the Matrix for Different Programs.

Program	Definitions	Lubbock veterans	Miami-Dade Serious Mental Illness (SMI) individuals
Engagement	Timing of diversion program	Post-booking	Pre-bookingPost-booking
Treatment venue	Within detention centerWithout incarcerationIn patientCommunity out patient	Treatment with incarceration	Treatment without incarceration
Length of service delivery	Active days, hours, monthsPassive days, hours, months	Varies 30–365 daysNormally long-term	VariesNormally short-term
Treatment penalties for non-compliance	Presence or absence of sanctions	None	Unknown
Treatment type	Range of treatment services	Out or in patient stabilization/treatmentCrisis responseHousing stabilizationTreatment for mental health, substance use, and traumaCase management with ongoing peer supportLegal servicesEmployment and vocational trainingFollow-up peer support	Adult crisis stabilization unit and addiction receiving facilityVarious levels of residential treatmentTreatment for co-occurring substance use and trauma-related disordersOutpatient behavioral health and primary care treatmentDay treatment and day activity programsCrisis respite servicesEmployment/vocational training services
Eligibility	Diagnosis	Veteran status required with any mental health or SUD Dx	Serious mental illness with recidivism history
Eligibility	Type of offense	Any	Must be multiple offenses
Eligibility	Crime classification (against property or person)	AnyAny misdemeanor and felony except capital murder	Non-violentMisdemeanor and less serious felonies
Budget	Internal costs	Average cost for housing $2,024/veteran	Average cost $3,726/person
Budget	Internal primary agency cost plus all collaborating agency costs	Cannot capture because over 30 agencies provide services	Average total cost $6,639/program enrollee
Staffing	QualificationsFTE	Unknown	Unknown
Outcome measures	As defined by program	Diverted n = 571;Lenght of Stay (LOS) went from 18.5 to 13 days; 30% reduction92% who successfully completed treatment, have not re-offended for the same offense in 5 years178 had bi-monthly check-ins11.25 days from homelessness to stable housing	Diverted n = 480;Average LOS 20.9; 52% reductionBefore program booked 4.5/year after program 1.2/year;Total cost savings associated with diversion $1.7 million
Stability	Years program has been in operation	7 years	>20 years
Context
Socioeconomic status	2019 Median household income/% below federal poverty line	$52,254 ± $2,403/20.6%	$55,171 ± $1,24615.6% ± 0.8%
Education	Percentage of population	88% > High School34% > Bachelor	80.4% > High School30.7% > Bachelor
Employment	Unemployment rate of person >16 years (2021)	5.8%	6.4%
Racial demographics	Census race categories	White 50.2% ± 1.1%Black 7.1% ± 0.7%Native 0.3% ± 0.2%Asian 2.6% ± 0.5%Hispanic 37.4% ± 1.1%	White 12.8% ± 0.1%Black 15.3% ± 0.1%Native 0.1% ± 0%^a Asian 1.5% ± 0.1%Hispanic 69.4% ± 0%
Homelessness	Department of Housing and Urban Development Point-in-time counts (2020)	283 (0.11%)	3,560 (0.13%)
Politics	2020 Presidential Election	65.27% Republican34.73% DemocratVoter turnout 66.14%	46.69% Republican53.31% DemocratVoter turnout 74.59%
Primary agency	Name and type facility	Lubbock County Detention Center (Jail)	Miami-Dade Corrections & Rehabilitation Depart. (Jail)
Number of collaborating agencies	Count	5 (actively involved)25 (referral options)	2
Types of collaborating agencies	Court, mental health, social service agencies	Mental Health AuthorityVetStar (Social Service agency)CourtVeterans Administration	Diversion FacilityCrisis Intervention Team
Stability	Funding sources	Government; state and federal grants	Government
Community system
Region	Census region	South	South
Rurality	Urban, suburban, rural	Rural	Urban
Size	Census category	Town-remote258,870 Pop.134.7 miles² 1,922 people/miles²	City-large2,716,940 Pop.1,899.8 miles² 1,430 people/miles²
Population stability	Rate out and in migration	6% out; 10.1% in	3.8% out; 3.7% in

The two program exemplars presented demonstrate the complexity of comparison. The two programs are both in the southern United States, serve similar SES communities, and are administered by jails. One program is in a geographically isolated, moderate size city surrounded by a large rural area, predominantly conservative politically and 50% White, 7% Black, and ~40% Hispanic. The other is in a large urban area, predominantly liberal politically with 13% White, 15% Black, and 70% Hispanic. The communities are different politically (conservative vs. liberal) and majority White versus non-White. Program jurisdiction policy choices for jurisdiction, treatment site, eligibility, funding, and services offered are not the same. The politics of crime come into play in the conservative rural community with punitive public opinion shaping the types and populations as deserving of diversion (Wozniak, 2016); veterans, for example, with any type of criminal charges are eligible for diversion. This contrasts with the politics of a predominantly liberal non-White community using the severity of mental illness as the guiding force of who should be treated by diversion (Nelson, 2010). Co-occurring SUD (Broner et al., 2004) complicates the understanding of outcomes. The number and types of collaborating agencies is very different (5 + 25 vs. 2 agencies); types of agencies for the one program are diverse (courts, mental health providers, social service providers, government agencies, large network of secondary collaborators) and the other has only two collaborating agencies, the Crisis Intervention Team and the Diversion Facility. Many factors are different between the programs: diversion timing (post-booking vs. pre- and post-booking), site (incarceration vs. no incarceration), and length of service (long vs. short term). Services are different but both include crisis stabilization and various levels of in- and outpatient treatment; however, the biggest difference is the follow up with legal and peer support services provided by one program.

Program eligibility is the biggest difference between the programs. One program only serves veterans regardless of mental health or SUD diagnosis; the other serves only individuals with a serious mental illness diagnosis. The other significant eligibility difference is one program serves veterans with any charge (except capital murder) while the other program only serves individuals with non-violent less serious misdemeanor and felony offenses. Mental health and SUD diagnoses and treatments cannot be fully separated, so it is difficult to determine the effect on outcomes. Some diversion programs such as the Miami-Dade, for example, restrict the definition of mental illness to serious mental illness versus broad definitions of mental illness (Hiday & Moloney, 2014). The eligibility criteria create very different service populations between the programs. Program outcomes cannot easily be compared, and the costs are reported differently. Interestingly, the number of diverted individuals being reported on is similar, even though the service communities have a 10-fold population difference; but we do not know the reporting timeframes. Both programs reduced length of stay, but this result needs to be tempered by the fact that the program with a lower reduction begins and provides treatment while the individual is still incarcerated.

The outcomes appear better for the program serving only individuals with serious mental illness, but the other program serves individuals with much more serious crimes. Thus, comparing outcomes is not possible because the population served is not the same and outcomes may differ between the program participants because of baseline differences that cannot be matched or accounted for (Petticrew et al., 2005). Program type and treatments offered are similar for crisis stabilization; employment, housing, and co-morbidity treatment but there are interventional difference in duration long versus short term; services wrap around services with peer support versus outpatient treatment; and case management versus outpatient primary care treatment. The matrix demonstrates program participants are not the same and more than likely programs are not comparable because of selection bias. Selection bias is confounded with both program eligibility, program type and treatments offered (Draine et al., 2005).

Limitations

The study limitations are the lack of previously published diversion program standards, the wide comprehensive collection of constructs that have not been tested empirically for relevance to a diversion program, and the rudimentary nature of the exemplar (because of the lack of published information that is the same between programs). Until more programs report the same set of constructs, it will be difficult to test the applicability of the matrix to compare programs. Nevertheless, our focus remains on standardizing a reporting system of program, context, and community system constructs rather than reporting program comparisons.

Conclusion

Using a standard reporting matrix to understand diversion of justice-involved individuals with serious mental illness has the potential to assist policy makers and practitioners with understanding diversion and the creation of sustainable diversion programs (L. Gittner & Dennis, 2021). The proliferation of diversion programs for justice-involved individuals with mental illness has far outpaced the development and use of common definitions and standard reporting techniques and subsequently, the language used to describe alternatives has become muddled (“No Entry a National Survey of Criminal Justice Diversion Programs and Initiatives,” 2013). Scrutinizing diversion program features, including purpose and place in a larger context (e.g., political pressures, strength of community involvement, social and health assets, financial capacity), can describe the way the program was intended to function and the way it was actually implemented (Milstein & Wetterhall, 1999).

The need for specificity when describing diversion programs will become of increasing importance in the growing exchange of ideas, innovations, and best practices for both local officials, practitioners, policy makers, and researchers. The proposed matrix provides standards for reporting diversion program evaluations and outputs/outcomes and can clarify nuances especially when there is a proliferation of similar programs. Standard reporting provides a path toward more generalizable assessment of diversion that would allow policy makers and practitioners to make informed decisions about how best to utilize existing programs within their jurisdiction.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Drs. Gittner and Dennis from the United States Department of Justice Bureau of Justice Assistance Award MO-BX-0031 and MO-BX-0030.

Author Biographies

Lisaann S. Gittner, PhD, is an associate professor at TTUHSC School of Population and Public Health. She uses big data to model factors contributing to healthy places. She is funded by NSF (#1238338) to model the etiology of lifestyle diseases, healthy spaces, and adverse social conditions. Previously, she was funded to build diversion collaboratives to reduce incidence of individuals with mental health issues in detention centers in rural areas by DoJ (MO-BX-0031 and MO-BX-0030).

Jeff A. Dennis, PhD, is an associate professor at Texas Tech University Health Sciences Center, School of Population and Public Health. His research examines social determinants of health and public mental health. Previously, he was funded to build diversion collaboratives and first responder mental health training in rural areas by DoJ (MO-BX-0031 and MO-BX-0030).

References

The 2022 Annual Homelessness Assessment Report (AHAR) to congress point-in-time estimates of homelessness . (2023). https://www.huduser.gov/portal/sites/default/files/pdf/2022-AHAR-Part-1.pdf

Agopian

M. W.

(1977). Evaluation of adult diversion programs: The California experience. Federal Probation, 41, 15–18.

Bardach

(2004). The extrapolation problem: How can we learn from the experience of others? Journal of Policy Analysis and Management, 23(2), 205–220.

Bardach

Patashnik

E. M.

(2019). A practical guide for policy analysis: The eightfold path to more effective problem solving. CQ press.

Blomberg

T. G.

(1983). Diversion’s disparate results and unresolved questions: An integrative evaluation perspective. Journal of Research in Crime and Delinquency, 20(1), 24–38.

Broner

Lattimore

P. K.

Cowell

A. J.

Schlenger

W. E.

(2004). Effects of diversion on adults with co-occurring mental illness and substance use: Outcomes from a national multi-site study. Behavioral Sciences & the Law, 22(4), 519–541. https://doi.org/10.1002/bsl.605

Campbell

M. C.

(2011). Politics, prisons, and law enforcement: An examination of the emergence of “law and order” politics in Texas. Law & Society Review, 45(3), 631–665.

Chambers

D. A.

Glasgow

R. E.

Stange

K. C.

(2013). The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implementation Science, 8(1), 117. https://doi.org/10.1186/1748-5908-8-117

Compton

M. T.

Shim

R. S.

(2015). The social determinants of mental health. Focus, 13(4), 419–425.

10.

Crisanti

A. S.

Case

B. F.

Isakson

B. L.

Steadman

H. J.

(2014). Understanding study attrition in the evaluation of jail diversion programs for persons with serious mental illness or co-occurring substance use disorders. Criminal Justice and Behavior, 41(6), 772–790.

11.

Data collection across the sequential intercept model . (2019). GAINS Action Brief (Section 22, Issue PEP19-SIM-DATA). https://store.samhsa.gov/sites/default/files/d7/priv/pep19-sim-data.pdf

12.

Development_Services_Group_Inc. (2017). Diversion from formal Juvenile Court processing literature review. Office of Juvenile Justice and Delinquency Programs. https://www.ojjdp.gov/mpg/litreviews/Diversion_Programs.pdf

13.

Draine

Blank

Kottsieper

Solomon

(2005). Contrasting jail diversion and in-jail services for mental illness and substance abuse: Do they serve the same clients? Behavioral Sciences & the Law, 23(2), 171–181.

14.

Eckhouse

(2019). Race, party, and representation in criminal justice politics. The Journal of Politics, 81(3), 1143–1152.

15.

Fox

Miley

L. N.

Kortright

K. E.

Wetsman

R. J.

(2021). Assessing the effect of mental health courts on adult and juvenile recidivism: A meta-analysis. American Journal of Criminal Justice, 46, 644–664.

16.

Frisman

L. K.

Lin

H. J.

Rodis

E. T.

Grzelak

Aiello

(2017). Evaluation of CT’s ASIST program: Specialized services to divert higher risk defendants. Behavioral Sciences & the Law, 35(5–6), 550–561.

17.

Geverdt

D. E.

(2019). Education Demographic and Geographic Estimates (EDGE) Program: Locale boundaries file documentation, 2017 (NCES 2018-115). U.S. Department of Education; National Center for Education Statistics. https://nces.ed.gov/programs/edge/docs/EDGE_NCES_LOCALE.pdf

18.

Gittner

K. B.

Gittner

L. S.

Dennis

J. A.

Matheny

L. M.

(2022). VetStar: Justice-involved Veterans successful mental health diversion. Journal of Veterans Studies, 8(1), 1–14. doi. org/10.21061/jvs.v8i1.246

19.

Gittner

Dennis

. (2021). A public health perspective on Diversion Programs for justice-involved individuals with mental health. In Scott-Hayward

C. S.

Copp

J. E.

Demuth

(Eds.), The handbook on pretrial justice (pp. 166–188). ASC Division on Corrections and Sentencing.

20.

GOB Project 193—Mental health diversion facility service capacity and fiscal impact estimates June 9, 2016 . (2016). http://napco4courtleaders.org/wp-content/uploads/2018/10/Diversion-Facility-Capacity-and-Fiscal-Impact-Estimates-06092016.doc

21.

Greenberg

G. A.

Rosenheck

R. A.

(2008). Jail incarceration, homelessness, and mental health: A national study. Psychiatric Services, 59(2), 170–177.

22.

Hiday

V. A.

Moloney

M. E.

(2014). Mental illness and the criminal justice system. In Cockerham

W. C.

Dingwall

Quah

( Eds.), The Wiley Blackwell encyclopedia of health, illness, behavior, and society, (pp. 1–5). Wiley. doi.org/10.1002/9781118410868.wbehibs513

23.

Klingele

(2019). Measuring change. The Journal of Criminal Law and Criminology (1973-), 109(4), 769–817.

24.

Kopak

A. M.

Frost

G. A.

(2017). Correlates of program success and recidivism among participants in an adult pre-arrest diversion program. American Journal of Criminal Justice, 42(4), 727–745.

25.

Lange

Rehm

Popova

(2011). The effectiveness of criminal justice diversion initiatives in North America: A systematic literature review. International Journal of Forensic Mental Health, 10(3), 200–214.

26.

Lattimore

P. K.

Tueller

Levin-Rector

Witwer

(2020). The prevalence of local criminal justice practices. Federal Probation, 84, 28.

27.

Lee

S. J.

Crowther

Keating

Kulkarni

(2013). What is needed to deliver collaborative care to address comorbidity more effectively for adults with a severe mental illness? Australian & New Zealand Journal of Psychiatry, 47(4), 333–346.

28.

Lowder

E. M.

Rade

C. B.

Desmarais

S. L.

(2018). Effectiveness of mental health courts in reducing recidivism: A meta-analysis. Psychiatric Services, 69(1), 15–22.

29.

Milstein

Wetterhall

S. F.

(1999). Framework for program evaluation in public health. Morbidity and Mortality Weekly Report, 48, 1–40.

30.

Morgan

Ziglio

(2007). Revitalising the evidence base for public health: An assets model. Promotion & Education, 14(Suppl. 2), 17–22.

31.

Nelson

C. A.

(2010). Racializing disability, disabling race: Policing race and mental status. Berkeley Journal of Criminal Law, 15, 1.

32.

No entry a national survey of criminal justice diversion programs and initiatives. (2013). The Center for Health and Justice at TASC. https://www.centerforhealthandjustice.org/tascblog/Images/documents/Publications/CHJ%20Diversion%20Report_web.pdf

33.

Øvretveit

Leviton

Parry

(2011). Increasing the generalisability of improvement research with an improvement replication programme. BMJ Quality & Safety, 20(Suppl. 1), i87–i91.

34.

Paccone

J. C.

(2020). Outcomes of national jail diversion programs for individuals with mental illnesses or substance use disorders: A comparison to the criminal justice system as is. University of California, Los Angeles.

35.

Perla

R. J.

Provost

L. P.

Parry

G. J.

(2013). Seven propositions of the science of improvement: Exploring foundations. Quality Management in Healthcare, 22(3), 170–186.

36.

Petticrew

Cummins

Ferrell

Findlay

Higgins

Hoy

Kearns

Sparks

(2005). Natural experiments: An underused tool for public health? Public Health, 119(9), 751–757.

37.

Pickett

J. T.

(2019). Public opinion and criminal justice policy: Theory and research. Annual Review of Criminology, 2, 405–428.

38.

Potter

R. H.

(2010). Jails, public health, and generalizability. Journal of Correctional Health Care, 16(4), 263–272.

39.

Remington

(2010). Functional outcome. In Stolerman

I. P.

(Ed.), Encyclopedia of psychopharmacology (pp. 546–547). Springer. https://doi.org/10.1007/978-3-540-68706-1_1596

40.

Rempel

Labriola

M. M.

Hunt

Davis

R. C.

Reich

W. A.

Cherney

(2018). NIJ’s multisite evaluation of prosecutor-led diversion programs: Strategies, impacts, and cost-effectiveness (No. 251665). National Criminal Justice Reference Service.

41.

Ritchie

(2013). Social determinants of health: A view on theory and measurement. Rhode Island Medical Journal, 96(7), 15–19.

42.

Rogers

M. S.

McNiel

D. E.

Binder

R. L.

(2019). Effectiveness of police crisis intervention training programs. The Journal of the American Academy of Psychiatry and the Law, 47(4), 414–421.

43.

Ryan

Brown

C. K.

Watanabe-Galloway

(2010). Toward successful postbooking diversion: What are the next steps? Psychiatric Services, 61(5), 469–477.

44.

Rychetnik

Frommer

Hawe

Shiell

(2002). Criteria for evaluating evidence on public health interventions. Journal of Epidemiology & Community Health, 56(2), 119–127. https://doi.org/10.1136/jech.56.2.119

45.

Sarteschi

C. M.

Vaughn

M. G.

Kim

(2011). Assessing the effectiveness of mental health courts: A quantitative review. Journal of Criminal Justice, 39(1), 12–20.

46.

Schwalbe

C. S.

Gearing

R. E.

MacKenzie

M. J.

Brewer

K. B.

Ibrahim

(2012). A meta-analysis of experimental studies of diversion programs for juvenile offenders. Clinical Psychology Review, 32(1), 26–33.

47.

The sequential intercept model . (2020). SAMHSA. https://www.samhsa.gov/criminal-juvenile-justice/sim-overview

48.

Singh

G. K.

Daus

G. P.

Allender

Ramey

C. T.

Martin

E. K.

Perry

Andrew

Vedamuthu

I. P

. (2017). Social determinants of health in the United States: Addressing major health inequality trends for the nation, 1935-2016. International Journal of MCH and AIDS, 6(2), 139–164.

49.

Sirotich

(2009). The criminal justice outcomes of jail diversion programs for persons with mental illness: A review of the evidence. Journal of the American Academy of Psychiatry and the Law Online, 37(4), 461–472.

50.

Sorenson

Sørensen

J. B.

(2001). Finding the right mix: Franchising, organizational learning, and chain performance. Strategic Management Journal, 22(6–7), 713–724.

51.

Steadman

H. J.

Redlich

A. D.

Griffin

Petrila

Monahan

(2005). From referral to disposition: Case processing in seven mental health courts. Behavioral Sciences & the Law, 23(2), 215–226. https://doi.org/10.1002/bsl.641

52.

Tonry

(2014). Remodeling American sentencing: A ten-step blueprint for moving past mass incarceration. Criminology & Public Policy, 13(4), 503–533.

53.

US_Census_Bureau. (2019). American community survey. https://www.census.gov/acs/www/data/data-tables-and-tools/

54.

US_Census_Bureau. (2020). Urban and rural. https://www.census.gov/programs-surveys/geography/guidance/geo-areas/urban-rural.html

55.

U.S. Census Bureau, net county-to-county migration flow (5-year estimate) . (2023). https://flowsmapper.geo.census.gov/map.html#

56.

VetStar. (2017). Help is here. Vetstar stakeholder report. Lubbock, TX: Author.

57.

VetStar. (2018). Help is here. VetStar stakeholder report. Lubbock, TX: Author.

58.

Wozniak

K. H.

(2016). Public opinion and the politics of criminal justice policy making: Reasons for optimism, pessimism, and uncertainty. Criminology & Public Policy, 15, 179–186.