Measurement-based care implementation by K-12 public school clinicians: A mixed-methods proof of concept study

Abstract

Background

This study assessed the feasibility and acceptability of a multilevel, multi-component implementation strategy for measurement-based care (MBC) called Feedback and Outcomes for Clinically Useful Student Services (FOCUSS). FOCUSS includes six components selected in our prior work with a national sample of school mental health stakeholders. This is among the first demonstrations of MBC with school-employed clinicians. We explored proof of concept by observing MBC adoption rates achieved by the end of the school year and other related implementation outcome data.

Method

A mixed-method, single-arm pilot study was conducted during one academic year with 10 school-employed mental health clinicians in two K-12 public school districts in Connecticut. Clinician adoption was assessed by monthly fidelity monitoring of measures clinicians entered in the feedback system. Clinician self-reported practices, attitudes, feasibility, acceptability, and appropriateness of using MBC with K-12 students was assessed by pre-training, 3-, 6-, and 9-month surveys. School year-end qualitative interviews explored clinician implementation experiences using MBC and FOCUSS implementation supports to inform future changes to FOCUSS in a district-wide trial.

Results

Clinicians were asked to implement MBC with five students; 60% of the clinicians achieved or exceeded this target, and MBC was adopted with 65 students. Other implementation outcomes were comparable to related studies. Qualitative feedback indicated that MBC is clinically valuable in schools by providing consistency and structure to sessions, is compatible with school mental health, and well regarded by students and parents. FOCUSS implementation supports were regarded as helpful, and individual performance feedback emails appeared to be a necessary component of FOCUSS to boost post-training implementation.

Conclusion

This is among the first studies of MBC implementation with school-employed mental health professionals in the United States. Results demonstrate proof of concept for MBC implementation with school social workers, psychologists and counselors and support subsequent district-wide use of FOCUSS to install MBC in schools.

Plain Language Summary

A Pilot Test to Learn How School Mental Health Clinicians, Students and Caregivers Can Monitor Student Progress Together The most common place that children and adolescents access mental health services is at school, but the quality of school mental health services varies. This study explores acceptability and feasibility of training school mental health clinicians to use a practice called measurement-based care (MBC) to improve the quality of mental health treatment provided in schools. MBC involves the use of student- and caregiver-reported progress measures to help the clinician, student, and caregiver(s) to monitor and adjust student services together depending on the student and family needs. School mental health clinicians (such as school social workers, school counselors, and school psychologists) are busy “front line” professionals with many responsibilities such as supporting student mental health crises in real time, regularly meeting with students on their caseload, providing teacher consultation, student assessment, leading and attending team meetings and helping with non-clinical tasks like lunch duty that come with working in a school. This poses challenges for MBC implementation, which is not common in schools. Our team provided 10 school mental health clinicians with training, group consultation, and other supports to start using MBC with a subset of their students. In one school year, MBC was used by nine out of 10 clinicians who participated in the study, and in end of year interviews, they reported that MBC was feasible, acceptable, and appropriate for schools. Interview data provide additional detail about the clinical value of MBC in schools, student and caregiver engagement in their services, and the clinicians’ experiences with this new practice. Future directions include larger-scale implementation of MBC in school districts.

Keywords

school mental health measurement-based care implementation strategies CFIR implementation determinants

Introduction

Schools are the primary setting where children and families in the United States receive mental health treatment (Duong et al., 2021). School mental health care is heterogeneous in terms of student diversity (Wilk et al., 2022), provider disciplines (social work, psychology, counseling), staffing ratios and clinical practices (Reaves et al., 2022; Zabek et al., 2022). School clinicians have large caseloads and are responsible for prevention to intervention, crisis response, and non-clinical duties (Zabek et al., 2022). Fortunately, there is growing interest in supporting school mental health professionals’ skills and capacity to deliver evidence-based mental health services (American Academy of Pediatrics, 2021; Zabek et al., 2022). Flexible, evidence-based strategies applicable to a wide range of students, such as measurement-based care (MBC), are needed to improve school mental health treatment quality.

MBC is the routine use of patient-reported progress measures to inform collaborative treatment adjustments in real time (Scott & Lewis, 2015). MBC core components include “Collect” (i.e., providing a clear, client-centered rationale for MBC and selecting and administering a progress measure aligned with treatment targets), “Share” (i.e., providing feedback and discussing scores in real time), and “Act” (i.e., adjusting individual sessions and/or overall treatment planning) (Barber & Resnick, 2023). MBC has evidence of effectiveness with children in traditional mental health care settings, but MBC implementation and effectiveness in schools is not well known (Parikh et al., 2020). In theory, MBC is highly compatible with school settings (Bohnenkamp et al., 2015; Connors et al., 2022; Lyon et al., 2022). MBC has the potential to promote consistent structure, student- and family-centered communication, and thus equity in the way students receive school-based interventions with progress monitoring and feedback. MBC-type practices are included in ethical standards and core competencies of national school professional organizations in the U.S. (American School Counselor Association, 2022, 2023; National Association of School Psychologists, 2020; National Association of Social Workers, 2012). Furthermore, since progress monitoring for academic subject areas is routine, especially in special education, school clinicians are expected to use progress monitoring (Connors et al., 2015).

MBC Implementation in Schools

To date, MBC has not yet been incorporated into education, but national and large-scale implementation efforts exist in healthcare systems (Interdepartmental Serious Mental Illness Coordinating Committee, 2023; Kaiser Permanente Institute for Health Policy, 2023). Yet, only 20% of providers in traditional mental health care settings report routine progress monitoring and feedback in their practice (Jensen-Doss et al., 2018), and MBC implementation among school clinicians is likely much lower (Cho et al., 2023).

There are various challenges to MBC implementation by school clinicians (Kelly, 2011; Parke, 2012). These include lack of access to measures, inadequate data systems or infrastructure, difficulty reaching caregivers, and lack of ongoing consultation or clinical supervision to support MBC practices (Connors et al., 2015). Beyond the school context, general barriers to MBC implementation include patient, practitioner, organizational, and system factors (Lewis et al., 2019). Implementation strategies to promote MBC uptake and implementation also vary widely by determinants in specific care settings (Boyd et al., 2018; Childs & Connors, 2022), with emphasis on clinician training as a standard strategy (Kassab et al., 2022), follow-up training, consultation, or supervision (Bovendeerd et al., 2019; Lyon et al., 2022), and tailoring strategies to site-based determinants (Lewis et al., 2022).

Despite these challenges, notable exceptions of MBC implementation in schools are emerging in several states, Canada, Northern Ireland, and Norway (Connors & Hoover, 2025). Domestically, MBC research in schools has been conducted in school-based health centers and with community-employed school mental health professionals (Borntrager & Lyon, 2015; Connors, Prout et al., 2021; Lyon et al., 2017; Sichel & Connors, 2022). Although these community-employed, school-based professionals expand school capacity to meet student mental health needs, school-employed mental health professionals are the foundation and “front line” for student mental health (Hoover et al., 2019). School psychologists, social workers, and counselors may face more challenges implementing evidence-based practices than their community-employed, school-based counterparts (Connors, Prout, et al., 2021; Langley et al., 2010). This is due to expanded roles and responsibilities that limit their time for clinical intervention, such as assessment, consultation, crisis management, whole-school mental health programming and non-clinical, unrelated duties including clerical and administrative work (Brown et al., 2006; Zabek et al., 2022). The largest U.S. study we are aware of with MBC implementation among 75 school-employed clinicians found significant effects in standardized and individualized assessment use following a brief online training and post-training consultation, but no significant effect on treatment modification (Lyon et al., 2022). Learning how to effectively support MBC among school-employed clinicians offers an opportunity for district-wide scaling of MBC.

Present Study

The goal of this study is to understand MBC acceptability and feasibility among school-employed clinicians with a multi-component implementation strategy package developed for this innovation and context. Feedback and Outcomes for Clinically Useful Student Services (FOCUSS) is an empirically derived implementation strategy package to promote school clinician MBC practices (Connors et al., 2022). FOCUSS strategies include: (1) assess clinician readiness; (2) identify and prepare champions; (3) develop a district implementation plan; (4) provide initial MBC training; (5) provide access to and training on measures; and (6) provide post-training group consultation (see Table 1). Performance feedback (PF) emails were added as a 7th strategy during this pilot (see Results for further explanation).

Table 1.

FOCUSS Implementation Strategies.

Strategy Name	Definition	Timing	Dose
Assess Clinician Readiness	Clinician-reported attitudes toward MBC, determinants of practice and implementation climate are aggregated and shared with district leaders to inform site-specific tailoring. For example, MBC training may target attitudinal barriers and/or discuss how MBC fits with other district initiatives. Results inform implementation plans (below).	Pre-training (summer)	Once
Identify and Prepare Champions	Approximately three site champions per district are identified. These are clinicians and/or supervisors who demonstrate commitment to MBC and are willing to support clinicians during FOCUSS. Champions liaise with the study team on a quarterly basis, including prior to initial trainings.	Pre-training (summer)	Once
Develop District Implementation Plan	A brief, usable, district-specific implementation plan includes purpose and goals of FOCUSS, its alignment with other local initiatives, training and consultation dates, data collection, clinician and caregiver participation activities, and contact information for MBC trainers and study team. School system leaders use this plan to communicate about FOCUSS during and after the study in their district.	Pre-training (summer)	Once
Provide Initial MBC Training	Clinicians attend a one-day MBC training about the purpose, goals, and evidence for MBC, practice-specific didactics, discussion, behavioral rehearsal, and planning activities. We use the Collect, Share, Act model, which emphasizes real-time use of patient-reported outcome measures with students and families to promote collaboration and shared decision making.	Start of school year	Once
Measure Access and Training	Clinicians receive access to a secure, web-based measurement feedback system that houses the Outcome Rating Scale and Session Rating Scale which are part of the PCOMS. De-identified PCOMS data are entered by clinicians, students, and caregivers. Training and consultation include how to introduce, collect, and use the PCOMS measures with students, caregivers, and on teams. Clinicians can request individual consultation at any point for additional training on the measures and/or feedback system.	Throughout the school year	Once per month and more as needed
Post-Training Group Consultation	Clinicians attend a 1-hr, post-training group consultation call each month for the entire school year. Consistent with best practices in practice-specific consultation, calls include case presentation and discussion, behavioral rehearsal/role play of MBC skills, and shared learning via peer consultation as well as expert trainer consultation.	Six months post-initial MBC training	Once per month
Performance Feedback (PF)	We use a PF protocol developed and piloted by our team for consistency and to promote reproducibility for other implementation scientists. We use best practices (e.g., data are individualized and feedback is simple, provided regularly, includes a specific performance target, and specific suggestions for improvement).	Starting approx. 1 month post-initial MBC training	Once per month

Note. Continuing education units were offered for initial MBC training and post-training group consultation, which could be considered a separate implementation strategy. Identify and Prepare Champions was the only strategy that we were not able to carry out as defined (see “Limitations” section). FOCUSS = Feedback and Outcomes for Clinically Useful Student Services; MBC = measurement-based care; PCOMS = Partners for Change Outcome Monitoring System.

This study was a small-scale, proof of concept to explore MBC acceptability and feasibility, based primarily on observed adoption of progress measures in one school year. We also observed how FOCUSS operates under typical conditions with school-employed clinicians as MBC implementers in K-12 public schools. Mixed-methods data on implementation outcomes and contextual factors answered the following research questions:

To what extent do clinicians adopt MBC using FOCUSS with a subset of their caseload in one school year?

How acceptable, feasible, and appropriate is MBC for school clinicians participating in FOCUSS?

How well does MBC fit in the school mental health context and for school-employed clinicians?

How acceptable, feasible, and usable are the selected patient-reported outcome measures and measurement feedback system for FOCUSS clinicians?

This study was designed to inform adaptations to MBC and/or FOCUSS implementation strategies for a larger-scale trial. We also sought to identify what facilitates MBC implementation in schools when school-employed clinicians adopt this practice during the first school year of training and support. Finally, we explored optimal research procedures and methods for a larger-scale trial (e.g., clinician recruitment, enrollment, retention, and study measure completion).

Method

Using a sequential mixed-methods design, we collected quantitative data throughout the school year, followed by qualitative data to expand our findings for complementarity (Palinkas et al., 2011). Data were merged after all analyses and conceptually were of equal weight. Adoption and implementation rates were observed to understand how likely this practice is to be used in this context by these providers. Additional clinician self-reported acceptability and feasibility and clinician attendance to training and consultation added context to observed adoption and implementation. Also, given that this was the first pilot of FOCUSS with a small sample, quantitative implementation outcomes are not intended for population inference to school clinicians. Instead, the quantitative findings were interpreted in reference to larger-scale implementation efforts and as a foundational data source expanded by qualitative data from implementers.

Participants

We partnered with two public school districts in Connecticut in the 2021–2022 school year to provide initial MBC training to all student services staff and recruit participants. Twelve clinicians voluntarily consented to and enrolled in the pilot, but two withdrew in early October, for a final sample of 10 clinician participants. Four were from one district and six from the other. All were Female. Most (N = 7, 70%) identified as White/Caucasian, one (10%) identified as Black or African American, one (10%) as Hispanic, and one (10%) preferred not to answer. Six (60%) reported being trained in Social Work, three (30%) in School Psychology, and one (10%) in professional counseling. Roles varied (i.e., five school social workers, two school psychologists, one social work intern, one school psychology intern, and one school counselor). On average, participants worked in mental health for 11.6 years (range = 0–29, Mdn = 10) and schools for 11.9 years (range = 5–26, Mdn = 9.5). All clinicians were licensed and credentialed to provide mental health interventions independently in schools except for one social work trainee.

Procedures

We assessed clinician readiness, developed a district implementation plan in partnership with district leaders, and provided initial MBC training (see Table 1). District-wide training was provided during a professional development day. Across two districts, 64 student services personnel were invited, and 36 attended. The district with lower attendance invited student services personnel broadly, including some not appropriate as MBC implementers (e.g., speech language pathologists).

In accordance with Yale University IRB-approved procedures, district-wide training was followed by clinician consent to access to the measurement feedback system (MFS), receive MFS training, attend monthly group consultation calls, and complete study measures. With grant funds, we purchased the Better Outcomes Now MFS, which includes the Partners for Change Outcome Measurement System (PCOMS). PCOMS includes two, 4-item measures rated on an analogue scale of 1–10 called the Outcome Rating Scale (ORS) and Session Rating Scale (SRS) (Duncan, 2012). The ORS tracks psychotherapy outcomes using four dimensions of individual well-being or distress, interpersonal or relational well-being, social relationships including school and work, and overall general well-being. The SRS tracks alliance and satisfaction with services. PCOMS was selected for its evidence-base in school systems primarily outside of the United States and application to wide range of student concerns and intervention goals. A live, virtual tutorial of the online system for the PCOMS measures (Child ORS, SRS) was provided by the study team, recorded, and disseminated with individual follow-up supports made available in consultation with measure developers. Monthly, post-training group consultation was delivered by the first and fourth authors two times per month to accommodate clinician schedules. Continuing Education Units were offered for training and consultation attendance. Individual performance feedback emails were added 3 months post-training to boost initial implementation (see Table 1).

Online surveys were collected at baseline, 3-, 6-, and 9-month intervals. Clinicians were invited at the end of the year to complete an optional “implementation experience interview” on Zoom. Clinicians received $25 Amazon gift cards for each survey completion and interview. This and the continuing education units were the only compensation participants received because training, post-training consultation, and MBC practices were approved by their employer as regular duties during work hours. Due to the small nature of this pilot, members of the study team supported implementation, collected study measures, conducted and coded interviews. Both primary coders (EHC, SS) were Female; one a Clinical-Community Psychologist with a PhD and almost two decades of experience conducting qualitative research, and the other a BA-level Research Assistant trained and supervised to conduct interviews and qualitative coding. This introduces possible response bias but also offered a strength of prior established relationships. Participants were told the reason for the interviews was to gather their honest experience with MBC to inform future district-wide implementation. Zoom interviews were videorecorded in private offices of clinicians and researchers, field notes were taken, and interviews lasted 20–40 min.

Measures

To answer research question one, adoption was collected by researchers in the MFS and clinician self-reported MBC practices were collected using baseline, 3-, 6-, and 9-month surveys. To answer research question two, clinician self-reported acceptability, feasibility, and appropriateness of MBC as a practice as well as clinician attitudes toward standardized assessment were collected using four surveys. Research questions three and four were answered by qualitative clinician interviews and survey write-in responses. Mixed-methods inquiry of implementation outcomes and innovation-context fit were used to expand and contextualize observed adoption rates as the primary outcome.

Adoption and Implementation of MBC was assessed via observational tracking of clinician ORS measures entered in the MFS each month. Although the PCOMS includes the ORS and SRS, and clinicians were trained on both, we focused on the ORS as the primary indicator of adoption and initial implementation, as it is the child outcome measure used to assess treatment progress. Through review of data trends over time and team consensus we defined “adoption” as ≥1 ORS measure collected for ≥1 case, at the clinician level, and “implementation” as ≥2 ORS measure collected for ≥1 case (see Figures 1 and 2). Clinician self-reported practices with MBC were also assessed via the Current Assessment Practice Evaluation—Revised (CAPER). This seven-item measure asks clinicians to self-report what percentage of their clients in the past month/week they administered and used (i.e., provided feedback about/discussed, changed the overall treatment plan, changed activities for a single session) standardized and individualized measures with. The CAPER has demonstrated good internal consistency (α = .72) and has been replicated with a sample of 95 clinicians (Connors et al., 2021; Lyon et al., 2019). CAPER scoring was adapted in consultation with developer Dr. Aaron Lyon from a Likert scale (which produces truncated ranges) to a more detailed continuous range of 0%–100% of a clinician's caseload for each item; in prior studies, it is reported on a 4-point scale (1 = “None [0%],” 2 = “Some [1–39%],” 3 = “Half [40–60%],” 4 = “Most [61–100%]”), which still offers points of comparison to our data.

Figure 1.

MBC Adoption by Clinician: Cases With 1 or More Outcome Rating Scale.

Figure 2.

MBC Implementation by Clinician: Cases With Two or More Outcome Rating Scales.

Acceptability, Feasibility, and Appropriateness of MBC as a clinical practice was assessed using clinician self-report on the Acceptability of Intervention Measure (AIM), Feasibility of Implementation Measure (FIM), and Intervention Appropriateness Measure (IAM) (Weiner et al., 2017). Structural validity, test–retest reliability, and sensitivity to change have been established (Weiner et al., 2017). We also tracked clinician attendance to monthly post-training consultation calls and requests for individual consultation as another indicator of acceptability and feasibility of FOCUSS supports.

Attitudes were measured via the Attitudes Toward Standardized Assessment Scale—Monitoring and Feedback (ASA-MF) which has three subscales (clinical utility, treatment planning, and practicality) and acceptable internal consistency (α = .81–.85) (Connors et al., 2021; Jensen-Doss et al., 2018).

Usability of PCOMS was assessed via the Usage Rating Profile—Assessment (URP-A; 3 months, 6 months, 9 months). This 28-item measure is part of a suite of measures developed specifically to assess intervention usage in schools (Miller et al., 2014). Items ask about acceptability, understanding, home–school collaboration, feasibility, system climate, and system support for any assessment tool inserted to item stems (ORS and SRS from the PCOMS, in this case). The six factors have good internal consistency (α = .62–.90) (Miller et al., 2014).

Clinician implementation experiences were assessed using qualitative data collected at the end of the implementation period in open-ended survey response items and individual interviews. Clinicians were asked about their experiences (1) as an interventionist, using MBC with students; and (2) as an intervention recipient, receiving FOCUSS implementation supports to do MBC. These experiences informed our understanding of the innovation-context fit of MBC as a practice in schools (RQ3), and the acceptability, feasibility, appropriateness, and usability of selected measures and measurement system in schools (RQ4). Specifically, the open-ended survey responses asked clinicians to write in comments, questions, or feedback about FOCUSS, what they liked about FOCUSS, what has been valuable to them and their students, and what advice they would have to improve the project. The semi-structured interview asked about motivations to consent to participation, implementation experiences with FOCUSS (with prompts about barriers experienced, comfort levels with MBC, student and parent response to MBC), MBC fit and value with school mental health services, barriers anticipated for their colleagues who didn’t consent, and advice for future MBC implementation in schools.

Analyses

Quantitative clinician self-reported survey data were downloaded from Qualtrics to SPSS Version 28.0, cleaned for usable variable names and descriptions, and inspected for errors. New variables were computed for scale and total scores per guidance from measure developers, normality and distributions were examined, and coded values were checked for accuracy across time points. Due to the small sample size, primary analyses were descriptive statistics displayed over time, visually inspected for trends, and compared to published means and standard deviations for each measure.

Qualitative content analysis was completed using grounded theory as a second stage (Charmaz, 2014; Elo & Kyngäs, 2008; Hsieh & Shannon, 2005). Because existing theory and research on MBC in schools is in its infancy, we were guided by an inductive approach to understand clinician experiences and contextual factors influencing observed implementation outcomes (Damschroder et al., 2022a; Hsieh & Shannon, 2005). However, our interpretation and categorization of final focus codes were guided by established theories of implementation outcomes. For example, the Implementation Outcomes Framework (Proctor et al., 2011) and the Consolidated Framework for Implementation Research, or CFIR 2.0 (Damschroder et al., 2022b) naming conventions for constructs were used to promote transportability to other research on this innovation (MBC) and context (schools).

First, two coders (EHC, SS) independently reviewed and open-coded all qualitative survey responses and detailed notes from all six interview transcripts. Next, open codes and memos for each segment of text were compared and discussed. Open codes were revised through iterative discussion and comparison with the data to develop focus codes with operational definitions. Survey responses were coded to consensus, and interview transcripts were divided for final focus coding. Sorting focus codes into categories or themes (referred to as re-contextualization) was mostly descriptive and somewhat interpretive, based on memos, contextual information, and elaboration when available (Lindgren et al., 2020). Final focus codes were also interpreted and categorized through consideration of implementation outcomes or determinants, and comparison to the CFIR Codebook Template (CFIR Research Team—Center for Clinical Management Research, 2024). The resulting theoretical and focus codes reflect a combination of terminology from predetermined determinants and outcomes frameworks and any innovation and/or context-specific language where applicable.

Results

MBC Adoption and Practices (RQ1)

Virtually all (nine of 10) clinicians adopted MBC for 65 students, resulting in 210 sessions with measures (M = 21.2, SD = 19.5, Mdn = 20.5, range = 1–49 sessions per clinician). One clinician never adopted MBC (see Figure 1). Clinicians were asked to implement MBC with five students, but 60% used MBC with more than five (range = 0–18). Baseline CAPER data indicate this is a notable practice change; 90% reported not completing or giving feedback about standardized progress measures prior to FOCUSS. Clinicians reported relatively more common baseline practices of tracking attendance or Individualized Education Program (IEP) goals, consistent with our prior findings (REDACTED). Although caseload-wide implementation was not a stated expectation or goal in the pilot, clinicians self-reported using measures with an average of up to 32% of their caseload (see Table 2), which is similar to rates reported in a national sample of 75 school mental health clinicians (Lyon et al., 2022).

Table 2.

Secondary Implementation Outcomes (Means and Standard Deviations).

	Baseline (n = 10)		3 Months (n = 6)		6 Months (n = 7)		9 Months (n = 9)
Measure	M	SD	M	SD	M	SD	M	SD
Attitudes
Clinical utility	3.46	0.69	4.10	0.47	4.05	0.46	3.74	0.67
Treatment planning	3.94	0.80	4.67	0.41	4.51	0.51	4.47	0.44
Practicality	3.66	0.80	4.27	0.68	4.14	0.64	3.87	0.68
Overall attitudes	3.65	0.69	4.31	0.38	4.21	0.48	3.98	0.57
Feasibility of MBC	3.95	0.51	4.29	0.66	4.07	1.16	4.08	0.64
Acceptability of MBC	4.05	0.67	4.54	0.51	4.64	0.63	4.56	0.51
Appropriateness of MBC	4.15	0.74	4.50	0.55	4.68	0.55	4.47	0.54
Usability of PCOMS
Acceptability	–	–	5.24	0.67	5.17	0.57	4.85	0.76
Understanding	–	–	4.89	0.54	4.95	0.52	4.70	0.63
Home–school collaboration	–	–	3.22	1.13	4.10	0.71	3.48	1.23
Feasibility			4.81	0.80	4.64	0.75	4.41	0.93
System climate			4.71	0.93	4.68	1.02	4.19	1.34
System support			3.56	1.38	3.19	1.48	3.30	0.98
Overall usability			4.62	0.51	4.70	0.54	4.38	0.59
MBC practices
CAPER total score (%)	2.64	4.88	24.33	24.63	31.59	37.55	21.29	23.21
Standardized assessment	0.40	1.26	35.22	34.61	31.58	37.60	20.04	25.80
Individualized assessment	8.30	16.97	30.80	40.07	35.42	50.11	32.06	34.26
Treatment modification	0.39	0.86	10.90	12.31	29.42	34.00	17.69	18.76

Note. MBC = measurement-based care; PCOMS = Partners For Change Outcome Management System, including the Outcome Rating Scale and Session Rating Scale which clinicians used as the measures to track progress; CAPER = Current Assessment Practice Evaluation—Revised.

As shown in Figures 1–3, ORS measure adoption and initial implementation notably increased between November and December for nearly half the sample. We attribute this to the introduction of individual performance feedback (PF) emails in late November. Despite promising feasibility, acceptability, and signals of adequate MBC adoption at baseline, only 50% of clinicians adopted MBC (≥1 student with ≥1 progress measure) by November 1st of the school year. Thus, PF emails were initiated to boost implementation and test augmentation of FOCUSS with a strategy rated in the “borderline” range in the FOCUSS development study (i.e., “monitor implementation progress and provide feedback”; Connors, Lyon et al., 2021). As of April 2022, 90% of clinicians adopted MBC (≥1 student with ≥1 ORS progress measure) with all six core FOCUSS implementation strategies plus PF.

Figure 3.

MBC Adoption by Clinician: Total Outcome Rating Scales.

Feasibility, Acceptability, Appropriateness of MBC (RQ2)

Table 2 shows mostly encouraging results for clinician attitudes toward the use of MBC; clinicians’ views of MBC acceptability, feasibility, and appropriateness; and the usability of the PCOMS measurement system. Specifically, acceptability, feasibility, and appropriateness of MBC scores were high overall and higher than observed in a prior study of 80 clinicians implementing MBC in public children's mental health services (Sichel & Connors, 2022). Over time, mean scores appeared to increase from baseline to 3 months and then were relatively stable through 6- and 9-month follow-up. Overall usability was half a standard deviation higher than reported for other school professional samples (Miller et al., 2014), but after an initial increase from baseline to 3 months, decreased below baseline levels at 9 months. The home–school collaboration domain was lower than the other subscales in our sample, which matches existing literature (Miller et al., 2014). Attitudes toward MBC were high overall and slightly higher than a prior study of standardized assessment implementation with 95 community-employed, school-based mental health providers (Connors, Lawson et al., 2021). Finally, all clinicians attended the district-wide MBC practice training, and 90% attended group post-training consultation. Consultation call attendance was very good (M = 3.2 calls per clinician, SD = 1.75; range = 0–5) for the five scheduled calls after one snow day cancellation. Half of the clinicians attended four or five calls and several requested individual consultation.

School Clinician Implementation Experiences (RQ3 and RQ4)

Qualitative categories and codes about school clinician implementation experiences are described below and displayed with illustrative quotes in Table 3.

Table 3.

Qualitative Categories and Codes From School Clinician Implementation Experiences.

Qualitative Category	Expands Quantitative Construct	Focus Codes	Illustrative Quote(s)
Clinical Value (MBC Outcomes)	Attitudes	MBC offers consistency and structure in sessions	“I think at the beginning, even though it was very difficult for most of my students to really stop and think how they were actually feeling, but it helped them learn that too.” (107)“It made for an interesting process that really opened up opportunity to make questions deeper … I don’t have that [with] you know, “Okay, how are you doing today?” but through that paper, “Okay, why is this…”, or, “why everything is okay?” … When we have that conversation, they start to open up, you know, the layers.” (111)“I see the program as an opportunity not only to collect data, it's also for me to teach them life skills. With the consistency, the organization. You do want to see progress and be kind of tangible somehow, and you need to learn to self-monitor yourself and this is a very gentle way to start to do that.” (111)“If some of the social workers are a little hesitant, I think they’ll be pleasantly surprised. I think there's nothing to be hesitant about. I think it's great information and it's something that we haven’t had as social workers and we definitely needed. We just don’t have these things.” (105)
		MBC helps adjust sessions Raises information not otherwise shared
		The data are clinically valuable
		MBC with subgroups of students
Clinical Value: Compatibility with School Mental Health	Appropriateness and attitudes	MBC compatibility with school mental health	“I’m thinking about how we set up 504 meetings and PPT [Planning and Placement Team] meetings, and we as the clinician report out on kind of our observations of student progress, and whether or not they’re making progress or not. I think this kind of brings into light the child's perspective and can also provide the parents perspective on an ongoing basis.” (100)
		Compatibility with IEPs (special education)
		Beginning of school year
		Use with “new” students
Clinical Value: Parent Engagement	Appropriateness and attitudes	Parents enjoy MBC	“[MBC] is a great tool and if you’re hesitant, it means maybe you haven’t had a parent yet who's questioned your goals and objectives and this is a great way for you to provide data for that and nobody wants to be stuck in that position. Cause even if you’re collecting your own data, which you should be, it's just another piece to go along with it and everybody loves data.” (105)
		Data offers transparency about services
		Parent engagement
Clinical Value: Student Engagement	Appropriateness and attitudes	MBC promotes engagement	“When I was running consistently with my students … my elementary school kids loved rating [themselves]. They loved moving that marker back and forth. They loved doing that and they loved rating how I did in the session and if they want to talk about it next time.” (107)“When [MBC] is used, it seems to empower the students.” (107)
		Challenges engaging students in MBC
		MBC empowers students
		Students enjoy MBC
Individual Implementer and Recipient Experiences	N/A—qualitative only	Individual clinician characteristics	“One of my big things is I don’t want to be the stagnant school psychologist … I see this as trying to learn something new every year.” (100)“Once we got students set up, it wasn’t too much at all. And maybe just because I’m younger and more tech savvy, I feel like it was easier.” (104)“Right now I probably have 38 [cases] … we all do other things too, of course, you know, the other programs, the IEP notes, the minutes of the meetings I sit in on.” (105)“It's endless! I have like a thing on my door that says like, you know, if I’m there or in a meeting and it- it makes no difference.” (106)“We had two students pass away and these big crises … For a while we had bomb threats coming in and just like … up until probably the end of January was so crisis driven that I had such a hard time starting cause I couldn’t, like, I felt like I couldn’t catch up, you know?” (106)
		Compatibility with workflow
		Forget to do MBC
		Learning curve for clinicians
		Learning curve for students/families
		Inner setting (school) barriers
FOCUSS Implementation Supports	N/A—qualitative only	Consultation is helpful or liked	“Not only having it, but also having you with it is huge too, because of just utilizing it and knowing [we could] ask questions and getting that feedback of how to use it in the correct way … I think that's huge too. You don’t feel necessarily alone.” (105)“It was very helpful in the beginning for all those trouble shooting things and kind of figuring out how it was going and what's changed and what to stay with. I think the monthly ones at the beginning are good. I mean, maybe an every-other-month thing more towards the half of the year once everybody figures things out.” (100)
		Peer consultation is helpful
		Liked training team supports
		Liked training on/access to measures
		Preference for more/different supports
The Innovation: Measures, Measurement Feedback System, MBC	Feasibility and usability	Feedback system usability	“I saw how easy it was to do. My initial reaction was oh, let me just get a piece of paper because I was having trouble with printing them out, initially. And then once you showed us how to do it, I’m like wait, so much easier.” (107)“I really like how it is tailored towards those who don't have a lot of time yet want to monitor the progress of their behavioral concerns or objectives.” (105)“I had them printed out just in case cause for a little while the computer system, like, for some reason it wasn’t working for me … but then it ended up fixing, so I was just able to do it on the computer. Plus … they [the students] would get to use my iPad and they really liked that.” (104)
	Feasibility and usability	MBC is quick and easy to use
		Procedures/logistics of “collect”
	Acceptability	Liked graphs	“And with the outcome [Outcome Rating Scale], being able to show them the graphing. I think it's a very good system that make easier and better, our interventions.” (111)“The students have enjoyed providing feedback about my performance.” (100)
		Liked having a program to collect data
		Seeing progress overtime
		Clinicians liked the measures
	Adoption^a	Intended future use of MBC	“I love gathering the data. The kids love it. The parents love it, you know, just the phone call and rating their kids, um, you know, I love the program. Hard to implement. I think it fits well, but I definitely get excited about it and want to continue doing it.” (107)
		General positive feedback about MBC/FOCUSS

Note. MBC = measurement-based care; IEPs = Individualized Education Programs; FOCUSS = Feedback and Outcomes for Clinically Useful Student Services.

Adoption was mixed with these qualitative codes refer to the likelihood that clinicians will continue MBC implementation after the pilot.

Clinical Value of MBC in School Mental Health

Across qualitative survey data and interviews, all six clinicians commented on at least one way MBC brought clinical value to school mental health practice. Students reportedly engaged in self-reflection because the measures are “black and white” and more specific than standard “open-ended” questions about functioning or progress. Two clinicians noted the opportunity to expand emotional self-awareness and vocabulary, as well as build “life skills” with MBC. Four clinicians reported that MBC offered consistency and structure in sessions (i.e., “consistency of the check-ins”; “consistency during counseling practice”; “the structure and the data the questions provide to both myself and the students”). We understand “consistency” in this context to refer to a planned, repeated practice that provides a routine for clinicians and all students in their sessions. One clinician noted that MBC helped adjust and “simple tweak” sessions. Another commented that MBC provides “great information and it's something that we haven’t had as social workers, and we definitely needed. We just don’t have these things.”

MBC With Subgroups of Students

School clinicians are generalists with diverse caseloads. Qualitative responses suggest that MBC may not be as meaningful for students on the autism spectrum and/or with lower cognitive functioning and may be most clinically useful for high schoolers or students with internalizing conditions. One clinician intentionally piloted MBC with students of Arabic background, African American, White American, and Hispanic/Latino descent to “test” its cultural relevance among her caseload. She noted that MBC was complicated with her immigrant families (especially undocumented) due to the student and family's legal considerations and concerns around privacy, despite her efforts to clarify that no student identifying information is entered into the feedback system or student record.

Compatibility with School Mental Health

All clinicians have students on their caseload in special education with IEPs. Integration of MBC with the IEP goals and objectives was noted as something that would improve MBC compatibility with school clinician workflows. In some cases, clinicians were able to achieve this integration. For example, report card conferences, 504 Plan meetings and IEP meetings were identified as times to share student MBC data and collect and discuss caregiver-reported data. One clinician noted the value of having “objective” data from “a structured tool to provide information” when she presents student progress in meetings. Four clinicians recommended starting school with an organized MBC process, especially with new students before “your numbers [caseload] increase and all the other sort of things that pop up.” This includes letting caregivers know at the beginning of the school year that MBC will be part of the student intervention and how important their participation is. One clinician noted this proactive approach is challenging in practice because “everything is chaotic in the beginning of the year so nobody's thinking about doing that.”

Caregiver and student engagement

In a handful of cases, caregiver data collection and discussion occurred in person; and “the majority were over the phone.” However, clinicians generally reported challenges with caregiver engagement in MBC. Clinicians expressed that families are sometimes overwhelmed with other life stressors. One clinician emphasized the importance to not “generalize” but “keep in mind that is because the population we work with. It's people that really struggle in many aspects of their lives. I think the intervention and data will be collected very differently if I worked with other kinds of kids, but that's not my job.” One clinician noted the impact of COVID on caregiver relationships during this 2021–2022 school year: “with COVID I feel like it's just been overwhelming, like, we can’t get parents to even do the rating scales for the evaluations we’re completing so it's difficult.” However, one clinician reported that lack of caregiver ratings were due to her own time limitations.

Four clinicians spoke about student engagement and enjoyment with MBC. Students reportedly “like rating themselves and their sessions” including “providing feedback about [the clinician's] performance” on the SRS as well as their own progress on the ORS. Challenges with student engagement pertained to logistics of collecting data routinely, such as clinicians not remembering, as well as figuring out whether the clinician's computer, student's chromebook, or paper and pencil was the most efficient method in the pace of individual and group sessions.

Individual Implementer and Recipient Experiences

The focus code in this category that came up most frequently was individual clinician characteristics, including clinicians’ style, preferences, or attributes that influenced their implementation experience. One clinician talked about her interest in any professional development. Others reported a natural interest in having data as a part of their practice. Years of experience came up as well; more experienced clinicians were more likely to forget to do MBC because they have an established routine. Another clinician newer to the field said, “it kind of gave me a routine that was helpful especially starting out in this field.” Personal comfort with technology was also raised as a determinant. The next most frequent focus code in this category was inner setting (school) barriers such as having limited time due to competing job demands, managing student crises, and staff turnover. Another frequent code that came up for clinicians and students (based on clinician report) was that this practice was a “learning curve,” referring to the initial time and effort to learn MBC; (i.e., “PCOMS is easy to use once you learn how to navigate the system.”) Less-frequently coded topics included compatibility with workflow (“how can I make this flawlessly fit into everything”) and forgetting to do MBC (“there were certainly times where we’d be ten minutes in and I’d say, ‘oh goodness, we forgot to check in’”).

FOCUSS Implementation Supports

Clinician participants’ qualitative responses in surveys and interviews that pertained to implementation supports were coded by five focus codes. The most frequent code was “consultation is helpful or liked,” referring to the monthly group consultation calls that were held. The next most frequent codes in this category were (1) liked training support teams, (2) liked training on/access to measures, and (3) preference for more/different supports. Recommendations ranged from every-other month consultation to weekly, 20-min group sessions because of “crisis moments” and “obstacles” that get in the way of attending consultation. One clinician said that consulting with peers was helpful, and another wished she had another social worker in her building to support implementation.

The Innovation (Measures, Feedback System, and MBC)

Feasibility and usability, specifically in terms of the logistics and procedures of collecting measures, was the most frequent code about the PCOMS measures, the feedback system, or MBC as a specific clinical practice. Some clinicians started with paper and pencil and one clinician never transitioned to using the online system, but those who used the MFS reported that it was easier once they learned it, and students enjoyed it more than doing paper measures. Most parent ratings were collected over the phone. Several clinicians reported measures were acceptable, for example: “I really enjoyed having a way to monitor progress that was consistent and quick.”

Acceptability of MBC and PCOMS measures was coded in qualitative data from approximately half of the clinicians. This included appreciation of “having an actual program to collect data!,” “seeing progress over time,” and liking the graphs produced in the feedback system. A few clinicians also noted liking the PCOMS measures themselves (“the students like rating themselves and their sessions”; “the CORS [Child Outcome Rating Scale] and the fact that it is age appropriate”). The SRS was particularly useful for a clinician to talk with an elementary school student about the purpose of school counseling, “cause you know I have that student who wants us to play games every time.”

Adoption was contextualized in the qualitative data by codes on general feedback about MBC and/or FOCUSS implementation supports, and the intended future use of MBC. Clinicians reported that MBC is “definitely needed” or “necessary,” “fitting” for their students and the school mental health context that “keeps me accountable” and “the kids like it.” Most clinicians reported using the PCOMS measures or “that sort of system” in future school years. Intentions for future use was expected to be facilitated by “already knowing how to navigate it.” At the same time, clinicians reported that due to the numerous practical and logistical challenges to MBC implementation in schools, their intentions for future use would include “starting fresh” to adjust their workflow based on what they learned.

Discussion

The purpose of this pilot was to explore initial MBC acceptability and feasibility among a school-employed mental health clinician sample, based primarily on observed adoption of progress measures, to inform a larger-scale trial. MBC adoption rates by the end of the school year (i.e., 60% of clinicians adopted MBC with five or more students; 90% adopted MBC with at least one student), serve as an initial proof of concept that MBC was acceptable to and feasible for a small sample of school-employed clinicians with FOCUSS implementation supports. Despite lack of prior experience with standardized progress measures, clinician ratings of acceptability, feasibility, appropriateness, and attitudes toward standardized assessment were positive in this sample. Furthermore, qualitative feedback suggests that MBC delivers on clinical value and is compatible with school mental health care delivery. These findings serve as a basis for future, district-wide implementation of FOCUSS to promote spread of MBC among entire student services departments.

Limitations

MBC adoption measures observed in the feedback system only captured partial MBC fidelity which was data collection. This is a common limitation in MBC implementation research that warrants mention as an area of future research. Additionally, these results should be interpreted as a proof of concept among a small sample of willing implementers. Relatedly, we may not have achieved saturation of qualitative themes for all school clinicians in this sample, and these two Connecticut districts may not be representative of districts more broadly, so a future multisite trial is needed to assess variation in school district size, region, and other inner and outer setting characteristics. We also focused on clinician-level implementation and thus did not track or report degree of implementation at the student (case) level. Due to the need for rapid year-end data collection before summer break, and rapid data synthesis to inform implementation in the subsequent year, we did not pilot test the interview guide, conduct repeat interviews, or return transcripts or results to participants for comment. However, a summary of interview findings was shared with district leadership to inform ongoing planning and involvement of clinicians as champions in the upcoming year.

Adoption improvement from 50% post-training to 90% at the end of the school year was a notable improvement, perhaps in part in part due to performance feedback emails. However, the effect of this strategy beyond the multilevel implementation support package was not isolated or empirically tested nor did we have a sample or design to do so. Clinicians were only asked to adopt MBC with a subset of cases but a future direction for understanding full implementation would be to assess adoption and implementation as a percentage of total cases or sessions. Furthermore, implementation mechanisms and outcomes were not assessed for all FOCUSS implementation strategies. Of strategies, “identify and prepare champions” was attempted prior to the training but challenging due to summer planning with districts. We expect that with a district-wide MBC implementation effort, it would be especially important to identify and prepare champions to promote buy-in and peer learning.

Future Directions

Several considerations and adjustments to FOCUSS may be necessary when going to scale as a quality improvement initiative. These include training early in the school year and explaining the initial learning curve of MBC to achieve workflow integration. Additionally, despite promising innovation-context fit in the current study, our findings underscore the importance of tailoring MBC implementation to schools (Lyon & Bruns, 2019). Conveying the value of MBC in a contextually relevant manner and ensuring MBC is integrated within workflows is imperative. Unique considerations around caregiver involvement, using MBC data in school team meetings, and making MBC part of routine practice also surfaced in our results.

Overall, FOCUSS appeared to promote adoption and initial implementation of MBC in this sample. Future research such as a Hybrid Type 1 trial with district-wide FOCUSS could offer opportunities to assess the effectiveness of MBC in schools and understand multilevel implementation determinants, strategies and outcomes with a larger sample. An empirical examination to optimize FOCUSS would also be useful, for example, in a Hybrid Type 2 trial. Sequential, multiple assignment, randomized trial designs could be used in the future to answer questions about relative effectiveness of implementation strategies in a multi-component package as well as optimal sequencing and dosage (Almirall et al., 2014; Lei et al., 2012).

Footnotes

Acknowledgments

We appreciate the dedicated partnership of all school mental health clinicians involved in this work as well as Stamford Public Schools and West Haven Public Schools.

ORCID iDs

Elizabeth H. Connors

Sophia Selino

Nicolina Fusco

Jacob K. Tebes

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was generously supported by the National Institute of Mental Health (NIMH K08MH116119; PI: Connors).

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

Almirall

Nahum-Shani

Sherwood

N. E.

Murphy

S. A.

(2014). Introduction to SMART designs for the development of adaptive interventions: With application to weight loss research. Translational Behavioral Medicine, 4(3), 260–274. https://doi.org/10.1007/s13142-014-0265-0

American Academy of Pediatrics. (2021). AAP-AACAP-CHA declaration of a national emergency in child and adolescent mental health. https://www.aap.org/en/advocacy/child-and-adolescent-healthy-mental-development/aap-aacap-cha-declaration-of-a-national-emergency-in-child-and-adolescent-mental-health/

American School Counselor Association. (2022). ASCA ethical standards for school counselors. https://schoolcounselor.org/getmedia/44f30280-ffe8-4b41-9ad8-f15909c3d164/EthicalStandards.pdf

American School Counselor Association. (2023). ASCA national model. https://schoolcounselor.org/About-School-Counseling/ASCA-National-Model-for-School-Counseling-Programs

Barber

Resnick

S. G.

(2023). Collect, share, act: A transtheoretical clinical model for doing measurement-based care in mental health treatment. Psychological Services, 20(S2), 150–157. https://doi.org/10.1037/ser0000629

Bohnenkamp

J. H.

Glascoe

Gracey

K. A.

Epstein

R. A.

Benningfield

M. M.

(2015). Implementing clinical outcomes assessment in everyday school mental health practice. Child and Adolescent Psychiatric Clinics of North America, 24(2), 399–413. https://doi.org/10.1016/j.chc.2014.11.006

Borntrager

Lyon

A. R.

(2015). Client progress monitoring and feedback in school-based mental health. Cognitive and Behavioral Practice, 22, 74–86. https://doi.org/10.1016/j.cbpra.2014.03.007

Bovendeerd

De Jong

Colijn

De Groot

Hafkenscheid

Moerbeek

De Keijser

(2019). Systematic client feedback to brief therapy in basic mental healthcare: Study protocol for a four-centre clinical trial. BMJ Open, 9(5), Article e025701 https://doi.org/10.1136/bmjopen-2018-025701

Boyd

M. R.

Powell

B. J.

Endicott

Lewis

C. C.

(2018). A method for tracking implementation strategies: An exemplar implementing measurement-based care in community behavioral health clinics. Behavior Therapy, 49(4), 525–537. https://doi.org/10.1016/j.beth.2017.11.012

10.

Brown

Dahlbeck

D. T.

Sparkman-Barnes

(2006). Collaborative relationships: School counselors and non-school mental health professionals working together to improve the mental health needs of students. Professional School Counseling, 9(4), 332–335. https://doi.org/10.1177/2156759X0500900413

11.

CFIR Research Team—Center for Clinical Management Research. (2024). Tools and templates. CFIR Guide. https://cfirguide.org/tools/tools-and-templates/

12.

Charmaz

(2014). Constructing grounded theory. Sage.

13.

Childs

A. W.

Connors

E. H.

(2022). A roadmap for measurement-based care implementation in intensive outpatient treatment settings for children and adolescents. Evidence-Based Practice in Child and Adolescent Mental Health, 7(4), 419–438. https://doi.org/10.1080/23794925.2021.1975518

14.

Cho

Cook

J. R.

Hawley

K. M.

(2023). A structural model of organization and clinician factors associated with standardized measure use in a national survey of youth mental health providers. Administration and Policy in Mental Health and Mental Health Services Research, 50(6), 876–887. https://doi.org/10.1007/s10488-023-01286-7

15.

Connors

E. H.

Arora

Curtis

Stephan

S. H.

(2015). Evidence-based assessment in school mental health. Cognitive and Behavioral Practice, 22, 60–73. https://doi.org/10.1016/j.cbpra.2014.03.008

16.

Connors

E. H.

Hoover

(2025). Advancing measurement-based care in school mental health (Publication No. PEP24-01-030). Substance Abuse and Mental Health Services Administration. https://library.samhsa.gov/sites/default/files/measurement-based-care-schools-pep24-01-030.pdf

17.

Connors

Lawson

Wheatley-Rowe

Hoover

(2021). Exploration, preparation, and implementation of standardized assessment in a multi-agency school behavioral health network. Administration and Policy in Mental Health and Mental Health Services Research, 48(3), 464–481. https://doi.org/10.1007/s10488-020-01082-7

18.

Connors

Lyon

A. R.

Garcia

Sichel

C. E.

Hoover

Weist

Tebes

(2022). Implementation strategies to promote measurement-based care in schools: Evidence from mental health experts across the USA. Implementation Science Communications, 3(1), 1–17. https://doi.org/10.1186/s43058-022-00319-w

19.

Connors

Prout

Vivrette

Padden

Lever

(2021). Trauma-focused cognitive behavioral therapy in 13 urban public schools: Mixed methods results of barriers, facilitators, and implementation outcomes. School Mental Health, 13(4), 772–790. https://doi.org/10.1007/s12310-021-09445-7

20.

Damschroder

L. J.

Reardon

C. M.

Opra Widerquist

M. A.

Lowery

(2022a). Conceptualizing outcomes for use with the Consolidated Framework for Implementation Research (CFIR): The CFIR Outcomes Addendum. Implementation Science, 17(1), 1–10. https://doi.org/10.1186/s13012-021-01181-5

21.

Damschroder

Reardon

C. M.

Opra Widerquist

M. A.

Lowery

J. C.

(2022b). The updated consolidated framework for implementation research based on user feedback. Implementation Science, 17(1), Article 75. https://doi.org/10.1186/s13012-022-01245-0

22.

Duncan

B. L.

(2012). The Partners for Change Outcome Management System (PCOMS): The heart and soul of change project. Canadian Psychology/Psychologie Canadienne, 53(2), 93–104. https://doi.org/10.1037/a0027762

23.

Duong

M. T.

Bruns

E. J.

Lee

Cox

Coifman

Mayworm

Lyon

A. R.

(2021). Rates of mental health service utilization by children and adolescents in schools and other common service settings: A systematic review and meta-analysis. Administration and Policy in Mental Health and Mental Health Services Research, 48(3), 420–439. https://doi.org/10.1007/s10488-020-01080-9

24.

Elo

Kyngäs

(2008). The qualitative content analysis process. Journal of Advanced Nursing, 62(1), 107–115. https://doi.org/10.1111/j.1365-2648.2007.04569.x

25.

Hoover

Lever

Sachdev

Bravo

Schlitt

Acosta Price

Sheriff

Cashman

(2019). Advancing comprehensive school mental health: Guidance from the field. National Center for School Mental Health.

26.

Hsieh

H.-F.

Shannon

S. E.

(2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687

27.

Interdepartmental Serious Mental Illness Coordinating Committee, D. E. W. G. (2023). Use of measurement-based care for behavioral health care in community settings—A brief report. https://www.samhsa.gov/sites/default/files/ismicc-measurement-based-care-report.pdf

28.

Jensen-Doss

Haimes

E. M. B.

Smith

A. M.

Lyon

A. R.

Lewis

C. C.

Stanick

C. F.

Hawley

K. M.

(2018). Monitoring treatment progress and providing feedback is viewed favorably but rarely used in practice. Administration and Policy in Mental Health and Mental Health Services Research, 45(1), 48–61. https://doi.org/10.1007/s10488-016-0763-0

29.

Kaiser Permanente Institute for Health Policy. (2023). Feedback informed care: Collaborating with patients to improve mental health care services. https://www.kpihp.org/wp-content/uploads/2023/05/0486_feedback_informed_care_051723_ADA.pdf

30.

Kassab

Scott

Boyd

M. R.

Puspitasari

Endicott

Lewis

C. C.

(2022). Tailored isn’t always better: Impact of standardized versus tailored training on intention to use measurement-based care. Implementation Research and Practice, 3, Article 26334895221087477. https://doi.org/10.1177/26334895221087477

31.

Kelly

M. S.

(2011). Data-driven decision making in school-based mental health: (How) Is it possible? Advances in School Mental Health Promotion, 4(4), 2–4. https://doi.org/10.1080/1754730X.2011.9715637

32.

Langley

A. K.

Nadeem

Kataoka

S. H.

Stein

B. D.

Jaycox

L. H.

(2010). Evidence-based mental health programs in schools: Barriers and facilitators of successful implementation. School Mental Health, 2(3), 105–113. https://doi.org/10.1007/s12310-010-9038-1

33.

Lei

Nahum-Shani

Lynch

Oslin

Murphy

S. A.

(2012). A “SMART” design for building individualized treatment sequences. Annual Review of Clinical Psychology, 8, 21–48. https://doi.org/10.1146/annurev-clinpsy-032511-143152

34.

Lewis

C. C.

Boyd

Puspitasari

Navarro

Howard

Kassab

Hoffman

Scott

Lyon

Douglas

Simon

Kroenke

(2019). Implementing measurement-based care in behavioral health: A review. JAMA Psychiatry, 76(3), 324–335. https://doi.org/10.1001/jamapsychiatry.2018.3329

35.

Lewis

C. C.

Marti

C. N.

Scott

Walker

M. R.

Boyd

Puspitasari

Mendel

Kroenke

(2022). Standardized versus tailored implementation of measurement-based care for depression in community mental health clinics. Psychiatric Services, 73(10), 1094–1101. https://doi.org/10.1176/appi.ps.202100284

36.

Lindgren

B.-M.

Lundman

Graneheim

U. H.

(2020). Abstraction and interpretation during the qualitative content analysis process. International Journal of Nursing Studies, 108, Article 103632. https://doi.org/10.1016/j.ijnurstu.2020.103632

37.

Lyon

A. R.

Bruns

E. J.

(2019). From evidence to impact: Joining our best school mental health practices with our best implementation strategies. School Mental Health, 11, 106–114. https://doi.org/10.1007/s12310-018-09306-w

38.

Lyon

A. R.

Liu

F. F.

Connors

E. H.

King

K. M.

Coifman

J. I.

Cook

McRee

Ludwig

Law

Dorsey

McCauley

(2022). How low can you go? Examining the effects of brief online training and post-training consultation dose on implementation mechanisms and outcomes for measurement-based care. Implementation Science Communications, 3(1), Article 79. https://doi.org/10.1186/s43058-022-00325-y

39.

Lyon

A. R.

Pullmann

M. D.

Dorsey

Martin

Grigore

A. A.

Becker

E. M.

Jensen-Doss

(2019). Reliability, validity, and factor structure of the Current Assessment Practice Evaluation-Revised (CAPER) in a national sample. The Journal of Behavioral Health Services & Research, 46, 43–63. https://doi.org/10.1007/s11414-018-9621-z

40.

Lyon

A. R.

Pullmann

M. D.

Whitaker

Ludwig

Wasse

J. K.

McCauley

(2017). A digital feedback system to support implementation of measurement-based care by school-based mental health clinicians. Journal of Clinical Child & Adolescent Psychology, 48(sup1), S168–S179. https://doi.org/10.1080/15374416.2017.1280808

41.

Miller

F. G.

Chafouleas

S. M.

Riley-Tillman

T. C.

Fabiano

G. A.

(2014). Teacher perceptions of the usability of school-based behavior assessments. Behavioral Disorders, 39(4), 201–210. https://doi.org/10.1177/019874291303900405

42.

National Association of School Psychologists. (2020). NASP 2020 professional standards adopted. https://www.nasponline.org/standards-and-certification/nasp-2020-professional-standards-adopted

43.

National Association of Social Workers. (2012). NASW standards for school social work services. School Social Work Association of America. https://www.sswaa.org/_files/ugd/426a18_a9bc671ba3d54d379163d7e9a541b82b.pdf

44.

Palinkas

L. A.

Aarons

G. A.

Horwitz

Chamberlain

Hurlburt

Landsverk

(2011). Mixed method designs in implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 44–53. https://doi.org/10.1007/s10488-010-0314-z

45.

Parikh

Fristad

M. A.

Axelson

Krishna

(2020). Evidence base for measurement-based care in child and adolescent psychiatry. Child and Adolescent Psychiatric Clinics of North America, 29(4), 587–599. https://doi.org/10.1016/j.chc.2020.06.001

46.

Parke

C. S.

(2012). Making use of district and school data. Practical Assessment, Research, and Evaluation, 17(1), Article 10. https://doi.org/10.7275/pqg3-3294

47.

Proctor

Silmere

Raghavan

Hovmand

Aarons

Bunger

Griffey

Hensley

(2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76. https://doi.org/10.1007/s10488-010-0319-7

48.

Reaves

Bohnenkamp

Mayworm

Sullivan

Connors

Lever

Kelly

M. S.

Bruns

E. J.

Hoover

(2022). Associations between school mental health team membership and impact on service provision. School Mental Health, 14, 672–684. https://doi.org/10.1007/s12310-021-09493-z

49.

Scott

Lewis

C. C.

(2015). Using measurement-based care to enhance any treatment. Cognitive and Behavioral Practice, 22, 49–59. https://doi.org/10.1016/j.cbpra.2014.01.010

50.

Sichel

C. E.

Connors

E. H.

(2022). Measurement feedback system implementation in public youth mental health treatment services: A mixed methods analysis. Implementation Science Communications, 3(1), 1–16. https://doi.org/10.1186/s43058-022-00356-5

51.

Weiner

B. J.

Lewis

C. C.

Stanick

Powell

B. J.

Dorsey

C. N.

Clary

A. S.

Boynton

M. H.

Halko

(2017). Psychometric assessment of three newly developed implementation outcome measures. Implementation Science, 12, 1–12. https://doi.org/10.1186/s13012-017-0635-3

52.

Wilk

A. S.

J.-C.

Wen

Cummings

J. R.

(2022). Recent trends in school-based mental health services among low-income and racial and ethnic minority adolescents. JAMA Pediatrics, 176(8), 813–815. https://doi.org/10.1001/jamapediatrics.2022.1020

53.

Zabek

Lyons

M. D.

Alwani

Taylor

J. V.

Brown-Meredith

Cruz

M. A.

Southall

V. H.

(2022). Roles and functions of school mental health professionals within comprehensive school mental health systems. School Mental Health, 15, 1–18. https://doi.org/10.1007/s12310-022-09535-0