Building a multicultural peer-consultation team: Planning,implementing,and early sustainment evaluation

Abstract

This article represents an implementation-focused evaluation of a multicultural peer-consultation team situated within a psychiatry department in a large academic medical center in the Southern United States. The evaluation comprised anonymous self-report questionnaires (n = 14) as well as individual (n = 3) or group interviews (n = 10) conducted by outside independent evaluators. Participants were current and former team members (i.e., graduate trainees, mental health care providers, clinical and research staff members) who voluntarily participated in this multimethod implementation evaluation. Results indicated that attendance on the team had several important impacts on members, and most notably an increased ability to provide multiculturally competent care, that is treatment that carefully and routinely considers the influence of culture and context on patients and therefore their clinical presentation. Further, no negative impacts from participating on the team were noted. A primary strength of the team's sustainability is that participation on the team was deemed to be relevant and useful by current and former team members. A major barrier to participation on the team is competing demands, such as high clinical loads. We conclude that this model for multicultural peer-consultation holds promise as an effective and implementable educational method for mental health care professionals. We discuss strengths, limitations, and future directions for research.

Keywords

implementation science multicultural education peer consultation psychotherapy

The need for multiculturally competent mental health care has been formally and repeatedly recognized (American Psychological Association, 2003, 2017; National Association of Social Workers, 2015; U.S. Department of Human Services, 2001). Despite the expectation for multiculturally competent care (that carefully and routinely considers the influence of culture and context on patients and their clinical presentation) and various mandates regarding training and implementation (e.g., National Association of Social Workers, 2015) and improvements in training thus far, robust and comprehensive methods are still needed to effectively train clinicians to attend to relevant cultural (e.g., idioms of distress) and contextual factors (e.g., overlapping systems of oppression; Benuto et al., 2019). Additionally, rigorous evaluation of innovative training models is essential to facilitate widespread dissemination of effective training strategies (Bhui et al., 2007; Peters et al., 2011). In this article, we describe efforts to sustain a model of multicultural peer-consultation, which has had the goal of helping team members provide mental health care that is multiculturally competent. To this end, we report on results from an evaluation of this team and propose guidelines for future teams seeking to implement this form of training across distinct settings.

Defining multiculturally competent care

Culture

Building on Eagleton’s (2000) conceptualization of culture, Kirmayer et al. (2014) note that culture has been defined in three ways: (a) as the place where human biology and experience originate, (b) as the ways in which groups of people with shared histories or identities are differentiated from others, and (c) as the expression of collective creative capacities (e.g., music, art, other media). To this end, the expression of psychiatric distress is influenced by culture, for example in that the ways we live and engage with others can have implications for our wellbeing and psychiatric distress, that there are patterns in the ways we communicate distress (i.e., idioms of distress that can vary among groups), and that there are certain expressions of psychiatric distress that carry social stigma differentially across groups of people.

Intersectionality of identities and systems of oppression

The ways in which people are categorized (whether this is by race, ethnicity, country of origin, religious affiliation, sex, sexual orientation, or gender identity, among others) have implications for their health, taking into account structural systems that systematically advantage certain groups while disadvantaging others, which represent social drivers of health that can give rise to health disparities. Thus, it is not only important to understand cultural dynamics in terms of the identities patients hold, but also to situate their identities, experiences, and mental health problems in the social, historical, and political contexts in which they reside. A great deal of work in this area has focused on recognizing cultural diversity in patients along a range of dimensions such as age and generational influences, developmental disabilities, disabilities acquired later in life, religion and spiritual orientation, ethnicity and racial identity, socioeconomic status, sexual orientation, national origin, and gender among others (Hays, 1996). As of late, there has also been recognition of the intersection of multiple dimensions of privilege and oppression shaped by current and historical systems, such as sexism, racism, xenophobia, classism, and LGBTQ intolerance, to name a few (Buchanan & Wiklund, 2020; Rosenthal, 2016).

Multicultural competence

There has been much debate about how to define multicultural competence within the field of mental health (e.g., Alizadeh & Chavan, 2016; Huey et al., 2014). Some define it as the awareness, knowledge, and skills within the practitioner that pertain to providing care to patients who are culturally diverse (Sue et al., 2019). Others have argued for process-oriented definitions that focus on what a clinician does within the clinical encounter (e.g., Lopez et al., 2020). Alternative explanations have focused on incorporating salient cultural content into treatment—these have come in the form of cultural adaptations to evidence-based treatments (e.g., Domenech Rodríguez & Bernal, 2012) and culturally commensurate therapies (Wendt & Gone, 2012). Others have focused on systems-level factors (e.g., community context, cultural characteristics of local populations, organizational infrastructure, and direct service support) and define cultural competence by the degree of compatibility among these factors (Hernandez et al., 2009).

We define multiculturally competent care as treatment that carefully and routinely considers the influence of culture and context on patients and therefore their clinical presentation. Stated another way, multiculturally competent providers appreciate that the clinical presentation is interwoven with culture and context (Kirmayer, 2006) rather than separate from it. Multiculturally competent providers must therefore explicitly and deliberately centralize cultural and contextual influences impacting their patients in the ways they conduct assessment, case formulation, treatment planning, and the treatment itself.

Impact of multiculturally competent care on patient outcomes

Quantifying the impact of multicultural competence on clinical care is hard to discern, in part due to diverse operationalizations and methodologies employed to train and evaluate its presence. Nonetheless, there are some important signals that multiculturally competent care represents an important endeavor and can lead to superior patient outcomes. The meta-analytic evidence suggests that attending to cultural and contextual factors in mental health care in the form of cultural adaptations to evidence-based treatments can lead to better therapeutic outcomes (Tao et al., 2015) and that multicultural competence is related to important clinical processes and outcomes (Soto et al., 2018). The benefits of multiculturally competent care are evidenced by stronger therapeutic alliances and greater patient psychological wellbeing (Tao et al., 2015), lower attrition rates (Lie et al., 2011), increased patient satisfaction (Govere & Govere, 2016), and better treatment adherence (Schilder et al., 2001). Taken together, this growing body of research suggests there are benefits to providing multiculturally competent care.

Routes to increasing capacity to deliver multiculturally competent care

Although there is broad recognition that providers need to deliver multiculturally competent care, there is great variability in the methods that are utilized to train mental health care providers in this area, with no reigning gold standard training method that is systematically and rigorously disseminated across training programs (Benuto et al., 2018). Training methods range from lecture, discussion, utilization of case scenarios, cultural immersion, role play, contact with diverse individuals, self-reflection of interactions with patients, journaling, and service learning that have resided in coursework integrated into graduate and medical education or continuing education credits (Benuto et al., 2018). However, this type of multicultural education has been criticized for relegating an emphasis on culture to a specified course rather than infused into all aspects of clinical training (Collins & Pieterse, 2007; Gregus et al., 2020; Neblett, 2019); for being time-limited and costly (Herschell et al., 2010); for being well-intentioned yet over-generalizing, simplistic, and impractical (Shepherd, 2019); and for changing knowledge but not attitudes, awareness, or skills (Benuto et al., 2018). In addition to coursework, clinical education in this area has resided within multicultural supervision whereby supervisors encourage reflection on cultural assumptions and facilitating a space where supervisees can be vulnerable (Ancis & Marshall, 2010), and can help supervisees implement and hone multicultural competence skills (Martinez & Holloway, 1997), which in turn can contribute to supervisees’ cultural responsiveness with patients (Leong & Wagner, 1994; Burkard et al., 2006) and strengthen the supervisory relationship (Ancis & Marshall, 2010).

Multicultural peer-consultation

An innovative training method to those previously described is that of cultural consultation. As noted by Kirmayer et al. (2003), cultural consultation aims to improve the ability of clinicians to meet the needs of their patients, especially those working with culturally diverse populations including immigrants, refugees, and ethnocultural minoritized groups, which can in turn lead to more comprehensive assessment, more effective treatment, and better clinical outcomes. Three general approaches to cultural consultation include (a) a consultation-liaison center staffed by experts in cultural psychiatry from whom other medical providers can request an evaluation of an identified patient; (b) clinics that deliver care to a particular cultural group (e.g., immigrants and refugees); and (c) multicultural peer-consultation whereby a group of peers meet on a regular basis to consult on the cultural and contextual aspects of ongoing cases.

To our knowledge, little has been written about multicultural consultation models whereby a consistent team of mental health care providers meet regularly to consult on ongoing cases. One well-known example of such a team, outside multicultural consultation, is included in comprehensive Dialectical Behavior Therapy (DBT; Sayrs, 2018; Swales, 2010). In DBT, the function of this team is to focus on improving and maintaining the motivation and capability of providers by serving as “therapy for therapists” (Sayrs, 2018). Essential components include creating a recurring structure and team agreements, setting an agenda, and having specified roles (e.g., team leader, observer; Sayrs, 2018).

The present study evaluates an existing multicultural peer-consultation team at a large academic medical center located in the Southern United States (see Nagy et al., 2019). This interdisciplinary multicultural peer-consultation team is composed of mental health care providers and staff from multiple disciplines (e.g., clinical psychology, social work, psychiatry) and various levels of training (e.g., trainees, licensed clinicians). This team meets weekly and rotates between meetings focused on peer-consultation (i.e., members of the team can request from and offer clinical input to others on the team), didactics (i.e., members of the team and outside experts lead discussions focused on multicultural topics), and a diversity action committee (i.e., a platform for the team to target organizational-level factors to make the clinic more multiculturally competent). This type of consultation team affords members a space to grow and learn, seek support, and further develop case conceptualization (a collaborative process that providers and patients engage in to describe and explain presenting problems) and treatment planning. Team members provide validation, resources, alternative perspectives, and problem-solving, as needed. Consultation requests can focus on assessing a specified clinical problem, formulating the solution, applying solutions or treatment plan, generating compassion, and validation.

Consultation recommendations are made in line with notions of culture that emphasize context and lived experience (e.g., Santos et al., 2021) versus solely relying on group-level knowledge, which has the potential to perpetuate stereotypes about distinct groups. Team members are also encouraged to highlight intersectionality of distinct and overlapping systems of oppression (e.g., sexism, racism, xenophobia, LGBTQ + intolerance) and the impact they can have on mental health. In addition, consultation leverages team members’ lived experiences which occupy a range of privileged and marginalized identities (pertaining to race, ethnicity, immigrant status, sex, gender identity, sexual orientation, sexual orientation, etc.), discipline (e.g., clinical psychology, social work, psychiatry), and their clinical training/theoretical orientation (e.g., cognitive-behavioral therapy, psychodynamic therapy).

Despite preliminary evidence indicating this model can be effective for improving the patient–provider relationship, as well as providers’ perceptions of patient engagement and adherence in treatment (Nagy et al., 2019), this type of consultation team is not widely available at academic training institutions. One reason might be the limited spread of knowledge about such consultation teams and how to launch one. Parallel to how training alone without more support does not lead to provider change (Valenstein-Mah et al., 2020), providing information on the consultation team alone is unlikely to lead to institutions starting one. Therefore, we need to describe clearly what our barriers and strengths were in launching and maintaining this consultation team for four years, over time (Aarons et al., 2011), so that there is more guidance for these practices in multicultural training.

Using implementation science to evaluate the multicultural peer-consultation team

In this study, we used implementation science principles to evaluate the initial implementation and early sustainment of our multicultural peer-consultation team. Implementation science is the study of what, how, and why an intervention, such as our multicultural peer-consultation team, is implemented and sustained in practice successfully (Bauer et al., 2015). Implementation science offers a framework for evaluating several stages of the implementation of a team, from planning to maintaining the team years later (Bauer et al., 2015; Stetler et al., 2006). This provided a framework to evaluate the early sustainment and training effort of the multicultural consultation team that was originally implemented in 2016, to ensure its continued utility and sustainment. In order for multicultural consultation teams to be effective over time, they need to be implemented in such a way that they retain core components and have features that are needed for the team to be sustained. Thus, information from an implementation evaluation of team members’ experiences informs wide-scale dissemination and implementation of the team across diverse settings (e.g., community clinics, the Veterans Health Administration), which is a goal of this study.

This evaluation was conducted keeping in mind its utility for future implementation and sustainment of multicultural consultation teams by other institutions. We assessed demographics of the multicultural peer-consultation team members, perceived impacts on clinical or research practice, the degree of member engagement in the team, and barriers to and strengths of early sustainment of the team over the past four years since its inception. From this, we list lessons learned as we overcame barriers and harnessed strengths in implementing and maintaining the team.

Methods

Design

At the time of data collection, the multicultural team had been implemented for four years. Two independent evaluators outside the team retrospectively documented the implementation process. The evaluators studied the time period in which original team leaders were planning to launch the team (pre-implementation phase), the first year of the team (implementation phase), through the fourth year of the team (early sustainment phase).

We used a quantitative → QUALITATIVE study design, collecting data in two phases for greater richness of data (Palinkas et al., 2011). To increase the reliability and validity of results, we interviewed previous team members, clarified evaluator bias during analysis, conducted a feedback session with team leadership (see Analysis section), and compared survey and interview data (Morse, 2015). We report on qualitative research according to reporting standards (Tong et al., 2007).

To design the research questions for this study, we integrated two frameworks applicable to the study aims—the Reach, Effectiveness, Adoption, Implementation Maintenance (RE-AIM) framework (Glasgow et al., 1999) and the Health Equity Implementation Framework (Woodward et al., 2019). The RE-AIM framework is an evaluation framework used to assess the impact of a newly implemented program (Glasgow et al., 2019). The five domains of RE-AIM are: Reach (i.e., description of members who participate), perceived Effectiveness on patient cases (i.e., degree of positive and negative consequences of participating on the team), Adoption of suggestions from consultation (i.e., the extent to which team members apply recommendations), and Implementation “fidelity” of the multicultural peer-consultation team (i.e., the extent to which the team was carried out as intended). The final domain of RE-AIM is Maintenance and we do not report on it in this manuscript. We conceptualized maintenance as not having occurred yet for the multicultural peer-consultation team as we have thus far focused on developing and refining this model. See Table 1 for a depiction of each RE-AIM domain and operational definitions in this study.

Table 1.

Four RE-AIM Framework Domains Aligned with Operational Definitions and Data Collection Methods in an Evaluation of a Multicultural Peer-Consultation Team.

RE-AIM Domain	Theoretical and Operational Definitions in this Evaluation	Survey Questions	Interview Questions
Reach	The absolute number, proportion, and representativeness of individuals who are affected or reached by a given initiative, namely the people participating in the multicultural peer-consultation team	1. Are you still attending the multicultural peer-consultation team through the Department of Psychiatry at Duke University? 2. How did you learn about this team?	1. What is your training background and your role in the institution? 2. What has your involvement been in the multicultural peer-consultation team?
Effectiveness	The impact of an intervention on important outcomes; team members’ perceptions of potential positive/negative effects, quality of life, or economic outcomes	---	3. Tell us about ways in which your patient or client cases have changed as a result of the multicultural consultation you received here a. Positive, negative, neutral? b. Any examples?
Adoption	The absolute number, proportion, and representativeness of agents who are willing to initiate a program. Extent to which team members sought consultation in the team. Information related to how people adopted it, why they adopted it, and variations of adoption (e.g., only attending didactics)	3. What made you decide to join the multicultural peer-consultation team? 4. To what degree do you feel comfortable seeking consultation from the multicultural peer-consultation team? Probe: What made you feel more/less comfortable?	4. What makes you more or less comfortable sharing or asking questions with the multicultural peer-consultation team? Probes: training level, personality, etc.
Implementation (fidelity)	At the setting level, implementation refers to the intervention agents’ fidelity to the various elements of an intervention's protocol. This includes consistency of delivery as intended; adaptations made to the multicultural peer-consultation team; level of adherence or drift to the intended components of the team; barriers to and strengths of implementing the team	5. How often are you able to attend the multicultural peer-consultation team? 6. Barriers/strengths (aligned to domains from the Health Equity Implementation Framework) a. What are the factors that facilitate you regularly attending, participating, or using information from the multicultural peer-consultation team? b. What are the factors that get in the way of you regularly attending, participating, or using information from the multicultural peer-consultation team?	7. To what degree do you utilize the suggestions or knowledge learned on the consultation team in your clinical work? Example—if you are newer to considering these issues, you might use them less in your practice. Probes: Do you incorporate all of the feedback into your practice? Why / why not? 8. Barriers/strengths focused questions from survey responses a. Anything about the patient population you have on your caseload? b. How do you feel the department affects your ability to participate or implement changes? c. Is there anything about the physical structure that affects you all—getting to the building, travel, enough space?

Additionally, the Health Equity Implementation Framework (Woodward et al., 2019) was used to design some data collection questions and inform qualitative analysis. The Health Equity Implementation Framework considers which factors, often at multiple levels in health care systems, may influence outcomes of implementing a program, and can help researchers identify barriers and strengths specific to areas where there are health care disparities (particularly relevant to multicultural training). The Health Equity Implementation Framework also assesses facilitators (referred to herein as strengths), which are specific strategies used by people in health care systems to successfully implement new programs, like the multicultural peer-consultation team (Harvey & Kitson, 2015). Thus, we asked questions about facilitation of developing the team, initial barriers, overcoming those barriers, and capitalizing on strengths in the system to encourage the team to continue operating.

The domains of the Health Equity Implementation Framework (Woodward et al., 2019) included in this evaluation are: the innovation (e.g., what is being implemented—in this case, the multicultural consultation team), culturally relevant factors (e.g., beliefs and biases of the patients, providers, and other staff), the clinical interaction (e.g., patient–provider exchange), the context (including the clinic, department, and health care system), and the societal context (including economies, physical structures or the built environment, and sociopolitical forces).

Participants

The study received a Declaration of Exemption from the Duke Health IRB under Category 2 of the Revised Common Rule. Therefore, participants were not required to provide informed consent. All current and former team members in the multicultural peer-consultation team received a description of the study prior to data collection. Participants were recruited via in-person team meetings and email correspondence from a team leader and evaluators. All participants were involved in mental health service, research, teaching, or mentorship. Current team members worked in a psychiatry department at an academic medical center and former team members worked in diverse settings (e.g., the Veterans Health Administration, private practice). Seventeen former members of the team were sent a recruitment email, four responded with potential interest in participation, and two completed interviews.

Procedures

In the first phase, participants received a link to an anonymous online survey via Qualtrics that assessed barriers and strengths to adoption and early sustainment of the multicultural peer-consultation team. The evaluators used survey data to refine interview questions.

In the second phase, evaluators conducted two virtual video qualitative focus groups. Evaluators conducted telephone qualitative individual interviews with team leadership and team members who no longer attended the team. During group and individual interviews, evaluators took detailed notes using the interview guide. Individual interviews were not audio recorded. Focus groups were audio recorded and transcribed verbatim.

Evaluators were a faculty member with advanced knowledge in qualitative methodology and a postdoctoral research fellow with novice knowledge in qualitative methodology from an outside institution. Both evaluators were clinical psychologists trained in implementation science and neither had been a member of the multicultural peer-consultation team. They both identified as women. One evaluator identified as Asian and the other White. Evaluators did not have prior relationships with study participants, with the exception of one team leader, and conveyed to study participants their training background with the purpose of evaluating the implementation impact, challenges, and strengths of the multicultural peer-consultation team.

Measures

We collected mixed-methods data through the use of an anonymous survey and individual and group qualitative interviews. Questions on the semi-structured interview guide were open-ended and evaluators queried with follow-up questions as necessary to gain clarity of responses from team members. Data comprised from these tools mapped onto the RE-AIM (Glasgow et al., 1999) domains of Reach, Effectiveness, Adoption, and Implementation as well as the Implementation domain of the Health Equity Implementation Framework (i.e., barriers to and strengths of implementing the team such as team members’ own views, department-level concerns, or support; Kessler et al., 2013; Woodward et al., 2019). See Table 1 for a full list of anonymous survey questions and interview prompts.

Which Members Participated? (Reach) . Team members self-reported the following demographic characteristics: age, race, sex, ethnicity, years in practice, and professional discipline (see Table 1).

How Did Participation on the Team Impact Clinical Practice? (Effectiveness) . Team members were asked in individual and group interviews to comment on the impact on clinical practice (i.e., “Tell us about ways in which your patient or client cases have changed as a result of the multicultural consultation you received here”) and probes were used to explore sub-categories mentioned by team members.

How Often Did Members Attend Consultation? (Adoption) . Attendance of enrolled team members was taken at all meetings from August 2019 to March 2020 (see Table 1).

What Factors Made It Easier or Harder to Engage in the Team? (Adoption) . In the qualitative interview, team members were asked “What makes you more or less comfortable sharing or asking questions with the multicultural peer-consultation team?” to identify reasons for adoption or lack thereof.

To What Extent Was the Multicultural Peer-Consultation Team Formed, Implemented, and Sustained as Intended? (Implementation) . In the anonymous survey, we included questions focused on barriers to and strengths of implementing the multicultural peer-consultation team. In the qualitative interviews, we included one question on perceived fidelity to the team, “To what degree do you utilize the suggestions or knowledge learned on the consultation team in your clinical work? Why/why not?”

Analysis

The two evaluators analyzed data using template analysis, a qualitative data reduction technique which involves developing a template used to summarize categories relevant to a topic of interest mentioned by participants (Hamilton, 2013). The template is used to organize the data in a meaningful and useful manner (Hamilton, 2013), and there is less interpretation from evaluators than in coding themes typical in other qualitative traditions. The purpose of template analysis is to gather content and create categories based on participants’ responses (Hamilton, 2013). Because the focus of template analysis is to gather content, this analysis technique does not focus on thematic interpretation or constructing theories while gathering data.

Given that the purpose of the project was to document the utility for future implementation of multicultural consultation teams by other institutions and to identify barriers and strengths, template analysis was a practical analysis to use. The research questions were expected to elicit information that was conducive to organization into categories (e.g., “What were the barriers and strengths to launching the team?”). Template analysis is also well-suited to applied research in this study because it is less time-consuming than other qualitative methods so quicker feedback could be given to the leaders of the multicultural peer-consultation team that allowed them to make adjustments as needed. Template analysis has been used for rapid qualitative analysis by several health services researchers (e.g., Abraham et al., 2021; Ecker et al., 2021). Evaluators used Microsoft Word to create and populate templates. They reviewed all qualitative data, and organized data from individual interviews and focus groups first, later merging any new data from surveys into one summary template of findings.

The evaluators created template domains a priori data analysis using domains from the RE-AIM framework (Reach, Effectiveness, Adoption, Implementation) and Health Equity Implementation Framework (embedded in Implementation). Evaluators reviewed data from the first focus group and refined their template. Evaluators added categories to the template as new categories were generated throughout focus groups, interviews, and surveys. The evaluators analyzed interviews independently then met to discuss any discrepancies. Discrepancies were resolved with discussion until consensus. By the final individual interview, evaluators agreed that saturation was occurring as participants were reporting similar categories from data analyzed earlier. Evaluators completed the summary template (the aggregate document containing all content that appeared in data collection) in tandem. To clarify, deepen, and broaden some findings, evaluators presented results virtually in vivo during interviews and to leadership of the multicultural peer-consultation team (Birt et al., 2016).

Results

Categories and subcategories of findings were organized by Reach, Effectiveness, Adoption, and Implementation. Within the domain of Implementation, barriers to and strengths of implementing the team were aligned with the Health Equity Implementation Framework. These findings provided lessons learned that may be helpful for other teams that aim to use a similar model as the one described herein.

Demographics of members on the multicultural peer-consultation team (reach)

Thirteen individuals participated in the focus groups and individual interviews, with 84.6% (n = 11) identifying as current members and 15.4% (n = 2) as former members. Participants were asked to respond to fill-in-the-blank questions related to their demographics. Of 13 individuals who completed focus groups and individual interviews, 11 participants identified as women (92.3%). When asked about race, 10 participants self-identified as White (76.9.2%), one identified as Black (7.7%), one identified as Latinx (7.7%), and one identified as N/A (7.7%). Regarding ethnicity, five participants self-identified as Non-Latinx/Hispanic (38.5%), two as Latinx (15.4%), one as Salvadoran and Mexican (7.7%), one as White, Jewish (7.7%), one as Caucasian (7.7%), one as American (7.7%), one as of Nigerian descent (7.7%), and one as Asian (7.7%). Despite our best efforts to capture independent dimensions of race and ethnicity, some respondents may have conflated them.¹ Average number of years in the mental health field was 5.85 (SD = 2.70; Range: 2–11). Table 2 provides a summary of their roles, disciplines, and their participation in the multicultural peer-consultation team.

Table 2.

Team Members Reached by the Multicultural Peer-Consultation Team.

	N
MC Team participants interviewed	13
Current member	11
Former member	2
Role^a
Staff or faculty	5
Trainee	8
Discipline
Undergraduate psychology	2
Graduate clinical psychology	9
Social work	2
Involvement in MC team consultation
Reported giving consultation	13
Reported receiving consultation	13
Involvement in MC team didactics
Reported attending a didactic	13
Reported giving a didactic	4

Notes. ^a Because some participants were no longer attending and may have switched roles since attending (e.g., graduated training, now faculty), each role was documented as the person's role during their most recent engagement in the multicultural peer-consultation team. For example, if they engaged as a trainee and no longer attend as a faculty member, they were categorized as a trainee.

Perceived impacts on clinical or research practice (effectiveness)

Participants mentioned mostly positive impacts of the multicultural peer-consultation team, even among former members. Evaluators asked explicitly about negative impacts on members’ work and all denied any negative impacts. There were four categories related to positive impacts on members’ work (e.g., clinical, research). First, participants reported that the team positively and directly impacted their work, as one participant in Focus Group 2 described: “I found consultation to be very helpful in both knowing what to say, but also in treatment planning, conceptualization, and building rapport.”² Second, participants reported receiving concrete information to improve multicultural approaches to their work; examples included, but were not limited to, specific clinical skills, information about different group identities, and how to work with a linguistic interpreter. Third, participants reported that the team expanded their knowledge of evidence-based practice beyond typical therapeutic approaches. A participant from Focus Group 1 described this well:

I am being exposed to more evidence-based practices than I have been exposed to outside of traditional models … that's been very insightful and helpful in growing my foundational knowledge. I think the [team has] exposed me to many approaches that I would not know otherwise. I think [the leader] in particular does a very good job when there are different identity factors and pieces.

One participant from Focus Group 2 who collected data for research studies (primarily diagnostic) also noted its impact, “I do think it's affected the clinical interviews and even though they’re structured it helps me know and think about perspectives and consider follow-up questions even for a really structured interview.”

The fourth category of effectiveness was that the team helped team members consider the systemic impact of marginalization on mental health conceptualization and treatment course. A participant from Focus Group 2 mentioned it was helpful by:

… bringing up case concerns about ways in which some of our systemic practices are adversely affecting some of our clients. Specifically, our African American clients or clients from specific cultures … I also talked about socioeconomic stress-related hurdles to accessing care.

Member attendance at team meetings (adoption)

We tracked attendance of enrolled team members³ (n = 14) from August 2019 to March 2020 for a total for 23 team meetings. Average attendance was nine out of 14 total members (64.3%) per session. There were no team meetings when all 14 members attended simultaneously. Some members attended infrequently (i.e., the lowest attendance member attended a total of three of 23 team meetings), while others frequently (i.e., the highest attendance member attended a total of 21 of 23 team meetings).

Member engagement in multicultural peer-consultation team (adoption)

Every team member who participated in the evaluation reported seeking and giving consultation during team meetings, although qualitative findings suggested two categories of factors that made this more difficult. First, some team members needed a warm-up period before asking or sharing in the team such that they were slower to adopt consultation. One participant in Focus Group 2 spoke to this, stating, “Something that inhibits me is that personally I tend to soak things up and get a feel for the group before jumping in. I think it's been some time that I’ve felt comfortable speaking up since I’ve gotten to know everybody.” A similar but different category for adoption was team members worrying about how they present or saying “the wrong thing.” This content was observed in the survey and the participant stated:

I would love to receive feedback on how I present information in team. Sometimes I wonder if I am phrasing things appropriately or not and I hesitate to contribute because I do not want to say the “wrong thing.” On the other hand, I do not know that it is the responsibility of the team to provide this feedback and do not want to burden the team.

Formation, implementation, and early sustainment of the multicultural peer-consultation team (implementation)

To provide very specific lessons learned for other institutions that may want to form a multicultural peer-consultation team, we documented barriers to and strengths of implementing the multicultural peer-consultation team as mentioned by participants. In Table 3, we organized barriers and strengths according to domains of the Health Equity Implementation Framework, including characteristics of the team itself, clinical encounter between patients and providers, patient factors, provider factors (including culturally relevant factors such as medical mistrust or bias), local clinic context, local departmental context, outer context of the academic medical center, and societal influences (i.e., physical structures, economies, and sociopolitical forces).

Table 3.

Barriers and Strengths to Implementation Framed by the Health Equity Implementation Framework.

	Theme	Demonstrative Quote or Summary
Characteristics of the multicultural peer-consultation team itself (e.g., activities specific to the team, length, topics)
Barriers	Balance of didactics and consultation (Individual Interview 1)	“I found the consultation to be the most helpful. I think some barriers would be the didactics. It felt like some of the information was duplicative of a parallel multicultural consultation we had with [co-leader]. I think figuring out what's the right balance of didactics in a consultation team—it's going to vary by setting but I think that's something to reflect on”
	Information could be overwhelming (Individual Interview 1)	“Sometimes when there was a bulk of info coming at me and I was mentally focused but other times it was like ‘Do I have the bandwidth to take in all of these very well-meaning suggestions?’ so sometimes that was an obstacle”
	Advocacy once monthly makes it difficult to get momentum (Individual Interview 1)	[In a month,] they would have consultation (wk. 1), didactic, (wk. 2), consultation (wk. 3), and advocacy (wk. 4) structure to the sessions. It ended up being there was an advocacy portion of it that was once a month. With that understandably it was hard to get momentum on things. Sometimes felt like there would be really good ideas and struggle to overcome inertia to advocacy because it was so infrequent”
	Lack of diversity of disciplines (Individual Interview 3)	“I don't think we had anyone besides psychologists or social workers after our third year of team. I think they were invited but never really participated”
	Time the team meeting is held (Survey)	“I just am not free during that time”
Strengths	Non-hierarchical structure (Focus Group 1)	“[I am] much more likely to share since there's people of different levels/disciplines. The team is very welcoming due to the leveled feedback and personality”
	Psychologically safe (non-judgmental, open space) (Focus Group 1)	“[There is a] safe and open culture the group has built. It feels very genuine and supportive. Everyone comes in with their skillset and their education also everyone is very open and nonjudgmental and vulnerable. It creates a helpful climate”
	Framing what each person wants out of consultation (Individual Interview 1)	“We were generally very good at framing what we wanted out of the consultation team. In my mind, I know that this person has specific expertise in [care for this population] so I am going to weigh their suggestions more heavily”
	Stable leaders (Individual Interview 2)	“[New co-leaders] help with the role that [former co-leader] used to have. [Former co-leader] and I consulted about the overall strategic plan for the team. When I wasn't there, she could run the meeting. We always had a partnership in that way … [Now] if I can't make a meeting I know one or two of [the other co-leaders] can make a meeting and we can maintain the same structure and format and agenda and maintain consistency across meetings”
	Frequency and content of meetings (Individual Interview 2)	- Weekly meetings are preferred - People appreciate varying content (didactics, consultation, diversity action committee)
	Shared knowledge and leadership (e.g., having more than one “leader” to distribute power) (Individual Interview 2)	“I would say early on one of the functions is to have another person in the room who is just as knowledgeable as another … such that there is some equal distribution of voices in the room. Otherwise what happens is a lot of people look to me [leader] and I in some ways monopolize the space and saying “here is what I think—here are my thoughts” which feels like not in line with the spirit of the team where everyone is equal. We still have some members who are early in their training who do not speak because they don't feel like they have much to add which I would disagree with. I think we have some people who show up and listen really well, but harder to contribute. We try to not have a vertical structure, non-hierarchical but very early on [co-leader] and I were seen as the experts”
	Handled foundational knowledge vs making sure not repetitive (Individual Interview 2)	“It's a hard balance. When we do consultation I try to bring in different frameworks and different models to back up my suggestions as an example—we will try to bring in those foundational topics in consultation. We also want to give team members an opportunity to share about their experiences. If we only taught the same content year to year we wouldn't be able to tap into the sustained knowledge base that our new team members have. We encourage everyone to lead a didactic and on a topic of their choosing. We may put a couple on there—usually front loaded with older members leading first few didactics—and we ask people if they want to lead didactics later on in the year”
Clinical encounter with patients/clients (e.g., something about how interacting with a patient/client)
Barriers	None noted	N/A
Strengths	Different identities in the clinical encounter (Survey)	“My clinical encounters sometimes leave me with questions about how I can do a better job to establish rapport with a client or address an “elephant in the room.” Often this is in the form of having a mismatched identity with a client, client's behavior in the room, or how they may be perceiving me. I find that this leads me to attend and participate in consultation more actively”
Patient/client factors (e.g., their beliefs, particular health problems, issues with treatment)
Barriers	Harder to apply to patients similar to [me] (Focus Group 1)	“Does seem harder to apply to patients that are similar to me (multicultural identities) if can't pinpoint a differing identifying factor then that's not a multicultural consultation it's just a consultation”
Strengths	Diverse caseload (Survey)	“Patient factors spark my consultation questions and the consultation requests of others, as well as ideas for didactics that would be helpful for the team and action items to pursue. I find that patient's beliefs as well as challenges that relate to life circumstances or demographic characteristics often raise questions about treatment planning that I think would benefit from consultation”
Provider factors (e.g., your knowledge, your beliefs, demands on your time, identities)
Barriers	Too many demands on Providers (Focus Group 1)	“There are days I didn't come to team because I have to finish notes or do this and I need this slot of time because I have to be at some of the other meetings”
Strengths	Trainees able to participate more easily (Individual Interview 1)	“I had the time on my schedule because I was an intern I was able to organize my clinical schedule to attend. There were not as many faculty on the team, but I think that was because patient demand makes it harder to participate. As a trainee I was able to participate”
	It's personal (Survey)	“As a minority female, I feel a personal stake in conversations surrounding multiculturalism. I think it is especially important to be conscious of multiculturalism in the context of therapy”
Other staff or providers (e.g., their knowledge or beliefs, demands, attitudes)
Barriers	None noted	N/A
Strengths	None noted	N/A
Inner context—local level (i.e., the clinic)
Barriers	Supervisors were not explicitly encouraging trainees to attend (Individual Interview 1)	“When evaluation of students and trainees is really a key part of it—if other faculty members are not showing it's a super important team and are not—even in their knowledge of you—giving you credit for it, like that validation or praise, there would be inherent reason that you would choose another team that they are on”
Strengths	Supervisors of trainees were supportive in participation (Individual Interview 1)	“Generally very supportive. It was my internship year, so supervisors that I had asked me about participation and were supportive. But when they found out I was going they would say oh that's wonderful—have you thought about this? Can you consult with them about that?”
	Early support from supervisors (Individual Interview 2)	“It has been four years [since the multicultural peer-consultation team was started] … very early on I had a lot of support from faculty to build this training”
Inner context—Organizational (i.e., the department of psychiatry)
Barriers	Lack of diversity of providers (Focus Group 1)	“There is no trans-identified clinician in the entire system”
	General lack of institutional support/emphasis on the team (Focus Group 1)	“The second year faculty members who were coming the first year no longer came because they were already under so much pressure because they already had so many supervision hours and this was another hour that wasn't clinic and postdocs had higher productivity targets so it became hard to do so”
	Trainees do not have as much power or knowledge of system (Individual Interview 3)	“I feel like the department definitely impacts me because a lot of my training is happening through the department and in psychiatry. But I’m only there 1–2 days a week so because of that I’m not interfacing directly with them. I feel like there's some interaction happening directly through my supervisor. And if there's things happening, they’re relayed through my supervisor. So, I’m sure there's things happening, but I don't have a lot of interaction”
	Leadership change at department shifted values to be productivity based (Individual Interview 2)	“Then the second year a new department chair came in and it seemed like a lot of things in the department changed at the time. One notable thing was productivity so the hour that people spent in team like the MCT or other teams and it made it harder for people to prove their productivity because people had to see clients. Their productivity had to be very, very high. [It] had to be in the 75th percentile after the new chair came. […] This meant the second year, faculty members who were coming the first year no longer came because they were already under so much pressure because they already had so many supervision hours and this was another hour that wasn't clinic”
	Too many other teams to compete for time (Individual Interview 3)	“When productivity is crunched, you had to choose which team because you couldn't go to all of them. I was on four teams (DBT, MCT, trauma, self-care) and those were important to me and I did all of them because it was important so I’d stay after or longer but not everyone wants to do that. There was at least a minimum of 7 teams. That could be a barrier. [If someone is] required to be on a team like DBT team so then it's another hour for MCT. Most people were on DBT and trauma so that's already 3 h. Interns had to do self-care, so they had to do some teams they couldn't get out of, 2 h DBT and 1 h for self-care and most interns did 1 h for trauma. Hours in teams and hour in supervision [it was] hard to find an hour for MCT especially when it's the only team faculty are not a part of”
Strengths	Other teams were models for people (Individual Interview 3)	“There are multiple other teams happening at [the department]. One potential benefit is that people may know what a team is and a consultation team specifically and so that may in some ways make it easier”
	Department allowed team to exist (Survey)	“Our department has allowed this team to be recognized as an entity and use clinic space and time, which facilitates my attendance of meetings during work hours and in our clinic”
Outer context (i.e., school of medicine and the health care system at Duke)
Barriers	Lack of resources and support (Survey)	“The health care system does not necessarily get in the way of attendance and participation, but in my experience it does not appear easy to make large scale policy and system changes to facilitate a more inclusive, open, and supportive environment to different types of patients. The health care system at Duke could be more supportive in terms of providing resources and support to the team to facilitate our work and distribution of information”
Strengths	None noted	N/A
Societal influence—Economies (e.g., Medicaid policies, insurance, how services are exchanged)
Barriers	Trainees can't work with patients with Medicaid (Survey)	“Medicaid policies get in the way of me being able to provide services to Medicaid patients as a trainee, which does not allow me to use new skills and apply information to the level and scope that I would like to. Barriers in terms of insurance reimbursement for Telemedicine also are a barrier to me working with patients from underserved communities (e.g., rural, chronic pain, limit transportation)”
Strengths	None noted	N/A
Societal influence—Physical structures (e.g., lighting, location of clinics and offices, quality of facilities and materials)
Barriers	Having it in one building makes it hard for others outside building to attend (Individual Interview 1)	“Where the building is located is a barrier because [people] didn't know it existed without intervention from us. Our day was booked from minute to minute in one-hour chunks so when I wanted to go to some lecture across campus I couldn't go unless there was some way I could get there. I think not being able to walk down the hall [is a barrier]. You will worry you will miss 15–20 min and that feels disrespectful so might as well not go”
	No parking (Individual Interview 3)	“There is no parking there. That is something clients would even complain about. Not necessarily even that close to where other people in psychiatry and psychology practice. If [the meeting room] is not down the hall it won't take like five extra minutes. [It] may take 10 to 15 and if it's across campus or if you have to drive. With driving you would have to have a parking pass for that particular lot [which is] $700 a year so not a lot of people had it. The wait list [for the parking pass] was 350 people long to get it”
Strengths	It's convenient in the same building where people worked (Individual Interview 3)	“It's very convenient for me. I just walk two doors down. Very low barrier for that”
Societal influence—Social and political forces (e.g., laws or social norms, eligibility for services based on law, immigration laws, political climate)
	Sociopolitical factors prevent marginalized populations from seeking care (Survey)	“The political climate and certain laws and social norms prevent historically marginalized populations from seeking and obtaining care to the same extent as other groups, which does not allow me to use the information from multicultural team to the same degree”
	Current sociopolitical climate makes this work even more important (Survey)	“With today's political climate, I think it is more important than ever to be conscientious of multiculturalism. People who get the short end of the stick in society involving factors surrounding their race/gender/identity will need providers who are aware of the complexities of multiculturalism”

We also documented implementation barriers and strengths in the process (or facilitation) required to launch the multicultural peer-consultation team, in other words, strategies co-leaders and developers used to create and implement the team. There was only one barrier reported during Individual Interview 3 to the process of launching the team at first—the co-developers were unsure if it was acceptable: “We didn't know if this would be a valued service by everybody else.” Results suggested the facilitation process of setting up the multicultural peer-consultation team was different by “phase” of implementation. That is, there were unique activities in exploring the need for and potential obstacles to starting the team and unique activities after launching the team (Aarons et al., 2011). Helpful activities are described by phase in Table 4.

Table 4.

Multicultural Peer-Consultation Team Phase Activities.

Phase	Activities
Pre-Implementation/Planning/Exploration	- Leaders initially did a “one-off training” to assess interest (i.e., a pilot training) - Decided to use a consultation approach because the department had prior experience with consultation teams for other psychotherapies - Conducted a formal needs assessment in department prior to starting
Implementation (after launching)	- Leaders presented to incoming trainees on multicultural peer-consultation team to advertise - Adapted frequency and content of meetings based on team members’ feedback - Leaders of team had check-ins outside the team for strategic planning - Leaders began to encourage others to start leading team or presenting didactics - Leaders were networking throughout department and leveraging their reputation and power to advertise multicultural peer-consultation team (e.g., speaking with directors of training programs)

Phase

Activities

Pre-Implementation/Planning/Exploration

- Leaders initially did a “one-off training” to assess interest (i.e., a pilot training) - Decided to use a consultation approach because the department had prior experience with consultation teams for other psychotherapies - Conducted a formal needs assessment in department prior to starting

Implementation (after launching)

- Leaders presented to incoming trainees on multicultural peer-consultation team to advertise - Adapted frequency and content of meetings based on team members’ feedback - Leaders of team had check-ins outside the team for strategic planning - Leaders began to encourage others to start leading team or presenting didactics - Leaders were networking throughout department and leveraging their reputation and power to advertise multicultural peer-consultation team (e.g., speaking with directors of training programs)

Discussion

Many clinicians and researchers today recognize the importance of striving towards APA's recommendations of multicultural competence (American Psychological Association, 2006; Liaison Committee on Medical Education, 2007). Different strategies have been employed to provide individuals with resources, education, and training experiences to develop competence such as coursework (Collins & Pieterse, 2007; D’Andrea & Daniels, 1991; Reynolds, 1995), time-limited workshops (Benjamin et al., 2019; Delphin & Rowe, 2008), and multicultural supervision (Leong & Wagner, 1994; Burkard et al., 2006). While mental health fields have attended more to multicultural education in recent decades, the innovations in this area need rigorous evaluation to determine their effectiveness (Benuto et al., 2019). Moreover, incorporating methods and evaluation of these methods from implementation science will facilitate wide-scale dissemination to meet the overarching desire to improve our field's ability to provide care to patients who have been traditionally underserved and underrepresented in clinical settings. Therefore, in the present study, we evaluated our multicultural peer-consultation team with implementation science frameworks and methods, focusing on the perceived utility (effectiveness), strengths, and barriers (i.e., challenges) of a multicultural peer-consultation service that has been implemented in an academic medical center over four years. By documenting and reporting interventions like multicultural peer-consultation services, institutions and individuals will have useful information to consider as they explore what kinds of multicultural competence interventions they might implement within their respective systems.

In the present evaluation, there were several noteworthy findings. Several positive impacts of participating on the team included strengthening various elements of the clinical practice, providing concrete information relevant to practicing with multicultural sensitivity and competency, expanding the knowledge base of evidence-based practice (relevant to intervention and assessment), and allowing a deeper consideration of the systemic impact of marginalization on mental health. These impacts are in line with the original goals of the multicultural peer-consultation team and needs identified within the clinic at the time of its creation (Nagy et al., 2019). When asked about the adoption of the team, which is the degree to which members participated in the elements of the team (i.e., giving and receiving consultation, providing and participating in didactics, and participating in diversity action committee initiatives), important barriers to adoption came to light. Some members expressed difficulty speaking on the team when they first joined, due to a need to warm up (to one another as well as the peer-consultation space) and imbibe information prior to participating themselves, and also expressed worry about not presenting themselves appropriately. Importantly, these findings signal that either participants were uncomfortable or did not have time, or that not everyone considered themselves expert enough to teach, but still found it beneficial to share and receive consultation.

Finally, this study examined barriers and strengths to the formation and implementation of the team. Consistent with steps outlined in Nagy et al. (2019), an identified strength of the formation of the team was that the leaders assessed needs of the clinic members prior to forming the team and based the structure of consultation on a familiar model. These identified strengths may explain why members continue to find the team relevant to their clinical needs. With regard to the implementation of the team, consistent themes emerged when identifying barriers and strengths. First, barriers at the levels of the clinic and department of psychiatry were implicated in making it difficult for members to attend as often as they would like or need. Specifically, some members reported that their attendance on the team was not adequately encouraged and supported by their supervisors, and members reported difficulty protecting the time needed to participate on the team, due to competing clinic and departmental responsibilities. This is consistent with barriers identified by Nagy et al. (2019) during the development of the team. Specifically, “competing demands for time” was identified as a barrier by 87.5% of participants, which is common in academic medical settings. Therefore, this appears to be a long-standing barrier to implementation. Other barriers of note included the monthly frequency of meeting for the diversity action committee, which led to limited momentum on advocacy goals. Further, our qualitative analyses indicated that a lack of diversity within the team with regard to disciplines and identities (e.g., members interviewed were trained in either clinical psychology or social work) was identified as a barrier to integrating diverse perspectives, from which we infer that team members believe that it would have been advantageous to have other team members from distinct disciplines (namely psychiatry and counseling psychology) and multicultural identities, which had been identified as a challenge in the department in which the team was housed. Strengths were largely related to the structure, content, and climate of the team. The familiarity and comfort with the case consultation structure, based on the model in DBT (Koerner, 2012), appeared to be a facilitator. The frequency of meetings, and content (i.e., consultation, didactic, and action committee) were further identified as facilitating implementation. A conscious effort on the part of the developers to foster a non-hierarchical structure and divide leadership tasks among multiple co-leaders was thought to add to a climate of comfort and stability. Members described a climate that was “open,” “nonjudgmental,” “genuine,” and “supportive”—in line with the principles behind the consultation team agreements laid out at its creation (Nagy et al., 2019).

Finally, some factors were identified as strengths or barriers by different members. For example, supervisors were described as being supportive by some, and less so by others. Members recognized that having a case load with diverse identities motivated their participation on the team, while barriers to such a case load (e.g., trainees are not able to see patients with Medicare) were also identified. While for some the physical location of the team was convenient, for others this was unfortunately a barrier due to distance from their usual place of work and lack of convenient parking.

Several limitations are acknowledged within the study. Data gathered during focus groups may have been biased by demand characteristics, that is that participants may not have wanted to say anything critical about the team out of respect or that they wanted to present themselves in a positive light. To offset this bias, one of the co-developers of the multicultural team (who was still at the institution) was not present during focus groups. This study is also limited in its generalizability due to the small sample size and setting (i.e., a high-demand academic medical center); the unique contextual factors representing strengths and barriers may not be present in other settings (e.g., private practice, the Veterans Health Administration, a graduate departmental clinic). It is possible that even with independent evaluators, former team members may have felt uncomfortable participating and reporting negative effects or impact of the multicultural peer-consultation team as some individuals still work in the department as team leaders. Of note, participants were also solicited two weeks into the COVID-19 pandemic and stay-at-home orders, which could have certainly limited their ability to participate. Additionally, this study did not capture data pertaining to each team member's prior multicultural training, which may have confounded some of the findings contained herein.

One important finding from this study was that attendance of team members was regular for some yet sporadic for others. Our results revealed that a particular barrier to participating on the team was the time the team meeting was held. We have reason to believe that, similarly to results noted in Nagy et al. (2019), team members have many competing demands that make it challenging to attend regularly. However, we did not specifically include questions in the survey or interviews inquiring about reasons for sporadic attendance. It is possible that the discrepancy in attendance is merely about having too many competing demands to be able to regularly attend the team, but sporadic attendance could also reflect ambivalence about or discomfort with the team. To explore these associations, future studies ought to directly ask for reasons for missing team meetings.

Lastly, effectiveness was represented by the perceived impact of the team by the team members, but not directly patient outcomes or patient experience nor subjective or objective measurements of multicultural competence in the provision of clinical care. Thus, the impact of consultation on patient care was entirely subjective to the providers themselves. Given that the focus of this study was on utility for future implementation of multicultural consultation teams by other institutions and identifying barriers and strengths, the focus was not on the efficacy of the team but rather its steps of implementation and early sustainment. Future studies should focus to identify further what are integral pieces of the multicultural team and other questions related to overall efficacy.

Despite these limitations, this study has several unique strengths. One strength is the use of mixed methods including surveys, focus groups, and individual interviews to gather data. The use of mixed methods helps account for the various limitations that generally accompany each individual method when used in isolation. Furthermore, this study used evaluators from outside of the program to reduce biases that are generally observed in groups. Another strength of this study is that it was designed to be accessible for as many current and former members of the team as possible, while also increasing the likelihood that participants’ opinions would be heard. Specifically, this study elected to have two focus groups, limiting them to five people per group in order to give more time and space for individuals to provide input. We intentionally took extra care to maintain confidentiality and limit coercion (e.g., group members were not required to participate in the groups).

Conclusion

In conducting this evaluation, there are several noteworthy “lessons learned” our team has gathered that represent recommendations to others who are interested in developing and implementing a similar service at their institutions. First, in order to increase adoption, we recommend that leaders take measures to increase the comfort of new members, to enable them to participate more actively. This could include talking openly about ways to increase psychological safety in the room and asking members what would allow them to feel more comfortable in sharing their thoughts in the beginning of each academic year during the “orientation to the team” portion (as new members join the team and others exit). This finding is consistent with the literature noting that discussing multicultural topics can evoke discomfort and many are wary to reveal prejudices and biases (Spanierman et al., 2006; Sue et al., 2009), which combined can pose a challenge toward developing the psychological safety needed to foster deep and nuanced learning in this area (Delphin & Rowe, 2008). Second, we recognize the value of making continuous efforts to include diverse members on teams to allow for multiple perspectives, particularly with regard to under-represented identities (e.g., individuals identifying as transgender) and those with diverse training backgrounds. Third, more frequent meetings to plan advocacy efforts in the clinic, department, and institution may enable momentum to make changes in this area. Such change tends to take consistent effort with awareness of and sensitivity to systemic barriers to change in general, which is difficult to achieve through monthly meetings. Finally, as a continued point of emphasis, it is important to engage leadership at the department and institutional level, to appropriately incentivize and allocate dedicated time for participating on the team (Nagy et al., 2019). This may include engaging leadership during the formation of the team, so that the goals of the team are viewed as being in line with the needs of the clinic or department as well as relevant stakeholders. Additionally, continual communication of the impact of the team on important patient outcomes to leadership can facilitate resource allocation for this service.

Footnotes

Acknowledgements

The work presented herein could not have been possible without the contributions of our team members. We would like to thank in particular team members Theresa Flores, Mary Triplett, Laura Bete, Angela Pisoni, Stephanie Schuette, and Paulina Ruiz who were instrumental in sustaining the MC team during the time of this evaluation, but who did not participate as authors for this manuscript. In addition, we would like to acknowledge the contributions of former team members who were instrumental in creating this team, in particular co-developer Dr. Kelly LeMaire.

Author note

Deepika Anand, CBT Center of Chicago, Chicago, IL, USA.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

Dr. Gabriela A. Nagy is a fellow with the Research in Implementation Science for Equity (RISE), at the University of California San Francisco's Center for Vulnerable Populations; through an award from the National Heart, Lung, and Blood Institute (5R25HL126146-07). Additionally, Dr. Nagy is supported by a Diversity Supplement from the National Institute on Minority Health and Health Disparities (R01MD012249-03S1). Dr. Nagy is also an awardee of an institutional Career Development Award through the Duke University REACH Equity Center; funded through the National Institute on Minority Health and Health Disparities (5U54MD012530-04). Dr. Eva N. Woodward is a fellow with the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis; through an award from the National Institute of Mental Health (5R25MH08091607).

ORCID iDs

Gabriela A. Nagy

Eva N. Woodward

Notes

Gabriela A. Nagy, PhD, is a clinical psychologist and Assistant Professor in the Department of Psychiatry and Behavioral Sciences at the Duke University School of Medicine and in the Duke University School of Nursing. Her research has broadly focused on achieving mental health equity in minoritized groups (with a particular focus on immigrants and refugees) through the use of implementation science, community engaged methods, and clinical science. In a related line of research, she is focused on optimizing learning methods in multicultural psychology and cross-cultural psychiatry.

Clair Cassiello-Robbins, PhD, is a clinical psychologist who specializes in the transdiagnostic treatment of emotional disorders with a focus on dysregulated anger. She completed her graduate training at Boston University and her postdoctoral training at Duke University Medical Center. Her interests include adapting evidence-based treatments to meet the needs of unique patient populations and treatment settings. She is currently a psychologist at Triangle Area Psychology (TAP) clinic in Durham, NC.

Deepika Anand, PhD, is a practicing clinical psychologist and founder of a private practice, the CBT Center of Chicago. Prior to this, she worked as a Medical Instructor at the Duke University Medical Center. Her published works include research on the role of emotion regulation for psychological outcomes such as substance use, depression and anxiety, cross-cultural adaptations of treatments and assessment tools; and strengthening multicultural competence in the field of psychology.

Macey L. Arnold, BS, is currently a doctoral student in the counseling psychology program at the University of North Texas, where she is specializing in sport and exercise psychology. Her research and professional interests focus on multicultural training and education, athlete wellness, and sports environments, with more specific interest in the experiences of 2SLGBTQ + athletes.

Jessica N. Coleman, MA, is a clinical psychology doctoral candidate in the Department of Psychology & Neuroscience at Duke University. Her research interests center on the development, evaluation, and implementation of psychosocial interventions to promote equity. She is particularly interested in promoting sexual and reproductive health among survivors of sexual violence and minoritized populations.

Joshua Nwosu, PhD, is a health psychology postdoctoral fellow at the Washington D.C. Veteran Affairs Medical Center. Dr. Nwosu is also a participating member of the Washington D.C. Veteran Affairs Psychology Diversity Committee. Dr. Nwosu's research interests and clinical service focus on brief culturally sensitive behavioral interventions for African Americans experiencing chronic and complex medical conditions.

R. Sonia Singh, PhD, is currently a psychology postdoctoral fellow at the South Central Mental Illness Research, Education and Clinical Center through the Central Arkansas Veterans Healthcare System and the University of Arkansas for Medical Sciences in Little Rock, Arkansas. Her main research interests are in implementation science and the integration between implementation science and health equity, specifically for gender and sexual minority populations.

Eva N. Woodward, PhD, is a clinical health psychologist. In her research, she uses implementation science and practice, community engaged research, and mixed methods to improve equitable health care delivery for groups experiencing disparities, such as people of color, sexual minority (lesbian, gay, bisexual, queer [LGBQ]) and low-income individuals. Dr. Woodward works at the VA Center for Mental Healthcare and Outcomes Research (Little Rock, Arkansas) and University of Arkansas for Medical Sciences. She is also a trainer for the Implementation Facilitation Learning Hub operated by Behavioral Health Quality Enhancement Research Initiative through the Veterans Health Administration.

References

Aarons

G. A.

Hurlburt

Horwitz

S. M

. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23. https://doi.org/10.1007/s10488-010-0327-7

Abraham

T. H.

Finley

E. P.

Drummond

K. L.

Haro

E. K.

Hamilton

A. B.

Townsend

J. C

., Littman, A. J., & Hudson

. (2021). A method for developing trustworthiness and preserving richness of qualitative data during team-based analysis of large data sets. American Journal of Evaluation, 42(1), 139–156. https://doi.org/10.1177/1098214019893784

Alizadeh

Chavan

. (2016). Cultural competence dimensions and outcomes: A systematic review of the literature. Health & Social Care in the Community, 24(6), e117–e130. https://doi.org/10.1111/hsc.12293

American Psychological Association. (2003). Guidelines on multicultural education, training, research, practice, and organizational change for psychologists. American Psychologist, 58(5), 377–402. https://doi.org/10.1037/0003-066X.58.5.377

American Psychological Association. (2006). Guidelines and principles for accreditation of programs in professional psychology . http://www.apa.org/ed/accreditation/about/policies/guiding-principles.pdf

American Psychological Association. (2017). Multicultural guidelines: An ecological approach to context, identity, and intersectionality . http://www.apa.org/about/policy/multicultural-guidelines.pdf

Ancis

J. R.

Marshall

D. S

. (2010). Using a multicultural framework to assess supervisees’ perceptions of culturally competent supervision. Journal of Counseling & Development, 88(3), 277–284. https://doi.org/10.1002/j.1556-6678.2010.tb00023.x

Bauer

M. S.

Damschroder

Hagedorn

Smith

Kilbourne

A. M

. (2015). An introduction to implementation science for the non-specialist. BMC Psychology, 3(1), 32. https://doi.org/10.1186/s40359-015-0089-9

Benjamin

J. Z.

Heredia

Jr Han

Kirtley

A. T.

Morrison

E. J.

Leffler

J. M

. (2019). Implementation of a cross-cultural simulation workshop: feasibility and training satisfaction. Training and Education in Professional Psychology, 45–53. https://doi.org/10.1037/tep0000300

10.

Benuto

L. T.

Casas

O’Donohue

W. T

. (2018). Training culturally competent psychologists: A systematic review of the training outcome literature. Training and Education in Professional Psychology, 12(3), 125. https://doi.org/10.1037/tep0000190

11.

Benuto

L. T.

Singer

Newlands

R. T.

Casas

J. B

. (2019). Training culturally competent psychologists: where are we and where do we need to go? Training and Education in Professional Psychology, 13(1), 56–63. https://doi.org/10.1037/tep0000214

12.

Bhui

Warfa

Edonya

McKenzie

Bhugra

. (2007). Cultural competence in mental health care: A review of model evaluations. BMC Health Services Research, 7(1), 15. https://doi.org/10.1186/1472-6963-7-15

13.

Birt

Scott

Cavers

Campbell

Walter

. (2016). Member checking: A tool to enhance trustworthiness or merely a nod to validation? Qualitative Health Research, 26(13), 1802–1811. https://doi.org/10.1177/1049732316654870

14.

Buchanan

N. T.

Wiklund

L. O

. (2020). Why clinical science must change or die: Integrating intersectionality and social justice. Women & Therapy, 43(3–4), 309–329. https://doi.org/10.1080/02703149.2020.1729470

15.

Burkard

A. W.

Johnson

A. J.

Madson

M. B.

Pruitt

N. T.

Contreras-Tadych

D. A.

Kozlowski

J. M.

Hess

S. A.

Knox

. (2006). Supervisor cultural responsiveness and unresponsiveness in cross-cultural supervision. Journal of Counseling Psychology, 53(3), 288–301. https://doi.org/10.1037/0022-0167.53.3.288

16.

Collins

N. M.

Pieterse

A. L

. (2007). Critical incident analysis based training: An approach for developing active racial/cultural awareness. Journal of Counseling & Development, 85(1), 14–23. https://doi.org/10.1002/j.1556-6678.2007.tb00439.x

17.

D’Andrea

Daniels

. (1991). Exploring the different levels of multicultural counseling training in counselor education. Journal of Counseling & Development, 70(1), 78–85. https://doi.org/10.1002/j.1556-6676.1991.tb01565.x

18.

Delphin

M. E.

Rowe

. (2008). Continuing education in cultural competence for community mental health practitioners. Professional Psychology: Research and Practice, 39(2), 182–191. https://doi.org/10.1037/0735-7028.39.2.182

19.

Domenech Rodríguez

M. M.

& Bernal

. (2012). Frameworks, models, and guidelines for cultural adaptation. In Bernal

Domenech Rodríguez

M. M.

(Eds.), Cultural adaptations: tools for evidence-based practice with diverse populations (pp. 23–44). American Psychological Association. https://doi.org/10.1037/13752-002

20.

Eagleton

. (2000). The idea of culture. Blackwell.

21.

Ecker

A. H.

Abraham

T. H.

Martin

L. A.

Marchant-Miros

., & Cucciare

M. A

. (2021). Factors affecting adoption of coordinated anxiety learning and management (CALM) in veterans’ affairs community-based outpatient clinics. The Journal of Rural Health, 37(2), 447–455. https://doi.org/10.1111/jrh.12528

22.

Glasgow

R. E.

Harden

S. M.

Gaglio

Rabin

B. A.

Smith

M. L.

Porter

G. C

., Ory, M. G., & Estabrooks

P. A

. (2019). RE-AIM planning and evaluation framework: Adapting to new science and practice with a twenty-year review. Frontiers in Public Health, 7, 1–9. https://doi.org/10.3389/fpubh.2019.00064

23.

Glasgow

R. E.

Vogt

T. M.

Boles

S. M

. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 89(9), 1322–1327. https://doi.org/10.2105/AJPH.89.9.1322

24.

Govere

E. M

. (2016). How effective is cultural competence training of healthcare providers on improving patient satisfaction of minority groups? A systematic review of literature. Worldviews on Evidence-Based Nursing, 13(6), 402–410. https://doi.org/10.1111/wvn.12176

25.

Gregus

S. J.

Stevens

K. T.

Seivert

N. P.

Tucker

R. P.

Callahan

J. L

. (2020). Student perceptions of multicultural training and program climate in clinical psychology doctoral programs. Training and Education in Professional Psychology, 14(4), 293–307. https://doi.org/10.1037/tep0000289

26.

Hamilton

. (2013). Qualitative methods in rapid turn-around health services research. HSRD Cyberseminar, US Department of Veteran Affairs. https://doi.org/10.1002/jclp.20605

27.

Harvey

Kitson

. (2015). Implementing evidence-based practice in healthcare: A facilitation guide. Routledge. https://doi.org/10.4324/9780203557334

28.

Hays

P. A

. (1996). Addressing the complexities of culture and gender in counseling. Journal of Counseling & Development, 74(4), 332–338. https://doi.org/10.1002/j.1556-6676.1996.tb01876.x

29.

Hernandez

Nesman

Mowery

Acevedo-Polakovich

I. D.

Callejas

L. M

. (2009). Cultural competence: A literature review and conceptual model for mental health services. Psychiatric Services, 60(8), 1046–1050. https://doi.org/10.1176/ps.2009.60.8.1046

30.

Herschell

A. D.

Kolko

D. J.

Baumann

B. L.

Davis

A. C

. (2010). The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology Review, 30(4), 448–466. https://doi.org/10.1016/j.cpr.2010.02.005

31.

Huey

S. J.

Jr Tilley

J.L.

Jones

E.O.

Smith

C. A.

(2014). The contribution of cultural competence to evidence-based care for ethnically diverse populations. Annual Review of Clinical Psychology, 10, 305–338. https://doi.org/10.1146/annurev-clinpsy-032813-153729

32.

Kessler

R. S.

Purcell

E. P.

Glasgow

R. E.

Klesges

L. M.

Benkeser

R. M.

Peek

C. J

. (2013). What does it mean to “employ” the RE-AIM model? Evaluation & the Health Professions, 36(1), 44–66. https://doi.org/10.1177/0163278712446066

33.

Kirmayer

L. J

. (2006). Beyond the ‘new cross-cultural psychiatry’: Cultural biology, discursive psychology and the ironies of globalization. Transcultural Psychiatry, 43(1), 126–144. https://doi.org/10.1177/1363461506061761

34.

Kirmayer

L. J.

Groleau

Guzder

Blake

Jarvis

. (2003). Cultural consultation: A model of mental health service for multicultural societies. The Canadian Journal of Psychiatry, 48(3), 145–153. https://doi.org/10.1177/070674370304800302

35.

Kirmayer

L. J.

Rousseau

Guzder

(2014). Introduction: The place of culture in mental health services. In Cultural consultation (pp. 1–20). Springer. https://doi.org/10.1007/978-1-4614-7615-3_1

36.

Koerner

. (2012). Doing dialectical behavior therapy: A practical guide. Guilford Press.

37.

Leong

F. T.

Wagner

N. S

. (1994). Cross-cultural counseling supervision: What do we know? What do we need to know? Counselor Education and Supervision, 34(2), 117–131. https://doi.org/10.1002/j.1556-6978.1994.tb00319.x

38.

Liaison Committee on Medical Education. (2007). Functions and structure of a medical school: Standards for accreditation of medical education programs leading to the MD degree. Liaison Committee on Medical Education.

39.

Lie

D. A.

Lee-Rey

Gomez

Bereknyei

Braddock

C. H

. (2011). Does cultural competency training of health professionals improve patient outcomes? A systematic review and proposed algorithm for future research. Journal of General Internal Medicine, 26(3), 317–325. https://doi.org/10.1007/s11606-010-1529-0

40.

López

S. R.

Ribas

A. C.

Sheinbaum

Santos

M. M.

Benalcázar

Garro

Kopelowicz

. (2020). Defining and assessing key behavioral indicators of the shifting cultural lenses model of cultural competence. Transcultural Psychiatry, pps. 594–609. https://doi.org/10.1177/1363461520909599

41.

Magaña López

Bevans

Wehrlen

Yang

Wallen

G. R

. (2017). Discrepancies in race and ethnicity documentation: A potential barrier in identifying racial and ethnic disparities. Journal of Racial and Ethnic Health Disparities, 4(5), 812–818. https://doi.org/10.1007/s40615-016-0283-3

42.

Martínez

Holloway

(1997). The supervision relationship in multicultural training. In Pope-Davis

D. B.

Coleman

H. L.

(Eds.), Multicultural aspects of counseling series 7: Multicultural counseling competencies: Assessment, education and training, and supervision (pp. 325–349). Sage Publications, Inc. https://doi.org/10.4135/9781452232072.n15

43.

Morse

J. M

. (2015). Critical analysis of strategies for determining rigor in qualitative inquiry. Qualitative Health Research, 25(9), 1212–1222. https://doi.org/10.1177/1049732315588501

44.

Nagy

G. A.

LeMaire

Miller

M. L.

Howard

Wyatt

Zerubavel

. (2019). Development and implementation of a multicultural consultation service within an academic medical center. Cognitive and Behavioral Practice, 26(4), 656–675. https://doi.org/10.1016/j.cbpra.2019.02.004

45.

National Association of Social Workers. (2015). Standards and indicators for cultural competence in social work practice. Washington, DC . https://www.socialworkers.org/LinkClick.aspx?fileticket=PonPTDEBrn4%3D&portalid=0

46.

Neblett

E. W.,

Jr. (2019). Diversity (psychological) science training: Challenges, tensions, and a call to action. Journal of Social Issues, 75(4), 1216–1239. https://doi.org/10.1111/josi.12357

47.

Palinkas

L. A.

Aarons

G. A.

Horwitz

Chamberlain

Hurlburt

Landsverk

. (2011). Mixed method designs in implementation research. Administration and Policy in Mental Health, 38(1), 44–53. https://doi.org/10.1007/s10488-010-0314-z

48.

Peters

H. J.

Krumm

A. J.

Gonzales

R. R.

Gunter

K. K.

Paez

K. N.

Zygowicz

S. D.

Haggins

K. L

. (2011). Multicultural environments of academic versus internship training programs: Lessons to be learned. Journal of Multicultural Counseling and Development, 39(2), 114–124. https://doi-org/10.1002/j.2161-1912.2011.tb00145.x https://doi.org/10.1002/j.2161-1912.2011.tb00145.x

49.

Reynolds

A. L.

(1995). Challenges and strategies for teaching multicultural counseling courses. In Ponterotto

J. G.

Casas

J. M

Suzuki

L. A

Alexander

C. M

(Eds.), Handbook of multicultural counseling (pp. 312−330). Sage Publications, Inc.

50.

Rosenthal

. (2016). Incorporating intersectionality into psychology: An opportunity to promote social justice and equity. American Psychologist, 71(6), 474–485. https://doi.org/10.1037/a0040323

51.

Santos

M. M.

Nagy

G. A.

Kanter

J. W.

López

S. R

. (2021). Applying a process-oriented model of cultural competence to behavioral activation for depression. Cognitive and Behavioral Practice, 28(2), 127–146. https://doi.org/10.1016/j.cbpra.2020.11.007

52.

Sayrs

J. H.

(2018). Running an effective DBT consultation team. In M. A. Swales (Ed.), The Oxford handbook of dialectical behaviour therapy. Oxford University Press. https://doi.org/10.1093/oxfordhb/9780198758723.013.10

53.

Schilder

A. J.

Kennedy

Goldstone

I. L.

Ogden

R. D.

Hogg

R. S.

O’Shaughnessy

M. V

. (2001). “Being dealt with as a whole person.” Care seeking and adherence: The benefits of culturally competent care. Social Science & Medicine, 52(11), 1643–1659. https://doi.org/10.1016/S0277-9536(00)00274-4

54.

Shepherd

S. M

. (2019). Cultural awareness workshops: Limitations and practical consequences. BMC Medical Education, 19(1), 1–10. https://doi.org/10.1186/s12909-018-1450-5

55.

Soto

Smith

T. B.

Griner

Domenech Rodríguez

Bernal

. (2018). Cultural adaptations and therapist multicultural competence: Two meta-analytic reviews. Journal of Clinical Psychology, 74(11), 1907–1923. https://doi.org/10.1002/jclp.22679

56.

Spanierman

L. B.

Poteat

V. P.

Beer

A. M.

Armstrong

P. I

. (2006). Psychosocial costs of racism to whites: Exploring patterns through cluster analysis. Journal of Counseling Psychology, 53(4), 434–441. https://doi.org/10.1037/0022-0167.53.4.434

57.

Stetler

C. B.

Legro

M. W.

Wallace

C. M.

Bowman

Guihan

Hagedorn

., … & Smith

J. L

. (2006). The role of formative evaluation in implementation research and the QUERI experience. Journal of General Internal Medicine, 21(2), S1–S8. http://dio.org/10.1111/j.1525-1497.2006.00355.x https://doi.org/10.1007/s11606-006-0267-9

58.

Sue

D. W.

Sue

Neville

H. A.

Smith

. (2019). Counseling the culturally diverse: Theory and practice. John Wiley & Sons.

59.

Sue

D. W.

Torino

G. C.

Capodilupo

C. M.

Rivera

D. P.

Lin

A. I

. (2009). How white faculty perceive and react to difficult dialogues on race: Implications for education and training. The Counseling Psychologist, 37(8), 1090–1115. https://doi.org/10.1177/0011000009340443

60.

Swales

M. A

. (2010). Implementing DBT: Selecting, training, and supervising a team. The Cognitive Behaviour Therapist, 3(2), 71–79. https://doi.org/10.1017/S1754470X10000061

61.

Tao

K. W.

Owen

Pace

B. T.

Imel

Z. E

. (2015). A meta-analysis of multicultural competencies and psychotherapy process and outcome. Journal of Counseling Psychology, 62(3), 337–350. https://doi.org/10.1037/cou0000086

62.

Tong

Sainsbury

Craig

. (2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19(6), 349–357. https://doi.org/10.1093/intqhc/mzm042

63.

U.S. Department of Health and Human Services. (2001). Mental health: Culture, race, and ethnicity-A supplement to mental health: A report of the surgeon general. U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, Center for Mental Health Services. https://www.ncbi.nlm.nih.gov/books/NBK44243/

64.

Valenstein-Mah

Greer

McKenzie

Hansen

Strom

T. Q.

Wiltsey Stirman

., Wilt, T. J., & Kehle-Forbes

S. M

. (2020). Effectiveness of training methods for delivery of evidence-based psychotherapies: A systematic review. Implementation Science, 15, 1–17. https://doi.org/10.1186/s13012-020-00998-w

65.

Wendt

D. C.

Gone

J. P

. (2012). Rethinking cultural competence: Insights from indigenous community treatment settings. Transcultural Psychiatry, 49(2), 206–222. https://doi.org/10.1177/1363461511425622

66.

Woodward

E. N.

Matthieu

M. M.

Uchendu

U. S.

Rogal

Kirchner

J. E

. (2019). The health equity implementation framework: Proposal and preliminary study of hepatitis C virus treatment. Implementation Science, 14(1), 1–18. https://doi.org/10.1186/s13012-019-0861-y