Insights From Community Organizations Collaborating With Government and Academia to Foster Health Literacy

Setting

Program activities were conducted in San Diego County, the second largest county in California and home to ~3.3 million individuals. The population is diverse: 35% are Hispanic/Latino, 6% are African American/Black, 1% are American Indian, Alaska Native, 13% are Asian, and 44% are non-Hispanic White; 23% of persons are foreign-born and in the 2021–2022 fiscal year, 5,055 refugees were resettled in San Diego. Importantly, 10.7% of persons are living in poverty (San Diego County Health and Human Services Agency, 2022; US Census Bureau, 2023).

Partnerships

A partnership involving local government, academic partners, and CBOs successfully obtained federal AHL funding. Of note, many of the partnering CBOs had previously collaborated with the coordinating university and they proposed that the university’s Center for Community Health—an entity with deep community ties and a history of co-leadership with community—lead the application to meet the diverse needs of network partners. It was expected that the initiative would create a community of partners with a shared vision to build health literacy locally; provide tailored technical assistance and resources directed toward health literacy; and support infrastructure, including capacity-building and tailored support for CBO partners in topics, including grant-writing, fund management, and evaluation support services. Monolingual CBOs might have been excluded due to language barriers but through this initiative were funded in part because of linguistic support from the university. These are challenges previously faced by CBOs in the network and which some CBOs described as barriers to competing for and obtaining federal funding.

In brief, upon receiving the notice of award, the city and university’s Center of Community Health convened a steering committee comprised representatives from local cities, jurisdictions and academics; they developed and disseminated an RFPs to the county’s non-profit sector (i.e., CBOs, ethnic CBOs, federally qualified health centers [FQHCs]) with the goal of distributing funds to CBOs who would implement community-led programs to build health literacy. Recognizing that local CBOs are best poised to implement public health strategies relevant to their communities in an accessible and effective way, the RFP was informed by a series of community conversations facilitated virtually for each of the participating cities to obtain input from community organizations about the greatest community needs, challenges, and strengths. The university coordinated these efforts, hosted a bidder’s conference, and provided voluntary one-on-one consultations to interested agencies as part of the project’s community-engaged approach to building capacity with local CBOs.

The grant selection committee—comprised 11 members from academia, government, and CBO sectors—selected 12 local CBOs and ethnic CBOs for funding for periods ranging from 10 to 12 months. CBOs designed and implemented community-focused health literacy interventions for locally underserved populations, including ethnic minorities, immigrants, and refugees. This approach sought to leverage existing partnerships and community trust of CBOs to achieve the broader program’s objectives. CBOs addressed health care access and health literacy for Limited English Proficiency persons (n = 5 organizations), nutrition health literacy (n = 5 organizations), and mental health literacy (n = 3 organizations). The funded projects were not defined as research studies. Instead, CBOs provided information, services, resources, and other community supports, often engaging community health workers (CHWs), including Promotoras (health promoters), as trusted messengers to build health literacy during the COVID-19 pandemic. Throughout the funding period and as part of the project’s capacity building efforts to address challenges faced by participating CBOs, the lead coordinating university convened CBOs and provided technical assistance through monthly one-on-one video-conferencing meetings between grantees and their program officer—a role held by the coordinating university’s staff. One in-person meeting was held. Per the funder’s requirements, data were collected by CBOs to support quarterly reporting and overall program evaluation. A second university led the evaluation and provided evaluation-focused technical assistance to CBOs.

Sub-Study Measures and Data Collection

Data for this sub-study are derived from a brief voluntary interviewer-administered survey conducted with CBO representatives at the conclusion of the service delivery period. This analysis explores how the AHL initiative could have better addressed communities’ and CBO grantees’ needs. In addition, participants were asked to reflect on how academic institutions could better support partnerships with CBOs and local government and to share lessons learned from this initiative. Finally, participants were asked to provide comments if not previously covered; responses were included in this analysis if pertinent to the aforementioned topics. Preliminary findings were shared with grantees during a member-checking event to obtain confirmation, clarification of findings, or further details not previously mentioned; any comments were documented verbatim and are included in this analysis (Koelsch, 2013). This evaluation was approved by the university’s Human Subject’s Research Protections Program.

Participants

To ensure consistency in the interview process, the first author interviewed the directors, co-directors or other representatives (n = 20 persons) of funded CBOs (n = 12 organizations) which ranged from grass-roots organizations to larger institutions, including an FQHC and a homeless shelter. Interviews were conducted between April and June 2023 and lasted ~60 minutes; participants’ responses were entered verbatim into the survey software by the interviewer. Eleven interviews were conducted in English and one in Spanish. The interviewer [first author] and co-author [senior author] are bilingual and questions were translated during the interview; responses were transcribed verbatim in Spanish; illustrative quotes were translated into English by the interviewer and reviewed by co-authors for accuracy.

Qualitative Analysis

We utilized the “Coding Consensus, Co-occurrence, and Comparison” method (Willms et al., 1990), which is based in grounded theory methods (Glaser et al., 1968). Participants’ responses, ranging from a few words to multiple phrases, were imported into an Excel spreadsheet for analysis. Preliminary codes were assigned to text segments ranging from phrases to several paragraphs. This open coding was based on a priori (deductive) and emergent (inductive) topics (Corbin & Strauss, 2014). Some responses were assigned to multiple themes. The preliminary results were shared with participants during one session and their feedback regarding the coding or interpretation of the findings is reflected in Tables 1–3—these provide the main themes, sample sizes for each theme, sub-themes, and illustrative quotes (Carlson, 2010; Miles & Huberman, 1994).

OPEN IN VIEWER

Table 1.

Community-Based Organization Grantees’ Perspectives on How the Local Advancing Health Literacy Program Could Better Address the Needs of Grantees (n = 20 Participants for 12 Organizations), 2023.

Theme and summary	Illustrative quotes
1. RESOURCES (n = 12) 1.1 Trainings (n = 2): CBOs may need ongoing trainings, including a “Grants 101” to address post-award grant management 1.2 Funding and Time (n = 6): Multi-year funding is needed (for community’s needs, new staff salaries, to achieve goals, to train community workers) 1.3 Tangible resources (n = 2): a. The program generated tangible and intangible resources that will help communities b. In-language resources need to be appropriate to the target community from the beginning of the grant processes/expectations 1.4 Student participation (n = 2): Ensuring students (including bilingual students) are a good fit for the CBO, understand the work assigned is important, and CBOs need to understand the university’s expectations of students and CBO	1.1 . . . I would have preferred ongoing trainings to support engagement because it was a new program; for example, providing trainings once per month 1.1. I personally need training, and this is our first restricted grant, and it has been difficult for me to ensure that things are being spent properly. It would be cool if you have a basic Grants 101 since this grant is a lot more accountable—I have not previously had to do meetings and this is a good thing—previously I just received the funding without restrictions. I can’t give enough Kudos to [Program Officer]—she has been amazing to us! 1.2. Multi-year funding approach would be very useful; this program is not funded for the coming year, and it will be absorbed into general operating expenses in order to continue. This program allowed us to create a new role which was funded through the program, and [former] participant’s time is partially funded as well, so now [we are] finding new resources to fund these components through current funds 1.2 A longer time period—it takes a while to figure out the right formula- it wasn’t even 12 months to get all of the [grant] pieces aligned. 1.3 We did not know how much training we needed to give to the community (we thought community members were on stage 2 of understanding the internet and how to navigate it) whereas they are on stage 1— [they need help with] how to turn on the computer, how to create an email, etc. So, if you look at our original proposal, we said we would provide 4 trainings and have actually provided 14 trainings. 1.3 Recognizing that the staff needs to work in Spanish, [it would be highly beneficial] if all documents and materials were provided in Spanish from the inception of the grant. This should be a standard operating procedure. Speaking Spanish is not equivalent to comprehending it, and having the most accurate translation is crucial. We don’t want to be excluded in the future, and we now understand the need for a bilingual evaluation expert on board. [Translated from Spanish] 1.4 Communication—with [monolingual] groups like ours . . . For example, when we expressed ourselves, we assumed that they [bilingual students] understood us but when they did the work, it was not what we had requested and intended. [Translated from Spanish]
2. EVALUATION ( n = 8) 2.1 Clear Communication (n = 2): Academic partners need to clearly communicate regarding the evaluation process (timelines, data needed, software used, outcomes measured) 2.2 Data Dashboards and Collection Tools (n = 3): CBOs need to understand the outcomes being measured and how to design data collection tools so the data can be accessed internally and also to have it made available to grantees at monthly check-ins 2.3 Time (n = 3) is needed to assess long-term impacts of programs	2.1 We had a delay in the evaluation and I wish we could have done it from the beginning– that could have helped us. I had made evaluation questionnaires and when they added the data to the system Survey Monkey, the questions were changed and they had to embed questions after [evaluation team] was added. . . That was a delay for us– we had to translate all of the surveys into 4 languages. They gave us the time to switch and then things went more smoothly 2.2 One thing would be software to simplify the data that we collected; we wanted to have it internally as an excel spreadsheet and also give it to the evaluation team and it would be good to have a better way to look at the data as a whole within our organization. We compile it manually and we asked the evaluation team how they analyze it to learn about how to analyze the data more efficiently 2.3 Looking at advocacy work is a long-term goal/process, so that is a conversation that was started w/ the evaluation team but did not make much progress and is an area of interest

OPEN IN VIEWER

Table 2.

Lessons Learned From Participating in the Local Advancing Health Literacy Program That May Benefit Other Communities, Policy Makers and Researchers Working to Advance Health Literacy Among Ethnic and Underserved Communities, Perspective of Community-Based Organization Grantees (n = 20 Participants for 12 Organizations), 2023.

Theme and summary

Illustrative quotes

1. RESOURCES FOR CBOs ( n = 7): 1. CBOs require more time to plan and implement new programs and engage community in new programs 2. CBOs may need access to health literacy resources [newsletters, toolkits] that can be tailored by for their community 3. Sustained funding is needed to ensure ongoing impact and that the community’s trust is not broken 2. COMMUNITY ENGAGEMENT AND TECHNICAL ASSISTANCE ( n = 9): 1. Cross-grantee collaboration could have supported broader impact and cross-learning 2. Community health workers can bring knowledge and resources to the community but may need specialized training to do their job effectively 3. Underserved communities have multiple needs but will engage if they are provided with information and tailored resources 3. OUTCOMES ( n = 4): 1. Community organizations can engage community members as credible messengers to increase positive health outcomes. 2. Structured programs that require monitoring and evaluation can benefit the CBOs 4. POLICY ( n = 3): Communities are diverse; policy-makers can be educated through data

1.1. . . . This whole project was a brand new—which is why I emphasize needing more time—it took time to get the word out and the incentives—it’s different from adding to an existing program rather than starting from the ground up. . . . We had to construct a new everything! 1.2 For example, Covered California has a toolkit—it would be helpful to have a health literacy toolkit that can be presented to the organizations as a starting point for their own work so they can adapt it to their organization. 1.3 In terms of capacity building and within the short time frame, we touched a lot of lives and people, but what will happen when the funding is gone—how can we continue the project which has been successful? It takes a while to build trust in the community and constituents– once that trust has been built, we want to let the community know that we are there to support you. . . . 2.1 I have also learned that collaboration with other like-minded agencies is a good thing—I tend to run solo—so I have learned to collaborate with persons that are also working in this area 2.2 [It is important to] improve the role of community health care workers (CHW) and increase their knowledge of the health care system through increased training because they have knowledge but they don’t know the health care system (for example, CHW was never in a clinic and does not understand how the clinic operates or understanding the Medicaid-Medicare patients). So, we give the CHWs a tour of the clinic as if they are the patient so they understand the processes and departments. For example, COVID-19 vaccination is for all but other vaccines are only for our patients 2.3 Of all things we learned, language is important, not just the English or Spanish, but also culturally didn’t fit. Some COVID materials were put out by the state of California but the partners said this is not how you would speak. The translations were awkward and not understandable. It recognized not just language fluency but literacy levels—it’s a reminder to be really mindful that materials need to be at a lower reading level for everyone to have access to them. 2.3 1—How to best access health literacy—a lot of people don’t know how to schedule appointments or select a provider. They didn’t understand insurance and how these companies work. 2—Persons with low health literacy experience major obstacles in navigating the health system. 3—Addressing the gap between health care challenges (for example language, region of service delivery) and the needs of underserved patients who have complex health related needs . . . 3.1 It’s not just about 100 or 200 persons; it’s about the impact we have. Our initial goal was to enroll 500 persons, but instead, we successfully enrolled 1,200 persons. When a promotora [community health worker] goes out into the field, it makes a significant difference. We are not only advocating for ourselves but also for other community groups. We want these groups to be appreciated for their direct work in the community. [Translated from Spanish] 3.2 . . . . I have learned that this format is more structured and accountability is good in this situation; there are things we don’t know about advancing health literacy 4. People in the underserved communities are not treated well- for whatever reason– this needs to be considered and part of the policy. . . . At the federal level and the city level underserved communities are not treated well. Everyone categorizes homeless but there are different levels of homeless—seniors can’t pay rent, because of domestic violence, chronic addictions, so many aspects but they are grouped into one and it’s unfair to each of these groups because they are grouped as one ‘community’ . . . They need to see the individual need of the person and not clump them together as the ‘underserved’. It needs to be broken down a bit and it starts at the top

OPEN IN VIEWER

Table 3.

What Academic Institutions Can Do To Support Collaborations Between CBO’s and Academic Partners, Perspective of Community-Based Organization Grantees (n = 20 Participants for 12 Organizations), 2023.

Theme and summary	Illustrative quotes
1. COMMUNITY ENGAGEMENT AND PARTNERSHIPS ( n = 6) 1. Universities need to have sustained and authentic relationships with CBOs to ensure projects meet the community’s needs, do not duplicate efforts, support student development as community-engaged leaders, and advance public policy 2. CBOs can benefit from 1:1 meeting with academics and partner-wide meetings for information sharing, networking to better serve the community	1.1 I go back and forth b/c our communities get evaluated all the time and provide feedback on what is going on in the community and there is not a result of this input provided—for example—this is how the changes have been made . . . The more we can build authentic relationships and the more that [UNIVERSITY] is participating in real time meetings to hear about what community is asking for– then relationships have been developed and community members can see how their participation is impacting a program or changes in the community. Sometimes there is duplication of existing efforts led by community or the efforts of the academic institution supersedes the interests of the community b/c the academics are taken more seriously by government agencies 1.1 I am thinking out loud. We have historically worked with foundations, donors, businesses, churches and an area that we have historically missed is universities and you have the next generation and I am excited to work with you to leverage your wisdom and to get the next generation of adults to get involved with community issues . . . We want community leaders to care about community and getting involved in something that means something to them. More of an ongoing relationship. It may result from quarterly volunteer positions (for ex having social workers volunteer in the program, or shadowing a social worker, or maybe an education piece where you invite different nonprofits to come in and discuss how they will address their community problems. We love to have volunteers, have people come out and to educate! . . . 1.2 Continuing the engagement and meetings helps. I do like having the individual engagement—because you can speak to the individual needs of your own organization, but having the collaborative come together is good because you can learn about other strategies that have been implemented that have been successful that your organization might implement. The universities can be the conduit to bring the CBOs together
2. GRANTSMANSHIP AND FUNDING ( n = 7) 1. Universities have the infrastructure to acquire funding and should secure flexible funding to support communities’ health literacy needs 2. Universities can engage CBOs to understand the intent of funding and relevance to their organization and evaluation to ensure programs meet the community’s needs, the funder’s objectives, and evaluation needs 3. Sustained partnerships with CBOs will ensure that a broader sector of organizations are aware of new funding opportunities	2.1 I think that academic institutions that are large and have capacity got a huge grant that our organizations could not have competed for or been awarded, and you were able to obtain it and deploy it to the community. 2.1 Looking for more funding! Funding is number 1—flexible funding that makes sense and meets the needs of the community—mental health is number 2, housing is number one, child development is number 3 and culturally competent services everywhere 2.2 I think we had to look up health literacy; we saw the funding opportunity several times and we didn’t think it was applicable to us; being clear on what is the funding and who it’s for in the RFP 2.3 More CBOs could be reached if the project intent is clear from the beginning. The preproposal conferences may have taken place before we learned of the opportunity. [Program Officer] held office hours and joining those helped us understand that we were a good fit for this project. Casting a wider net could help. I don’t believe the RFP conference was made available afterward (for later viewing) on a website
3. TECHNICAL ASSISTANCE ( n = 8) 1. Universities can bring their expertise to the community through technical assistance and trainings 2. Help CBOs build digital literacy to support their work 3. Help build CBO’s evaluation skillset (e.g., survey design, accessing/using data)	3.1 [Universities can] make experts available. I know all of the universities have adjuncts and they are looking for classes to teach and that they could have CBOs and be paid as if they are teaching a class . . . for a CBO this is professional development and workforce development 3.1 Also training on addressing future pandemics and natural disasters 3.2 . . . .For example, there is a QR code—I just learned what a QR code is and some of our clients are 80 years old, so there is a computer literacy issue 3.3 [Universities can] Provide relevant trainings: for example, how to use technology to collect surveys the way that large organizations do; now we have an evaluation consultant and she will help us with this since you have data and want data [Translated from Spanish] 3.3 A lot of people think that brining this knowledge that academic people know to the community . . . for example, the meeting we had with the mapping was very useful because we were not aware about [this resource] before and it helped me navigate the mapping system (Healthy Places Index) in specific towns. Now we can map our own city! Collect information about the cities that we serve—COVID vaccination, cases, diagnoses by gender

Local Health Literacy Programs Can Address Communities’ Needs

Table 1 describes how the local AHL initiative can respond to communities’ needs, highlighting community-engaged strategies for addressing challenges faced by participating CBOs. Regarding the “Resource Needs” theme, respondents indicated that CBOs can benefit from pre-award trainings, such as grant-writing to increase competitiveness for funding as well as post-award activities pertaining to implementation and evaluation activities (see 1.1). CBOs reported needing multi-year funding to support program development and implementation that would have a lasting impact on health literacy goals (see 1.2). Since CBOs vary in their size and in-house resources, grants that fund tangible resources (e.g., computers), staff trainings, or which provide foundational resources for limited English proficiency persons can be especially helpful to smaller CBOs (see 1.3). Student involvement in health literacy programs was viewed favorably, though student roles, time, and nature and level of supervision expected of CBOs should be clearly defined at the outset by partnering universities (see 1.4). In addition, one CBO reported that it is important to confirm that bilingual students’ language skills align with the CBO and community member cultural and linguistic needs (see 1.4).

The second emergent theme pertained to evaluation activities. CBOs reported that having strong evaluation protocols would benefit grantees and their communities and they identified a need for detailed communications from grant-makers vis-à-vis type of data needed, timelines, and grant-related procedures when initiating a new program (see Table 1, 2.1). CBOs also consistently identified a need for assistance in developing data collection systems, including technical assistance to select appropriate outcome measures and data collection tools to ensure internal program monitoring (see 2.2). CBOs working with external evaluators would like real-time access to their data dashboards. Finally, CBOs indicated that longitudinal evaluations are needed to document the long-term impacts of investments in health literacy programs that integrate community-based approaches and collaborations with CBOs (see 2.3).

Lessons Learned From Participating in the AHL Grant Program

CBOs reflected on lessons learned from participating in this initiative; Table 2 describes emergent themes and three sub-themes. CBOs consistently noted the need for additional resources, including time to navigate organizational administrative processes related to program planning and implementation (e.g., coordinating across departments); grantees that launched new programs needed additional time to recruit and train new staff and develop client-facing materials (see 1.1). CBOs’ familiarity with health literacy concepts varied, thus, having access to foundational materials that could be tailored, including in-language resources, could facilitate program implementation (see 1.2). Finally, CBOs noted that funding time-limited programs may constrain building health literacy in the community; instead, sustained funding may support long-term change and foster trust between CBOs and communities served (see 1.3).

Regarding community engagement and capacity building, CBOs noted that cross-grantee collaboration can benefit CBOs and the communities they serve; information can be shared across CBOs and community members can be better served through cross-agency referrals to holistically address social determinants of health and related needs (see 2.1). CBOs noted that CHWs (e.g., promotoras) are very successful in engaging community members, however, subject-matter expertise should not be assumed, and programs should prepare to tailor trainings regarding health literacy and other areas (e.g., eligibility for public programs, health systems operations) to enhance staff members’ effectiveness (see 2.2). Participants noted that community members will engage with services if they are accessible (i.e., geographically, in-language, appropriate, providing foundational information and resources) and tailoring of resources to local community’s characteristics should be considered to support resource uptake (see 2.3).

CBOs consistently expressed a strong interest in assessing programmatic outcomes, noting that CBOs and CHWs can help funders and programs, respectively, achieve their intended outcomes due to being credible messengers and trusted sources of information and resources for the diverse communities they serve (see 3.1). Furthermore, grants that require structured monitoring and evaluation activities can enhance CBOs’ abilities to achieve their immediate and long-term goals (see 3.2). Finally, lessons learned related to policy emerged; some participants reported that data can be leveraged to inform policy-makers about the unique needs of population subgroups so that funding streams can be designed to achieve the greatest impact (see 4).

Academic Institutions Can Support Collaborations With CBOs

Three main themes emerged when participants were asked to reflect on how academic institutions can support partnerships with CBOs (see Table 3). The importance of meaningful community engagement and partnership with CBOs and community members was a significant theme. Participants indicated that engaging early and often with the community can support relationship-building and ensure that academic partners are familiar with the community’s needs and that community needs are centered during intervention planning, delivery, and evaluation. This approach would facilitate multisectoral responses to funding opportunities, avoid duplication of efforts, support the development of students whose training has a community-focused lens, and help inform public policy (see 3.1). Meaningful relationships between academic institutions and CBOs can be cultivated and strengthened through regular meetings that involve bi-directional information sharing, co-leadership and networking opportunities (see 3.2).

Regarding grantsmanship and funding, participants mentioned that they face challenges in competing for grant funding due to limited capacity and experience in grantsmanship. They noted that this is an area of expertise for universities due to the resources needed to compile a successful application (e.g., federal DUNS number, fiscal management teams, use of data in proposals, evaluation expertise in pre- and post-award phases) (see 2.1). CBOs felt that they can benefit from technical assistance and training to support all aspects of grant procurement (e.g., gaining awareness of funding opportunities, understanding RFPs, determining how projects of interest align with funders’ goals, proposal development and post-award grants management and evaluation support, sustainability planning). In addition, CBOs reported a need for flexible funding to address health literacy projects’ diverse needs (e.g., staffing, training, program supplies, evaluation support) (see 2.2). CBO representatives expressed that universities that regularly engage with CBOs can readily disseminate funding opportunities and strategize to build effective partnerships (see 2.3). Finally, while CBOs vary in their scope and size, participants reported an interest in having universities share expertise by supporting capacity-building efforts around digital literacy and evaluation for local nonprofits (see 3.1, 3.2, and 3.3).