Intersection of Mixed Methods and Community-Based Participatory Research: A Methodological Review

Abstract

Although mixed methods research (MMR) and community-based participatory research (CBPR) have been employed to investigate complex research questions to improve the reach, rigor, and relevance of research, little is understood about the intersection of the approaches. We conducted a methodological review of studies (n = 129) using both MMR and CBPR, an advanced application we refer to as mixed methods community-based participatory research (MMCBPR). We systematically examined published MMCBPR studies to identify the methodological features and use in current research. Findings demonstrate that the components of MMR were not adequately described although some detail was provided about the use of CBPR. This study contributes to the evolution of advanced applications, and we offer recommendations for future applications of MMCBPR.

Keywords

mixed methods research community-based participatory research research paradigms methodological review

Mixed methods research (MMR) has gained prominence as a comprehensive methodological approach that incorporates both qualitative and quantitative methods in all phases of a single study. Though MMR designs can vary greatly, the core principle of the methodological approach is the integration of qualitative and quantitative methods in data collection, analysis, and inferences to generate a deeper understanding of a research topic. The integration of objective, generalizable quantitative data with descriptive, contextual qualitative data are emphasized in a contemporary paradigm that values diverse philosophical perspectives as a means to enhance the researcher’s ability to interpret very complex phenomena (Creswell, Klassen, Plano Clark, & Smith, 2011; Johnson & Onwuegbuzie, 2004; Teddlie & Tashakkori, 2003, 2009). A great deal has been written about the utility and implications of MMR; however, further examination of integrated paradigms or perspectives is necessary to advance its application in MMR studies (Shannon-Baker, 2016).

To expand the field of MMR, leaders have called for additional study of how the methodology intersects with other research approaches (Hesse-Biber & Johnson, 2013; Plano Clark & Ivankova, 2016). These intersections have been referred to as embedded, nested, and advanced designs due to the way in which a core MMR design is integrated with another research design, methodological approach, or theoretical framework, such as evaluation research or experimental designs (Creswell, 2015; Creswell & Plano Clark, 2018; Plano Clark & Ivankova, 2016). In their recent book, Plano Clark and Ivankova (2016) refer to these hybrid approaches as “advanced applications of mixed methods designs” (p. 136) and note an additional level of complexity within them, especially since integration within mixed methods itself is not always achieved (Bryman, 2006). Given the dynamic components and variation in theoretical and philosophical viewpoints of advanced designs, a key recommendation in the National Institutes of Health’s Best Practices in Mixed Methods Research in the Health Sciences (2011) is to clearly articulate and portray how the multiple components work together and to what end. Creswell (2016) also notes that the real-world implementation of these advanced designs warrants new ways of conceptualizing and actualizing hybrid designs.

One such advanced application needing further investigation is the intersection of MMR with participatory and action research approaches (Hesse-Biber & Johnson, 2013). Plano Clark and Ivankova (2016) describe the emergence of applications that integrate MMR core designs with these methodological approaches and note a need to understand their use across disciplines. Ivankova (2014) has examined designs that intersect MMR and action research and offered standards for implementation and quality based on an initial literature review of the hybrid design. Her search included several terms that span a broad continuum of community engagement in research; however, it excluded a major participatory methodological approach in the health and social sciences, community-based participatory research (CBPR).

CBPR is an orientation to research that equitably engages communities in the design and implementation of research that benefits the local setting (Israel, Eng, Schulz, & Parker, 2013). Partnering with community members, community-based organizations, community leaders, and practitioners allows for the research design to be grounded in diverse perspectives and expertise and to be more responsive to the needs of the community. CBPR is built on core principles that foster equitable partnership, capacity building, co-learning, and action toward health and social justice. CBPR falls on the far end of the continuum of community engagement in research, where community members are fully involved as research partners who engage in shared decision making and “co-research” as opposed to simply cooperating with or serving in an advisory role to research led by academicians (Jacquez, Vaughn, & Wagner, 2013; National Institutes of Health, 2011; Vaughn et al., 2017a; Vaughn et al., 2017b).

Though initially developed within public health, CBPR is increasingly being promoted in health and social behavioral fields (Israel et al., 2013; Minkler & Wallerstein, 2010), and scholars have noted the potential for MMR to advance the CPBR approach and vice versa within these disciplines (Lucero et al., 2018). Thus, we believe that this is the optimal time to initiate a review of studies that explicitly cite the use of MMR and CBPR to examine the current intersection of the two approaches within published studies, to fill the current gap in the literature, and to inform future research. Specifically, there is a need to understand the ways in which researchers are intersecting MMR with CBPR, identify the rationales for using this design, and describe current challenges in order to guide future researchers who use this advanced application.

In this methodological review, we present the results of a literature search of peer-reviewed articles where both MMR and CBPR were used—an approach that we refer to as mixed methods community-based participatory research (MMCBPR). The purpose of this review was to systematically examine and report how the two approaches intersect in empirical studies, including methodological features and the reported rationale for using MMCBPR advanced designs in the health and social sciences. Specifically, we seek to answer the following research question: How is the advanced application of mixed methods research and community-based participatory research being applied in the health and social sciences? Our aims are to:

describe the nature and scope of MMR that intersects with an explicit CBPR approach.

articulate the purpose and design of published MMCBPR studies.

characterize the methodological contribution of MMCBPR to mixed methods methodology and participatory research designs.

We intend to provide students, researchers, and reviewers of scientific manuscripts with an understanding of MMCBPR practices that are currently being applied in the health and social sciences, in order to identify opportunities for strengthening MMCBPR study implementation and publications.

Method

For the purpose of this study, we followed guidance put forth by Molina-Azorin and Fetters (2016) to examine studies that explicitly used both an MMR and a CBPR approach in a single study. We considered MMR to be a methodology that employs qualitative and quantitative data collection and analysis techniques and explicitly combines the methods to draw on the strengths of each approach (Creswell et al., 2011; Creswell & Plano Clark, 2018). Studies that collected and integrated qualitative and quantitative data without using the terminology of “mixed methods research” were also included in the analysis. CBPR was defined as an approach to research that partners with community members throughout the full research process (Israel et al., 2013). For this review, we included studies that collaborated with stakeholders in any phase of the research process (e.g., priority setting, research design, data collection, data analysis, interpretation, dissemination, etc.) and that explicitly claimed the nomenclature of CBPR. Though similar terms are used for related approaches across a variety of disciplines, we excluded studies that did not name their approach as CBPR to understand the prevalence of this widely used discipline in particular.

Procedures

We searched PubMed, Scopus, and Web of Science databases using keyword searches. Search terms were (mixed method* AND community-based participatory research) and (quantitative AND qualitative AND community-based participatory research). The key term mixed method with a wildcard (*) was used and suggested in previous studies (Coyle et al., 2016; Plano Clark, 2010). We conducted the search twice between August 2016 and June 2017. The research team manually screened the resulting records for duplication, full-text availability, and language.

Selection Criteria

We included all empirical articles (no conference proceedings, protocols, book chapters, theses) that were published in English in peer-reviewed journals through June 30, 2017. Articles were included in the review if they met the definitions of both MMR and CBPR.

Data Screening and Eligibility

The searches resulted in 1,145 total records. We extracted author(s), title, journal, and publication year from each manuscript identified in the database searches. After removing duplicates, there were 381 publications. Twenty-one records were unavailable in full text, and two were unavailable in English and were removed from the data set. The remaining 358 were assessed for eligibility (Figure 1).

Figure 1.

Application of MMR in CBPR Phases.

We created a codebook to assess eligibility and to conduct the review, with categories for the extracted data displayed in columns (authors, title, journal, year). We added the following column categories for each publication to address our methodological aims: general components of the study (i.e., type of study, overall purpose, population, location, topic), components of MMR (i.e., MMR approach; MMR rationale; MMR integration at the design, methods, and reporting levels; quantitative data source; qualitative data source), components of CBPR (i.e., CBPR approach, phase in CBPR process, CBPR rationale, research partnership), and MMCBPR impact on study findings. Prior to analysis, initial definitions and codes were developed for each of the categories based on existing literature (Table 1).

Table 1.

Definitions of Categories and Codes.

Category	Description
Empirical study	An empirical study includes data collection and analysis and may include experiments, interventions, assessments, and evaluations. Literature reviews and methodological studies are not included.
Purpose	Overall aim(s) of the study
Population	Group of people or community being studied
Location	City and country where data were collected
Topic	Content area of the research questions
MMR approach	A mixed methods study includes the data collection and analysis of both quantitative and qualitative data and the purposeful integration of the two data sets (Creswell & Plano Clark, 2011).
MMR rationale	Stated purpose for integrating both quantitative and qualitative data in the same study
Integration	Intentional mixing of quantitative and qualitative databases to gain a deeper understanding of the research questions. Integration at the methods level was coded using the following definitions (Fetters, Curry, & Creswell, 2013):
Integration at the design level	• Convergent: Quantitative and qualitative data are collected and then merged following analysis • Explanatory sequential: Quantitative data are followed by qualitative data • Exploratory sequential: Qualitative data collection and analysis are followed by quantitative data collection and analysis • Multistage: Three or more stages, with any combination of sequential and convergent designs
Integration at the methods level	• Connecting: Linking one database to another through sampling • Building: Using one database to determine the data collection approach of the other • Merging: Bringing the two databases together for comparison • Embedding: Linking the two databases at multiple points through data collection and analysis
Integration at the interpretation and reporting level	• Narrative: Describing the quantitative and qualitative findings in one or more reports • Data transformation: Converting one type of data to the other type (e.g., quantitative to qualitative) and then integrating with a data set that has not been transformed • Joint display: Representing integration through the visual linking of quantitative and qualitative data
Quantitative methods	Source(s) for quantitative data
Qualitative methods	Source(s) for qualitative data
CBPR approach	The term community-based participatory research is used, referring to an approach to research that equitably involves community members in all aspects of the research process (Israel et al., 2013).• Engagement includes, at minimum, getting feedback on topic, research questions, procedures, or findings • Engagement does not include only data collection from a community sample
CBPR phase	The CBPR approach to research includes seven components (Israel et al., 2013):1. Forming a CBPR partnership 2. Assessing community strengths and dynamics 3. Identifying priority local health concerns and research questions 4. Designing and conducting etiologic, intervention and/or policy research 5. Feeding back and interpreting the findings 6. Disseminating and translating research findings 7. Maintaining, sustaining, and evaluating the partnership
CBPR rationale	Purpose for using a CBPR approach
CBPR partnership	Community members who collaborated with academic researchers in the study
MMCBPR benefit	Statement of added value of integrating MMR and CBPR in a single study

Note. MMR = mixed methods research; CBPR = community-based participatory research; MMCBPR = mixed methods community-based participatory research.

After defining the codebook, we selected 10 articles to review and code independently. We then met to review our codes, resolve any disagreements, and reach consensus. We continued to review independently coded articles at each of our remaining meetings. For each batch of coding, members of the research team were assigned 10 unique articles (to be only coded by one individual) and 5 shared articles to be analyzed by the group (coded independently and then discussed as a group). The iterative process allowed us to modify our initial codebook by refining category definitions and codes and ensured coding consistency among team members. New definitions and codes were used when reviewing subsequent articles.

Of the 358 records, the research team reviewed, discussed, and reached consensus on the codes assigned to 35 articles (10% of the total reviewed). Records were then divided among the research team, and each member was assigned 20% of the total (approximately 72 articles per person). For each of the 358 records included in the analysis, we examined and coded the full articles. Through this systematic process, 229 articles were removed for not meeting the inclusion criteria (28% were nonempirical, 51% were not CBPR, and 21% did not meet the definition of MMR). A total of 129 articles met our inclusion criteria and were included in the data synthesis.

Data Synthesis

Patterns in the codes were analyzed inductively and summarized to examine and describe the methodological components of MMCBPR studies and characterize the contribution and challenges of MMCBPR. Following synthesis of the data, we reviewed all MMCBPR studies to identify exemplars. Exemplars were selected for their inclusion of statements for each of the components defined above (rationale, MMR approach, CBPR approach, etc.).

Results

A total of 129 articles were included in the methodological review. We intentionally did not limit the time frame in our search to provide a broad review of MMCBPR to date. As a result, we found that MMCBPR studies were published from 2004 to 2017 and were primarily published after 2010 (n = 112; 90%). However, after a steady rise in MMCBPR studies, the largest number was published in 2013, and there has been a subsequent downward trend since then (Figure 2).

Figure 2.

MMCBPR studies by year of publication.

In the following sections, we present findings from the methodological review, including frequencies and descriptions of methodological components. We also include qualitative excerpts from exemplar studies to highlight key results.

MMCBPR Topics and Population

Although there is evidence that researchers in a variety of settings have used MMR in CBPR approaches, the MMCBPR studies included in the current review predominantly focused on topics located in the health sciences (n = 97; 75%), with the most represented topics including obesity, food, nutrition, and physical activity (n = 19; 15%); behavioral health (n = 19; 15%); and sexual and reproductive health (n = 13; 10%). Other health-related topics and those located within the social sciences—such as aging, homelessness, and criminal justice—were also represented in MMCBPR studies and are further highlighted in Table 2. Additionally, 18 (14%) of the MMCBPR studies focused on the community-academic partnership itself, including evaluations of the collaboration, partnership functioning over time, and outcomes for community partners. It should be noted that these partnerships may have been established to address a specific health or social science issue; however, for the purposes of this review, when the focus of the reported MMCBPR study was on the partnership itself, it was coded as such.

Table 2.

Topics Represented in MMCBPR Studies.

Topic	Examples	n	%
Obesity related	Food, nutrition, physical activity	19	15
Behavioral health	Mental health, substance use	19	15
Community-academic partnerships	Research methods, evaluation	18	14
Social sciences	Education, criminal justice, civic engagement, homelessness	15	12
Sexual and reproductive health	HIV prevention, STD prevention, reproductive health services	13	10
Chronic disease	Asthma, cardiovascular disease, diabetes	9	7
Cancer	Cancer screening	9	7
Health needs and disparities	Community health needs and priorities	9	7
Health systems and policy	Transitions in care, acute care policy, primary care, medical education	8	6
Health knowledge, beliefs, and behaviors	Health promotion, health education	6	5
Environmental and occupational health	Built environment, lead exposure, safety	5	4

Note. MMCBPR = mixed methods community-based participatory research; STD = sexually transmitted diseases.

The large majority of the MMCBPR studies were conducted in the United States (n = 102; 79%) or Canada (n = 17; 13%) and investigated research questions related to historically marginalized and vulnerable populations. With regard to racial and ethnic minorities, MMCBPR studies mostly focused on the health and well-being of predominantly African American (n = 19; 15%); Latino (n = 12; 9%); American Indian, Inuit, or Aboriginal (n = 9; 7%); and immigrant or refugee (n = 8; 6%) communities. Of the studies not explicitly focused on minority or immigrant populations, other marginalized or vulnerable populations were represented, including low-income (n = 8; 6%), LGBTQ (lesbian, gay, bisexual, transgender, queer; n = 5; 4%), and rural (n =4; 3%) communities. Furthermore, the largest number of studies (n = 28; 22%) included more than one of the categories listed above—for example, low-income African American and Latino families. Other populations were also represented in the MMCBPR studies, including representative individuals from professional organizations (n = 10; 8%) and topic-specific populations such as individuals who were homeless or living with HIV/AIDS (n = 9; 7%). Notably, 40 studies (31%) had a primary population of children and/or adolescents.

Mixed Methods Language and Rationale

Table 3 displays the mixed methods features identified in the studies. Of the 129 studies included in the current analysis, 52 (40%) indicated the use of quantitative and qualitative data in their studies but did not use the term mixed methods nor any other related terms. More than half of the studies (n = 77; 60%) identified their study as mixed methods, but only 64 of these (50% of total studies) provided some description of how mixed methods was employed. The other 13 studies (10%) only mentioned mixed methods in the abstract or list of keywords. Surprisingly, only 20 of the total 129 studies (16%) cited mixed methods references when naming the approach.

Table 3.

Mixed Methods Features of Published MMCPBR Studies.

Mixed Methods Features	n	%
Named MMR by author(s)	77	60
Description/definition provided	64	50
Cited MMR literature	20	16
Abstract/keywords only	13	10
Provided MMR rationale	44	34
MMR design stated by author	6	5
MMR basic designs
Convergent	47	36
Explanatory	16	12
Exploratory	16	12
MMR other designs
Intramethod	25	19
Multistage	17	13
Not adequately described	7	6
Integration
Merging	110	85
Connecting	93	72
Building	40	31
Embedding	13	10
Interpretation and reporting integration
Narrative	115	89
Data transformation	19	15
Joint display	17	13

Note. MMCBPR = mixed methods community-based participatory research; MMR = mixed methods research.

In one exemplar, John and Gunter (2016) name their mixed methods design and used mixed methods language throughout the article. For example, the authors wrote,

Data were collected using mixed methods and an explanatory sequential design (Creswell & Plano Clark, 2011). A quantitative representative sample survey of the Clackamas County adult population was followed by qualitative methods involving middle-aged and older adult residents in mapping environmental assets in their local communities using participatory photographic surveys. The qualitative phase was given priority because a key objective of community mapping was to partner with and engage older residents in explaining how and why environmental assets contribute to their lived experience of community age-friendliness, developing collaborations and designing assessment-driven projects to improve local age-friendly options. (pp. 1098-1099)

In contrast, a notable proportion of MMCBPR studies did not use mixed methods language or state a reason for integrating quantitative and qualitative data (n = 85; 66%), including half of the 77 studies (n = 39; 51%) that directly identified using MMR. Among studies that described a rationale, the most common rationales were to obtain a more complete understanding of a phenomenon, develop a more appropriate or effective intervention, build a more relevant instrument, enhance interpretation of results, or add rigor to the study design.

For instance, Christopher et al. (2008) used a mixed methods evaluation design to assess changes in knowledge, comfort, and cancer awareness levels following an intervention that used lay health workers to generate knowledge and awareness about cervical cancer prevention among Native American community members. Their rationale for an MMR design was that it “provides a more complete view of change in the community” and “can strengthen the validity of findings compared to using one type of result” (p. 822). In their study, the authors used analysis and integration of field notes, meeting minutes, and conversational data to add context to the statistical results of their evaluation design. In their conclusion, the authors returned to their rationale and concluded that the study was strengthened by the integration of quantitative and qualitative data:

. . . because this study employed a nonexperimental one-group pretest–posttest design, we are limited in our ability to unequivocally conclude that our CBPR and LHA intervention caused changes in knowledge and awareness on the reservation. However, a one-group pretest–posttest design used in conjunction with qualitative methods can be an effective way to make inferences about potential changes that take place in a population because of an intervention. The qualitative findings provide contextual information on changes resulting from the project and are consistent with our quantitative findings. (p. 831)

MMR Design

The studies included in this analysis were examined for their use and description of research designs that align with common MMR typologies: convergent, explanatory sequential, exploratory sequential, and multiphase designs (Creswell & Plano Clark, 2018; Fetters, Curry, & Creswell, 2013). Only six of the authors (5%) explicitly named their MMR core design: two were convergent, one was explanatory sequential, and three were exploratory sequential designs. Two additional studies were stated by the authors as being “sequential,” but they did not explicitly state which type of sequential design. Of the total studies, 47 (36%) used a convergent design, where quantitative and qualitative data were collected and analyzed separately and then merged together for data interpretation. Additionally, 16 (12%) were explanatory sequential, where the quantitative strand was conducted first and followed up with the qualitative strand, and 16 (12%) were exploratory sequential where the qualitative strand was followed by the quantitative strand. Finally, 17 (13%) of studies described multiple stages of mixed methods data collection, analysis, and integration throughout the course of a partnership.

Many of the MMCBPR studies used an intramethod design (n = 25; 19%), which has been defined as the collection of quantitative and qualitative data within a single method (Johnson & Turner, 2003). While there are differing opinions on whether these studies are truly mixed methods designs due to the use of a single data collection technique and data set, we included these studies based on the extent to which they appeared in articles included in this review. Of the intramethod designs, 14 (11% of total, 56% of intramethod) used a single questionnaire with both open- and closed-ended items, and one study conducted multidimensional scaling of photos and ratings collected as part of the photovoice method. Additionally, 10 (8% of total studies, 40% of intramethod studies) used concept mapping where qualitative data are transformed into quantitative data using multidimensional scaling and hierarchical cluster analysis. Windsor (2013) demonstrates the potential benefits of using concept mapping early in a community-academic partnership. In collaboration with substance users, service providers, and community residents, the research team used concept mapping to build capacity within their partnership and understand the impact of substance abuse on the community to inform future research.

MMR Data Collection

To understand how the MMR designs were implemented, we analyzed the different quantitative and qualitative methods used in MMCBPR studies. The collection of quantitative data consisted primarily of self-report instruments, such as surveys and pretest/posttest questionnaires (n = 102; 79%), ranking or rating tasks (n = 15; 12%), physiological measurements (n = 12; 9%), analysis of administrative and secondary data sources (n = 5, 4%), and observations or audits (n = 4; 3%). Qualitative data consisted primarily of focus group interviews and group discussions (n = 72; 56%), followed by individual interviews (n = 65; 50%), open-ended questionnaires (n = 17; 13%), observations/field notes (n = 11; 9%), documents or records (n = 10; 8%), and photovoice (n = 6; 5%). Of the 129 total studies, 44 (34%) used more than one type of qualitative method, including 5 of the 7 exemplar studies. In comparison, only 10 (8%) used more than one type of quantitative method.

MMR Integration

In addition to the type of integration that occurs through different MMR designs, MMCBPR studies were coded for integration at the methods and interpretation and reporting levels using definitions developed by Fetters et al. (2013) (Table 1). Of the 129 studies, nearly all (n = 110; 85%) described merging the quantitative and qualitative databases for analysis. The second most common integration strategy was connecting (n = 93; 72%), which involves linking one database to the second database through sampling. Approximately one third of the studies used building (n = 40; 31%), which occurs when one database informs the collection of the second database, such as using themes from focus groups to develop culturally relevant questionnaires. Last, 13 studies (10%) integrated data collection and analysis at multiple points, which is known as embedding. Although we described here the results of each integration approach separately, it should be noted that a large majority of the studies used more than one approach (n = 87, 67%). Far fewer studies used more than one approach when integrating at the interpretation and reporting level (n = 21; 16%). At this level, integrating through narrative in the discussion and conclusion sections of articles was by far the most common approach (n = 115; 89%). This was followed by data transformation (n = 19; 15%) and joint display (n = 17; 13%) approaches.

CBPR Rationale

In addition to MMR, we reviewed the 129 MMCPBR articles for their use and description of the CBPR approach. Notably, many of the articles were limited in their definitions and descriptions of CBPR implementation; therefore, for the purpose of this review, we did not analyze whether every principle of the CBPR was fulfilled as we discovered it was difficult to accurately assess the extent of the collaborative partnership in most articles. Instead, we relied on whether authors explicitly referred to their partnerships as CBPR and indicated involvement of community partners in addition to academic researchers.

Another way in which we examined the extent of CBPR implementation and potential benefits to MMCBPR research studies was by finding out whether a CBPR purpose statement or rationale was clearly stated. Of the total MMCBPR studies, 21 (16%) did not directly or indirectly state a CBPR rationale, while 108 (84%) included a rationale. Of these 108, 47 (44%; 36% of total studies) provided more than one rationale. The most frequently stated rationale was increasing the study impact (n = 40; 31%), particularly through the future development of sustainable and relevant interventions and programs resulting from the research. Other stated rationales included increasing participation and collaboration (n = 35; 27%); tailoring interventions or programs based on the input of community members (n = 28; 22%); obtaining local knowledge and feedback on the research process (n = 23; 18%); identifying community needs, resources, or priorities (n = 15; 12%); capacity building (n = 12; 9%); enhancing research quality (n = 10; 8%); and establishing trust with the community (n = 4; 3%). In the same study by John and Gunter (2016), the authors state, “We employed CBPR tools to empower people in communities to determine what supports residents have, need, and want to age actively and intentionally in their place of choice, and to use their voices and vision to encourage and enact change” (p. 1100). Through their description, we are able to discern that CBPR was selected to tailor programs based on the input of community members, obtain local knowledge, and increase the study impact through change.

CBPR Partnership

Equitable partnerships between academic researchers and various members of the community are the crux of CBPR. In our review of partnerships described in MMCBPR studies, we found that community partner involvement in the research process varied considerably across studies, and just over half (n = 73; 57%) indicated the involvement of more than one type of community partner. Regarding the specific types of partners, the largest percentage of studies (n = 61; 47%) described their community partner as “community members,” including community residents, patients, and individuals with a shared identity/experience to the defined community (e.g., construction workers, people with mental illness, medical residents, etc.). Thirty-four percent of studies (n = 44) partnered directly with community-based organizations, including faith-based organizations and nonprofits, whereas 24 (19%) of the research teams partnered directly with service providers, which were overwhelmingly health care providers in hospital systems. In 21 studies (18%), partners were described as an advisory board or council, whose purpose was typically to provide feedback or guide the research. Table 4 provides a complete summary of stakeholders represented in the community-academic partnerships.

Table 4.

Community Stakeholders in CPBR Partnerships.

Stakeholders	n	%
Community members (e.g., community residents, patients, individuals with shared experience)	61	47
Community-based organizations	44	34
Service providers (e.g., health care providers, social workers)	24	19
Advisory boards and councils	23	18
Youth	16	12
Government representatives and policy makers	14	12
Community partnerships and coalitions	14	11
Community leaders and advocates	13	10
Schools	7	5
Families and caregivers	6	5
Lay health advisors, community health workers, peer educators	6	5
Community research team	4	3

Note. CBPR = community-based participatory research.

MMR in a CBPR Methodological Approach

To understand the intersection of MMR and CBPR, we examined where mixed methods is used within the seven core phases of CBPR described by Israel and colleagues (2013): (1) forming a CBPR partnership; (2) assessing community strengths and dynamics; (3) identifying priority local health concerns and research questions; (4) designing and conducting etiologic, intervention, and/or policy research; (5) feeding back and interpreting the findings; (6) disseminating and translating research findings; and (7) maintaining, sustaining, and evaluating the partnership. First, we start with the results that describe the phases in which mixed methods was present, then we follow that with a brief highlighting of the phases in which mixed methods was not represented. Figure 3 displays each of the CBPR phases and the number of MMCBPR studies within each phase.

Figure 3.

Intersection of MMR and CBPR Designs.

Assessing Community Strengths and Dynamics includes research questions related to existing resources within a community, relevant community-based organizations or influential stakeholders, and power dynamics. It is common for this phase to occur in the early stage of a CBPR partnership. Six (5%) of the studies used MMR designs to assess community dynamics. For example, Traube, Kerkorian, Cederbaum, Bhupali, & McKay (2013) assessed factors influencing informed consent, initiation, and ongoing involvement of African American children in HIV-focused CBPR research. Quantitative and qualitative survey data were integrated to understand motivation, trust, and assent in CBPR partnerships with youth. The meta-inferences from their convergent mixed methods design formed the basis of practical guidelines for researchers conducting CBPR studies with African American youth.

Identifying Priority Local Health Concerns and Research Questions, includes identifying and prioritizing health or social problems that affect the community, identifying factors that contribute to health or social problems, and developing research questions that can be used to understand these problems (Israel et al., 2013). Fifteen (12%) MMCBPR studies used MMR designs to conduct a needs assessment, identify local problems, and understand community priorities. For example, Weathers et al. (2011) used a convergent design that integrated focus groups, key informant interviews, and community health surveys to identify health concerns of African American residents living in an urban setting. The researchers explained the importance of using both data sets to gain a thorough understanding of community needs and priorities: “Although quantitative approaches yield concrete evidence of community needs, qualitative approaches provide a context for how these issues can be addressed” (p. 2087).

Among the studies included in this analysis, the majority (n = 94; 73%) used an MMR design when in the Designing and Conducting Etiologic, Intervention, and/or Policy Research phase of the CBPR framework. This component of the CBPR model includes decisions about which research methods to use, what intervention strategy may be appropriate, and intervention implementation. The studies represented in this category included mixed methods assessment of intervention effectiveness (28; 22% of total studies) and randomized controlled trials featuring both quantitative and qualitative data (n = 2; 2%). Some of these studies also included instrument development (13; 10% of total studies) or intervention development (n = 15; 12%) prior to data collection. Bellows, Howard, Boekeloo, and Randolph (2015) utilized an exploratory sequential mixed methods design using a CBPR approach to identify and validate HIV stakeholders’ perceptions about challenges and opportunities for HIV prevention in a high-prevalence metropolitan area in the United States. Using the guidance and feedback of a community advisory committee, they first collected and analyzed focus group and interview data from HIV stakeholders about the needs of the HIV community and the expertise of local providers. Findings from the focus group and interview data were used to build a quantitative survey that was administered to HIV-prevention organizations.

Sixteen (12%) of the studies were in the central phase, Maintaining, Sustaining, and Evaluating the Partnership. This phase is considered an ongoing process that is integral to successful CBPR work and is often placed at the center of models representing a CBPR cycle because it is based on the relationships developed within the partnership. Maintaining and evaluating the partnership includes investigations of effectiveness and capacity in addition to investigations of facilitators and barriers to the research partnership. One exemplar was the work of Christopher and colleagues (2008), who intersected MMR and CBPR to evaluate the effectiveness of their CBPR partnership to improve cervical cancer prevention efforts in a Native American community. The research team integrated qualitative data from field notes, meeting minutes, and conversational data to explain the quantitative impact of their lay health worker program, focusing on community acceptance, community engagement, community attendance at meetings and events, capacity building among university and community partners, and health-seeking behavior and health knowledge among local residents. The authors described the importance of intersecting MMR and CBPR in evaluation designs, which purposefully integrate generalizable estimates of program effects, processes, and experiences during the program and contextual findings of a local community.

The absence of mixed methods in three of the CBPR phases is worth noting but is not necessarily surprising given the specific aims of each phase. For example, the first phase, Forming a CBPR Partnership, is central to the success of collaborations between community and academic entities, but it does not directly relate to the traditional phases of the research process wherein quantitative or qualitative data would be collected. This CBPR phase includes identifying potential partners, building trust, establishing norms, considering power dynamics, and strategies to build capacity. Feeding Back and Interpreting the Data involves participatory data analysis, where all partners are engaged in making sense of the data that were collected. Finally, Disseminating and Translating Research Findings includes sharing findings with the community. In this phase, research partners may make decisions about how to publish the results, share findings, and influence future interventions and clinical or policy change. While it is possible that each of these past two phases occurred in the studies included in this review, it was not clear from the details provided in the articles.

Discussion

The articles included in the current review indicate that the largest overall number of MMCBPR studies were published in 2010 or later and the large majority have focused on health-related issues associated with marginalized populations—a finding that is congruent with other reports of participatory advanced frameworks (Fetters et al., 2013). The increase in MMCBPR studies coincides with a general growth of studies that use MMR and CBPR approaches separately (DiClemente, Salazar, & Crosby, 2015; Mertens et al., 2016; Viswanathan et al., 2004). As scholars continue to develop advanced applications to promote more rigorous research that is relevant to target populations, this review provides an important understanding of the first decade of MMCBPR studies while identifying significant opportunities for the advancement of practice and publication in MMCBPR.

Although some detail was provided in the reviewed studies about the use of CBPR, by and large, the primary components of MMR were not adequately described. For example, only 6 of 129 studies explicitly identified an MMR core design, and 66% provided no discernable rationale for selecting an MMR approach. Conversely, 84% of studies identified a rationale for using CBPR, with the most cited being its utility for creating sustainable and relevant interventions. On one hand, these findings may suggest that MMCBPR studies prioritize the CBPR approach and apply MMR within that framework; however, they may also suggest that, to date, researchers who are publishing MMCBPR studies have become more familiar with the language, practices, and literature associated with CBPR compared with those of MMR. Additionally, they may choose to prioritize CBPR descriptions within the limitations set by journal article length. This may especially be the case in the health sciences, which were predominantly represented in MMCBPR studies, because participatory approaches have been increasingly used in public health and are regarded for transforming research findings on complex health and social problems into actionable change strategies (Ahmed & Palermo, 2010; Israel et al., 2010; Minkler & Wallerstein, 2010; National Institutes of Health, 2011; Viswanathan et al., 2004). Additionally, public health and health sciences researchers may be using mixed methods designs for pragmatic reasons without knowing the language of the methodology; that is, they are conducting MMR without having a name for it.

If the field is going to expand the use of advanced MMR applications as a means to improve science and intervention implementation, common language between the two approaches needs to be established, including detailed but concise descriptions of MMCBPR designs. It is also necessary to improve researchers’ understanding and descriptions of the primary components within both approaches and how the two are intersected in complex real-world applications, which supports a key recommendation made by Creswell (2016). This should include examination of levels of integration in MMCBPR in general, as well as within specific topic areas or disciplines, to develop recommendations for a MMCBPR typology.

The most common CBPR phase that utilized an MMR design was the Designing and Conducting Etiologic, Intervention, and/or Policy Research, which includes decisions about which research methods to use, what intervention strategy may be appropriate, and intervention implementation. In addition, research teams also employed MMR designs for preliminary work in partnerships, including needs assessments or the development of priority areas and research questions as well as the evaluation of partnerships throughout the span of their collaboration. While this finding seems particularly important because it highlights the CBPR phases in which mixed methods has been used to date, it also emphasizes phases in which mixed methods has not been readily apparent, such as in data interpretation and dissemination. Other findings from this review, such as a lack of joint displays of mixed methods results, further suggest that integration is not extending into data interpretation and reporting. This does not necessarily mean that mixed methods is not happening during these phases, but it may seem that at the very least it is not being written about. If the field is going to fulfill its argument that advanced applications provide a more comprehensive approach to research, then it would benefit from a deeper exploration of how MMR and CBPR intersect at these phases in the research process and how this affects local communities and the scientific enterprise.

Related to the issue raised above, MMR and CBPR scholars have asserted that each of these approaches enhance the reach, relevance, and rigor of research studies and have called for additional studies that help better understand the pathways from methodological approaches to outcomes (Bryman, 2006; Creswell & Plano Clark, 2018; Minkler & Wallerstein, 2010; Sandoval et al., 2011). Of the studies included in this review, it seems that researchers are clearer about their use of CBPR to improve translational research compared with their use of MMR for those purposes. Additional study into how researchers use MMR within a CBPR approach are warranted to better understand their selection of MMR and how they view the integration of the two approaches. Furthermore, while the current review did not examine the impact of a MMCBPR approach on individual or community-level outcomes, a future review of the effects of MMCBPR on these outcomes would greatly benefit both fields separately and when applied together.

Finally, we chose to include intramethod designs—the collection of quantitative and qualitative data through the same method—in our review and found that 19% of the articles met these criteria. We suspect that intramethod designs are popular in CBPR projects because of the popularity of participatory approaches, wherein participants are engaged in both the generation and the analysis of data. Such approaches (e.g., concept mapping, social network analysis, environmental mapping or audits, photovoice) are aligned with the principles of CBPR in general and, therefore, may be frequently selected as effective strategies for including community partners in the research process. As the prevalence of MMCBPR projects increases, we anticipate a similar growth in the use of participatory intramethod designs and, as a result, a need to consider the role of these designs in MMR and discuss implications for the field.

Limitations

The empirical studies included in this review do not necessarily represent all that has been published with regard to MMCBPR. In addition, it includes studies that did not explicitly identify as MMCBPR but at least referenced the use of quantitative and qualitative methods, as well as partnerships between academic researchers and community members. Although the current review reported on who participated in the partnerships and at what stage in the CBPR process MMR was used, we were unable to determine the strength of the partnerships and whether they represented comprehensive implementation of CBPR principles.

In addition to the points above, we were only able to analyze the studies based on what the authors reported. As such, there may be details related to MMR and CBPR that were not adequately represented in the written article, perhaps including vital information that would have identified a research design or described a rationale for using MMR and CBPR. Furthermore, the extent to which authors have thus far reported on MMCBPR studies prevented us from being able to determine how a MMCBPR approach improved the rigor, relevance, and reach of studies or determine whether MMR design benefited the CBPR approach, or vice versa. Further examination of MMCBPR studies compared with MMR and CBPR only studies would provide additional understanding of the potential benefits of MMCBPR.

Recommendations for Best Practices

Hybrid research approaches that combine multiple philosophical and methodological paradigms to address “wicked problems” (Mertens, 2015) that span multiple systems and influential factors are inherently complex. We acknowledge that MMR itself is multifaceted and requires skill, expertise, and understanding of both quantitative and qualitative methods as well as how the two may be integrated at various points in the research process. While individual researchers may not be equally facile in both methods, collaboration with others who are able to contribute expertise to the various parts is expected to result in a greater sum.

At the same time, we also know that CBPR is very complex, time-consuming, and resource intensive. It requires ongoing, extended collaboration and navigating competing needs, resources, and commitments of academic and community partners (Becker, Israel, & Allen, 2005; Minkler, 2005; Wolff & Maurana, 2001). When MMR is used within a CBPR approach, we anticipate that research designs will be even more complicated than using either approach alone and certainly more so than conventional research designs. To continue moving advanced applications forward, this means that not only must researchers be able to develop designs that help answer multidimensional research questions, but they must also be able to understand and articulate the components of the CBPR approach as well as the MMR design when disseminating results in publications. Doing so will ensure that others can better assess the empirical evidence and the field can develop a deeper understanding of the impact of advanced designs on health and well-being.

The current review provides an important contribution to the evolution of advanced applications by articulating current methodological features, uses, and challenges of intersecting MMR with another methodological approach, like CBPR. Based on the results, we offer recommendations for future empirical applications of MMCBPR. We have also highlighted the exemplar articles discussed qualitatively in this review as real-world applications of these recommendations (see Table 5).

Existing MMR and CBPR literature: On the surface, it may seem easy to intersect MMR and CBPR; however, researchers who are aware of the intricacies of each approach know that they are complex when fully implemented. To continue building the evidence for MMCBPR, researchers should read, understand, and cite the existing CBPR and MMR literature, including hallmark pieces.

MMR and CBPR rationale: MMR and CBPR are deeply theoretical and take into account the potential for generating a greater understanding of wicked problems from multiple viewpoints. As such, both approaches are highly principled and necessitate a clear understanding of why their usage is applicable for a given research problem. Researchers using MMCBPR need to clearly articulate a rationale for each approach, and those rationales should be grounded in the respective literature.

MMR and CBPR research questions: As we have cited previously, MMR and CBPR scholars have written a great deal on the contributions of these approaches to transformative scientific endeavors. As with the rationale for using MMCBPR, research questions should reflect MMR and CBPR principles and research designs. For MMR, this includes at least one research question for each data collection method and an integrated question (Creswell & Plano Clark, 2018; Plano Clark & Ivankova, 2016). For CBPR, there should be at least one question that asks how the community partnership will strengthen the rigor, reach, and/or relevance of the research based on the nine key principles outlined by Israel et al. (2008) and Israel et al. (2013).

MMR design: Researchers utilizing MMCBPR need to be knowledgeable about the different MMR designs and select the one that fits the purpose of their research and research questions. Additionally, published MMCBPR studies should clearly articulate the design and the rationale for selecting the design.

Comprehensive description: In addition to the recommendation for more details about MMR design decisions, both the field and the individual MMCBPR empirical studies would benefit from providing additional detail about data collection, data analysis, and integration of methods, interpretation, and reporting. Journals publishing MMCBPR studies should also encourage, and allow for, adequate space for these components of studies to be addressed.

Value of MMCBPR: To understand the potential impact of MMCBPR as an advanced application, published studies should provide explicit evidence or discussion of the intersection between the two approaches and the translation of MMCBPR findings in real-world settings. This should include discussion of how the MMR design benefited the CBPR approach, or vice versa. The further development of a typology for the MMCBPR-advanced application would provide a best practice framework for researchers using MMCBPR so that they can clearly identify and articulate the value of MMCBPR in their study publications.

Table 5.

Exemplars of MMCBPR.

Author(s)	Topic and Population	Levels of MMR Integration			CBPR Approach		Phase of MMR in CBPR Cycle	MMCBPR Benefit
Author(s)	Topic and Population	Design	Methods	Interpretation and Reporting	CBPR Partners	CBPR Rationale	Phase of MMR in CBPR Cycle	MMCBPR Benefit
Bellows, Howard, Boekeloo, and Randolph (2015)	HIV/AIDS;people living with HIV/AIDS	Exploratory sequential	BuildingMerging	Narrative	Advisory board	Identify prioritiesCommunity feedback	Data collection	“Using a systematic, sequential mixed methods study design with a CBPR approach, we learned that the concerns of our community advisors and stakeholders were validated. . . The community partnership also facilitated interpretation of findings and their implications for current and future programmatic and research efforts.” (p. 325)
Christopher, Gidley, Letiecq, Smith, and McCormick (2008)	Cancer;Native American women	Convergent	Merging	Narrative	Advisory boardLay health advisors/community health workers	Tailor intervention	Evaluate partnership	“For the sake of public health programs and public health practitioners and researchers, we encourage those working cross culturally and those using a CBPR approach to participate deeply in the communities in which they work and to employ both quantitative and qualitative evaluation methods to gain an enhanced understanding of program effects.” (p. 833)
John and Gunter (2016)	Aging;Middle-aged and older adults	Explanatory sequential	ConnectingMerging	Narrative	Community partnership/coalition	Identify prioritiesSocial change	Data collection	“[MM] and [CBPR] activities were used to engage and mobilize stakeholders, gather and share information about the interplay between attributes of people (i.e., individual, social, cultural) and attributes of place (i.e., local, county, region) that were experienced . . .” (p. 1097)
Johnson, Ali, and Shipp (2009)	Community-academic partnerships;Somali refugee women	Convergent	Merging	Narrative	Community membersService providers	Establishing trust	Intervention development	“CBPR using [MM] has emerged as a crucial strategy in partnering with refugee communities, particularly when working with vulnerable populations less commonly researched who uphold cultural standards that diverge from Western philosophies and values.” (p. S231)
Traube, Kerkorian, Cederbaum, Bhupali, and McKay (2013)	Community-academic partnerships;African American children in CBPR studies for HIV prevention	Intramethod	ConnectingMerging	Narrative	Youth	Local knowledge	Assess community strengths and dynamics	“We adopted a mixed-methods approach to analyze the data based on the belief that it is important to express qualitative perspectives, methods, perceptions, and conclusions, bolstering our understanding of the quantitative analysis.” (p. 82)
Weathers et al. (2011)	Health needs and preferences;African American adults	Convergent	Merging	Narrative	Community-based organizations	Identify prioritiesCommunity feedback	Identify priorities and research questions	“[CBPR] is a long-term endeavor, and the data obtained from both the quantitative and qualitative methods provide critical information that community and academic members of our partnership can use to develop relevant intervention strategies.” (p. 2092)
Windsor (2013)	Substance abuse prevention;Primarily African American adults	Intramethod	ConnectingBuildingMerging	Data transformationJoint display	Community membersSubstance usersService providers	Study impactQuality of researchTailor intervention	Data collection	“The current article argues that [MM] are useful in CBPR research because they combine the prolonged community contact and depth of qualitative research with the breath of quantitative work” (p. 275)

Note. MMCBPR = mixed methods community-based participatory research; MMR = mixed methods research; CBPR = community-based participatory research; MM = mixed methods.

Conclusion

As with other advanced applications, researchers conducting an MMCBPR project face significant challenges when designing an MMCBPR study. Both MMR and CBPR are complex fields and conducting research that fully integrates the two approaches requires considerable time and development. Though we recognize the difficulty of including descriptions of both MMR and CBPR components in a single manuscript, we were surprised by the limited use of MMR language, definitions, and literature in the studies included in our review. To understand the impact of MMCBPR, we encourage researchers to fully describe the components of MMR in their study, the components of CBPR in their study, and challenges and benefits of implementing MMCBPR projects in their field.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Melissa DeJonckheere

References

Ahmed

S. M.

Palermo

A.-G. S.

(2010). Community engagement in research: Frameworks for education and peer review. American Journal of Public Health, 100, 1380-1387.

Becker

A. B.

Israel

B. A.

Allen

A. J.

(2005). Strategies and techniques for effective group process in CBPR partnerships. Methods in Community-Based Participatory Research for Health, 2, 69-94.

Bellows

Howard

Boekeloo

Randolph

(2015). Investigating community concerns regarding HIV prevention organizations’ expertise in serving HIV-vulnerable populations. Progress in Community Health Partnerships: Research, Education, and Action, 9, 315-326.

Bryman

(2006). Integrating quantitative and qualitative research: How is it done? Qualitative Research, 6, 97-113. doi:10.1177/1468794106058877

Christopher

Gidley

A. L.

Letiecq

Smith

McCormick

A. K. H. G

. (2008). A cervical cancer community-based participatory research project in a Native American community. Health Education & Behavior, 35, 821-834.

Coyle

C. E.

Schulman-Green

Feder

Toraman

Prust

M. L.

Plano Clark

V. L.

Curry

(2016). Federal funding for mixed methods research in the health sciences in the United States: Recent trends. Journal of Mixed Methods Research. Advance online publication. doi:10.1177/1558689816662578

Creswell

J. W.

(2015). Educational research: Planning, conducting, and evaluating quantitative and qualitative research. Upper Saddle River, NJ: Pearson Education.

Creswell

J. W.

(2016). Reflections on the MMIRA: The Future of Mixed Methods Task Force Report. Journal of Mixed Methods Research, 10(3), 215-219. doi:10.1177/1558689816650298

Creswell

J. W.

Klassen

A. C.

Plano Clark

V. L.

Smith

K. C.

(2011). Best practices for mixed methods research in the health sciences. Bethesda, MD: National Institutes of Health.

10.

Creswell

J. W.

Plano Clark

V. L.

(2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage.

11.

Creswell

J. W.

Plano Clark

V. L.

(2018). Designing and conducting mixed methods research (3rd ed.). Thousand Oaks, CA: Sage.

12.

DiClemente

R. J.

Salazar

L. F.

Crosby

R. A.

(2015). Community-based participatory research in the context of health promotion. In Research methods in health promotion (2nd ed., pp. 313-336). San Francisco, CA: Jossey-Bass.

13.

Fetters

M. D.

Curry

L. A.

Creswell

J. W.

(2013). Achieving integration in mixed methods designs: Principles and practices. Health Services Research, 48(6 Pt. 2), 2134-2156. doi:10.1111/1475-6773.12117

14.

Hesse-Biber

Johnson

R. B.

(2013). Coming at things differently: Future directions of possible engagement with mixed methods research. Journal of Mixed Methods Research, 7(2), 103-109.

15.

Israel

B. A.

Coombe

C. M.

Cheezum

R. R.

Schulz

A. J.

McGranaghan

R. J.

Lichtenstein

. . . Burris

(2010). Community-based participatory research: A capacity-building approach for policy advocacy aimed at eliminating health disparities. American Journal of Public Health, 100(11), 2094-2102. http://doi.org/10.2105/AJPH.2009.170506

16.

Israel

B. A.

Eng

Schulz

A. J.

Parker

E. A.

(2013). Methods for community-based participatory research for health. San Francisco, CA: Jossey-Bass.

17.

Israel

B. A.

Schulz

A. J.

Parker

E. A.

Becker

A. B.

Allen

A. J.

III Guzman

J. R.

(2008). Critical issues in developing and following CBPR principles. In Minkler

Wallerstein

(Eds.), Community-based participatory research for health: From process to outcomes (pp. 1-23). San Francisco, CA: Jossey-Bass.

18.

Ivankova

N. V.

(2014). Mixed methods applications in action research: From methods to community action. Thousand Oaks, CA: Sage.

19.

Jacquez

Vaughn

L. M.

Wagner

(2013). Youth as partners, participants or passive recipients: A review of children and adolescents in community-based participatory research (CBPR). American Journal of Community Psychology, 51, 176-189. doi:10.1007/s10464-012-9533-7

20.

John

D. H.

Gunter

(2016). engAGE in community: Using mixed methods to mobilize older people to elucidate the age-friendly attributes of urban and rural places. Journal of Applied Gerontology, 35, 1095-1120.

21.

Johnson

Turner

L. A.

(2003). Data collection strategies in mixed methods research. In Tashakkori

Teddlie

(Eds.), Handbook of mixed methods in social & behavioral research (pp. 297-319). Thousand Oaks, CA: Sage.

22.

Johnson

C. E.

Ali

S. A.

Shipp

M. P. L.

(2009). Building community-based participatory research partnerships with a Somali refugee community. American Journal of Preventive Medicine, 37, S230-S236. doi:10.1016/j.amepre.2009.09.036

23.

Johnson

R. B.

Onwuegbuzie

A. J.

(2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14-26.

24.

Lucero

Wallerstein

Duran

Alegria

Greene-Moton

Israel

. . . White Hat

E. R.

(2018). Development of a mixed methods investigation of process and outcomes of community-based participatory research. Journal of Mixed Methods Research, 12(1), 55-74.

25.

Mertens

D. M.

(2015). Mixed methods and wicked problems. Journal of Mixed Methods Research, 9(1), 3-6.

26.

Mertens

D. M.

Bazeley

Bowleg

Fielding

Maxwell

Molina-Azorin

J. F.

Niglas

(2016). The future of mixed methods: A five year projection to 2020. Indianapolis, IN: Mixed Methods International Research Association. Retrieved from https://mmira.wildapricot.org/resources/Documents/MMIRA%20task%20force%20report%20Jan2016%20final.pdf

27.

Minkler

(2005). Community-based research partnerships: Challenges and opportunities. Journal of Urban Health, 82, ii3-ii12.

28.

Minkler

Wallerstein

(2010). Community-based participatory research for health: From process to outcomes. San Francisco, CA: Jossey-Bass.

29.

Molina-Azorin

J. F.

Fetters

M. D.

(2016). Mixed methods research prevalence studies: Field-specific studies on the state of the art of mixed methods research. Journal of Mixed Methods Research, 10(2), 123-128.

30.

National Institutes of Health. (2011). Clinical and Translational Science Awards Consortium Community Engagement Key Function Committee Task Force on the Principles of Community Engagement. Principles of community engagement. 2nd ed. Bethesda (MD): National Institutes of Health.

31.

Plano Clark

V. L

. (2010). The adoption and practice of mixed methods: US trends in federally funded health-related research. Qualitative Inquiry, 16, 428-440.

32.

Plano Clark

V. L.

Ivankova

N. V

. (2016). Mixed methods research: A guide to the field (Vol. 3). Thousand Oaks, CA: Sage.

33.

Sandoval

J. A.

Lucero

Oetzel

Avila

Belone

Mau

. . . Wallerstein

(2011). Process and outcome constructs for evaluating community-based participatory research projects: A matrix of existing measures. Health Education Research, 27, 680-690.

34.

Shannon-Baker

(2016). Making paradigms meaningful in mixed methods research. Journal of Mixed Methods Research, 10(4), 319-334.

35.

Teddlie

Tashakkori

(2003). Major issues and controversies in the use of mixed methods in the social and behvioral sciences. In Tashakkori

Teddlie

(Eds.), Handbook of mixed methods in social & behavioral research (pp. 3-50). Thousand Oaks, CA: Sage.

36.

Teddlie

Tashakkori

(2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Thousand Oaks, CA: Sage.

37.

Traube

D. E.

Kerkorian

Cederbaum

J. A.

Bhupali

McKay

M. M.

(2013). African American children’s perceptions of HIV-focused community-based participatory research. Journal of Empirical Research on Human Research Ethics, 8, 79-90.

38.

Vaughn

L. M.

Jacquez

Lindquist-Grantz

Parsons

Melink

(2017a). Immigrants as research partners: A review of immigrants in community-based participatory research (CBPR). Journal of Immigrant and Minority Health, 19, 1457-1468. doi:10.1007/s10903-016-0474-3

39.

Vaughn

L. M.

Jacquez

Zhen-Duan

Graham

Marschner

Peralta

García

Recino

Maya

Cabrera

(2017b). Latinos Unidos por la Salud: The process of developing an immigrant community research team. Collaborations: A Journal of Community-Based Research and Practice, 1, 1-22.

40.

Viswanathan

Ammerman

Eng

Garlehner

Lohr

K. N.

Griffith

. . .Whitener

(2004). Community-based participatory research: Assessing the evidence. (AHRQ Evidence Report Summaries). Rockville, MD: Agency for Healthcare Research and Quality. Retrieved from https://www.ncbi.nlm.nih.gov/books/NBK11852/

41.

Weathers

Barg

F. K.

Bowman

Briggs

Delmoor

Kumanyika

. . . Halbert

C. H.

(2011). Using a mixed-methods approach to identify health concerns in an African American community. American Journal of Public Health, 101, 2087-2092.

42.

Windsor

L. C.

(2013). Using concept mapping in community-based participatory research: A mixed methods approach. Journal of Mixed Methods Research, 7(3), 274-293. doi:10.1177/1558689813479175

43.

Wolff

Maurana

C. A.

(2001). Building effective community–academic partnerships to improve health: A qualitative study of perspectives from communities. Academic Medicine, 76, 166-172.