Is Collaborative,Community-Engaged Scholarship More Rigorous Than Traditional Scholarship? On Advocacy,Bias,and Social Science Research

Rigor Not Advocacy

In our view, the real question to ask when determining the quality and usefulness of CCES, is not whether it exhibits advocacy, but rather, whether it is rigorous—a question for all forms of research. Many forms of research have an outcome of interest to the investigators, and the research findings describe processes that lead to this outcome or analyze factors that produce these outcomes. This is particularly true for policy-oriented research in fields like education. Many education researchers, for example, develop programs and then test them in experimental trials or in other ways (Coalition for Evidence-Based Policy, 2003). CCES is not different in substance from these other forms of research, although it is often seen as different and more suspect as biased (Tittle, 2004).

No research is purely objective. Values and personal standpoints affect all forms of research, as many scholars, whether community-engaged or not, have long pointed out (Collins, 2000; Milner, 2007). Researchers bring underlying assumptions that shape the questions asked, the data considered relevant and the methods used. Researchers have an obligation to identify the biases they bring to their research and take steps to mitigate such bias (Maxwell, 2005).

In our view, the real issue is not whether collaborative community-based research is rigorous but rather how rigor is defined. It is a given that rigorous research should use appropriate and systematic methods, stand up to critique by knowledgeable parties, and consider contrary evidence and alternative hypotheses (Maxwell, 2005). But rigor is often used as a code word for a set of practices that align themselves with detached research, rather than engaged research. As Davies and Dodd (2002) note,

If rigor is understood only in terms of a structured, measurable, systemized, ordered, uniform and neutral approach, then other research methods that allow flexibility, contradictions, incompleteness, or values will always appear “sloppy,” epitomizing everything that is “nonrigor” and therefore lacking in credibility. (p. 280)

Many critics perhaps suspect that this type of research is designed to produce results that “the community wants.” But community organizations have an interest in the credibility and legitimacy of the research they rely upon to improve their practice or use to advocate for changes in policy or practice. Research that is not credible will not advance their cause; research that proves what they already think will not improve practice. Groups do not learn from research that simply confirms their agenda or justifies their grant proposals. True advocacy research that helps community partners is critical research.

We are not alone in trying to reclaim the concept of rigor so that it applies to community-engaged research. Shor and Freire (1986) argue that the traditional meaning of rigor needs to be redefined, calling for a “creative rigor” that critiques the authoritarian way of transferring knowledge “which mechanically structures education, and discourages us from the responsibility of recreating ourselves in society” (p. 77). Instead, Shor and Freire propose “a creative pedagogy which seeks to reinvent knowledge situated in the themes, needs, and language of the students (and communities), as an act of illuminating power in the society” (p. 81). This notion of creative rigor resonates with the main principles elaborated in other works on CCES like Community-Based Research in Higher Education (Strand, Cutforth, Stoecker, Marullo & Donahue, 2003).

We also agree with other scholars (e.g., Fine, 2008; Hale, 2008; Shor & Freire, 1986) who have argued that research can actually be more rigorous when it is engaged because it is accountable to input and critique from a more diverse set of actors—including those both in the academy and in the community. Newer work on “impact validity” raises similar criteria, that research be designed with consideration for its contributions to social and political change (Massey & Barreras, 2013).

Indeed, some types of traditional, detached research have no accountability beyond the individual researcher. While most are accountable to a scholarly community, these scholarly communities can be insular. They may endorse stereotypes that are shared widely within the research community but lack validity in relevant communities. There is a long history of White researchers studying Black communities, for example, and reinforcing stereotypes even though their findings were published in peer-reviewed journals and accepted as rigorous (Zuberi & Bonilla-Silva, 2008). Many indigenous scholars critique traditional anthropology and its widely accepted findings for its contribution to colonizing assumptions (L. T. Smith, 1999).

From this point of view collaborative research can be considered more rigorous than more traditional forms because it must demonstrate its credibility to a broader audience that brings a more diverse set of questions and standpoints to bear. In collaborative work, there is accountability to partners and to the demands of practice. In this article, we intend to advance this discussion by elaborating multiple ways in which CCES can be considered more rigorous than detached scholarship and bring to light new knowledge that would otherwise go unnoticed.

This is not to say that collaborative researchers do not face some tensions between rigor and advocacy. Research is the specialized focus of scholars; community participants have many demands on their time and, in the end, focus on action or practice and getting results (Stoecker, 2012). Activists willing to engage in research see it as important, but as one among many important practices. In addition, community organizations may have an immediate, short-term interest in the publication of findings or accounts that benefit their agenda, and therefore resist research findings that identify weaknesses in their efforts even if analyzing these weaknesses contributes important lessons for improving practice in the broader field of organizing for change.

There may be cases where publishing the results of research may do real harm to community participants. These are difficult ethical issues for community-engaged scholars. However, they are not unique to this research approach. Standard human subjects protection asks researchers to limit harm and gives participants the right to withdraw from research at any time without penalty. If community participants in collaborative research believe the harm is too great, institutional review boards and the ethical obligations of scholars give them the right to withdraw their participation as well. In some cases that could mean the research is not published.

In the following pages, the authors describe research projects that combine rigor and advocacy. They illustrate ways that CCES has led to rigorous research that creates knowledge that supports social action. We then draw from these cases to discuss ways in which CCES can be more rigorous than detached scholarship and draw some lessons for addressing the tensions that inevitably arise when scholars and activists collaborate to build knowledge and advocate for change agendas.

Insurance Redlining: A Strategic Collaborative—Gregory Squires

In 1988, a sales manager for the American Family Insurance Company told one of his agents, in a tape recorded discussion, “Very honestly, I think you write too many blacks . . . You got to sell good, solid premium-paying white people . . . They own their own homes, the white works” (Lynch, 1997, p. 159). This conversation reveals the deep-seated racial bias that existed in the home insurance industry that both motivated me to conduct research in support of antidiscrimination campaigns and, when revealed, helped one campaign win a significant victory in Milwaukee.

Since the late 1970s, I have conducted research on insurance redlining practices, which, in conjunction with the work of other scholars, the organizing activity of several community-based organizations, and creative legal advocacy, has led to some significant changes in the way the home insurance industry serves communities of color. The following pages describe the involvement of collaborative research, organizing efforts, and law enforcement that culminated in a favorable settlement of one lawsuit and set the table for subsequent victories. The context in which the research was conducted—particularly the array of other available financial and non-financial resources—proved to be a key factor in these outcomes. But it was the actions of a variety of actors exploiting that context which produced important transformations in the way this vital financial service is provided in the nation’s metropolitan areas.

Respecting Coproducers of Knowledge—Celina Su

Over the years, I have heard consistent complaints of “research fatigue” from some community leaders. Such leaders stated that they were tired of reporters, academics, and foundation officers conducting “drive-by” interviews, with a set agenda—only to never be heard from again. We were getting diplomas and promotions with these publications, and what were they getting? I have tried to take a different approach, to build deep and sustained collaborations with community organizations and this required open discussion and trust building.

When I approached some community organizations working on education reform in the South Bronx in the early 2000s, for instance, they demanded to know what I might offer in return for their time. I offered to present books on community organizing and lead discussions (without academic jargon), to report back whatever analyses I performed along the way, and to give them opportunities to respond to my findings. Ensuring that my research was relevant to their interests in these ways immediately increased access to and helped to establish trust with these community partners (Jagosh et al., 2012).

I also made a case for why my research would not simply replicate what had already been published. At one organization, board members granted me access because they were impressed with one of my research questions, which focused on an aspect of their work that interested them and that they had not reflected upon before.

These board members also wondered whether, given the fact that most of the books they had read were written by older White men, a scholarly inquiry by a younger woman of color might highlight different insights into their work. It did not seem to me that they held essentialist views on scholars by race (as if White men necessarily thought X, and a Brazilian Chinese American woman like me necessarily thought Y); but rather they thought that our different positionalities (by race, gender, disciplinary training, and other axes of social position) shaped our work (Hale, 2008) and might produce some different findings. In response, I had to articulate potential biases and assumptions in my research early on.

When I think about rigor in my research, I think about the approach I take to collaboration. My analyses remain simultaneously both sympathetic with and critical of the community-based organizations and institutions with whom I work. I have, for instance, examined strengths and weaknesses of different strategies with members of a social change organization, ultimately coming to a conclusion in agreement with some members, but not others. Here, I reflect on the ways in which collaborative research efforts helped me gain insights I believe I would not have otherwise.

Community-Engaged Research With Community Organizing Groups—Mark R. Warren

For most of my career, I have studied community and youth organizing efforts to advance educational justice, racial equity, and community development. I have often chosen to study relatively successful models of organizing to document and analyze organizing strategies and processes—identifying how the organizers do their work and why, and the results achieved. Most researchers and educators hold deficit views of low-income communities of color, perceive people as passive victims of oppression who need to be helped by professionals, and believe that elites drive social improvement and social change processes. By documenting the struggles of people on the ground, and highlighting the creative and sophisticated strategies groups have developed to organize for change, my research is intended to shift the dominant paradigm toward respect for grassroots leadership. I also believe it is important to analyze these organizing strategies to build theory and practical knowledge about effective ways communities can organize for change.

My approach to community engagement is to collaborate with the organizations I study. I consult with the groups throughout the process—discussing with them research questions, research design, and data collection. I discuss findings and insights with partners and receive their feedback. I share a draft of the written work. I listen carefully to feedback at all stages but, in the end, I am responsible for the content of the final product.

When I was at the Harvard Graduate School of Education, I co-led with Karen Mapp a group of 15 doctoral students who formed teams to conduct case studies of six community organizing groups that had achieved significant success in affecting public education in their localities (Warren, Mapp, & the Community Organizing and School Reform Project, 2011). We intended to document the value of low-income communities of color building the power to create equity-oriented change in public education. In contrast to top-down school reform models, we were advocating for the kind of transformational change in public education that would only come with active participation and meaningful roles for parents, young people, and other members of the communities most affected by educational inequities and failures. I think our partners hoped to get recognition and publicity for their hard work from our project. They also hoped to learn new lessons about their organizing because an outside set of eyes examined their work.

Collaborative Research in Boston’s Youth Justice Movement—Luke Aubry Kupscznk

Between 2005 and 2008, Boston students and their adult supporters in youth organizing groups joined in a citywide alliance to demand jobs for young people as a way to prevent violence in their neighborhoods. They took measures that included turning their backs on city councilors at a city council meeting and staging a “die-in” on the steps of City Hall. Through these efforts, they won significant increases in city and state funding for youth jobs. Adult staff members at some of these organizations believe this period is a strong example of the exertion of power by youth and that it helped spur even greater levels of collaboration across youth and adult allies in Boston that continue today.

In what follows, I describe the development and process of conducting a collaborative research project between researchers at the University of Massachusetts in Boston and three of these youth organizations about this high point of the youth justice movement in Boston. I contend that the collaborative nature of this project not only aided in the development of rich research questions and the facilitation of interviews but also added to the rigor applied in analyzing and delivering our findings.

Community-Based Participatory Research and Action—José Calderon

The concept of advocacy can have many meanings. For some, it can mean the researcher coming from the outside and using research as a means of advocating for a particular group or population. This can be a disempowering form of research. Another form is one where the scholar works alongside community participants in carrying out research around issues, needs, or problems that are pertinent to the community. In this method of research, the community participants have a voice in the research, in diagnosing and defining the problem, in carrying out research on the problem, in analyzing the outcomes of the research, and in using the research to present and implement solutions. The latter is the type of research that I attempt to carry out in my work. In much of my research, I ground theory in data collected through participant observation and interviewing, and develop my analysis through the writing and coding of field notes and interview transcripts. My work, however, has not entirely followed the grounded-theory approach, because I have been more than a participant observer in the process. My involvement as a leader in various community, neighborhood, and civic groups has made it impossible for me to be a neutral observer. Gathering data in the dual roles of researcher and activist, however, has provided special insight into activities and trends in the community.

In this type of research, there is a reciprocal process where the researchers and community participants learn about each other and the history, culture, and foundations of the community in which they are both participating. The research is also seen as an avenue for taking research outcomes and using them to implement strategies that can address the issues that the community is facing. This includes a practice where, rather than “expert” solutions being predefined, the results of the research are interpreted and used as guides for action and advancing social change.

For example, I teach at Pitzer College and work with community activists in Pomona, California, a majority Latino and African American city, where a bill, Measure T, was placed on the city’s ballot to replace the elections of city councilmembers by district to at-large elections. Together with Pitzer students, including Jared Calvert and Kathy Cabrera, I worked with councilmember Cristina Carrizosa and other community members to carry out research on the measure and the history of voting rights in the city. The research revealed that Measure T was a sinister attempt by conservative forces, including the police, to turn back the will of the people in Pomona who, back in 1990 (after law suits by the Mexican American Legal Defense and Education Fund and the Southwest Voter Registration Project), voted to scrap citywide elections in favor of single-member districts to bolster minority representation. After the research exposed that the police association had given $50,000 to efforts to support the passage of this bill and that they were affiliated with a leaflet depicting a White hand extended upward over Brown hands reaching from below, a multiracial coalition of community members and organizations held a press conference, walked door to door, and on election day defeated Measure T and helped elect two supportive councilmembers.

An essential component of this style of learning and research is a commitment to promoting an equal relationship between the interests of the academics and the community participants. Traditionally, academics have had a tendency to “parachute” into a community or workplace for their own research or funding interests without developing the kind of long-term relationship and form of collaboration that it takes to create concrete change. In working to move beyond traditional research models, participating students and faculty collaborate to create what Kenneth Reardon (1998) has described as “social learning processes that can develop the organizational, analytical, and communication skills of local leaders and their community-based organizations” (p. 59).

I have learned that it is essential for faculty members to make a long-term commitment to the sites and communities where they are situated or where they have placed their students. Although students can only commit for a semester or until graduation, faculty participants are in a better position to sustain campus–community partnerships. As these long-term partnerships are developed, students and faculty can become an influential force in their communities. They no longer are placed in the role of travelers passing by. Instead, they see themselves as participants with a stake in the decisions being made.

As research efforts are used to create and change policies, the divide between campus and community is being diminished. Our communities do not see the campus as an island, and, more important, we don’t see ourselves as an island. We see ourselves as an appendage of a larger community.

The participants in the many coalitions and organizations that I have been involved with have not been as concerned with bias in research but more concerned with finding solutions to the many problems that they confront in the community and in the schools. I have been considered an “insider” by these organizations and coalitions, as I usually carry out research in communities where my family lives or where I have established a long-term foundation. By working actively in these organizations and coalitions, I am able to develop an ongoing dialogue with the participants who allow me to serve as both an active participant and researcher. In this dialogue, we not only engage in analysis and reflection but also challenge each other and begin to develop theories and strategies for dealing with the emerging problems they are facing. In my view, these theories are important but mainly when they lead to strategies for change.

As in the Measure T example, a problem was identified that led to research on the forces behind a movement to turn at-large elections back to single districts. As a result of the research, the community participants theorized that the real intent of the measure was to curtail the increasing power of the growing Latino community and to defeat councilmembers who opposed traffic checkpoints and defended the rights of immigrants. A plan was implemented to hold a press conference, expose the measure as an attack on voting rights, and get the vote out to defeat the measure. The outcome resulted in a defeat of Measure T and a continued voter movement that also elected councilmembers who were more supportive of immigrant rights.

My entry point comes from my activist background. Coming from this background, I work hard to support students who get involved with community leaders and other participants in finding solutions to practical problems in their communities. There are many students coming out of high school these days who have a history of community involvement. The higher education experience can put a damper on their passion. Some of these students have a tendency to turn away from the academy and drop out. I think community-based participatory action and research can make a real difference for these students. I know that it makes a difference for faculty who have come out of an activist history and are trying to find a means to exist in academia without being co-opted and without losing the values that give social meaning to their research or teaching.

Time and time again, I have found that being involved as an organizer alongside the community participants allows me to develop a trust that I would not normally develop as a neutral researcher. In the Measure T example, I had already developed a trust with community members and some councilmembers as a result of being an activist in previous community struggles. This trust allowed for using the research on the origins of district elections in the city as a foundation for developing a plan of action and its implementation.

I combine critical pedagogy, participatory action research, and community engagement as a means of bringing students and faculty together with community-based organizations to work on common issues and to effect social change. These collaborative efforts are examples of policy-making models that go beyond charity and dependence on experts to “get at the root causes of problems, and focus directly or indirectly on politically empowering the powerless” (Morton, 1995, p. 23). The research and learning described here focus on the sources of inequality and what can be done about it. The dominant understanding of inequality tends to blame individuals for their inadequacies. Instead, the practices described here focus on the historical and systemic foundations of inequality and challenge students and faculty to find common ground with community institutions, unions, organizations, and neighborhood leaders to arouse social consciousness and long-term structural change.

Discussion

The above cases demonstrate many ways in which CCES is rigorous and, in fact, sometimes more rigorous than traditional scholarship. First, conducting this kind of research requires making explicit personal biases and standpoints and considering their influence on the research process. When Celina Su’s community partners demanded a clear explanation for the value of her proposed research to justify the time they would spend on it, she had to articulate potential biases and assumptions in the research early on. The efforts of Mark Warren’s team to build relationships with community participants required that researchers interrogate their personal experiences and values so that they can “tell their story” to the partners. This process helps to clarify the influence of those experiences and values on the research project and in some ways make it more objective; Charles Hale (2008, p. 11) has called this positioned objectivity.

Second, participant contributions often strengthen research design and methods up front. The input from Su’s partners helped create a survey for a participatory budgeting process that was more likely to be filled out and include useful information—from undocumented immigrants, for example. Luke Kupscznk’s partners shaped the research focus and interview protocols for youth leaders in ways that helped reveal their deeper experiences.

Third, community-based research involves forms of accountability and validity tests to research findings that go beyond peer-review. Warren’s community participants questioned him sharply when he shared some initial findings on race relations and that process pushed him to develop more nuanced and complex analyses of interracial unity in organizing processes. Luke Kupscznk’s partners insisted on the importance of race and class analysis in youth leadership. Gregory Squires learned that the targets of community-based action bring an additional level of accountability beyond what occurs in typical peer-review processes. When the target of research is a powerful organization, like the insurance company in Squires’s American Family expert report, it has the resources to hire its own researchers and mount a media campaign to attempt to discredit its opponents. This threat pushes collaborative researchers to be even more careful about the rigor of their research.

Fourth, collaborative research creates the kind of relationships and trust necessary for valid ethnographic research. Luke Kupscznk’s case reveals the step-by-step manner in which researchers on youth organizing engaged with organizers in the design, conduct, and products of the research. This partnership process led to access and trust with young people to be interviewed by the research team so they were willing to open up and tell their stories to university-based researchers.

Moreover, previous relationships of trust often facilitate collaborations in the first place. In the American Family case, the fact that the attorneys had long worked with civil rights groups in Milwaukee on other issues (e.g., school desegregation) helped them come together more easily on the insurance case. José Calderón’s case also speaks to the value of creating and sustaining long-term relationships of trust capable of producing multiple research-action collaborations.

In the end, CCES approaches can sometimes lead to knowledge that would be overlooked in traditional scholarship. Community activists are often the first to identify critical issues of inequality and oppression because they experience them directly. The results of Squires’s research contributed to a larger knowledge base on insurance redlining, which supported subsequent legal actions. While, redlining by home insurance companies was not entirely a secret, little attention was paid to this issue by academics before community organizations made this a public policy issue. Research about this form of redlining developed not out of academic, disciplinary interest but as a result of the demands of community activists to which collaborative, equity-oriented researchers responded. Similarly, Kupscznk and his colleagues would not have identified and studied how youth activism in Boston led to large increases in spending on youth jobs and reductions in violence if community partners with local, contextual knowledge had not raised the issue. Calderón’s community partners were concerned with growing attacks on Latino immigrants and called for research and action on Measure T and the suppression of Latino voting rights.

Despite the potential advantages of CCES both for knowledge production and action, tensions can arise in these university–community partnerships. One challenge arises from the different priorities and trainings of organizers—focused on action—versus scholars—focused on research. In Kupscznk’s case, researchers faced many challenges in working with community partners to focus research questions. These activists had a deep passion for their work and they “develop ideas faster than they can communicate them.” Many meetings and iterations were required to create a focused research design that would be rigorous.

Perhaps more fundamentally, community activists have their primary accountability to advancing the interests of their constituencies. Scholars have a responsibility to a wider audience, building knowledge in the academic disciplines and in the public sphere. Community organizations are also interested in self-reflection and exploring tensions in their work to learn from them to advance their own practice and to contribute lessons to a larger organizing world. But sometimes, there is tension between short-term and long-term interests. Warren, in particular, faced some of the tensions that can occur when his responsibility to produce independent scholarship clashed with the immediate needs of community participants. Warren revealed weaknesses when community organizations wanted to emphasize strengths at that time.

Throughout the above cases, we learn about the importance of building relationships of trust and mutual respect. These relationships are the foundation for conducting CCES. Even if the tensions identified above can never be fully resolved, it appears that strong, mutually respectful, and long-lasting relationships can create the context for mitigating tensions and allowing both knowledge and action to proceed.

Conclusion

CCES represents a partnership between researchers and community change agents designed to create knowledge that helps to advance social justice. In that sense, it is decidedly not neutral. It critiques systems of inequality and injustice and plays a role in advocacy efforts to advance social change. The best interests of these advocacy efforts lie in conducting rigorous, systematic research that stands the test of critique from experts representing multiple perspectives in the academy and in the community. Many kinds of tensions can arise in these collaborations, including when short-term organizational interests conflict with the long-term needs of the broader social justice movement. In the end, the relationships created through CCES create the space through which to address inevitable tensions and build a larger movement for social justice. It is this larger movement that is the ultimate goal of CCES.