Abstract
This article details the challenges we faced in collecting sensitive information in an ethnically and religiously divided community that has experienced recent violence. The discussion is based on a summer 2016 survey experiment we conducted in Jos, Nigeria, to gather information regarding residents' perceptions of local communal violence. We discuss the challenges of such research and our approach to randomized sampling, constructing treatments that minimize the stress to respondents, debriefing to lower the possibility of spreading rumors of conflict, and utilizing computer tablets to increase access to the study for respondents with varying languages and levels of literacy. In particular, we discuss a geographic sampling method used for randomization, which we hope will prove useful to others facing similar randomization challenges.
In the summer of 2016, we conducted a survey in Jos, Nigeria, in an attempt to ascertain the perceptions of individuals living in a locality divided along ethnic lines where such divisions had defined recent communal violence. We agree with Wood’s (2006) and Longman’s (2013) observations not only that people, even those who have experienced conflict and violence, are more open to and desirous of talking about their experience or views than one might expect, but that it is important that elite views are not generalized to the whole population and that the perspectives of the everyday people affected by such conflict are not overlooked in conflict research. As Longman (2013:167) observes, “Average citizens have a clear understanding of how conflicts take place in their communities and why people choose to participate or not. Much of the national-level literature on conflicts tends to treat the general population as an undifferentiated mass, completely prone to manipulation.” Numerous challenges, however, are present when working in societies divided along ethnic lines and by communal conflict. Despite this, many scholars discussing the challenges and potential solutions focus on qualitative field research (such as interviews; e.g., Browne and Moffett 2014; Campbell 2017; Fujii 2008; Helbardt, Hellmann-Rajanayagam, and Korff 2010; Longman 2013; Moss, Uluğ, and Acar 2019; Shesterinina 2018; Wood 2013), with less direct methodological reflection on the challenges and solutions related to quantitative surveys and experiments in these societies. 1 We hope to address this shortcoming by presenting the significant challenges—such as developing representative sampling techniques, avoiding the spread of dangerous rumors, and protecting researchers and subjects from harm—alongside the strategies we developed to address these issues.
For scholars of communal violence and civil wars, Jos, Nigeria, represents a promising and intriguing “laboratory” for testing theories of ethnicity, conflict, and peacebuilding. Straddling Nigeria’s northern and southern regions, a variety of religious and tribal identities can be found in the Jos area. Along with the majority Christian population, there is a significant Muslim minority (particularly in Jos North), as well as adherents to more traditional religious beliefs. There are also differences among the religious sects themselves, with Christians divided among a variety of denominations, including “mainline” and “evangelical” Protestant churches. A number of ethno-tribal affiliations also exist both within and between these communities, where the Muslim minority is primarily Hausa–Fulani and the Christian majority is made up of Berom, Anaguta, and Afizere groups. Due to the constellation of ethnic identities, Jos provides scholars the ability to leverage the diversity to better understand identities and conflict.
Unfortunately, Jos’ ethnic groups have become highly segregated, with ethnicity a salient division in a series of communal conflicts. Competing claims to ethnic dominance or political control of Jos date back to the colonial era and settlement of Jos when new waves of Hausa migrants from the north (and minorities from other parts of Nigeria) settled in Jos to work in the tin mining industry (e.g., Best 2008; Krause 2011; Kwaja 2011; Madueke 2018; Milligan 2013; Ostien 2009). After 1994, following the creation of new local government areas dividing Jos and disputes over political control, political tensions began to threaten communal peace. The first major violent clash between indigenous Christian ethno-tribal groups and Muslim Hausa–Fulani (considered nonindigenous by the dominant Christian ethno-tribal groups) broke out in 2001 (Madueke 2018; Ostien 2009:11-12; Osaretin and Akov 2013:353; Vinson 2017:218). Subsequent major clashes sparked by political events or religious offenses followed in 2002, 2008, and 2010, along with numerous smaller incidents of violence. The 2001 Jos violence spurred the segregation of city neighborhoods into largely Hausa Muslim and Christian ethno-tribal enclaves, based on the ethno-tribal/-religious dynamics of the violence (Krause 2018:213; Madueke 2018; author omitted).
The history of violence coupled with the continued segregation and tensions along ethnic divisions present a variety of challenges for conducting safe and effective survey experiments in the area, and we developed numerous strategies to address these (see Table 1 for a summary). The importance of abiding by strong ethical standards to reduce risk for the researcher, local partners, and participants has received increasing attention in recent years as scholars reflect on their challenges and choices in fieldwork settings (see Browne and Moffett 2014; Campbell 2017; Cohen and Arieli 2011; Cronin-Furman 2018; Duggan and Bush 2014; Fujii 2008; Gerharz 2017; Greenwald 2009; Helbardt et al. 2010; Höglund and Öberg 2011; Kovats-Bernat 2002; Mazurana, Jacobson, and Gale 2013; Moss et al. 2019; Nakray, Alston, and Whittenbury 2016; Shesterinina 2018; Sriram 2009; Wood 2006).
Summary of Challenges, Strategies, and Conclusions of Methods Utilized.
Another fundamental methodological concern is how to effectively conduct a survey experiment at all in a divided society such as Jos, where the preferred procedure of random draws from a population list is impossible. Prior researchers have developed methods for dealing with this randomization challenge, the most common of which is a “random walk” procedure (see Adida et al. 2017a, 2017b; Blair and Roessler 2018; Linke et al. 2015, for recent examples). As Lupu and Michelitch (2018:199) note, however, “human beings are notoriously bad at randomizing, and recent research shows that enumerators’ instructions on conducting a random walk systematically affect both selection probabilities and survey results” (Bauer 2014, 2016). We believe there may be specific reasons why a “random walk” procedure may be problematic within a divided society that has experienced conflict, and so we developed an alternative geographic sampling procedure. This geographic sampling method involves randomly selecting locations within Jos based on publicly available satellite imagery of structures and then training research assistants (RAs) to conduct surveys at these locations. As with our discussion of the ethical considerations, we hope that this geographical sampling methodology serves as an inspiration for those attempting to develop survey experiments in areas where divisions are salient for potential conflict.
Protecting Researchers and Subjects in Societies Divided by Conflict
Conflict Context and Mitigating Risks for Research Personnel
We believe the first priority of research in divided societies such as Jos is to minimize risks to both participants and researchers, but the challenges for research in divided conflict communities can vary considerably, depending on the type of ongoing or recent conflict and the research methods employed. Some scholars emphasize how the particular risks to researchers and research subjects depend on context—for example, the target of the violence, the number of warring parties or insurgent groups, whether the conflict is ongoing, whether the state is one of the parties to the conflict, and the type of violence employed (Duggan and Bush 2014; Helbardt et al. 2010; Mazurana, Gale, and Jacobson 2013; Paluck 2009; Wood 2006, 2013). For example, if the population is the subject of government repression (either in the context of violent conflict or its aftermath), cooperation with outside researchers on sensitive subjects entails risks to those choosing to participate in a study or agreeing to be interviewed, even if informed consent is followed (Browne and Moffett 2014; Campbell 2017; Cronin-Furman 2018; Eck 2011; Fujii 2008; Helbardt et al. 2010; Kovats-Bernat 2002; Longman 2013; Moss et al. 2019; Shesterinina 2018; Wood 2013). As Shesterinina (2018:192) notes, “Participation in research in these highly politicized conditions can result in retraumatization, local retaliation, and state investigations that can subject interlocutors to imprisonment, torture, and even death for sharing politically sensitive or potentially compromising information.” Alternatively, conflict zones (even during times of peace) may involve dealing with insular communities that are wary of outsiders given the potential return to conflict—that is, what Cohen and Arieli (2011) discuss as a “cold peace.” Hence, the design of this survey work addressing Jos conflict involved carefully establishing the legitimacy of the research and credibility of survey enumerators (hereafter referred to as RAs), obtaining informed consent and debriefing subjects, and, importantly, ensuring and monitoring the security and safety of the survey RAs themselves. We also were not conducting the experiment during an election year when sensitivity or tensions might run higher, and the time in which the survey was conducted was one of relative calm. While the success of the research process and methods in the field can never be known in advance, scholars note the importance of significant local knowledge and familiarity with the conflict context in order to reduce negative consequences (Campbell 2017; Cronin-Furman 2018; Eck 2011; Fujii 2008; Longman 2013; Moss et al. 2019; Shesterinina 2018; Sluka 2012). We hope that the thought put into these elements and the lessons learned, including the unintended benefits of the research design, will be helpful to other scholars considering survey work in conflict settings.
Suspicion and Trust
A main challenge of working in a community affected by conflict is the suspicion between members of the warring groups, suspicion of outside researchers, and the challenges of establishing trust as a researcher. As previously noted, divisions in Jos between Christian and Muslim populations have led to a segregation of the urban area into geographically clustered communities and enclaves. Consequently, outsiders to local communities can be viewed with suspicion, particularly if outsiders are believed to be the members of rival ethnic groups. Strangers visiting neighborhoods may lead to distress among local residents and may pose safety issues for researchers.
To address this challenge, we adopted a number of strategies to carry out the survey work with sensitivity to the local dynamics and respect for communal norms. First, we recruited local RAs from Jos who represent the ethnic and religious identities of the various communities. We considered working with a third-party survey firm, but this was prohibitive not only because it was unaffordable but because we could not guarantee the level of monitoring and sophistication we believed this kind of survey project required (see Eck 2011, for discussion). Having already established connections with the University of Jos and its Center for Peace & Conflict Management during previous fieldwork, we were able to make arrangements in the months preceding the survey work to recruit RAs recommended by trusted faculty, and we used an application to gain background and identity information of the RA applicants, who were largely a mix of masters, undergraduate, and PhD students from the University of Jos. 2 Local universities can provide enumerators who are motivated to conduct research and sympathetic to the importance of research standards and ethics. Furthermore, due to the diverse student body at the University of Jos, we were able to recruit Christian and Muslim RAs who were accustomed to working in a cooperative academic environment. Local survey enumerators also understand communal norms and the best cultural practices for respectful interaction, a factor considered critical for the success of field research (Cohen and Arieli 2011:432; Eck 2011:169-70; see also Moss et al. 2019).
Drawing on local knowledge and collaboration to assess context and changes in the local environment can enable the success of the research and ensure the safety of locals recruited to assist in the research (Campbell 2017; Longman 2013; Moss et al. 2019; Paluck 2009; Wood 2013). Our relationship with the RAs was critical to the success of the project in terms of both their identity and their local knowledge. We could confirm whether the neighborhoods they would be entering were acceptable for them to visit, given their ethnic (tribal/religious) identity or whether there were any subtle dangers or risks. As Paluck (2009:45) notes, “It is an important methodological and even ethical consideration to compose a research team that will invite the confidence and frankness of the whole range of participants in the sample,” as this can “make them more or less likely to be perceived by participants as biased, foreign, or unfriendly, which would hamper their ability to work…and in some cases put the researchers in danger.” While our sense of the ethno-tribal/religious boundaries within Jos communities was generally on target, there were times when we reassigned a selected survey site to a pair of RAs with a different tribal/religious makeup. For example, if the community RAs would be entering was predominantly Muslim, we would make sure the RAs were likewise Muslim. If the community was mixed, RAs would be paired Muslim/Christian and so on. Given that members of the Muslim community in Jos could find it unacceptable for a male RA to administer a survey to a female, we generally sent our Muslim RAs out in mixed male/female pairs. Despite our fairly extensive local knowledge from previous fieldwork, the RAs’ deeper local knowledge about where exactly community boundaries lay was essential for avoiding missteps that would endanger the security of our RAs or create any communal confusion and tensions. 3 Nonetheless, they were encouraged to leave any neighborhood in which either RA felt uncomfortable. Daily debriefings were important, not only for RA reflections on participant encounters and surveys but also to emphasize to some RAs with more outgoing/risk-taking personalities not to compromise their security. Based on our experience, Eck (2011:175) offers sage advice in noting that “Researchers should not equate willingness on the part of enumerators to enter dangerous areas with the idea that this is then an acceptable ethical practice,” and researchers should “rein them in” as necessary. While the safety of research participants is often emphasized, it also is critical to take precautions to protect the local research team/RAs (Cronin-Furman 2018; Shesterinina 2018).
We also sought to reduce suspicion by providing RAs with copies of letters of support (backing the academic study and the RAs themselves) provided by the University of Jos’ Head of the Department of General & Applied Psychology, the Head of the Department of Political Science, the Dean of the Faculty of Social Sciences, and the Director of the Center for Conflict Management & Peace Studies. With each potential survey subject, RAs allowed subjects to view these letters of support (which RAs also offered to read to participants) prior to obtaining informed consent, and RAs carried with them picture identification from the university. While RAs reported that this documentation was generally not of paramount importance, there was the occasional case in which a participant or community leader asked for documentation establishing the veracity of the academic research and then carefully inspected the letters of support before agreeing to participate or allowing the RAs to continue with their work. Documentation of independent institutional backing is not a substitute for other precautions in the design of field research in conflict communities, but it can be an extra assurance for potential participants that the research is legitimate and RAs are properly carrying out their duties.
RA Training and Protocol for Reducing Risks
An important component of the survey experiment’s success was rigorous training to ensure that RAs understood research ethics, risks or problems they might face, and how to address challenges. 4 First, to prioritize safety, we emphasized that if RAs sensed any hostility in a community, they were instructed to immediately leave the household or area, go on to the next survey location, and note the issue on their survey completion sheet, so that other RAs could be directed to avoid that specific neighborhood. Training sessions also included role-playing in which RAs were tested on their ability to respectfully and effectively navigate or remove themselves from a range of situations in an interpersonally and culturally sensitive manner. Daily debriefings with each pair of RAs allowed us to ensure that protocol was being followed and concerns were addressed. RAs were also provided with phone credit and encouraged to call while out in the field if questions of any type or concerns arose. Thankfully, threats, hostility, and a general sense of insecurity were nearly never reported by the RAs (only three cases out of roughly 2,200 required use of safety protocols). 5
Second, a critical factor was having RAs (particularly Muslim RAs) first make themselves and the nature of their research known to the local chief/district head or leadership presiding over a community, an approach consistent with local cultural norms and, in some cases, a security measure. As Cohen and Arieli (2011:425) observe, “The researcher, whether a party to the conflict or not, faces the challenge of gaining familiarity with and cooperation from the research population in this environment, which may be closed to and mistrusting of outsiders” (see also Browne and Moffett 2014; Cohen and Arieli 2011; Eck 2011:175-76; Shesterinina 2018:196). Due to past communal violence, leaders of some Jos neighborhoods (including youth vigilante groups) are careful to monitor the arrival of visitors. Local community members would often personally direct the RAs to the local chief, so they could explain the purpose of their visit, share the research documentation, and receive the leader’s approval to continue with their work. For example, in one community, after one Muslim RA met the local Imam and went to pray at the local mosque, the Imam announced after prayers to those gathered at the mosque that the RAs were doing important research and the people should be welcoming. 6
The task of recruiting, selecting, training, deploying, and monitoring RAs in the field can be a delicate one in conflict communities. Jos had been relatively calm in recent years, and we were not overly concerned about the sensitivity of our specific survey experiment design. However, we preferred to err on the side of utmost caution given the uncertainties of carrying out conflict research in a conflict community. Despite (or perhaps because of) these precautions, RAs encountered few challenges or concerns and more often reported enjoying the interactions with a wide array of Jos residents. Although being in the field and daily monitoring and debriefing with RAs was a more exhausting way to carry out fieldwork, it helped gain access to communities through effective outreach and ensured the safety of our RAs.
Survey Experiment Design Considerations
Randomization
Turning to the survey experiment design, a problem posed by ethnic segregation in Jos and its environs is that traditional sampling techniques may lead to inherent bias, such as starting researchers at polling stations within a city and then having these researchers walk in random directions from a polling place using a prescribed protocol (see Adida 2012; Adida et al. 2017a; Blair and Roessler 2018; Conroy-Krutz 2013; Finkel, Horowitz, and Rojo-Mendoza 2012; Humphreys and Weinstein 2012; Linke et. al. 2015). In divided local communities with a history of ethnic violence, however, it can be dangerous to send researchers to randomly walk a city when a random turn can lead an individual across a boundary into a community that, given that researcher’s ethnic or religious identity, can be dangerous. For example, during previous fieldwork, while traveling by car through the city of Jos with two RAs, the driver took a shortcut through a Muslim community, and the Christian RAs became noticeably nervous. They explained that this was a predominantly Muslim neighborhood where there had been past violence and hence one they avoided. To ensure the safety of researchers and residents, those conducting a survey or randomized fieldwork will have to “break” the random walk protocol in ways that will lead to bias in the resulting sample, which limits the efficacy randomization provides. A further potential advantage of polling places—that such locations will follow the population density of the area—may be a problematic assumption. Some communities may be underserved by polling locations for political or security reasons.
We developed a geographic randomization technique using GPS and satellite photographs, allowing us to demarcate sampling sites where local knowledge could be used to identify the specific communities. To conduct the random sample, a large polygon was placed over Jos using GIS software (QGIS Development Team 2016), and a number of grid cells defined by a latitude and longitude were drawn within this polygon, with each cell representing a potential sampling area.
Randomly sampling from these cells without further adjustments is problematic, however, as geographic randomization does not necessarily mean population randomization. As Jos is an urban area, some locations are more densely populated than others, and without further weighting, a cell from an outlying area of Jos will have the same probability to be sampled as a densely populated urban community or a busy outdoor market. Reliable data on the cell-by-cell population density were impossible to obtain (and would nonetheless change throughout the day dynamically), so a proxy had to be developed to increase the weight of particularly population-dense cells.
Satellite photography provided such a proxy, as Jos was relatively well-photographed with easily accessible and publicly available data. Specifically, we used Google Earth to access this data, as the software (1) aggregates a number of satellite sources at different levels, meaning that there would be good coverage of the entire polygon, (2) is updated regularly with new images, and (3) allowed us to easily overlay our constructed grid cells over the relevant satellite photography (Google 2016). The grid cells were overlaid over the satellite imagery, so a randomly sampled grid cell was then associated with overhead photographs allowing for the identification of distinct structures. The number of distinct structures within each cell provided a proxy weighting for density. Due to inconsistent spacing between buildings and roofing overhangs, it could be challenging to be certain of every demarcation between buildings, though the inconsistent use of roofing materials can aid in the demarcation task. Gaps between buildings and clear delineations in roofing material thus formed the basis for identifying individual buildings on the satellite maps. 7
The final sampling technique thus involved randomly sampling grid cells within the polygon and subsequently selecting random structures within each of the cells. Each distinct structure within selected cells was numbered, and the total number of structures used to determine how many structures would be sampled within a cell, with one structure sampled for every five structures. Each randomly selected sample structure could be associated with a pair of latitude and longitude coordinates and placed within the political and ethnic context of the local community using the expertise of local residents prior to arrival. The resulting series of GPS coordinates was associated with either Muslim or Christian (or mixed) areas, meaning that Muslim and Christian RAs could be sent to appropriate locations.
An additional benefit of this method was that we were not restricted to residential areas. As surveys needed to be conducted during daylight hours for safety purposes, it was possible for particular groups of individuals to be inadvertently excluded from the sample if they engage in work or conduct business in commercial parts of the city during the day. Our sampling method allowed RAs to encounter individuals as they went about their day, regardless of whether they were in their residence. Hence, RAs would occasionally conduct interviews with individuals working in their small street shops or at larger businesses. RAs, out of their own volition on a number of occasions, assisted participants with their duties to enable them to participate, including assisting a tailor with his sewing, playing with/distracting children or holding a baby, or finishing the ironing.
Although providing directions to each location was possible, we believe the use of traditional paper maps would have prevented RAs from successfully completing their tasks. Simply providing directions and maps could also lead researchers through communities where they might be viewed with suspicion. Therefore, RAs were provided with a GPS device with the locations for which they were responsible and given wide latitude in determining their path to each location. These devices were loaded with locations every morning and purged of these locations at the end of each day to prevent the researchers from sampling locations multiple times. These devices could also be checked at the end of the day to ensure that researchers reached the indicated points, as the GPS devices would track their movements during the day. 8 This was a critical feature of this method, as we would otherwise have no way of being sure RAs reached the designated sites. Ultimately, by providing geographic coordinates, identifying particular communities ahead of time, and giving RAs wide discretion in the route taken to these locations, we hoped to avoid the potential pitfalls of existing methods for developing a random sample.
Sensitivity of Treatments and Questions
Our interest in communal violence as part of this project meant that extra care had to be taken when dealing with individuals who were familiar with local violence (Campbell 2017; Cronin-Furman 2018; Fujii 2008; Helbardt et al. 2010; Longman 2013; Shesterinina 2018; Wood 2013). The most important challenge was presenting individuals with hypothetical conflict scenarios as a randomized treatment without causing misunderstanding resulting in distress or rumors, the latter of which is known to play a role in conflict dynamics (Bass 2008; Bhavnani, Findley, and Kuklinski 2009; Bubandt 2008; Espeland 2011; Greenhill and Oppenheim 2017; Larson and Lewis 2018). Safety was paramount in this regard, both for RAs and for residents. However, we ultimately agreed with Wood (2013:304): “Yet this uncertainty should not mean that research cannot be conducted…but that the researcher should engage continuously in critical analysis, self-reflection, and dialogue in order to remain alert to unintended consequences of the project.” Another challenge is that many residents of Jos have borne witness to past violence, and we did not want individuals to relive these experiences. Finally, residents needed to feel comfortable answering truthfully rather than tactfully or politically (i.e., “social desirability bias”; see Mutz 2011:29), as the censoring of answers would not only bias any results but would be largely impossible to detect later.
The first set of precautions was largely conventional. First, RAs identified themselves and noted their affiliation with the University of Jos. Second, sampled residents were instructed that although their responses were important for the research project, anonymity would be maintained. 9 Third, if a resident was uncomfortable with a particular question, they could forgo answering this and subsequent questions altogether. Finally, regardless of whether individuals completed the full survey, every RA was instructed to debrief the respondents that any scenarios presented in the survey were hypothetical and did not actually occur, allaying fears that actual violence had taken place. This was one of the most emphasized protocols in training, and RAs were instructed to obtain a verbal assent that the participant understood that the news prompt was fictitious. To reinforce this protocol at each site, we also provided the RAs with a “Survey Completion Form” where RAs had to check a box to indicate that they provided the debriefing (in addition to supplying the GPS reference code for each site, documenting the number of eligible participants at each location, indicating whether the participant completed the survey, and providing a reason code if a survey was not conducted or was incomplete). These Survey Completion Forms were reviewed each day with the RAs upon their return. The logic of sending out RAs in pairs was also to give each RA a partner who could not only help share in survey administration but also to remind one another of the protocol. Ideally, we would have included a debriefing statement about the fictional nature of the prompt in the conclusion of the survey itself. However, given that participants could choose to end the survey at any time, the limits of the survey software did not allow us to tie the debriefing information to whatever point a participant might choose to end the survey. Hence, while both an electronic and verbal debriefing would have been ideal, we had to rely on verbal debriefing alone.
The construction of the treatments and research questions were of particular concern, as the survey included a randomized treatment that would prime respondents on particular identity dimensions in relation to conflict. Individuals, however, would already hold particularly strong opinions about conflict and other communities, making it difficult to separate an individual’s opinion from a larger narrative that a community might construct to explain violence. The most straightforward way to prime respondents was to present a treatment about hypothetical violence that indicated either a religious or a tribal motivation. By not referring to past violence, we could avoid preconstructed explanations or triggering traumatic experiences related to previous episodes of past violence. Therefore, the hypothetical scenario required plausibility as any scenario that lacked credibility would not provide any leverage for questions tied to the treatment. On the other hand, any hypothetical scenario that included particular details could lead to distress and rumors, a dangerous reality in areas with ethnic conflict (e.g., Bass 2008; Bubant 2008; Espeland 2011; Greenhill and Oppenheim 2017). We erred on the side of caution because even though this might decrease the “power” of the treatments, any evidence associated with the treatments would be a “harder” test of any of our hypotheses. This was a compromise we were willing to make, and we feel it is the most ethical one in this case, as any hypothetical scenario would involve deceiving participants and had to be balanced against other considerations (see The Belmont Report 1979, for a clear statement of such trade-offs; see Mutz 2011, for a discussion of deception and manipulation in survey experiments). We believe we were following the suggestion of Fujii (2008:3) that “The question of what constitutes ethical behavior depends in part on the social, political, and cultural context in which the study is taking place and the researchers’ sensitivities and abilities to respond to and operate within that context in ways that minimize risks to participants and to themselves.”
Ultimately, the treatment referenced a recent bout of deadly violence, as well as previous conflicts (broadly speaking), so that those receiving the treatment connect this violence to historic patterns in the area. Outside of this, however, the treatment is vague in terms of the location of the violence. There is no reference to Jos, or Plateau State (the federal unit for which Jos is the seat), thus minimizing the fear that the violence would be a threat to the reader. The treatments are also vague in terms of the parties involved. Alternative treatments mention religious or tribal violence, but not the specific religious or tribal identity of the victims or the perpetrators. Including such information would certainly increase the leverage of the treatment but would also personalize the violence for the survey taker in ways that might lead to anger, distress, or further revisiting of trauma.
It is important to note here that the process of constructing the treatments was also informed by previous fieldwork and in-depth knowledge of Nigerian media’s reporting of ethnic, political, and terrorist violence in the country. We therefore knew from these experiences that individuals often discussed conflict in a variety of contexts (including places of worship, universities, nongovernmental organization organized peace meetings, in theatre productions) and that participants would likely be willing to talk about conflict with the RAs in relation to our survey. In prior research, one of the authors created an original data set on ethnic and religious violence in northern Nigeria from hand coding over 500 cases of violence between 1979 and 2011 (from over 13,000 editions of a major Nigerian newspaper), as well as studying and collecting other local sources/magazines reporting on inter-group violence (see Vinson 2017, 2020). This background knowledge gave us much more confidence in designing a basic and believable news story prompt and also reassured us that reading such a story would be ordinary for participants exposed to similar daily news stories (that are often more detailed regarding the violence and suspected actors). We would have been more concerned about the ethical boundaries of the treatment design without this prior local knowledge/research. We also vetted the treatments with longtime Western professors at the University of Jos who understand both the local culture/context and research methods, and we also hired a local expert to provide back translation and comment on problematic wording or questions.
A dry run of an earlier version of the survey instrument was also conducted with 104 students in an Experimental Methods class at the University of Jos approximately two years prior. The survey was more limited in a number of respects—for example, (1) it was not a randomized sample or representative of the sociodemographic and religious makeup of the Jos population, (2) it could only be administered as a hardcopy survey (hence more constraints on length), and (3) only two prompts/treatments could be used given the small sample size. It was useful, however, in addressing any hiccups with translation or question wording, survey length, and question order that would need to be addressed in the larger study. Although we conducted no focus group discussions, we did not receive reports of concern with the treatment wording or design from the University of Jos professor who administered the survey to the class on our behalf and who led follow-up discussion of the survey experiment and methods.
Reassuringly, the RAs in daily debriefings reported few signs of stressed or disturbed participants as a result of survey participation; 10 rather, people who refused to participate or ended the survey early did so most often because they were too busy and had chores to attend to. 11 A number of participants commented, following the RAs debriefing them about the fictional nature of the news story, that they thought it was a real news story, which speaks to the importance of debriefing. We must be cautious, however, in concluding that this study resulted in no significant discomfort, reliving of trauma, or undue stress, as these may not always be visible or verbalized. There are two additional steps that we would have adopted in hindsight that we would recommend. First, conducting small focus group sessions with volunteers from relevant cross sections of the population (e.g., different ethnic identities, ages, and gender) to discuss reactions to or concerns with the treatment and survey questions is another way to assess the sensitivity of the treatment. We believe this would offer more direct insight into the sensitivity of the survey design than a general pretest of the survey questions with some sample of participants (although a pretest or pilot study has its important uses as well). Second, we recommend the inclusion of brief follow-up survey questions for this type of survey experiment. For example, a question about how realistic or believable participants felt the prompt was, whether they felt uncomfortable reading about the event described and answering questions, and an open-ended question from RAs about whether participants have any concerns about what they read that could be addressed. 12
Language and Literacy
The use of tablets for survey completion raised a number of additional issues of literacy and language. Based on a 2010 National Literacy Survey, the overall adult (15 and above) English literacy rate of Nigeria is around 58 percent, with considerable urban versus rural difference (around 70 percent and 39 percent, respectively). The estimated literacy rate for Plateau state, of which Jos is the capital, is nearly 60 percent, and the three Jos local government areas of Jos East, Jos North, and Jos South all have a 73 percent literacy rate (National Bureau of Statistics 2010). Therefore, a nontrivial portion of the population of Jos would be unable to complete the survey without a translator. 13 Use of translators may, however, introduce potential bias into the sample along educational cleavages. Furthermore, any tablet survey would need to be reliably translated for different languages, as different communities use different languages. As English is our own first language and the official language of Nigeria, this meant that only one translation was necessary and could be readily accomplished with translation and then independent back translation (with translators recommended by local contacts, including University of Jos faculty). 14
Distributing tablets with written text presents two simultaneous challenges to respondents: (1) difficulty understanding the written instructions and (2) the ability to use software for response entry with which they have no prior experience. We required a software program that could be installed on each tablet that would allow for audio prompts to be delivered alongside written questions and answers, along with visual indicators of how to enter responses on the device. The software eventually chosen was the Nova Research Company’s (2016) Questionnaire Development System with audio computer-assisted self-interviewing (ACASI) capability, which boasts both the ability to add audio and to sync audio clips with visual indication of entry buttons. The research team then hired a local assistant recommended by the University of Jos faculty to record each of the research questions and responses, in both English and Hausa, so that participants could choose their preferred language at the beginning of the survey. While a time-consuming process to prepare the translation work, record the audio, and upload it with the survey software, RAs were able to present the tablet device to willing participants and then briefly, through a short tutorial, 15 illustrate how questions and answers are presented in both written and audible form and how responses are entered. Respondents could then read and/or use earphones to listen to prompts and answer the questions with some privacy.
Tablets, however, added logistical burdens to the research team while in the field. First, tablets need to record data locally to the device, as unreliable wireless access will always be a concern (not to mention the additional security concerns of broadcasting answers to sensitive questions over wireless networks). Furthermore, the device’s battery needs to last an entire day, especially when researchers are being sent over long distances and cannot easily return to a central location or find ways to recharge during the day. Likewise, the research team needs to charge tablets at the end of each day or overnight. 16 While these logistical burdens added to the complexity of the project, the technology minimized stress on participants and increased the likelihood of accurate measurements of perceptions. One of the significant benefits of the use of tablets, however, was decreasing the burdens on RAs while in the field. After initial training on the devices, RAs were able to focus their attention on safely traveling from location to location rather than the sorting and storing of completed surveys or ensuring the randomization of any experimental prompts. By offloading as much of the conduct of the survey to the software, 17 RAs were free to focus on the sampling and safety protocols—a clear advantage in communities that have experienced conflict.
Lessons and Conclusions
Initially, 2,209 completed surveys were collected from May 23 to June 30, 2016, through the efforts of 18 RAs. The technological elements of the survey experiment and the training of the RAs proved critical to the success of the project. After two days of GPS device training, only one of the 18 RAs needed to be retrained. By the end of the survey experiment, RAs were quite familiar with the process of moving from location to location using the devices and completed their tasks with impressive efficiency and dedication. As discussed, one key benefit of using GPS devices was that the devices tracked the movements of RAs, such that we could examine the paths taken to ensure that RAs indeed arrived at the locations they indicated surveys were completed. While GPS devices recorded movements of RAs, the tablets recorded the survey timing (specifically when surveys were begun and when they ended), 18 which became critical when, upon analysis, it became clear that some surveys took a suspiciously short amount of time to complete and that these surveys were associated with a single RA. After sharing this information, at least one RA admitted that they had witnessed this partnered RA simply taking a survey once they had reached a location and found that a participant was not available or home. This meant that there was an unreliable RA, and we had to drop 284 surveys associated with this RA.
The novelty of the technology may have inadvertently increased the willingness of participants to complete the survey, and the ACASI capability also meant few barriers to participation. RAs reported that anyone, no matter their education level, was able to complete the survey. Participants, if literate, could complete the survey in either English or Hausa and those who were illiterate could listen to the audio version. Participants often both read through the survey and used the audio to ensure maximum understanding. Several RAs reported that participants were excited to use the tablet software. In a mock training session on the tablets among residents within the University of Jos compound, RAs reported that one woman was proud to have participated because her teenage children saw she was capable of using advanced technology. Likewise, a number of participants over 65 had no trouble interacting with the tablet and survey software. RAs reported some older participants not only could use the tablet but were enthusiastically engaging with the questions—for example, nodding their heads as they listened to the questions, gesturing with enthusiasm as questions and answers appeared, and exclaiming aloud their views as they went along (despite RA reminders that the device does not record what they say and they need not express their views aloud). Some participants expressed a wish for even more questions (despite it already being a long survey) or wanted to further expound on their personal experience of losing homes, family members, friends, and so on. Although anecdotal, RAs indicated that it was the individuals who expressed how much they lost in past violence that also seemed to be the happiest about the work the RAs were doing. In particular, they expressed gratefulness that regular people of Jos were able to express their views and opinions rather than research that only includes the voices and opinions of the “big men” elite. In general, we were struck by the enthusiasm of participants. Similarly, Longman (2013:256-57) notes in his discussion of research in conflict zones, the unexpected discovery that, despite the dangers, “[p]eople are often much more willing to speak openly than one would expect.…Average people—farmers, day laborers, market women—not only are much better informed than most elites (whether domestic or international) assume, but are also often very willing to reveal what they know.”
Although the geographical sampling method was successfully completed in Jos, there are several caveats to this success. First, while an adequate number of surveys were completed in a timely fashion, there is no way to verify that the sample collected is any more representative than one collected using traditional “random walk” techniques. In the end, both this geographical method and previously used methods are efforts to ameliorate the significant challenges posed by local contexts. Second, Jos is particularly well suited to this geographical approach. Satellite photography is readily available, and the architectural landscape of Jos means that basing population density weighting on the number of buildings is relatively effective as there are few large apartment complexes or multiple story buildings in Jos. Other metropolitan areas are not conducive to using building numbers as a proxy for population density: A high-rise apartment building in the middle of a major city in the United States would be weighted similar to a suburban home using this method. One limitation of this method is that buildings only serve as a proxy for population density. It is still possible that in one area of the city, a single building may house a single family, while in another area, a distinct building may be the residence of several families. Furthermore, recent satellite photography is readily available for Jos, which may not be the case for all locations where one may attempt such a method of geographic randomization. Despite the increase in the last several decades of satellite imagery of sufficient quality to attempt this method, rural areas and the areas immediately surrounding metropolitan areas may not have recent imagery available in all parts of the world.
One of the major lessons we learned was the significant trade-offs that exist when developing treatments meant to frame a particular aspect of conflict in areas that have recently experienced such conflict. In addition to avoiding treatments that might imply immediate threat and the dangers of reactivating trauma (see discussions above), treatments will always be translated through local contexts of conflict and identity. We suspected that Muslims and Christians might respond differently, as their identity and position within the community would lead to different ways of thinking about the treatments, and this turned out to be the case. Although the treatments did lead to interesting differences, these differences were not consistent between the Muslim and Christian subsamples (Vinson and Rudloff, forthcoming). These important differences highlight the importance of understanding the context in a community in order to anticipate how the intersection of conflict and identity may influence how individuals interpret conflict treatments that will potentially interact with these identities and experiences. This is where prior fieldwork and a pilot study can, yet again, play a valuable role in a study’s successful conclusion—knowledge of the context in a community can help to craft appropriate treatments prior to when the survey experiment enters the field, as well as inform how data is analyzed and conclusions drawn after work in the field has been completed.
The tainting of surveys by one RA could be seen either as a failure, since despite all of our efforts it still occurred, or as a success of the technology and methods used, since we were able to detect the problem. This could also be seen as an unavoidable hazard of fieldwork that relies on third parties: Despite training and monitoring, unreliable data can still infect a sample. Ultimately, we reached an unexpected conclusion—a skepticism for third party contracted surveys that are entered into the field (see also Eck 2011:168). Certainly, some perform admirably in a variety of conditions, and it certainly decreases the logistical burden on researchers when the details can be handled by individuals with experience and local knowledge. A good survey firm will no doubt deliver a sample that carefully follows protocol and indicate any issues that arose in the field. There is likely, however, great variation in the quality of these firms and the attempts taken to monitor enumerators. What is troubling is that there may be little way of knowing with clarity the quality of data provided by a contracted firm without careful oversight or a trusted independent survey manager. We learned the value of being constantly present in the field to ensure not only that protocol was followed, but that any necessary adjustments could be made after daily debriefings, that no unnecessary risks were being taken by our research team, and that we were available when questions arose. We agree with Paluck’s (2009:54) conclusion that it is critical to data collection in the field that the principal investigators be present and have “intimate engagement with all aspects of the field research.”
Supplemental Material
Supplemental Material, sj-pdf-1-smr-10.1177_0049124120986175 - Surveys in Communities Divided by Ethnicity and Conflict: Challenges, Possible Solutions, and Lessons Learned from a Survey in Jos, Nigeria
Supplemental Material, sj-pdf-1-smr-10.1177_0049124120986175 for Surveys in Communities Divided by Ethnicity and Conflict: Challenges, Possible Solutions, and Lessons Learned from a Survey in Jos, Nigeria by Peter Rudloff and Laura Thaut Vinson in Sociological Methods & Research
Supplemental Material
Supplemental Material, sj-pdf-2-smr-10.1177_0049124120986175 - Surveys in Communities Divided by Ethnicity and Conflict: Challenges, Possible Solutions, and Lessons Learned from a Survey in Jos, Nigeria
Supplemental Material, sj-pdf-2-smr-10.1177_0049124120986175 for Surveys in Communities Divided by Ethnicity and Conflict: Challenges, Possible Solutions, and Lessons Learned from a Survey in Jos, Nigeria by Peter Rudloff and Laura Thaut Vinson in Sociological Methods & Research
Footnotes
Acknowledgments
The authors wish to thank the University of Jos, including the Dean of the Faculty of Social Sciences, the Department of Political Science, and the Centre for Conflict Management and Peace Studies for their support and Katrina Korb, Danny McCain, and the many faculty at the University of Jos for the significant help they provided to this study while in the field. Special thanks to the research assistants for their tireless dedication the project: Abubakar Nabeel Abdulkareem, Fariatu Yahuza Ahmad, Chidozie Diamond Anyalewechi, Mafeng David Bot, Bulus Jonathan Charles, Salihu Madinatu, Omeka Matthew, Bot Polycarp Moses, Ruqayya Sulaiman Nayaya, Ibrahim Musa Nuhu, Temitope Olamide Osoba, Sanni Moses Peter, Abdullahi Salmanu Muhammad, Francis Emmanuel Tsaku, and Abdullahi Yusuf. We further thank these many research assistants who carried out the research in the field. Finally, we wish to thank Eve Ringsmuth and Matthew Motta for sharing their advice about the paper, and Stephen Nemeth for feedback and thoughts during the development of this project.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the Peace Research Grant Program of the International Peace Research Association Foundation.
Supplemental Material
Supplemental material for this article is available online.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
