Member checking: A feminist participatory analysis of the use of preliminary results pamphlets in cross-cultural,cross-language research

Introduction

Participatory research methodologies are increasingly common in geography (Chilvers, 2009; de Leeuw et al., 2012; Wynne-Jones et al., 2015). Often overlapping academic work and activism, these methods encourage the creation of shared knowledge among researchers and research participants in an attempt to challenge the existing social order and facilitate greater social justice and equality (Kinpaisby, 2008). This set of norms and routines aims, through reflexivity, to subvert the power relations existing in the research process and to foster the creation of a new type of knowledge that is not gender neutral, rational and objective but instead, local and useful for research participants (Code, 2006).

Adding another nuance to participatory research methodologies, there are feminist participatory research methodologies. Although both approaches stand on the same epistemological ground and are empowerment-oriented collective endeavors, feminist participatory methodologies depart from an analysis of gender relations. This inquiry helps materialize the masculine bias present both in local communities and in the process of knowledge construction. These practices are centered on women’s range of voices and capabilities to contest the unequal gender relations embedded in local institutions for the collective good (Reid et al., 2006).

Participation, dialogue and collaboration have become buzzwords within qualitative geographical methodology (Chilvers, 2009; de Leeuw et al., 2012; Wynne-Jones et al., 2015). However, how do we methodologically put participation, dialogue and collaboration into practice? Is their operationalization rigorous? These questions are often difficult to answer given not only the shortness of methods sections in most articles but also the lack of a common understanding regarding the meaning of these terms.

This article addresses these concerns by presenting a methodological tool—a pamphlet used repeatedly during a four-year research project on gendered agricultural practices in East Africa.

The article starts by presenting the overall research topic and context, followed by an explanation of the pamphlet and its modalities of use. The two subsequent sections debate the advantages of this method in relation to participation and validity and the barriers encountered in the employment of this tool. Concluding remarks then follow.

Research context

The participative method presented here was utilized in my PhD project which aimed to disclose local gender structures and how they were manifested in the agricultural landscape and the water management of two smallholder irrigation farming systems in Eastern Africa: Sibou, Kenya, and Engaruka, Tanzania. The results highlighted the local organization and gender structures as ‘hydropatriarchies’, with men having material and discursive control over water as a productive resource (see Caretta, 2014).

Fieldwork was conducted for a total of seven months between 2011 and 2015. The fieldwork included four long—at least seven weeks—sessions and four short visits. The study sites are located in dry-lands rural remote settings where subsistence farming is prevalent. English is not the first language in these sites. While most of the locals are conversant in Swahili, the local languages of Kimarakwet in Sibou, Kenya, and Kimaasai in Engaruka, Tanzania, are prevalently spoken. At the beginning of the research project, I learned Swahili to ensure that I could make myself understood, and I did not need translators during the interviews in Swahili. The participants, especially older ones, preferred discussing issues in their local languages, as it made them feel more at ease. Local research assistants were hired to arrange the research logistics and to translate (for an in-depth analysis on positionality, the role of language and assistants in my research, see Caretta, 2015).

A combination of qualitative methods was employed repeatedly during the research: focus groups, semi-structured interviews, participatory mapping, observation and member checking in the form of the pamphlets. Member checking, a method used to countercheck preliminary data in order to complement data triangulation (Cho and Trent, 2006), is the focus in this article. There were 13 focus groups with a minimum of seven participants per group, and 11 participatory mapping exercises were conducted by consulting almost 200 people in Siboual. Semi-structured interviews were carried out with a Siboual of 52 women and 31 men.

The pamphlet

The pamphlet consisted of an A4 paper sheet divided into 4 columns, which could be folded into two. The previously gathered data were categorized according to themes (e.g. labor, crops and climate); in simple language, the pamphlet reported the information that the participants had shared with me (see all figures).

Several photos were included, so that analphabets could recognize familiar faces and follow along as the content was being read aloud. Although it was far from a perfect solution for communicating these data to analphabets, as it could reinforce an unequal exchange in the research process, photo elicitation helped the participants to better remember the instance of our previous meeting and the discussions we had had. Moreover, as the vast majority of the participants are not exposed to the image culture, photos elicited their curiosity and memories, which stayed with them (see also Harper, 2002; Stone-MacDonald and Stone, 2013). Instead of reading the content immediately, literate participants reviewed the pictures one by one and tried to determine who was depicted in them and where they were taken. ¹ I created these pamphlets for my second and my third fieldwork trips in 2012 and 2013, respectively. In the first instance, rather than using it as a tool to check the preliminary findings, I employed the pamphlet as a means of reminding the participants about who I was, re-introducing myself and outlining the research objectives. The pamphlet was not only distributed to participants during focus groups or interviews but also to anyone who would stop my assistant and me during our walks and observations to talk to us. People were eager to obtain a copy. In the first pamphlet, the pictures of me dressed in their traditional ornaments often made people laugh, which opened the way for a discussion of the celebration in which they had given me their beads.

Triggering these memories created a relaxed atmosphere. The participants that we met randomly in the fields were sometimes available to take the time to peruse the pamphlet and discuss the themes of the research project. Those, on the other hand, who did not have time to stop to discuss the pamphlet would take a copy and then share it with their family members, including children. It was not unusual to have school pupils call me by my first name, saying, ‘I read about you!’ or to see farmers whom I had never met reading through the pamphlet. In the remote locations that did not have access to newspapers and where only a limited number of school books were available for pupils, distributing a written text generated an interest that made the pamphlet spread rapidly throughout the community. It can thus be argued that reaching out to villagers using an unusual artifact sparked curiosity and prompted awareness about my presence and the work I was doing. Notably, the text stated, ‘Martina will work on this project for five years, and she will come back from time to time’, which initiated the process of participation and collaboration with the local communities in a positive way.

OPEN IN VIEWER

Figure 1.

The first pamphlet in Kiswahili employed in 2012. The title read, ‘Current modes of irrigation in dry-lands in East Africa: understanding work, soil and climate’. 1. Agriculture; 2. Groups; 3. Weather.

I translated the first pamphlet into Swahili, thinking that it would be accessible to most participants; however, during its dissemination, research assistants in Kenya had to read it aloud in their local language, Kimarakwet. Moreover, the first pamphlet summarized short findings about both locations in Kenya and Tanzania under the same titles. There were pros and cons in this simplification. On one hand, it was confusing for assistants to clarify the distinction between the two study sites. On the other hand, it invited comparisons and questions from the participants about the agricultural practices of the other location. These insights guided the composition of the second pamphlet for the third fieldwork session of 2013. Then, two pamphlets were produced. One in Kimarakwet, translated by one of the local research assistants, and one in Swahili to be distributed in Tanzania. The amount of data gathered and the difficulty of summarizing these data in just two pages also influenced the decision to create two different pamphlets.

The data presented in the second pamphlet were more complex and reflected my last fieldwork session. I used contrasting pictures of women’s and men’s tasks (see Figure 2) and included discussion-provoking statements, such as ‘Without irrigation, the soil could not be fertile’ and ‘Women cannot irrigate … but things have changed. Women also assist men in repairing irrigation channels’, to test some of my inferences based on the data gathered and to harness the participants’ feedback.

OPEN IN VIEWER

Figure 2.

The second pamphlet in Kimarakwet employed in 2013. The title read, ‘Current modes of irrigation in dry-lands in East Africa: understanding work, soil and climate’. 1. Women’s and men’s work; 2. Weather and agriculture; 3. Agricultural practices.

Participation in practice

In order to get the conversation going participants were always presented with the pamphlet, as an icebreaker. Beginning in this way was useful to establish the research context and to recap what we had discussed previously. Repeated focus groups provided me with the opportunity to refine and verify data, and the participants had time to reflect on previous meetings and to react to the information that they had obtained (Morgan et al., 2008). By conducting repeated focus groups, mutual understanding and trust were created and further enhanced by adding on previous knowledge which was presented in the pamphlet. Several participants, whether they agreed or disagreed with the content, wanted to add details:

Green gram and watermelon seeds were first introduced in 2003. Tomatoes started being planted in the 1990s, while bananas and cassava were already being cultivated in the 1980s. (Water users’ committee focus group (only men), Sibou, Kenya, 20 July 2013)

My grandfather started cultivating in the 1950s; then there was no oxen plough, but he would use only the hoe. At that time, there were no horticultural cultivations, and we had no water users’ association. (Mbacha, Engaruka, Tanzania, 26 August 2013)

In the first statement, the participant provided a clarification of some data; the second statement complemented the text with a spontaneous piece of information, which I had not asked for. This instance, as highlighted by Stone-MacDonald and Stone (2013) in their use of feedback video interviews in Tanzania, illustrates that photos elicit unexpected responses and prompt new information that the researcher’s direct questions might have missed. Moreover, in transcultural interviews, misunderstandings tend to occur more frequently, but, according to Lagesen (2010) the use of a ‘boundary object’—in this case, the pamphlet–can establish common ground that facilitates an informal exchange and creates a space for inclusion and reciprocal understanding between two cultures.

When presenting the pamphlet, my assistants and I repeatedly stressed that thepamphlet did not contain ‘the truth’ about the participants’ community; it instead presented my interpretation of what they had told me. Hence, I could be wrong, and it was their responsibility to ensure that their community was rendered in its complexity and its entirety. In line with feminist epistemology (Code, 2006), I acknowledged my standpoint by opening myself to the participants’ contributions. In fact, while the design of the pamphlet could have implied authority, I stated, ‘You have cultivated all your life; I am here to learn your practices directly from you. I am just a student’. This statement called for the participants’ collaboration and support in fulfilling the research objectives. Thus, my presence and the production of the pamphlet became a catalyst to discuss local issues that could be of use to participants, and I conceded that I was not an expert (see Lather, 1986).

OPEN IN VIEWER

Figure 3.

Administration of the pamphlet. On the lower left-hand side and on the upper right-hand side, assistants read out the pamphlet to participants.

It can be argued that leaving participants to freely interpret and select whatever part of the pamphlet that they considered most relevant or interesting shows that this tool can potentially reverse the research power hierarchy, as I was forced to absorb unforeseen feedback and to adapt to new ways of seeing. On the other hand, asking participants to correct my understanding was an equalizing methodological approach because the participants and I shared the power of data interpretation (see also Chilvers, 2009; de Leeuw et al., 2012). Hence, the pamphlets not only enabled a process of continuous reflection throughout the research project; they also sparked mutual learning.

I designed the pamphlet with ethical concerns in mind. Participation is not a privilege with which to endow informants and respondents; in feminist critical research, it is instead a moral principle that seeks to create a space for inclusion and dialogue. The participants must be given a chance to improve their first contributions–to have a second say and to make the most out of the research by, for instance, suggesting societally relevant research aims and outcomes. For example, one of the original research aims was to disclose the local gender division of labor. In my discussions with participants, it became evident that I had not approached this aim critically or historically. Cash crops, which are generally seen as a positive opportunity for livelihood improvement and diversification, had actually added a burden to women’s labor routines. Women raised this issue when reading the second pamphlet, which depicted cash cropping as an advancement for the local community. Their indications led to a redesigned research aim that took into account the negotiations and historical changes of the gender division of labor. These moments also exemplified the lack of spatial and temporal separation between data collection and analysis. The participants were giving answers and adding and providing interpretations of the data, but they were also posing questions that expanded the inquiry, which I noted.

The researcher must be accountable to participants (Chilvers, 2009), as results should also be of value for non-academic partners, and the participants encouraged this accountability:

Keep writing these pamphlets. Make them into a book for children to learn in school. And make sure your name is on the first page so we can remember you. (Elders, Sibou, Kenya, 1 August 2013)

This statement signifies the inclusiveness but also the novelty of this research tool. Through collaboration, reciprocal questioning and challenging, a different type of collaborative knowledge was produced, one that participants encouraged and wanted to benefit their children’s education. Additionally, research assistants played a pivotal role in framing the research aims and gathering and counterchecking data. They were depicted in the second pamphlet to indicate pamphlets’ co-authorship.

A tool to overcome biased understanding

Triangulation is most commonly used to validate studies; however, it does not always achieve this end, as there can be misunderstandings in data collection, which may lead to imprecise findings. Member checking, which the pamphlet is one example of, can thus serve to check preliminary findings with informants. By continuing all throughout the research process, member checking is a reflexive mode of knowledge production that establishes rigor by completing triangulation (Cho and Trent, 2006). Member checking is grounded in a constructivist view of validity, i.e. to reach validity, data must be continuously proven (Seale, 1999). To achieve this aim, the researcher must act as an ‘active interviewer’ and not merely gather information in a neutral way. In an interactive fashion, s/he should serve as a catalyst for interviewees’ answers, thus building a space for common understanding (Holstein and Gubrium, 1995). Re-examining the findings, through the method of member checking, allows for more accuracy which is defined as member validation (Seale, 1999).

For instance, during my first fieldwork visit, the elderly men stated the following:

Men are fencing, clearing the fields before cultivation can start and are responsible for repairing the canals. Women cannot help in these practices. (Focus group with men (40+ yrs), Sibou, Kenya, 6 August 2011)

OPEN IN VIEWER

Figure 4.

The second pamphlet in Kiswahili employed in Engaruka, Tanzania, in 2013. The title read, ‘Current modes of irrigation in dry-lands in East Africa: understanding work, soil and climate’ 1. Agriculture; 2. Irrigation; 3. Groups; 4. Ndimi. On the top right-hand side, my assistant and I jointly conduct a focus group.

This information was summarized in the pamphlet that was distributed during my second fieldwork visit. Reading my summary of the gender division of labor, women participants disagreed:

Women currently climb trees, cut thorns to fence their plots and clear plots before cultivating any time their husband is not around. (Focus group with women (20–45 yrs), Sibou, Kenya, 30 July 2013)

Therefore, it can be argued that administering the pamphlet exposed my erroneous and gender biased understanding of community practices. According to the participants, the data that I had gathered during my first fieldwork session described labor arrangements and crops that had profoundly changed in the last 10 years. Moreover, the participants had apparently reported an ideal picture of their community based on traditional customs and crops that had been renegotiated and replaced by new ones.

Coming to this realization was certainly a fundamental turning point in my research because it made me rethink my research aims and the way that I was asking questions. Indeed, there was an underlying misunderstanding between the respondents and me: I wanted to know about the present and how the current labor arrangements and crops had become part of their community, not about the past. Some people, particularly elders, had a pessimistic impression of the current state of their community and the new phenomena (i.e. climate change, cash economy and marketization) that were influencing their habits and customs; they preferred to refer to an idealized picture of the past.

Men, however, tended to depict a quite strict division of labor, with which women did not agree:

It is not true that women do not take part in canal repairs! They used to carry cement, and earlier they carried grass and sand on their back! (Focus group with young women (20–35 yrs), Sibou, Kenya, 29 January 2012)

These refinements show that strong member validation was reached (Seale, 1999). Checking preliminary findings was not performed only for the sake of attaining convergent and confirmed accounts; it also elicited research questions about the discursive representation of the gender division of labor. Reading only the men’s discursive rendering of local gender structures, i.e. the dominant version, women reacted strongly:

Women do more work than men. We pound the millet, we grind it, we cook, we dig, we plant, we weed, we harvest, we carry the harvest home together with firewood, while men just clear the plots and irrigate. (Focus group with young women (20–35 yrs), Sibou, Kenya, 25 July 2013)

Women can carry also the mangoes, but if their husband is around, traders deal only with him, and he is the one pocketing the money. (Linette, Sibou, Kenya, 24 July 2013)

Again, the pamphlet allowed participants to voice the unjust circumstances of daily living. This gut reaction energized women who understood the need to assert another discourse about their role in society. The pamphlet was wielded against men by women in public spaces, who said, ‘You told Martina only one part of the story! Where are we here?’ (Author’s notes, August 2013). These are instances of catalytic validity; in other words, through the validation process, participants were empowered, challenging me and contesting my data, which forced me to re-orient the study (Lather, 1986). In order words, member checking was a feminist participatory method. In fact, besides enhancing the study validity, encouraging and triggering participants’ spontaneous contributions, member checking was functional in rectifying and moving beyond a binary gender biased understanding of agricultural practices.

A tool ‘under improvement’

In contemporary geography, participatory methodologies are typically heralded as sound ways of conducting research (de Leeuw et al., 2012). Conversely, I argue that exercising critical reflexivity to acknowledge the limitations of our participatory methods is crucial, rather than considering them the next fail-safe solution in the social sciences.

The very first version of the pamphlet was a trial that allowed room for many improvements. The need to produce the pamphlet in a second local language became apparent afterwards, along with the importance of avoiding an oversimplification of the findings. Conducting member checking several times over the course of a research project will ensure the refinement of the format and the language used. Major discrepancies were avoided thanks to a first check performed through a translation by a research assistant. While the typical research hierarchy was retained in the phrasing of the content, as I had the power to decide what should and should not be included in the pamphlet, the research assistant provided input on potential oversimplifications and missing information. Nevertheless, the brevity of the text and the tendency to include photos over text to attract the participants’ attention do not allow for the counterchecking of detailed data, which must always be discussed following the presentation and reading of the pamphlet.

In the absence of pictures, word choice also proved to be a pitfall of this tool:

Jembe [hoe]? Which jembe do you mean? The small one? The big one? Women do use the jembe, but only the small one. (Muindi, Engaruka, Tanzania, 22 August 2013)

This misunderstanding emerged after I had already distributed the pamphlet to dozens of participants. Until then, no one had noticed this ambiguity, and I had missed an important clarification of the materiality of everyday labor. Drawing the two hoes and then having participants show them to me allowed for improvement.

A word of caution: participation and the participants’ interest in the pamphlet were not a given.

Here in Engaruka, there seems to be general agreement about the contents of the booklet. I wonder whether people are afraid to voice their dissent or if they really agree. I should ask more detailed questions about the content. When interviewed, it is more difficult to get articulate answers from women who have not gone to school. It is very frequent that women answer that they do not know. And this is not the case, as they are the ones carrying out the practices. (Author’s diary, 22 February 2012)

There might be several reasons for this apparent constant agreement and lack of engagement with this methodological tool. First, as highlighted by Reid (2000), receiving feedback is more difficult when participants are illiterate or have low levels of education, as they might be intimated by the researcher’s authority. This was certainly the case in Engaruka, Tanzania, where personal observations confirmed a preference towards first sons in schooling and the practice of early marriage for daughters (see also Archambault, 2011; Bonini, 2006).

Second, participants might not see the advantages of taking part in discussions that do not directly reveal solutions, instead leaving them to face the harsh reality of social problems (see Wang and Burris, 1994). Finally, in other studies (Bradshaw, 2001; Tsouvalis and Waterton, 2012), participants did not provide member-checking feedback if the data were presented and analyzed with a critical perspective.

Conclusion

The participatory tool presented in this article was a novelty to me, the research assistants and the participants, and had three primary advantages. First, it created considerable curiosity in the participants, who were rarely exposed to written texts and pictures. Copies of the pamphlets were widely distributed among participants, respondents and passersby who wanted to get acquainted with the content and have a chance to discuss it. I stated that the content was not the truth, but rather my interpretation of the facts that they had shared with me during my previous fieldwork. This statement appeared to create a relaxed environment, and most participants were not afraid to correct me. I received a considerable amount of diverse feedback. The participants clarified and added details to the data; they introduced new issues and expressed their expectations for the research project. I followed up on those new aspects and considered those expectations when rethinking my research aims and interviews questions.

Second, the pamphlet could be considered a member checking method that completes triangulation. The pamphlet was a tool to countercheck preliminary data with the participants to provide ‘an accurate reflection of reality (or at least participants’ construction of reality)’ (Cho and Trent, 2006: 322), while facilitating a less hierarchical relationship between the researcher and the research participants. In this way, the previous gender biased understanding was overcome and the study’s validity was enhanced. In some instances, the pamphlet turned into catalytic validity, as it empowered participants to contest unjust local circumstances and forced me to re-orient my thinking and my research plan in light of these reactions (see Lather, 1986).

Third, as a ‘boundary object’ (Lagesen, 2010), the pamphlet produced a cross-cultural space for mutual exchange and learning, which facilitated a process of collective reflexivity. I was not the only one holding the knowledge; I asked for assistance in acquiring and improving my knowledge. This process triggered a shift in everyone’s positionality. Data collection, analysis and distribution were made possible thanks to the engagement of research assistants (see also Caretta, 2015) and participants. They lent their time to share information with me, to help correct the data and to provide interpretations; in addition, they spread news around the community about me and my research project, thus attracting more participants.

True reflexivity also calls for the acknowledgement of a method’s limitations. The pamphlet was not a perfect method. Several repetitions improved the format to enhance participants’ comprehension both in terms of the language used and the content and pictures included. Not all those who took part in the focus groups and interviews were equally enthusiastic about this tool. Some did not show interest or provide feedback. The reason for this lack of interest might be analphabetism, which hindered some participants’ full understanding of the content (see also Reid, 2000) and their ability to contest the researcher’s presumed authority.

In sum, the pamphlet is not the next fail-safe tool for participative research; however, I argue, it can effectively create a space for inclusion, discussion and reciprocal learning between researchers and participants, leading to collective reflexivity. It is also ethically just towards participants who can physically see their contributions, whether they are adding content or correcting and improving the data collected. Member checking is thus a powerful tool to enhance validity; particularly in pamphlet form, this tool, whether successful or not, can generate catalytic validity by empowering participants and re-orienting researchers.