Evaluations,international agencies and censorship: A field doer’s viewpoint

Abstract

This article presents a radically empirical and descriptive account of the most often neglected viewpoint: that of the ‘doer’. Using field autoethnographic examples from remote Northern Brazil, the article portrays the clash between the universalistic programs of international agencies and doer perspective at the fieldwork level. The see-judge-act tool helped to chart the heuristic field of daily life as a reflective practice. Three moments illustrate the confrontation of the action planned: (a) a projection of abstract theoretical assumptions based on a universalistic framework; (b) a silent period of accommodation of local realities to the imposed, descending logic of the formal discourse; and (c) the production of an exportable version of the partially hidden local situation. All three moments show the struggles to make the ‘doer viewpoint’ communicable beyond the local geographic region. In the end, fieldwork dangers were censored and inconsistencies were evened-out.

Keywords

censorship evaluation participation public health universalistic program

Introduction

The problem of collecting empirical evidence in remote regions

To what extent might centralized international agencies’ directives conflict with grassroots fieldworkers’ input to evaluations? On the one hand, participation stresses a relationship to the lived experiences of participants; the wider the participation in a health intervention evaluation, for example, the more varied will be the social actors who want to contribute, and the more empowered they will feel to defend their contribution. On the other hand, the very nature of international agencies places native participation in the broader context of political decisions that define the aims, activities and results expected. An initial review of several interventional agencies’ prescriptions and procedures reveals serious restrictions on researchers’ and evaluators’ freedom. Moreover, these agencies – including the United Nations Development Programme (UNDP), the World Bank (WB) and the Joint United Nations Programme on HIV/AIDS (UNAIDS) – situate local contexts within a universalistic framework (see UNDP, n.d., 2005, 2009a, 2009b, 2012; UNAIDS, 2009; UNAIDS/WB/UNDP, 2005).

This tension needs to be considered from the perspective of the fieldworkers on the frontline, but few publish their observations about the life-threatening settings in which they work. Cohen (2012) identified ‘contested terrains’ as environments in which violence and conflict impinge on fieldwork, evaluation and research. He concluded that fear restrains people’s ability and willingness to participate in an evaluation. (Undoubtedly, the safety and wellbeing of participants and researchers should take priority over the evaluation or research goals.) Meanwhile, Pérouse de Montclos suggests that a report’s ‘sanitization’ of any critics might be another consideration (2012: 156). That is, self-censorship might be a reason to avoid including compromising statements about colleagues or organizations. Furthermore, Byrne points to issues with the often unattainable ‘universal/nomothetic accounts of causality’ in evaluating a complex intervention (2013: 217).

To what extent are international agencies concerned with the constituency of influential social actors within their political framework? To what extent do they consider the risks for local ‘doers’ who work in remote and dangerous regions? How might the experiences of a doer help in comprehending the socio-political challenges of a participatory evaluation in a field where the evaluator has been living, working and practicing? This article tries to tell the story from a doer’s perspective, a viewpoint rarely discussed academically.

Mintzberg and Srinivas (2010) distinguish ‘doers’ from ‘helpers’ as follows: doers are local insiders, while helpers are cosmopolitan outsiders. Zúñiga (1988) has described as ‘amphibious management’ the in-house struggle between the doers – professionals who have been recruited from among those ostracized for their political positions – and the helpers – highly qualified public figures who take it upon themselves to ensure collective survival through grants and international aid.

Drawing on a case example, this article describes the challenges faced at a local level, where peer educators carry out prevention and workers (the doers) live permanently in the same neighbourhood. It describes the long road from the ‘project’ as a proposal to its implementation as a field action intended to provide help in understanding effective change. An autoethnography reconstructs the experiences of a doer located in the setting of a riverside municipality in northern Brazil. There, the field actions encompass leprosy and HIV/AIDS care and prevention. Finally, in this article, the term ‘social actors’ is used in place of the more commonly used ‘stakeholders’ in order to avoid the limitation of analysing participation only in terms of project-directed interests.

A doer’s groundedness

For six years, I lived in a remote region where I also worked as a volunteer and public health nurse. Before becoming a part of this community, I received intensive training (1989–94) in cross-cultural insertion and in the use of the ‘see–judge–act’ method (Beaudin, 1996; Gotto, 2004; Rocher, 2006; Warren, 2002) for assessing a social movement perspective. The region was a very poor one, subsisting on minimal farming and fishing resources. An involvement with an ecclesial community (Boff, 1978; Dawson, 2006) and with other Latin American colleagues immersed our group as a collective within the marginalized neighbourhoods and riverside towns and communities. Part of a larger Latin American movement, the idea of ‘a preferential option for the poor’ (Gutiérrez, 2004) is to live within marginalized populations. This means collaboration with women’s associations, youth groups, rural cooperatives, health and sanitation movements (social pastoral) and local community centres, and allied manifestations for public services access (health, water, electricity, political decision making, etc.).

Together, we used the see–judge–act strategy (Dawson, 2006: 117) as a tool to analyse collectively the social and political dimensions of our shared living as members of a community. ‘See–judge–act’ is an informal, critical sociological method that guides the foundations of social practices (Rocher, 2006). It has a long history in Francophone Belgian and Latin American popular social action, especially with workers’ unions (Beaudin, 1996). Its aim is to ensure a methodical progression from observation to evaluation – based, if possible, on collective assessments – in order to ground action decisions (Warren, 2002). It stresses the importance of immersion in the living conditions, which positions would-be researchers both in the experience itself and in the generation of solutions (Gotto, 2004) from a ‘pragmatic solidarity’ perspective (Farmer, 2003). In our case, for example, it helped to explain how primary care and preventive services might be provided in local communities when caring for HIV/AIDS patients in Latin America.

Data collection through narrative reports as a project worker

Naroll (1962: 11–15) argued that the major determinants for data quality control are the extent and duration of an ethnographer’s placement, the type of observation, and the ethnographer’s personal participation in local culture. Moreover, the longer researchers stay in contact with and exposed to the local culture and develop a familiarity with its language, the better the chances that constructed facts will be closer to a shared vision of social reality (Naroll, 1962: 87–94).

As part of my placement, I became a member of the neighbourhood committees of a women’s association (1994–2000) and worked as a volunteer trainer of community health agents in institutional and non-formal organizations (Pastoral da saúde [health]). From 1997 to 2000, my involvement with public health programmes continued in the same setting via a joint coordination of the municipal leprosy programme and the municipal STDs/HIV/AIDS programme (rural extension actions from a Latin American regional reference centre for the Amazonian region). The leprosy-control programme activities involved systematic visits to more than 100 riverside communities as well as an urban campaign. The STDs/HIV/AIDS municipal programme included a peer-education pilot project with sex workers in prostitution areas (the floating port, bars, hotels, public parks, streets, schools, etc.) in the municipality in which I lived.

Heuristics

The guiding methodological orientation was intended as a means of meeting the challenge of articulating and understanding highly contextualized, tacit knowledge from personal experiences. Personal knowledge is an ‘intellectual commitment’ that involves the participation of the knower in an objective process of understanding a ‘hidden reality’ by establishing a contact with it (Polanyi, 1964: vii–viii). A ‘heuristic field’ gives access to opportunities to know a reality while living in a particular context; the individual acquires knowledge and uses it without needing strict rules of justification and evidence (Polanyi, 1964: 403). The underlying premise of ‘tacit knowing’ is that a person knows more than he or she can tell. This knowledge can be recovered and incorporated into a more representative totality.

‘Being there’ during extended periods brings a wealth of information about a situation and evaluations from different viewpoints (Borneman and Hammoudi, 2009). The ideas underlying the present article were first presented and discussed at an international public health conference (Laperrière and Zúñiga, 2006), than in a colloquium about evaluation, translation and administration of knowledge in health research in Brazil (Laperrière, 2013). Audience feedback (users, students, professors, volunteers, Brazilian health ministry officials and municipal politicians, some linked in international cooperation partnerships) has been used as ‘inter-judge correlation’, thereby increasing the ‘fitness’ and ‘plausibility’ of the article’s conclusions (Miles and Huberman, 1994).

Degree of independence

When Pérouse de Montclos constructs the pros and cons of evaluations according to the evaluator’s degree of independence, he states that self-evaluations offer easier access to internal information, but they lack critical distance and diffusion outside the institution (2012: 158). Conversely, analysis undertaken by researchers provides a good critical distance as well as the potential for publication, but it might be perceived as being a tool for control that is inefficient in helping to learn from mistakes.

The micro-context of sex work was affected by a large variety of recognized and behind-the-scenes actors (e.g. politicians, traffickers and police and army contingents). I had to remain open to all avenues, while being both impartially detached and publicizing this detachment. Socio-environmental factors limited the predictability and required personal capacities for coping with unforeseeable dimensions. These included culturally based differences in reactions to evaluation, socio-economic inequalities and a context characterized by political instability and its violent consequences.

I believe that my unique position as a ‘doer’ and a ‘researcher’ sheds light on the gains and neutralizes the limits of a balanced picture of the reflexivity process. I have refrained from naming organizations, people or places as, even after more than seven years, I still fear the negative consequences for the local participants and for myself.

Method

The act: Sharing a human context for a long period of time

Community-based outreach through a leprosy-control programme in riverside populations

The geographic area of this study’s small, isolated riverside town located 147 km from the state capital covered roughly 9617 km². Because the town is on the Amazonian Plateau, the population had to cope with floods in July and droughts in October (I lived through six floods between 1994 and 2000, sharing the consequences with the local population). These ‘cyclical’ natural disasters (Ministério da Saude, 2009) constrained both health and commercial activities. The river level directly influences transportation between the riverside town, the second-largest municipality and the capital. Its level rose 3 to 5 cm every day, and the difference between the lowest and highest river levels varied by more than 20 m. This situation forces the riverside populations to move their houses to solid ground, to raise its floor or build another one above the water line. River transportation (e.g. recreo, or public boats, and launches) was the only way to reach the remote villages.

The intensive leprosy-control actions included house-to-house visits incorporating skin examinations, health education and basic health assistance. A typical health services trip by boat could last from 15 days to a month, living in the boat and improvising meals using available resources. The health team always shared its resources with the isolated families, even when these were barely sufficient for their own needs. On one such trip, it took two days to reach the farthest riverside communities situated on a tributary of the main river (author’s narrative in unpublished report, 6 October–31 March 1998). We completed our visit into little river ramifications with a smaller voadeira (launch) to reach out to all the communities, isolated families and people encountered on the riverside. It sometimes took two to three hours to reach a single family. We faced risks as extreme as the sinking of our boats with no possibility of any help reaching us, tropical diseases (malaria, hepatitis A and B, dysentery, cholera, etc.), dangerous animals and fish, physical injuries, sunstroke, limited food and water – and all with no communication with rescue teams or municipal authorities.

In five months, we managed to make three rural trips by boat (lasting 20 days each) during the intensive flood period. We reached nearly every family in this region’s 95 riverside communities (author’s report, 20–25 April 1999). Water levels rose to the highest ever recorded levels for this region (1999; see similar descriptions provided by DeCastro and McGrath, 2001; Moura and Peres, 2000; Moura and Moreira, 2001). Most homes were flooded and entrance could only be gained through their windows. New floors had been built out of branches and tree trunks (author’s report, 20–25 April 1999). The flooding brought with it caimans, poisonous snakes and piranhas, which represented a serious threat of mutilation and even death to small children.

Dwelling on conditions such as these had no place in the well-circumscribed objectives of leprosy detection, and, as such, were left out of reports by the regional health team. They sent these ‘sanitized’ reports to the Brazilian capital and to the higher leprosy-control programme levels (international organizations). They documented only the quantitative statistics about the number of skin examinations and newly identified leprosy cases. It was a first lesson in reporting terrain experiences to fit constraints and expectations imposed from above: I was told by the regional centre that context is only anecdotal and is best left out of a report.

We listened to the riverside peasants’ fears, despair and worries. At one point, there was no place to land the boat because of the extensive flooding. We witnessed several situations of extreme food deprivation and lack of transportation or places to live, health emergencies and unsafe living. Despite the flooding, however, the intensive leprosy-control actions led to the diagnosis of 23 new leprosy cases. In addition, the health team reached untreated patients and remote ones. Overall, our six-member health team conducted 8734 skin examinations and 1722 educational actions and family visits (author’s report).

HIV/AIDS prevention in urban prostitution zones

Whether providing care for people with leprosy or STDs/HIV/AIDS prevention to riverside communities, these actions took place in conditions of uncertainty, insecurity and institutional violence. But, living in the poorest neighbourhoods in the outlying areas of Brazil’s Amazon region made it possible to acquaint marginalized groups and collectives with health services and support. A typical night’s work as a project worker involved walking with peer educators in areas and bars associated with prostitution (author’s fifth report, 21 September–24 February 2000). There were many ‘floating bars’ open day and night serving sex workers, gangs, fishermen and ‘people in transit’ in the harbour; sex workers walked on the floating corridors to follow clients’ movements. When working in a bar, a sex worker is sometimes obliged to ask for permission to leave for a STD clinical appointment. During intensive fishing periods, sex workers will travel weeks or months on the boat, which meant that we had to negotiate common arrangements and schedules with the bar owners.

The educational activity comprised giving short explanations about STD prevention, each week with a different theme. When working in the floating bars, we needed to stop singers and bands or turn off the radio so we could get the clients’ attention. Peer educators used a microphone and gave live educational talks. Bar owners were willing to allow our team to operate during their night programme. Indeed, they even encouraged the peer educators to do so. The regional health team from the capital visited the local project on a biweekly basis. When they left at the end of the morning, we remained behind.

Because of our presence and proximity in these marginal areas, we had to maintain our professional commitment to the principle that life must be preserved while coping with numerous social problems beyond the project’s official interventions. Walking in areas associated with prostitution meant also hearing gunfire and encountering substance-addicted populations, smelling fish odours emanating from the docks, giving primary care assistance to sex workers (cuts from suicide attempts or fights) and, on one occasion, running from a knife-wielding client (author’s notes, fifth report). Looking at a letter from the agency dated 9 May 2000, we noticed that such details were absent in the final report to regional, national and international levels of health organization.

Implementing health programme interventions with the participation of community members is inevitably intertwined with political actions by governmental, military and civil society organizations. I lived half a kilometre from the major prostitution areas – I knew that the official research and intervention programming ignored significant social actors, whereas my immersion made it possible to observe:

direct local actors – users, peer educators, the social activity structure of prostitution areas, including clients, bar owners, etc.;

indirect local actors – socio-sanitary network; health, administrative and service professionals; political and legal apparatuses; public administration; police, military, and cultural apparatuses – representatives of the community in general; religious sects; and

regional and national actors – reference centres, politicians, the research community).

Many (drug traffickers, members of the military and of the police force) were both difficult and dangerous to identify.

The evaluation of the act: Returning to live there and doing evaluative research

I returned to this area four years later to conduct an evaluative research study of the project. I felt ethically engaged to the poorest sectors, where I had been involved previously, and committed to communicate their viewpoints to the best of my abilities. To follow the path of the area’s social movements, I opted to use an ethnographic perspective with a participatory approach. This made it possible to observe changes over time. The initiative was mine. I wanted to go beyond the available epidemiological surveys and look at the multiple social actors’ points of view of the sex workers’ peer-education project (House and Howe, 1999). I underlined the value of the beneficiaries’ (sex workers) and peer educators’ points of view as being those most easily ignored.

Historically, the sex workers’ peer-education project had been sponsored by a foreign funding agency with regional and local cooperation. In 2002, the value of the interventions was verified by epidemiological studies and sexual behaviour questionnaires. From the moment of its implementation, the project’s designers favoured evaluation activities that would demonstrate the project’s validity and would contribute to decisions about continuing with it locally. After three years of implementation, responsibilities were transferred entirely to the municipal health networks.

The evaluation research was linked to my Master’s degree research in nursing. The ecclesial community offered me a home (for five months). The Brazilian regional centre signed the required ethical agreement and the municipal health authorities accepted the proposal – but no monetary contribution was offered or requested. There was no direct sponsor for the research.

The evaluation would be performed ‘on the basis of the points of view of the various actors involved’ (House and Howe, 1999): health network authorities, municipal authorities, peer educators, sex workers and other voices often forced into silence (i.e. marginalized communities). Part of the research objective was to underline how context imposed various limits as to predictability, which renders the production of evaluative research difficult within the established, rigorous, Western scientific criteria of predictability regarding outcomes and researcher control of process.

The evaluative research was conducted during a pre-election municipal campaign, during which period it was quite impossible to defend its political neutrality. Several people questioned me about my potential involvement in active vote-getting activities. I had to explain my position as an ‘outsider’ who preserves a point of neutrality amid the two principal political parties. Nevertheless, such neutrality appears impossible to maintain for many for whom the ‘right’ political option is crucial for their basic survival. One participant commented: ‘You came back because you are with the candidate “X” [in place] or “Y” [opposition]: you have to choose.’

For the most part, the evaluation gathered qualitative evidence in three peer-education projects (sex workers, men who have sex with men, adolescents in peripheral neighbourhoods). The fieldwork was carried out over a five-month period: 23 intensive participant–observation sessions in four major prostitution zones (day and night) and two prisons (men and women); regular visits to project sites, clinics, sex workers and even to the family homes of local educators, municipal health services and community organizations; and 21 individual and 18 collective semi-structured interviews. All these formal research activities took place on site and were summarized using field notes, which furnished a background understanding of the formal evaluative research activities.

The research strategy had to tread on slippery and uncertain ground. Because I was living as a member of the community, people were less reluctant to speak openly on the meaning of their comments and opinions. Nevertheless, during the interviews, I suggested that to participants that they should not reveal information about underground or informal organizational structures related to local drug dealers, names of actors in illegal actions, and so on. Revelations about specific names related to the sex work trade might have endangered us all, for example. I had to consider the risk to life of obtaining too much information about participants’ relationships with other influential social actors in the community. Giving the impression of ‘knowing too much’ was just as dangerous as meddling in obscured or illicit knowledge.

‘Objectivity’ of programme guidelines versus ‘subjectivity’ of fieldwork experience

During the course of the evaluative research, I gathered data and recorded observations in a personal fieldwork journal. I analysed these later, when I was outside the local municipality. Once I felt safe, a critical analysis of the evaluative research questioned my reticence: Why the self-censorship? From where was it coming?

This second reading of the research notes underlined three successive moments in the production of the final written report, from the initial plans to attempting to satisfy the demands and expectations of both the health programme promoters and the social actors involved. A time-ordered display (Miles and Huberman, 1994) illustrates the generalized understandings from the historical chronological flow of fieldwork events.

The return to the field as a researcher and evaluator might be seen as an effort to objectify the researcher’s subjectivity (Bourdieu, 2001); a unique situation in which one might live the contradiction between the ideal of participation and its concreteness in remote fieldwork. The three moments mentioned above illustrate stages in the self-reflexive process of visualizing changes and their implementation. They are (a) a projection of abstract theoretical assumptions based on previous experiences, (b) a silent period of accommodation of local realities with the descending logic of the formal discourse and (c) the production of an exportable version of the partially hidden local situation. All three show the difficulties with making the ‘doer perspective’ public outside of the local geographic region.

The first moment: Verticality respect in a top-to-bottom view of actors’ power and influence

At the direct action level, local ‘doers’ (including me, in the case of this research) must apply the prescribed actions assigned to them – and the evaluation results should confirm the idealistic expectations. For my study, both the university (in Canada) and the local institution (Brazil) approved the protocol for ethical and scientific merit, and then the municipal health institutions accepted the evaluative research with the peer-education projects. The tension started when the higher levels of these organizations (municipal and regional) sought confirmation of success by way of a formal evaluative report, while the beneficiaries of the project saw the evaluative research as an action project aimed primarily at improving their living conditions. My previous experiences here had increased my awareness of the living conditions. It also served to help me understand the matter of confidentiality among the evaluative research participants. Formal and informal interactions raised new issues – poverty, daily food challenges, family responsibilities, friendships and work availability – all of which had to be incorporated into the participatory research process.

The pressures from the Brazilian institution were explicit: a broad conception of participation was to be dropped, as it was considered to be too political. However, we (the doers) resisted the translation of our life into foreign words through which we could no longer recognize ourselves, or which glossed over the life experiences that shaped our concerns. When asked, the peer educators spontaneously expressed a willingness to share their experiences in groups. Later on, once the process had gotten underway, some asked whether municipal authorities had authorized their participation and what information about their comments would be made public. It was clear that the political context was never too far from their participative strategies. The observations showed that political agents inside the research groups maintained the informal social organization within the health programme’s local political framework. Participants referred privately to the presence of municipal-level informants in our discussion group. They admitted that political agents had infiltrated the AIDS prevention projects. This imposed constraints that, unless taken into account, would distort the responses received in the group interviews and increase the risk of being considered to be making politically critical comments.

The second moment: The silence produced by formal expectations of rigour

Remaining silent appeared to be a survival option for escaping the accusation of being an informant who transmits internal local information. Surrounded by international borders (Brazil, Colombia, Peru, Bolivia), the Amazonian region attracts smuggling and drug trafficking, an influx of clandestine people and land invasions. Intimidating methods reinforced secrets (e.g. disappearances and murders), and I had to avoid making public some inopportune evidence. I intentionally excluded information related to hidden networks between opponents. At one point, some peer educators and I talked openly about the ethical considerations of exposing unwanted truths, and it became clear that material about gangs, drug dealers, politicians and the armed forces had to remain confidential.

Thus, I faced an idealization of participation: How will it be possible to write and publish about the field incoherencies between official evaluation reports, scientific thesis and peer-reviewed publications? I became aware of the imposed silence provoked by both the research presentation format and the official discourse related to the project. At one point, some peer educators and sex workers raised questions about what it meant to participate by sharing information, and what would be the impact of this sharing in their political contexts.

Field reality was a disquieting excursion into the unforeseen. The potential consequences of unguarded spontaneity worked against full participation. The field was traversed by socio-political danger, which affected relations among sanitary, police, legal, government, cultural (educators and the clergy) and criminal actors. The pre-election period in which we were working exacerbated these tensions. Furthermore, the peer-education beneficiaries were sex workers who remained the women at the bottom of the local social ladder. In that remote region, they had had long and painful experience of the costs of honesty. In the evaluative research, the sex workers used very subtle and sophisticated codes for ‘testing the waters’. They tested my capacity for secrecy by trying to identify clues that would allow them to identify the hidden dangers lurking in unguarded communication.

When walking together through prostitution zones at night, some peer educators and I shared a similar sense of danger, which never explicitly emerged in formal interviews. At one point, prior to an interview, a ‘central’ (politically powerful) participant said: ‘Yes, I remember you. You are a good friend of …’ (2004, personal conversation). This was a veiled and subtle warning; both of us knew that the woman’s son had been killed in questionable circumstances. Private conversations added to or contradicted claims about topics such as the dangers of working as a peer educator in areas associated with prostitution. We lived with the ever-present fear of losing our jobs in the power shifts that followed the municipal election campaign. We were well aware of the links between the municipal health authorities and the electoral candidates.

This awareness and interpretation of uneasy silences presented a significant challenge. Such silences took the form of unanswered questions: What will I say to higher levels about the actual field experiences? Should I trouble the report with the discrepancies between actual practices and the initial project, with the twists and turns of the original idea amid the miasma of what actually happened? What will I transmit to international meetings that expect confirmation of theories or theoretical modifications purified of the murkiness of action – local, anecdotal, contradictory and not always pretty or politically palatable?

Should I acknowledge that not everything went ‘as planned’, and that I negotiated an uneasy path between acting according to the plan and attaining the expected results? Will my audience hear a call to modify their beliefs, or will they think that I am just justifying my inadequacies? Which do I choose: honesty and realism or evasive fast-talking and ingenuity?

The third moment: The painful production of an ‘ascending discourse’

Researchers, evaluators or consultants’ will usually collect the data first, then return to a secure place to analyse them; here, I intentionally stayed with the study’s participants. The social environment influenced my thoughts. Analysing the data, I lived in my skin and guts the daily insecurity of the minimal living conditions of most inhabitants (wooden shacks, hammocks, no direct transportation, limitation of electricity and water), the political meddling (pre-election, civil servants’ demobilization), the lack of indispensable resources for research (computer, internet, photocopier, papers), the vulnerability to climatic calamities (the rise of river levels, extreme heat, heavy rain, malaria and dengue mosquitoes), the criminality of our neighbourhood (street gangs related to some of the evaluation participants) and the corruption (municipal and regional politicians). A trial had just started involving most of the local police; another exposed an extensive prostitution trade in the Brazilian state of Amazonas. ‘I am writing. The noise of military helicopters flying low over our heads and heading to the land invasion where my colleagues are living’ (author’s autoethnographic journal in Laperrière, 2007: 48) These dimensions added significant variations to how the ‘intellectual field activity’ was conducted.

A third ‘moment’ was necessary to overcome my fears of publishing uncomfortable details. The task was to produce a convergent discourse that would be both scientifically acceptable and close enough to the lived and shared experiences. However, the discourse needed to be sufficiently de-socialized, de-politicized and universally exportable to be published. A coherent report would disqualify the explorations and detours that would read and be judged as evidence of my incompetence and disobedience. Even when agreeing to tone down idiosyncrasies and singularities, I remained committed to a mission to nourish low-level grounding, which would, hopefully, influence the higher levels, and which might, one day, inform the decision-making apparatuses.

Should I include anecdotal details (e.g. corruption and drug dealing), interview excerpts and vignettes in the official report and in international presentations of the study? Their exclusion would limit the evaluation to a mere verification and acceptance of the peer-education project’s expected outputs. Strongly influenced by the scientific committee, the final report was not to be disturbed by determinants such as resource availability, bureaucratic or political delays and changes, extreme weather conditions and the impacts of unplanned economic situations on plans prepared two or three years in advance. These contingencies undermined evaluation goals that entailed a perfect correspondence of planned inputs and expected outputs. A decision was to be made: truth, untruth or partial truth? Remaining evasive appeared to be the healthy choice.

Discussion

The problem of censoring evidence to fit a socially desirable public discourse

At the highest levels of research reporting, international and national aid agencies prescribe the publication of results through a universalistic discourse of participation and decentralization. Websites reveal that ‘winning’ researchers and evaluators better translate their intentions in ‘worldwide’ terms. This image of universality and globalization depicts a world in which conflict and danger are conspicuously absent from the reporting health programmes, as seen habitually in the report style of UNDP (n.d., 2005, 2009a, 2009b, 2012), UNAIDS (2009) and UNAIDS/WB/UNDP (2005) publications. Moreover, international agencies prescribe their expectations of governance: ‘[D]ecentralization refers to the restructuring of authority so that there is a system of co-responsibility between institutions of governance at the central, regional and local levels according to the principle of subsidiarity’ (UNDP, 2004: 4). The directedness is evident, too: ‘Decentralization policies are influencing change at national and local level with a direct impact on institutions, communities and individuals’ (UNDP, 2008). The UNDP provides funds for projects that emphasize democratic practices. One example is a project in Columbia, which received funding for the ‘expansion of capacities of national, regional and local levels of governance to reduce conflict and manage the equitable delivery of public services’ (UNDP, 2012: 60).

While the Brazilian government’s public health AIDS programmes recognize the importance of involving participants from local communities (Chequer et al., 2006), they also expect that the methodology should be impartial and scientifically rigorous. At an international scientific conference (Laperrière and Zúñiga, 2006), I was talking openly about dangerousness when a Brazilian stopped me, stipulating that such dangers are non-existent! In fact, there were known murders of foreigners: two Americans, one a religious practitioner and the other a social anthropologist (Acrítica, 2005a, 2005b). Further, as I write this article, Brazilian human rights commissions are investigating paedophilia and human trafficking in that area (Acrítica, 2013). A recent statistical study shows that Brazil, a country without civil war or ethnic, religious or overtly terrorist conflicts, had up to 206,005 homicides in four years (2008–11) – more than resulted from the 12 biggest armed conflicts in the world and almost the same as from the 62 wars happening during that same period of time (Waiselfisz, 2013: 20–1). The northern and northeast regions feature the most significant increase in homicides per 100,000 inhabitants.

It is naïve to expect that informants will convey only the whole and unvarnished truth. Why else would local participants be unwilling to report political incidents such as corruption, drug and human trafficking? As Edwards (1953) first conceptualized in the middle of the last century, social desirability invites a participant to give a response about behaviours, opinions or attitudes according to social conventions.

When results are expected not to match outcomes set out in an initial plan, to discuss frankly what did happen is almost unbearable. Evaluations can identify not only recipes for programme success, but also the skeletons in the closet that expose only a virtual programme (Brousselle and Champagne, 2004). Pérouse de Montclos delineates this dichotomy as follows: ‘NGOs promote quantitative analysis to “please donors”, insisting on success rather than failures, and international agencies such as the World Bank and World Health Organization will avoid “reports that compromise anyone” for the sake of their “independence”’ (2012: 156).

Despite a growing discourse on local participation and democracy practices (UNDP, 2004, 2012), few studies underline the dangers involved in conducting peer education, evaluation and research when living in the community (Laperrière, 2008). Informal structures must be taken into account in order to understand and work on the relationships between formal structures (institutions) and informal social networks, which establish a symbolic-cultural logic in Latin American contemporary societies (Lomnitz, 2008). International agencies impose a hierarchical, vertical alignment and power relationships that colour a socially desirable discourse, which restrains the establishment of trusting relations between the ‘insiders’ and the ‘outsiders’ (Laperrière, Potvin and Zúñiga, 2012). We cannot analyse answers given within group encounters if we pay no heed to the political and social context in which these communications took place.

Conclusion

In their reports and evaluations, doers have to juggle the dissonance and the partial fit of the two worlds of universalistic global aid agencies’ guidelines and dangerous fieldwork contexts. Equally, evaluators and researchers have to present publications that are socially acceptable. What, then, is the discourse that would provide the doer’s viewpoint with a grounded voice, one that could convey the meaning of local experiences to a public far removed from them in location?

This article offers three lessons. First, it reveals how a formal evaluation might obscure work done by doers living in the setting. A flood disaster has severe social, psychological and health impacts on a population (Carroll et al., 2008); it also has an impact on the workers who give support to that population, especially when members of their families live in a similar situation. The foregoing discussion draws out the enhancement of data quality when analysis is based on prolonged immersion in the situation studied and on social roles only locals feel and understand.

Second, the article encourages open discussion of the dangers confronted by local inhabitant doers, whose actions are only possible through a strong personal engagement. Over a long period of time, doers and inhabitants share the same neighbourhood and political structures, which, in turn, means that doers will receive secret information that might reorganize the planning of a project.

Third, it reappraises the universalistic programming of health actions and evaluations in remote regions and contested terrains. Specifically, it gives a voice to the censored everyday local activities that feeds the input of an imported intervention at a global level.

Santos (2014) observes that science is not a unique, valid way to know the world. We can be naïve in our reading of research reports, such as by not realizing what they don’t include. Local political information is conventionally removed from research publications. Through decontextualizing evaluation and research, the inconvenient knowledge emerging from a doer’s action is discarded. Furthermore, several issues concerning evaluations, international agencies and censorship remain untouched, such as those first raised by Stavenhagen in the 1960s: ‘How many studies do we have about the political elites and the decision-making process […] of regional coronelismo [bosses, military or civilian]; of the influence of foreign diplomatic missions in the national political scene […] from which the dominant groups used to maintain the status quo?’ (1981: 187) One way to reduce the ideological conflicts in international relationships may be to limit studies without socio-political account or context (Casanova, 2002). Otherwise, the message is always tacitly present: Better avoid problems that are politically unacceptable… or be prepared for the negative repercussions of breaking the rule of silence.

Footnotes

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Hélène Laperrière is an associate professor at the School of Nursing, University of Ottawa. Her research is grounded in a Laboratory of Social Practices and Popular Education in Health. This ‘open field’ laboratory explores innovative forms of cooperation among researchers, students and local communities to produce hybrid knowledge.

References

Acrítica (2005a) Morte de missionária é apurada [Death of a missionary is studied]. Acrítica, 13 February 2005.

Acrítica (2005b) EUA podem enviar agentes para ajudar na investigação da morte [USA can send agents to help in death investigation]. Acrítica, 15 August 2005.

Acrítica (2013) Pesquisa aponta que aeroporto é utilizado como rota do tráfico de mulheres em Manaus [Investigation shows that airport is used as a connection for women traffic in Manaus]. Acrítica, 4 March 2013.

Beaudin

(1996) Les mouvements d’Action catholique comme pratique et théologie critiques. Laval théologique et philosophique 52(1): 67–84.

Boff

(1978) Comunidade eclesial: comunidade política [Ecclesial Community: political community]. Petrópolis: Vozes.

Borneman

Hammoudi

(2009) Being There. The Fieldwork Encounter and the Making of Truth. Berkeley: University of California Press.

Brousselle

Champagne

(2004) How was the UNAIDS Drug Access Initiative implemented in Chile? Evaluation and Program Planning 27: 295–308.

Byrne

(2013) Evaluating complex social interventions in a complex world. Evaluation 19(3): 217–28.

Carroll

Morbey

Balogh

Araoz

(2009) Flooded homes, broken bonds, the meaning of home, psychological processes and their impact on psychological health in a disaster. Health & Place 15(2): 540–7.

10.

Casanova

(2002) Exploração, colonialismo e luta pela democracia na América Latina [Exploitation, colonianism and fight for democracy in Latin America]. Petropolis: Vozes.

11.

Chequer

Simão

Cortes

(2006) UNGASS Monitoring: A joint construction between the Brazilian Government and the Civil Society. Paper presented at the XVI International AIDS Conference, Toronto, 13–18 August 2006.

12.

Cohen

(2012) Conducting evaluation in contested terrain: challenges, methodology and approach in an American context. Evaluation and Program Planning 35: 189–98.

13.

Dawson

(2006) The origins and character of the base ecclesial community: a Brazilian perspective. In: Rowland

(ed.), The Cambridge Companion of Liberation theology, 5th edn. Cambridge: Cambridge University Press, 109–28.

14.

DeCastro

McGrath

(2001) O manejo comunitário de lagos na Amazônia [The community management of lakes in Amazon]. Parcerias estratégicas 12: 113–26. URL (consulted 24 July 2013): http://novo.cgee.org.br/arquivos/pe_12.pdf

15.

Edwards

(1953) The relationship between the judged desirability of a trait and the probability that the trait will be endorsed. Journal of Applied Psychology 37(2): 90–3.

16.

Farmer

(2003) Pathologies of Power. Berkeley: University of California Press.

17.

Gotto

(2004) Voir, juger, agir ou l’ABC de l’action citoyenne. Points de Repères, 18.

18.

Gutiérrez

(2004) The Power of the Poor in History. Eugene, OR: Wipf & Stock Publishers [trans. from Spanish by RR Barr, originally published in 1982].

19.

House

Howe

(1999) Values in Evaluation and Social Research. Thousand Oaks, CA: SAGE.

20.

Laperrière

(2007) Evaluating in political turmoil: nursing challenges in prevention programs. Nursing Inquiry 14(1): 42–50.

21.

Laperrière

(2008) Evaluation of STD/HIV/AIDS peer-education and dangerousness: a local perspective. Ciência e Saúde Coletiva 13(6): 1817–24.

22.

Laperrière

(2013) Potential and challenges of participative evaluation for the public health politics. Presentation at the Colloquium Evaluation, translation and administration of knowledge in health research, ENSP-Fiocruz, Rio de Janeiro, Brazil, 11 November 2013.

23.

Laperrière

Potvin

Zúñiga

(2012) A socio political framework of evaluability assessment for participatory evaluations: how to make sense of the power differences in programs that involve the State and Civil society? Evaluation 18(2): 246–59.

24.

Laperrière

Zúñiga

(2006) The community evaluation in public health: vertical conflicts and ambiguous methodologies. Presentation at the 11th International Congress of Public Health, Rio de Janeiro, Brazil, 23 August 2006.

25.

Lomnitz

(2008) Lo formal y lo informal en las sociedades contemporáneas [The formal and informal in contemporaneous societies]. Santiago, Chile: Centro de investigaciones Diego Barros Arana.

26.

Miles

Huberman

(1994) Qualitative Data Analysis, 2nd edn. Thousand Oaks, CA: SAGE.

27.

Ministério da Saúde (2009) Emergência de Saúde Pública de Importancia national - ESPIN Enchentes no Norte e Nordeste do Brasil 2009 [Public health emergency at the national level – ESPIN, Floods in North and Northern Brazil 2009]. Brasilia: Secretaria de Vigilância em Saúde. Comitê das Enchentes. URL (consulted 24 July 2013): http://portal.saude.gov.br/portal/arquivos/pdf/informe_enchentes_norte_nordeste_2009.pdf

28.

Mintzberg

Srinivas

(2010) Juxtaposing doers and helpers in development. Community Development Journal 45(1): 39–57.

29.

Moura

EAF

Peres

LVC

(2000) Aspectos demográficos, sócio-econômicos e de saúde da população ribeirinha durante a enchente de 1999 na Reserva de Desenvolvimento Sustentável Mamirauá (região do Médio Solimões- Amazonas) [Demographic, socioeconomic and health dimensions of riverside populations during the flood of 1999 in the Reserve of sustainable development of Mamirauá]. IDS, Mamirauá, Brazil. URL (consulted 22 July 2013): http://www.abep.org.br/?q=publicacoes/anais/anais-2000-população-e-meio-ambiente

30.

Moura

Moreira

(2001) A população da região Norte: Processos de ocupação e de urbanização recentes [A population from North region: process of recent occupation and urbanization]. Parcerias estratégicas 12: 215–38. URL (consulted 22 July 2013): http://novo.cgee.org.br/arquivos/pe_12.pdf

31.

Naroll

(1962) Data Quality Control. A New Research Technique. Prolegomena to a Cross-Cultural Study of Culture Stress. New York: Glencoe.

32.

Pérouse de Montclos

(2012) Humanitarian action in developing countries: who evaluates who? Evaluation and Program Planning 35: 154–60.

33.

Polanyi

(1964) Personal Knowledge. New York: Harper & Row.

34.

Rocher

(2006) Être sociologue-citoyen. Communication at the Colloque Présences d’un sociologue en hommage à Guy Rocher, Université de Montréal, Montréal, Canada 24 February 2006. URL (consulted 22 July 2013): http://hdl.handle.net/1866/1303

35.

Santos

(2014) Epistemologies of the South. Justice Against Epistemicide. Herndon: Paradigm.

36.

Stavenhagen

(1981) Between Underdevelopment and Revolution. A Latin American Perspective. New Delhi: Abhinav.

37.

The Joint United Nations Programme on HIV/AIDS (2009) Joint Action for Result Outcome Framework 2009-2011. Geneva: UNAIDS.

38.

The Joint United Nations Programme on HIV/AIDS/World Bank/United Nations Development Programme (2005) Mainstreaming HIV and AIDS in Sectors & Programmes. An Implementation Guide for National Responses. Geneva: UNAIDS/World Bank/PNUD.

39.

United Nations Development Programme (n.d.) The Role of Participation and Partnership in Decentralised Governance: A Brief Synthesis of Policy Lessons and Recommendations of Nine Country Case Studies on Services Delivery for the Poor. [Robertson Work, chief Editor]. New York: UNDP. URL: http://unpan1.un.org/intradoc/groups/public/documents/un/unpan006230.pdf

40.

United Nations Development Programme (2004) Decentralised Governance for Development: A Combined Practice Note on Decentralisation, Local Governance and Urban/Rural Development. New York: Democratic Governance Group, Bureau for Development Policy. URL (consulted 20 January 2010): http://www.undp.org/governance/docs/DLGUD_PN_English.pdf

41.

United Nations Development Programme (2005) National Development Planning and Implementation: Strategy Note and Guide. URL (consulted 20 January 2012): http://www.undp.org/hiv/focus02.htm

42.

United Nations Development Programme (2008) Decentralisation and Human Development. HDR Networks 18. URL (consulted 24 July 2013): http://hdr.undp.org/en/media/hdinsights_march2008.pdf

43.

United Nations Development Programme (2009a) The Millennium Development Goals Report. URL (consulted 20 January 2012): http://www.undp.org/mdg

44.

United Nations Development Programme (2009b) Handbook on Planning, Monitoring and Evaluating for Development Results. New York: UNDP.

45.

United Nations Development Program (2012) Democratic Governance Thematic Trust Fund. 2011 Annual Report. New York: UNDP. URL (consulted 20 July 2013): http://www.undp.org/content/dam/undp/library/Democratic%20Governance/Democratic%20Goverannce%20Thematic%20Trust%20Fund/DGTTF_2011AR.pdf

46.

Waiselfisz

(2013) Mapa da violência 2013. Homicídios e juventude no Brasil [Map of violence 2013. Homicids and youth in Brazil]. Rio de Janeiro: CEBELA-Flacso Brasil.

47.

Warren

(2002) La découverte de la ‘question sociale’: sociologie et mouvements d’action jeunesse canadiens-français. Revue d’histoire de l’Amérique française 55(4): 539–72. URL (consulted 12 November 2011): https://dx-doi-org.web.bisu.edu.cn/doi:10.1522/24910383

48.

Zúñiga

(1988) La gestion amphibie. Revue international d’action communautaire 19(59): 157–68.