From Absolute Poverty to Participatory Poverty Assessment and Back Again? A Critique of Poverty Scorecard through a Case Study from Khyber Pakhtunkhwa,Pakistan

Abstract

This paper critiques the poverty scorecard (PSC), a monetary-based tool for measuring poverty that has been adopted by community development programmes and participatory non-governmental organizations (NGOs) in 66 countries including Pakistan. Drawing on 15 interviews and five focus group discussions conducted with the staff members and community members of Frontier Organization for Rural Development, a participatory NGO in Pakistan, I argue that it uses clouds intra-household power and resources distribution thereby furthering gender, class, and ethnic inequalities. Instead of solely relying on the PSC, the paper suggests supplementing it with participatory methods.

Keywords

Rural poverty poverty scorecard participatory poverty assessment gender blindness methodological issues Khyber Pakhtunkhwa

Introduction

This paper critiques the poverty scorecard (PSC), a relatively new poverty assessment tool designed by Mark Schreiner (2009, 2010). It draws on the case study of Frontier Organization for Rural Development (FORD),¹ a participatory non-governmental organization (NGO) that has been working for rural development and democratization for more than 30 years in the rural areas of the province of Khyber Pakhtunkhwa, Pakistan. While field working FORD to investigate historical, politico-economic and socio-cultural factors that have influenced its efforts for bottom-up democratization and empowerment, several staff members and community members expressed reservations about the accuracy of PSC-based findings. Their comments became the basis for investigating the reasons for its adoption as well as its implications.

Since their emergence on the development scene of the developing countries such as Pakistan, Bangladesh, and India, participatory NGOs have been applying various participatory tools and techniques for identifying and mapping the needs and problems of rural poor. Participatory rural appraisal (PRA) is ‘an approach and methods for learning about rural life and conditions from, with and by rural people’ (Chambers, 1994a: 954; see also Chambers, 1994b, 1994c, 2002a, 2002b), but partly due to the methodological limitations of participatory approaches (Cooke and Kothari, 2002; Kapoor, 2002) and partly due to donors’ influence to take it as a technical issue for essentially political problems (Holland et al., 2004; Williams, 2004), a number of NGOs in Pakistan have gradually switched to the application of PSC in identifying poor households in rural areas.

The PSC is an expenditure-based tool for the measurement of poverty, which its designer considers methodologically and analytically superior to the participatory poverty assessment (PPA) and expenditure-based survey questionnaire for collecting reliable and accurate data quickly (Schreiner, 2010). This paper counters these claims by arguing that, being based on the monetary approach to poverty measurement, the tool might work more as a means for the oppression of the poor and socially excluded, particularly women. This could be so because while in PPA households are disaggregated to collect data from men, women, and children from diverse backgrounds, data in PSC are collected from household adults. Based on their opinion about the household conditions, each household is assigned a poverty score. While a one-page PSC-based questionnaire might be easy to quickly administer to a large number of households, the exercise seriously clouds intra-household circumstances, for instance, concerning resource and power distribution between adults and children or between men and women. Finally, the tool might be a preferable choice for application in mapping eligible households for a social safety net programme such as the Federal Government of Pakistan’s Benazir Income Support Program (BISP) (Gazdar, 2011). For NGOs that strive for bottom-up democratization and reach out to the socially excluded, it may turn them into ‘ladles in the global soup kitchen’ (Commins, 1999). This could be arguably so because by following pre-fixed criteria for identifying the poor, the tool assumes a unitary model of rural households (Kabeer, 1999).

To warrant the issues raised above, the paper is structured as follows. The first section briefly describes the key features of the PSC. Since the PSC broadly falls in the domain of monetary approach to poverty, therefore, the section also dwells on key defining features of monetary poverty. Likewise, PSC’s adoption is grounded on the critique of PPA and the like. Therefore, I also present defining features and limitations of PPAs to help us in judiciously assessing the empirical evidence. This is followed by a methodological account of data collection procedures adopted in the field. Finally, the qualitative data analysis is presented to warrant the claim made above and to identify the implications of adopting PSC for rural development in Pakistan and more widely in the Global South.

PSC and monetary poverty

The PSC is an expenditure-based tool for poverty assessment (Schreiner, 2010). The tool has gained quite an acceptance as it has been used in 66 countries so far.² It entered Pakistan through the BISP,³ the first ever national social safety net programme of the Federal Government of Pakistan, which began in 2009 (Durr-e-Nayab and Farooq, 2014; Gazdar, 2011). The tool uses the proxy means test to estimate the welfare status of households on a scale of 0 to 100 as shown in Table 1.⁴ It collects data on twelve indicators that include ‘household size, type of housing and toilet facilities, education, child status, household assets, agricultural landholding and livestock ownership’ (Rural Support Program Network, 2018: 15).

Table 1.

Categories of households in Schreiner’s poverty scorecard.

Poverty category/band	Score range
Extremely poor	0–11
Chronically poor	12–18
Transitory poor	19–23
Non-poor	24–100

Source: Rural Support Program Network (2018: 15).

The PSC’s focus on expenditure places it largely in the domain of monetary poverty. The monetary measure, often regarded as ‘absolute poverty’, is the oldest and perhaps the most widely-used of the four approaches to poverty (Mencher, 1972). The other three approaches are: (a) capability poverty; (b) social exclusion poverty; and (c) PPA (Laderchi et al., 2003; see also Kwadzo, 2010, 2015, Waglé, 2002).

In measuring absolute poverty the standard approach entails the development of a nutritional standard to draw a poverty line by taking income or expenditure as a proxy indicator for nutrition using food energy or food share method (Lister, 2004).⁵ This way, an ‘objective’ measure is developed to distinguish the poor from the non-poor. However, firstly, based on gender, class, climate, occupation and other aspects, people’s lives and hence their needs vary (Alcock, 2006). Secondly, caloric estimates are judgemental and do not reflect diversity in terms of class, race, and gender, no matter which level they are measured at (Baulch, 1996). Thirdly, it identifies a single (low cost) food plan, thus assuming every housekeeper to be ‘efficient’ in managing the nutritional and other needs of the household members with a limited amount of money (Rein, 1971). Fourthly, the ‘physical’ or ‘nutritional’ definition is narrow as well as arbitrary because it does not account for changes in the types of goods in the market and/or changes in people’s habits (Abel-Smith and Townsend, 1972; see also, Alcock, 2006: 82–94; Baulch, 1996; 36–42; Rein, 1971: 46–63). Finally, in gendered terms, the approach assumes a ‘unitary model of household’ that has dominated ‘. . . economic approaches to poverty analysis and [has] informed many of the policies intended to address poverty’ (Kabeer, 2003: 104; see also, Kabeer, 1991; Agarwal, 1986). The distribution of ownership and use of material resources (money, food, clothing, etc.) and non-material resources (decision-making, allocation of tasks and responsibilities, etc.) in households is seldom equal between men and women.

These critical aspects of absolute poverty made way for the mainstreaming of PRA. However, PRA appears to be on the backfoot now as a new and purportedly efficient tool of the PSC has gained popularity. The tool has been used in 66 countries including Pakistan, where in addition to government agencies a number of participatory NGOs have subscribed to it (Rural Support Program Network, 2019). Two features have been avowedly made declaring PSC as distinctively superior to survey questionnaires and participatory approaches. Firstly, according to Schreiner (2010: 327), the tool is ‘tailored to the capabilities and purposes . . . of local, pro-poor organizations'; secondly, the results obtained from NGOs’ use of participatory wealth ranking or ‘blunt’ rules such as land-ownership or housing quality ‘. . . are typically subjective and relative. . .’. Likewise, in contrast to the 45 pages-long questionnaire of the 2005/2006 Pakistan Social and Living Standards Measurement Survey – a survey that the government regularly undertakes – the PSC is asserted to be:

simple, quick, and inexpensive. [Moreover, it is an] . . . easy-to-use poverty scorecard that pro-poor programs in Pakistan can use to estimate the likelihood that a household has expenditure below a given poverty line, to monitor groups’ poverty rates at a point in time, to track changes in groups’ poverty rates over time, and to target services to households. . . . [moreover, it is an] objective tool with known accuracy. . . . [e]xtra work is minimized; non-specialists can compute scores by hand in the field in 5 to 10 minutes because the scorecard has only 10 indicators, only categorical indicators, and only integer points that require no arithmetic beyond basic addition (Schreiner, 2010: 326–34).

Since a social safety net programme such as the BISP is only for women, these claims about the tool’s efficacy seem valid, and hence it would be absurd critiquing the tool for failing to capture intrahousehold inequality. However, for participatory NGOs the tool is questionable for some reasons. Firstly, participatory NGOs do not operate as distributors of monies among the poorest. Rather, in as much as it possible for them, their vision is to transform structures that are formed or sustained by inequalities based on gender, class, ethnicity, etc. Secondly, taking a multidimensional view of poverty, they go beyond the monetary domain to address issues such as lack of entrepreneurial skills, lack of marketing skills, and lack of time. Thirdly, NGOs work with and through a diverse pool of men, women, and children, who are not only economically poor but could also lack adequate access to other opportunities in life. To demonstrate how sole reliance on the PSC is problematic for NGOs I give methodological details about the qualitative data collected from FORD’s staff-members and communities, which is followed by data analysis.

Methodology

The data for this paper were collected as a part of an extended case study (Burawoy, 1998) of FORD. FORD is one of the nine members of the Rural Support Program Network (Rural Support Program Network, 2019), which claims to have the largest community outreach in 145 out of 154 districts in Pakistan (Rural Support Program Network, 2018: 7). The research aimed to conduct a process-based analysis of historical, political-economy and socio-cultural factors inhibiting NGOs’ objectives for democratization and empowerment in rural areas. Such a broadly conceived research agenda took the researcher eight months to conduct 63 in-depth interviews and 20 focus group discussions (FGDs), and collected documentary evidence and observational data about three different projects run by FORD. However, this paper draws on only five FGDs and 15 interviews of the corpus, which were specifically conducted to investigate PSC-related issues using interview guides. I always carried the one-page research information sheet to share it with literate research participants; for illiterate participants, I spelt out the intent of research in the vernacular (Arksey and Knight, 1999). Initially, I conducted 15 individual interviews to collect the narratives of the staff and community members regarding the adoption of PSC and its administration. To compare and contrast the views expressed therein, the same interviewees were requested for participation in FGDs (Kamberelis and Dimitriadis, 2014). The same interviewees were recruited for FGDs because recruiting new participants and engaging them in pro-active and open discussion in group situations could have been considerably challenging. Both the interviews and FGDs were digitally recorded with prior verbal permission from the illiterate research participants and written approval from the literate participants.

It is important to note that the data were collected only from male community members. Pashtun society rather strongly observes norms regarding women’s purdah (veiling); culturally, it is almost a taboo for strangers to speak with women in rural areas. Yet, it is also worth noting that NGOs have adequately trained female workforce to interact with female community members. Whether it is the task of PPA, identification of deserving women for training in livestock, agriculture, or enterprise development, an NGO’s female workers do it. This way NGOs navigate around the cultural barrier.

The data were analysed in a free-floating manner using NVivo, a qualitative data analysis software (Bazeley and Jackson, 2013). That is, each interview and FGD transcript was annotated individually into various independently hanging analytical categories (i.e., nodes) in the NVivo database. Then, all the nodes were re-read for grouping them thematically into ‘tree nodes.’ The process was recursive as it involved reviewing and reflecting both on the poverty literature in the light of participatory theory (e.g., Chambers, 1994b, 1994c) and on the empirical data, the analysis of which is presented below.

Data analysis

The research context

The data for this paper were collected on ‘The livelihood development programme (LDP) project,’ which FORD implemented in the three districts of Hall, Fall, and Gall in Khyber Pakhtunkhwa Province (Pakistan). The LDP was a three-year multi-sectoral project (June 2010–May 2013) funded by the Australian Aid for International Development with an amount of approximately US$ 5 million. To identify the deserving community members for the delivery of various interventions, FORD conducted a survey using the new tool of PSC. The LDP aimed ‘to reduce rural poverty through reviving livelihoods and empowering communities’ (Frontier Organization for Rural Development, 2009: 3) in the three districts. This was to be achieved through four ‘programme components’: ‘[1] to empower poor/vulnerable groups, reviving community collective action and rebuilding community institutions to enable them to make claims to their rights; [2] improve livelihoods of the poor and vulnerable, especially women in the target area; [3] advocacy and lobbying initiatives to bolster support for the long-term strategic needs of the poor and vulnerable, especially women; and [4] developing the capacity of state actors in participatory community development’ (Frontier Organization for Rural Development, 2009: 3–4). The project covered six Union Councils⁶ to deliver interventions in the areas of community physical infrastructure (CPI) development, enterprise and value chain development, microfinance, natural resource management, and human resource development (Table 2).

Table 2.

Goals, objectives and interventions in the livelihood development programme project.

Objectives of the programme	Categories of interventions*	Sets of specific interventions
Objective 1 of the project To empower poor/vulnerable groups, reviving community collective action and rebuild community institutions to make claims for their rights	Identification of poor	1) Poverty scorecard survey
	Social mobilization	2) Community dialogues
		3) Formation of men/women community organizations
		4) Community management skills training
		5) Leadership management skills training
		6) Development of community resource persons
		7) Formation of village development organizations/village organizations and local support organizations
		8) Manager conferences and exposure visits
Objective 2 of the project Improve/strengthen livelihoods of the poor and vulnerable, especially women, in target area	Community physical infrastructure	9) Initiation and completion of community-built physical infrastructure schemes
	Community physical infrastructure	10) Social/technical feasibilities and preparation of programme (community-identified) digests
	Microcredit and community investment fund (CIF)	11) Revolving CIF
		12) CIF management trainings
		13) Development of community credit extension workers
	Natural resource management	14) Agriculture demonstration plots
		15) Training in improved agriculture practices
		16) Training of agriculture extension workers, poultry extension workers and livestock extension workers
		17) Distribution of improved seeds and fertilizers
		18) Vaccination and de-worming campaigns for livestock
	Enterprise development	19) Trainings in enterprise and entrepreneurship development
	Human resource development	20) Trainings in technical and vocational skills
Objective 3 of the project Advocacy and lobbying initiatives to bolster support for long-term strategic needs of the poor and vulnerable, especially women, and developing capacity of state actors in participatory approaches/community development	Social mobilization	21) Capacity building of government officials in various disciplines
		22) Advocacy and awareness workshops/seminars on participatory development, developing pro-poor plans and policies
		23) Coordination meetings between government and civil society organizations
		24) Hardware/technical support and systems development at District Council and Union Council levels

Note: adapted from Frontier Organization for Rural Development (2009: 3–18).

To identify eligible poor male and female community members for the interventions, FORD conducted the PSC survey. The original PSC for Pakistan comprised 10 indicators (see Figure 1). In LDP, however, FORD used a modified version designed by the World Bank (Frontier Organization for Rural Development, 2011: 6) (see Figure 2, section D). It comprises 13 statistically weighted questions that are believed to be simple, objective, easy to use for the collection of data quickly, and the results from which can be aggregated at higher levels (Frontier Organization for Rural Development, 2011). Data were collected by surveying an adult member of each household. Each household was categorized into one of the four bands of poverty as given in Table 1. Adult males and females of households scoring 0–11 were eligible for any type of intervention of the project and household members scoring 11–18 were eligible for selected interventions only.

Figure 1.

Schreiner’s ‘simple poverty scorecard for Pakistan’.

Figure 2.

Poverty scorecard used in the livelihood development programme.^*

Rationale for the adoption of PSC

FORD was established in the 1980s and like most Southern NGOs, it used participatory methods, such as problem ranking, wealth ranking, and poverty ranking (Chambers, 2002a; Rietbergen-McCracken and Narayan, 1998). FORD claims itself to be uniquely belonging to that group of Pakistani NGOs that uses the ‘social guidance’ approach. The approach entails holding frequent ‘dialogues’ with community members to persuade them to sign terms of partnership for development with the organization. A series of dialogues initially wind up in the establishment of community organizations (COs) comprising 20-25 people. When most COs in a particular village remain intact for a while, they are then federated into a village development organization. This is a trans-neighbourhood body comprising representatives from constituent COs. Thus, the process of social guidance initiates as ‘supply-driven’ because FORD reaches out to people without their demand for its presence or services (Frontier Organization for Rural Development, 2009). However, by holding lengthy and frequent community dialogues, villagers get sensitized to form COs to begin demanding services and interventions not only from FORD but also from other NGOs and the government’s line departments of education, irrigation, livestock, etc. In the painstaking process of dialogues, staff-members would regularly rely on participatory tools and techniques of problem ranking, preference ranking, etc. The tools not only helped in the identification of the needs of rural poor (men and women) and the formation of COs, but these also allowed them to garner a constant loop of dialogue and interaction based on trust, equality, and partnership for long-term sustainable alleviation of rural communities’ problems.

Yet, developing a sense of partnership and ownership in the communities and winning their confidence and trust in the organization was never as straightforward as it appears in the theory of participatory development. For instance, in poverty ranking, villagers would categorize their households as ‘destitute,’ ‘very poor,’ ‘poor,’ ‘better off,’ or ‘well to do,’ but this created two problems: ‘First to facilitate the formation of a CO [community organization], the perceived pro-poor bias of RSPs [rural support programmes] can inflate the number of members regarded as poor and very poor. Second, the number of poor and very poor cannot be compared across villages (or regions) and aggregated because the assessment is location-specific’ (Khan, 2004: 4).

These and other limitations of participatory NGOs have been quite well-treated in the scholarly literature. For instance, we know that participation as a ‘good fund-raising device’ (Rahnema, 2010: 131) helped NGOs grow their businesses but it also resulted in PRA being ‘misused or abused’ (Kapoor, 2002: 115). PRA facilitators’ attitudes could overshadow or altogether ignore questions of legitimacy and power, including questions of gender. In many cases, participatory tools for the measurement of poverty such as preference ranking, wealth ranking, seasonal calendars, time-use analysis, FGDs, and in-depth interviews might treat ‘communities’ as homogeneous and harmonious, thereby missing intra-communal power-relations and gender hierarchies (Guijt and Shah, 1998; Mosse, 1994). Similarly, the socio-psychological processes, for example, groupthink, coercive persuasion, etc. in participatory exercises might go unnoticed if PRA is conducted without critical self-awareness by the facilitators (Cooke, 2002).

The ex-project coordinator of FORD, Mr Sarmad (age: 50s), who had worked in the organization for more than 15 years, echoed these limitations. While strongly supporting FORD’s adoption of PSC, he narrated his experience of working with the communities thus:

Earlier we would ask community members about poverty. That was based on their perception. We would ask the community who is poor among you and they would say we all are poor. When we would probe more and more, then they would say that these and those individuals are poor but that was criticized. We were required to develop a scientific approach [emphasis added].

Participatory methods are empowerment-oriented and aim at capturing the meaning of social phenomena (e.g., poverty, powerlessness, and inequality) from the perspective of the research participants, but the results cannot be compared over time or from one area to another (Schreiner, 2009). To overcome these limitations, FORD at one time adopted the quantitative ‘consumption expenditure-based survey’ used by the government of Pakistan and donors (Frontier Organization for Rural Development, n.d: 4). However, that too turned out taxing in terms of time and resources as household circumstances could not be realistically captured either. The search for a ‘scientific’ solution finally ended in 2009 when FORD began using the PSC to ‘strike a balance between qualitative and quantitative approaches’ (Frontier Organization for Rural Development, n.d: 4).

PSC in LDP: Issues and implications

The staff members were trained in data collection through the PSC. They, in turn, trained 98 enumerators hired from the very villages where the data were supposed to be collected (Frontier Organization for Rural Development, 2011: 6). According to a community specialist at FORD, Mr Daaman (age: 30s), locals were hired for data collection because:

even if we are not there, they can still perform their duty, as 10 or 20 days are given to us for poverty scorecard. Second, they know the people by name as they are the members of the same village and they do not have any problem while filling the forms.

Moreover, to ensure accuracy in the data collection, the staff members were mandated to supervise the enumerators and cross-verify 8–10% of the data (Frontier Organization for Rural Development, 2011: 9–10). Despite these measures, there was a discrepancy in the data. This was due to both the administrative oversight to conduct the survey quickly and methodological issues in the PSC.

From the claims made about the PSC (Schreiner, 2010: 326–334), it would follow that the 98 enumerators hired for the survey should have collected objective, and accurate data quickly. However, some interviewees complained about the high poverty score due to which they or other community members became ineligible for financial grants and training in agriculture, livestock, poultry, etc. For instance, Mr Ustaz (age: 30s), a participant in an FGD, complained that ‘unfortunately, the survey was conducted in such a way that poor people’s score is high. So, when training⁷ or grants are announced and we give their [i.e., poor community members’] names, the LDP staff say their score is high and they are not eligible.’ Another participant, Mr Taqweem (age: 30s), a community activist for FORD from village Shahbaz, confirmatively added that ‘our Union Council’s survey has been done and we are on the same page. . . . They have shown the poor as rich and the rich as poor.’

As mentioned above, each response in the 13 questions had a (statistical) weight: ticking yes/no about the availability of household items such as a stove, television, washing machine, fridge, etc. could decrease/increase the poverty score of the households. ‘There could be two reasons,’ Mr Mamdu (age: 50s) from village Siyal added, ‘either the person who was conducting the survey had made a mistake [emphasis added] in noting the responses or the respondent gave him wrong information.’ Since discrepant data had implications for the non/eligibility of villagers for project interventions, it is worth-explaining as to how the two types of ‘mistakes’ occurred.

Although the interviewees confirmed FORD’s practice of hiring local enumerators for data collection, they raised questions about the accuracy of the data. For instance, Mr Bajang (age: 50s), a community resource person from the village of Dawarkhel in Gall, who had been working with FORD for more than 10 years, confirmed that ‘I sent two persons for that training. They both were trained on form-filling. I directed them to cover half the village each from the opposite side. I had monitored them.’ Similarly, Mr Marjan (age: 20s) from village Taluq said that ‘the survey was conducted by the president of our community organization.’

According to the survey report, each field supervisor ‘typically submitted between 35 and 190 completed PSC [poverty scorecard] forms daily to the office’ (Frontier Organization for Rural Development, 2011: 11). However, firstly, the possibility of enumerators filling-in data by themselves was raised by some interviewees. Secondly, some also contended that it might have been done to make money by filling in as many questionnaires as possible without regard to the accuracy of the data. This could have been done due to the reason that enumerators were paid a modest amount of money (Pakistani Rupees [PKR] 15.0) for each PSC questionnaires that they filled. Thirdly, while FORD’s survey report is silent on these issues, it mentions the challenges, such as the problem of locating and recruiting respondents at the time of harvesting season, hot weather, fabrication of responses by the respondents, etc. (Frontier Organization for Rural Development, 2011). Fourthly, the enumerators might have personal or familial biases against some households to deliberately enter incorrect data. Finally, these problems, coupled with the pressure to do the task quickly, might have contributed to enumerators’ collection of data from irrelevant people or inputting data on their own.

To verify whether or not the above views expressed by the community were ‘biased,’ the staff members were consulted. Unsurprisingly, the management did not acknowledge the issue of non-supervision in data collection, but a financial consultant, Mr Pindaar (age: 30s), confidently said that ‘90% of the survey data will be correct.’ Although the field-staff acknowledged discrepancy in the data, they complained about the shortage of time and non-cooperation by the management as reasons for inconsistency in data. For instance, Mr Daaman (age: 30s), the community specialist, remarked that

when the project began, we were given only two months and the PSU [Project Support Unit] and head office said that the targets for the first year needed to be completed. Poverty scorecard, the formation of community organizations and the intervention delivery were also included. We had to do all these in two months.

A social organizer from the Fall district, Mr Cheema (age: 50s), complained that

in the poverty scorecard training, we were told that the social organizers would be supervisors and the PSU team would monitor them, but the PSU team came to us not even a single time. They have not monitored the field visits. They have not paid any attention to our problems.

It appears that the administrative oversight and the rush to fill out a high number of questionnaires each day forced enumerators to collect data without clarifying the questions appropriately, probing into the responses of the respondents, and/or asking questions from relevant individuals. As such, FORD is criticizable for collecting discrepant data. This, nonetheless, is only a partial explanation for the inaccuracies in the data. Part of the reason for the discrepancy in the data lay in the methodological limitation of the tool. Since this aspect of the tool has significant implications not only for FORD but equally and more widely for pro-poor NGOs in the Global South, therefore, I dwell on this aspect of the PSC in the remainder of the paper.

It is claimed that the PSC is an ‘. . . objective tool with known accuracy’ (Schreiner, 2009: 2, 2010: 327). In reality, however, the scorecard is ‘simplistic’ rather than ‘simple’ as Schreiner (2009, 2010) has intriguingly titled his paper. Firstly, question items and response categories were not appropriately customized to the social and economic conditions of the areas. The interviewees contested the idea of asking about the household items as a viable criterion for determining poverty. They said that nowadays even poorer households had small gas-stoves, refrigerators, or water pumps, which they might have purchased either by installments or as second-hand items. Mr Taqweem (age: 30s) criticized the question items as:

Every house has a washroom. We would tell them we have bathrooms and they tick marked it. Doesn’t matter whether it would be a proper bathroom or mud-built but they marked it as yes. For example, if I had a small picnic gas cylinder and they asked: ‘do you have a stove?’ As farmers call a small picnic cylinder as stove, so they said yes and the enumerators marked it as yes. If men were not at home, then women would answer the questions who are not as sharp as men and as women like to show-off⁸ they would answer every question as yes.

Secondly, FORD strictly advised enumerators and monitoring teams to never disclose the purpose and intent of the survey, which also contributed to the discrepancy in the data. The policy of non-disclosure owes its origin to FORD’s experience about communities’ tendency to fabricate responses about their household circumstances. PPA was abandoned because it captured ‘subjective’ aspects of poverty and every participant would claim to be poor (see Mr Sarmad’s remarks above). However, as it is clear from the interviewees’ remarks, the purportedly ‘objective’ and ‘accurate’ nature of the PSC could not bypass these issues either. Thirdly, questions about the non/availability of a toilett, washing machine or the number of school-going children are viewed differently by different sets of adults based on their education, class, and gender. For instance, as per Mr Taqweem’s (age: 30s) remarks noted above, if most women have the tendency to ‘show off’ while giving information about household items, then just one circumstantial factor of the harvesting season, when male household members are busy in the farms, would affect both accuracy and objectivity of the data. As mentioned above, the project was implemented in Khyber Pakhtunkhwa where Pashtuns are the major ethnolinguistic group. The Pashtun social structure, particularly in rural areas, is characterized by strong affinal ties of communitarian living and yet they also strongly espouse ‘tarboorwali, or cousin rivalry, and siyali, or status envy . . . so that even within small family groups there are sources of tension and centrifugal pressure’ (Ginsburg, 2011: 96). The ‘tribal’ sense of communitarian living and simultaneously entertaining milder or serious envy towards some or all of one’s kin group is commonly found in rural Pashtun society. Although, no interviewee or FGD participant mentioned or acknowledged this, nonetheless, an evaluation study of PSC-based data collection for the BISP is quite revealing for our purpose here. A beneficiary from the Muzaffargarh district of Pakistan is reported thus:

Shirin’s family consists of 13 individuals and has no assets except their house and a cow. Shirin’s PSC score is above 27 and hence she is ineligible for the cash transfer. According to Shirin, the survey team did not come to her doorstep and took down her details in a central location in the village. She believes that the survey team which included residents ‘favoured their own people’. Shirin is not clear on how the score is calculated and holds the mistaken belief that the survey team itself assigned her a score of 27 (Gazdar and Zuberi, 2014: 19).

As in this case, it would not be surprising to find a Pashtun to misreport about his neighbour or a cousin, especially in cases when an adult representative from the household is not available on the spot. Thus, if ‘small errors in reporting – either on the part of the respondent or the enumerator – can make the difference with regard to eligibility’ in BISP (Gazdar and Zuberi, 2014: 20), so can we expect this to happen with the PSC survey by organizations such as FORD.

Similarly, in rural Pakistan, it is common to find three or four married brothers living together in an extended family. They might have pooled money to build a single flush toilet for use by household members. Likewise, it is also quite common to observe that a relatively richer brother in the joint family builds a toilet or buys a bike or refrigerator for personal use but following cultural norms regarding strong sense of in-group loyalty, he would allow his married brothers and their children to use the items he has bought or built. An old peasant interviewee, Mr Daad (age: 60s), whose son was a member of the village CO, made an insightful comment in this regard:

as you [might] know, our people mostly live together [in joint families] under a single roof but kitchens are separate. See there is poverty. . . . we often have one toilet that is shared by all people [belonging to the same joint family] but if you ask the people [i.e., member of the joint family] each would say yes I have a concrete toilet; . . . [relatively] rich [brother] gives bike or fridge when needed [by poorer brother]. Who knows if those using someone else’s bike, fridge in the [larger joint] family says, ‘yes’ I have it.’

If an adult male member of the household says ‘yes’ to the questions regarding the presence of a concrete toilet used by all in a joint family or a gas stove that each kitchen-user regularly borrows for cooking, then the score of those who actually do not own such item would also go high. In a way, just as in PPAs, if conducted repeatedly with different members of the household, produced temporally and geographically non-comparable data, so might be the case with PSC-based data collection.

The preceding point brings up the final and arguably the most palpable critique of the tool from a gender perspective. It obscures the fact that women’s poverty is often hidden whenever it is measured at the household level or the family level (Millar and Glendinning, 1992). The measurement of poverty in income or expenditure terms at household level has two biases. Firstly, by estimating poverty at a single scale level, that is, household income, it suffers from a ‘measurement bias’ for it assumes equal distribution of intra-household resources. Secondly, the measure suffers from an ‘institutional bias’ for it prioritizes market as distributor or mechanism for satisfying needs and thus bypasses gender inequality in the fulfilment of basic needs (Kabeer, 1996).

Collecting data through the PSC at the household level exposes it to the above critique in that it does not measure ownership, control, and usage of intra-household assets. These issues have far-reaching implications not only for FORD and other NGOs in Pakistan, but for all those participatory NGOs which aim for rural upliftment in the Global South and who might be adopting this tool.

PSC or participatory methodology? Breaking the deadlock

The foregoing analysis could be questioned on the ground that if the data collection process using the PSC is improved then there might be nothing to critique it for. Some might also say that similar types of operational issues could also arise in the application of participatory methods. Thirdly, the direct assessment process of participatory methods is costly, time-consuming and subjective in nature while the PSC, besides being quick and inexpensive, is minimally subjective. Hence, instead of rejecting the PSC, it would be arguably better to improve its data collection process. However, these contentions are answerable as follows.

In contrast to the practice of temporarily hiring enumerators for the collection of data using the PSC, it is an NGO’s regular field-staff who apply participatory methods. Even if some team members do not have the expertise and in-depth knowledge on assessing poverty or do not know the theory and practice of a participatory approach, the mentorship of experienced staff-members and office-based discussions add to the rigour of the data and quicken organizational learning amongst the staff-members (Ebrahim, 2003; Pearson, 2011). The PSC’s application would not give this opportunity to the field staff because data collection is done through the enumerators hired from the community. As stated earlier, participatory methods are based on the approach of learning ‘from, with and by rural people’ (Chambers, 1994a: 954). Its dialogical interactive processes have been found to reflect the realities of rural poor more accurately and in detail than the quantitative tools of data collection (see, for example, Chamber, 1994a, 1994b, 1994c).

The ultimate purpose of quantitative and qualitative tools is to produce stories (Denzin, 2019). If an organization’s agenda is to collect stories of those suffering from monetary poverty and give them financial support, then using the PSC could be enough. But if an NGO’s agenda is inclusively aiming at identifying those who are not just monetarily poor but whose rights are violated, for example, in the form of domestic violence, child labour, and under-payment in labour services, then PSC-based results would not be adequate. Although such an agenda, besides being time-consuming, has at times appeared elusive, NGOs are still considered relatively more effective than state agencies in many developing countries (Bebbington et al., 2008).

Questions in the PSC (Figure 2, Section D) collect data about aspects of the households’ monetary poverty, but they do not in any direct sense measure intra-household inequalities. For brevity, I problematize only three questions in the tool. Data on question 4 cannot show the multiple factors due to which the school-aged children might not attend school. Since there is no space provided for noting the reasons for not attending the schools, it would be narrow to assume that it is only due to monetary poverty that, for example, some children are attending schools while others do not. Since many participatory NGOs in Pakistan and other developing countries work for increasing children’s enrolment in schools, PSC-based data on this aspect would not tell us the reasons behind children’s non-enrolment/attendance in schools and, therefore, an NGO could be guessing to minimize or alleviate the factors due to which children do not go to schools. In a similar vein, the presence of television (question 11) in a household is indeed reflective of the economic status of a household, but we cannot reasonably conclude that all members of the household would get equal opportunity to watch it for recreational purposes. Likewise, the presence of a certain number of cattle (question 12) in the household is a plausible proxy indicator to determine a household’s non/poor status. Yet, we would not be in any sense close to knowing whether the income earned from their products (e.g., milk) or their sale goes to the consumption needs of the whole household or the household head. Although participatory methods are time-consuming and costly, nonetheless, I would argue that saving time and money should not be prioritized enough to leave out the socially excluded or to bypass the collection of data on intra-household power dynamics.

Finally, in many developing countries there is a dearth of commitment to collecting accurate data. For instance, as mentioned above, the PSC’s use in the BISP was dictated by the goal to identify deserving poor women for the government’s tri-monthly financial support. Besides the reported practice of community members favouring their kin (see p. 12), in January 2020, 82,0165 fake beneficiaries reportedly consumed BISP money of which 14,730 were government employees (Hussain, 2020). This instance cannot be used as justification for stopping at making the data collection processes of the PSC more rigorous. Yet, in a general environment of malpractices at the communal and government level, collecting accurate data could be quite intractable.

We cannot solely rely on the PSC to collect data on ownership, control, and usage of intra-household assets and resources. Yet, rather than taking the PSC and participatory methods as dichotomous, it might be better to weld both together. The tools could not only turn out mutually supplementary, but these could also help NGOs address challenges of accountability and transparency for which they have been recently critiqued.

Conclusion: The way forward

The NGOs have recently come under scrutiny for relative lack of or deficiencies in accountability and transparency, the very goals for which they burgeoned in the 1980s and onwards. NGOs have been using various development jargon (Pearce, 2010) mostly to attract funding rather than focusing on the actual work to be undertaken (Bano, 2008, 2012).⁹ A process of NGOization has taken place (Banks et al., 2015: 709) whereby they have been increasingly engaging in the delivery of public services and other tangible benefits (Hearn, 1998; Yacobi, 2007). In such a scenario, we might echo Sahoo’s (2013) conclusion that people’s participation in development activities cannot be taken as equivalent to awareness about their development.

Nevertheless, governments in the Global South rely on NGOs for the provision of various developmental (poverty alleviation, enterprise development, etc.), relief and humanitarian services (Aldashev and Navarra, 2018). Without going tangentially to ponder whether a country such as Pakistan can muster decentralized and people-centred development or not, the Global South does need participatory NGOs to work on issues of vital rural concern, be it poverty, employment, education, women’s empowerment, etc. For instance, achieving the United Nations’ Sustainable Development Goals is beyond Pakistan’s capacity and that of most developing countries. There is no doubt that participatory approaches, though in no sense a panacea for the ills of development, nonetheless weld congruently well with NGOs’ goals. Thus, keeping aside the debate whether participation is a tyranny (Cooke and Kothari, 2002) or means for transformation (Hickey and Mohan, 2004), the use of participatory methods, however practiced, does allow NGOs to develop some rapport and understanding with and between community members.

In contrast to the participatory exercises where the field-staff would intensively interact with the villagers, the PSC might gradually change this into a one-time interaction. More importantly, the administering of the survey through the enumerators from the community is a kind of ‘outsourcing’ of the fieldwork through which the field-staff and the community members used to develop a level of affinity and collaborative engagement with each other. Secondly, keeping the gendered critique in view, if a household is categorized as ‘non-poor,’ but its female members do not have or are given limited access to the household assets, then it might have two negative implications. Women in such households would be deprived of the opportunity to access interventions. Moreover, working on the basis of gender-blind results, organizations such as FORD might operate for the perpetuation of gender-based inequalities. In other words, the tool would further the organizational contradictions of participatory NGOs as while they envision to garner grassroots change in rural areas, in reality using PSC-based data they would mostly turn blind to gender, ethnic, class and other power-based hierarchies in the rural social structure. Finally, for participatory NGOs believing in bottom-up social change through the social guidance of the poor, the tool’s narrow and exclusive focus on quantitative information would deprive them of documenting subjective factors implicating the well-being of the poor. Hence, like state agencies, participatory NGOs might merely become top-down operational mechanisms for the provision of welfare items thereby truncating the very agenda for which they emerged in the 1980s in the first place.

Rather than creating an ‘either–or’ situation between participatory methods and the PSC, it would be better to integrate both. The micro-details on intra-household circumstances collected through participatory methods along with the documentation of monetarily poor households through the PSC could help NGOs design appropriate strategies for tackling multiple challenges that the poor and the socially excluded face in rural areas. However, its feasibility is something that could be addressed through further research.

Footnotes

Acknowledgements

I thank the anonymous reviewer for insightful comments on an earlier draft of the paper. I would also like to thank Dr Noor Sanauddin (University of Peshawar) and Dr Sanaullah (AWKUM, Mardan) for critical comments and suggestions on the paper.

Funding

The author declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: I thank Charles Wallace Pakistan Trust (United Kingdom) for a grant of £1000 towards the completion of the Doctoral project from which this paper is drawn.

ORCID iD

Syed Owais

Notes

Author biography

Syed Owais is an Assistant Professor at the Department of Sociology, University of Peshawar. His main research experience and interests are in the application of sociological and historical institutionalists arguments for analysing NGOs’ efforts for grassroots change.

References

Abel-Smith

Townsend

(1972) The poor and the poorest. In: Roach

Roach

(eds) Poverty: Selected Readings. London: Penguin Books, 138–150.

Agarwal

(1986) Women, poverty and agricultural growth in India. The Journal of Peasant Studies 13(4): 165–220.

Alcock

(2006) Understanding Poverty. London: Palgrave Macmillan.

Aldashev

Navarra

(2018) Development NGOs: Basic facts. Annals of Public and Cooperative Economics 89(1): 125–156.

Arksey

Knight

(1999) Interviewing for Social Scientists: An Introductory Resource with Examples. London: SAGE Publications.

Banks

Hulme

Edwards

(2015) NGOs, states, and donors revisited: Still too close for comfort? World Development 66: 707–718.

Bano

(2008) Dangerous correlations: Aid’s impact on NGOs’ performance and ability to mobilize members in Pakistan. World Development 36(11): 2297–2313.

Bano

(2012) Breakdown in Pakistan: How Aid is Eroding Institutions of Collective Welfare. Stanford: Stanford University Press.

Baulch

(1996) Neglected trade-offs in poverty measurement. IDS Bulletin 27(1): 36–42.

10.

Bazeley

Jackson

(2013) Qualitative Data Analysis with NVIVO. London: SAGE Publications Ltd.

11.

Bebbington

Hickey

Mitlin

(eds) (2008) Can NGOs Make a Difference? The Challenge of Development Alternatives. London: Zed Books.

12.

Burawoy

(1998) The extended case method. Sociological Theory 16(1): 4–33.

13.

Chambers

(1994a) Participatory rural appraisal (PRA): Analysis of experience. World Development 22(9): 1253–1268.

14.

Chambers

(1994b) Participatory rural appraisal (PRA): Challenges, potentials and paradigm. World Development 22(10): 1437–1454.

15.

Chambers

(1994c) The origins and practice of participatory rural appraisal. World Development 22(7): 953–969.

16.

Chambers

(2002a) Participatory Workshops: A Sourcebook of 21 Sets of Ideas and Activities. London: Earthscan.

17.

Chambers

(2002b) Relaxed and Participatory Appraisal: Notes on Practical Approaches and Methods for Participants in PRA/PLA-related Familiarization Workshops. Available at: http://courses.washington.edu/pbaf531/Chambers_PRA_Notes.pdf (accessed 14 July 2016).

18.

Commins

(1999) NGOs: Ladles in the global soup kitchen? Development in Practice 9(5): 619–622.

19.

Cooke

(2002) The social psychological limits of participation. In: Cooke

Kothari

(eds) Participation: The New Tyranny? London: Zed, 102–121.

20.

Cooke

Kothari

(2002) Participation: The New Tyranny? London: Zed Books.

21.

Denzin

(2019) The Qualitative Manifest: A Call to Arms. New York: Routledge.

22.

Durr-e-Nayab Farooq

(2014) Effectiveness of cash transfer programs for household welfare in Pakistan: The case of the Benazir Income Support Program. The Pakistan Development Review 53(2): 145–174.

23.

Ebrahim

(2003) NGOs and Organisational Change: Discourse, Reporting and Learning. New York: Cambridge University Press.

24.

Frontier Organization for Rural Development (2009) Livelihood Development Program (Project Proposal). Islamabad: Frontier Organization for Rural Development.

25.

Frontier Organization for Rural Development (2011) Annual Report: Livelihood Strengthening Program. Islamabad: Frontier Organization for Rural Development.

26.

Frontier Organization for Rural Development (n.d) Poverty Score Card: Four Districts Under Poverty Alleviation Program. Islamabad: Frontier Organization for Rural Development.

27.

Gazdar

(2011) Social protection in Pakistan: In the midst of a paradigm shift? Economic and Political Weekly 46(28): 59–66.

28.

Gazdar

Zuberi

(2014) Final Report of the Beneficiary Feedback Survey, National Cash Transfer Program – Pakistan. Karachi: Collective for Social Science Research. Available at: http://www.researchcollective.org/Documents/Updated_Anonymised_BFS_Final_Report_03022014.pdf (accessed 12 September 2019).

29.

Ginsburg

(2011) An Economic Interpretation of the Pashtunwali. The University of Chicago Legal Forum 2011(1): 89–114.

30.

Guijt

Shah

(1998) The Myth of Community: Gender Issues in Participatory Development. London: IT Publications.

31.

Hearn

(1998) The ‘NGO-isation’ of Kenyan society: USAID & the restructuring of health care. Review of African Political Economy 25(75): 89–100.

32.

Hickey

Mohan

(2004) Participation: From Tyranny to Transformation? Exploring New Approaches to Participation in Development. London: Zed Publishers.

33.

Holland

Brocklesby

Abugre

(2004) Beyond the technical fix? Participation in donor approaches to rights-based development. In: Hickey

Mohan

(eds) Participation: From Tyranny to Transformation? Exploring New Approaches to Participation in Development. London: Zed Publishers, 252–268.

34.

Hussain

(2020) PTI govt sacks 4 grade-17 employees over BISP scam. The Express Tribune, 19 January. Available at: https://tribune.com.pk/story/2140069/1-pti-govt-sacks-4-grade-17-employees-bisp-scam/ (accessed 22 February 2020).

35.

Jamal

Baldwin

(2017) Angels of mercy or smiling western invaders? Community’s perception of NGOs in Northwest Pakistan. International Social Work 62(1): 89–104.

36.

Kabeer

(1991) Gender dimensions of rural poverty: Analysis from Bangladesh. The Journal of Peasant Studies 18(2): 241–262.

37.

Kabeer

(1996) Agency, well-being and inequality: Reflections on the gender dimensions of poverty. IDS Bulletin, 27.

38.

Kabeer

(1999) Targeting women or transforming institutions? Policy from NGO anti-poverty efforts. In: Eade

(ed.) Development with Women. London: Oxfam, 32–45.

39.

Kabeer

(2003) Gender Mainstreaming in Poverty Eradication and the Millennium Development Goals: A Handbook for Policymakers and other Stakeholders. Ottawa (Ontario): Commonwealth Secretariat, International Development Research Centre (IDRC).

40.

Kahlon

TUM

(2015) Governance in Pakistan: Legitimacy regime of NGOs. NDU Journal XXIX: 181–196.

41.

Kamberelis

Dimitriadis

(2014) Focus group research: Retrospect and prospects. In: Leavy

(ed.) The Oxford Handbook of Qualitative Research. Oxford: Oxford University Press, 1–48.

42.

Kapoor

(2002) The devil’s in the theory: A critical assessment of Robert Chambers’ work on participatory development. Third World Quarterly 23(1): 101–117.

43.

Khan

(2004) Methods of assessment of rural poverty, projects and program impact: A handbook for practitioners in rural support programs. Rural Support Program Network (RSPN). Available at: http://www.rspn.org/wp-content/uploads/2013/04/Methods-of-Assessment-of-Rural-Poverty-Projects-and-Programmes-Impacts-A-Handbook-for-Practitioners-in-Rural-Support-Programmes-2004.pdf (accessed 2 April 2014).

44.

Kwadzo

(2010) Conceptualisation and measurement of poverty: A comparative analysis. Unpublished PhD Thesis, Indiana University of Pennsylvania, Indiana, PA.

45.

Kwadzo

(2015) Choosing concepts and measurements of poverty: A comparison of three major poverty approaches. Journal of Poverty 19(4): 409–423.

46.

Laderchi

Saith

Stewart

(2003) Does it matter that we do not agree on the definition of poverty? A comparison of four approaches. Oxford Development Studies 31(3): 243–274.

47.

Lister

(2004) Poverty. Cambridge: Polity Press.

48.

Mencher

(1972) The problem of measuring poverty. In: Roach

Roach

(eds) Poverty: Selected Readings. London: Penguine Books, 71–85.

49.

Mezzera

Aftab

Yusuf

(2010) Devolution Row: An Assessment of Pakistan’s 2001 Local Government Ordinance. Available at: https://www.clingendael.org/sites/default/files/pdfs/20101119_CRU_publicatie_mmezzera.pdf (accessed 1 April 2014).

50.

Millar

Glendinning

(1992) ‘It all really starts in the family’: Gender divisions and poverty. In: Glendinning

Millar

(eds) Women and Poverty in Britain: the 1990s. London/New York: Harvester Westsheaf, 3–10.

51.

Mosse

(1994) Authority, gender and knowledge: Theoretical reflections on the practice of participatory rural appraisal. Development and Change 25: 497–526.

52.

Owais

Sanauddin

Nawaz

, et al. (2015) Negotiating transparency and accountability of non-governmental organizations (NGOs) in Khyber Pakhtunkhwa, Pakistan. PUTAJ – Humanities & Social Sciences 22(2): 47–61.

53.

Pearce

(2010) Is social change fundable? NGOs and theories and practices of social change. Development in Practice 20(6): 621–635.

54.

Pearson

(2011) Integrating learning into organisational capacity development of Cambodian NGOs. Development in Practice 21(8): 1037–1049.

55.

Rahnema

(2010) Participation. In: Sachs

(ed.) The Development Dictionary: A Guide to Knowledge as Power. London: Zed Books, 127–144.

56.

Rein

(1971) Problems in the definition and measurement of poverty. In: Townsend

(ed.) The Concept of Poverty. London: Heinemann Educational Books Ltd, 46–63.

57.

Rietbergen-McCracken

Narayan

(1998) Participation and Social Assessment: Tools and Techniques. Washington, DC: World Bank.

58.

Rural Support Program Network (2018) Annual Report 2018. Available at: http://www.rspn.org/wp-content/uploads/2019/04/AR%20RSPN%202018.pdf (accessed 29 August 2019).

59.

Rural Support Program Network (2019) Meet Our Members. Available at: http://www.rspn.org/index.php/about-us/our-members/ (accessed 24 January 2019).

60.

Sahoo

(2013) Doing development or creating dependency? NGOs and civil society in India. South Asia: Journal of South Asian Studies 36(2): 258–272.

61.

Schreiner

(2009) A simple poverty scorecard for Pakistan. Available at: http://www.simplepovertyscorecard.com/PAK_2005_ENG.pdf (accessed 3 May 2016).

62.

Schreiner

(2010) A simple poverty scorecard for Pakistan. Journal of Asian and African Studies 45(3): 326–349.

63.

Sen

(1983) Poor, relatively speaking. Oxford Economic Papers 35(2): 153–169.

64.

Thomas

(2000) Poverty and the ‘End of Development’. In: Allen

Thomas

(eds) Poverty and Development into the 21st Century. London: Oxford University Press, 3–22.

65.

Townsend

(1979) Poverty in the United Kingdom: A Survey of Household Resources and Standards of Living. Middlesex: Penguin Books Ltd.

66.

United Nations Development Programme (1997) Human Development Report. Available at: http://hdr.undp.org/en/content/human-development-report-1997 (accessed 2 February 2015).

67.

Waglé

(2002) Rethinking poverty: Definition and measurement. International Social Science Journal 54(171): 155–165.

68.

Weiss

(1998) The slow yet steady path to women's empowerment in pakistan. In: Haddad

Esposito

(eds) Islam, Gender and Social Change. New York: Oxford University Press, 124–143.

69.

Weiss

(1999) Women, civil society and politics in Pakistan. Citizenship Studies 3(1): 141–150.

70.

Weiss

(2003) Interpreting Islam and women’s rights: Implementing CEDAW in Pakistan. International Sociology 18(3): 581–601.

71.

Williams

(2004) Evaluating participatory development: Tyranny, power and (re)politicisation. Third World Quarterly 25(3): 557–578.

72.

Yacobi

(2007) The NGOization of space: Dilemmas of social change, planning policy, and the Israeli public sphere. Environment and Planning: Society and Space 25(4): 745–758.