Economic Evaluation of Public Health Interventions: An Application to Interventions for the Prevention of Violence Against Women and Girls Implemented by the “What Works to Prevent Violence Against Women and Girls?” Global Program

Abstract

Violence against women and girls (VAWG) has important social, economic, and public health impacts. Governments and international donors are increasing their investment in VAWG prevention programs, yet clear guidelines to assess the “value for money” of these interventions are lacking. Improved costing and economic evaluation of VAWG prevention can support programming through supporting priority setting, justifying investment, and planning the financing of VAWG prevention services. This article sets out a standardized methodology for the economic evaluation of complex, that is, multicomponent and/or multiplatform, programs designed to prevent VAWG in low- and middle-income countries (LMICs). It outlines an approach that can be used alongside the most recent guidance for the economic evaluation of public health interventions in LMICs. It defines standardized methods of data collection and analysis, outcomes, and unit costs (i.e., average costs per person reached, output or service delivered), and provides guidance to investigate the uncertainty in cost-effectiveness estimates and report results. The costing approach has been developed and piloted as part of the “What Works to Prevent Violence Against Women and Girls?” (What Works?) program in five countries. This article and its supplementary material can be used by both economists and non-economists to contribute to the generation of new cost-effectiveness data on VAWG prevention, and ultimately improve the allocative efficiency and financing across VAWG programs.

Keywords

intimate partner violence violence against women and girls women’s empowerment community mobilization costing cost-effectiveness outcome measurement

Introduction

Globally, 35% of women have experienced intimate partner violence (IPV) (Devries et al., 2013). Violence against women and girls (VAWG) has important social, economic, and public health impacts, in addition to being a human rights concern. Exposure to IPV is associated with an elevated risk of a number of adverse health outcomes, such as HIV, induced abortion, and preterm birth (World Health Organization [WHO], 2013), as well as a number of adverse mental health outcomes (Ferrari et al., 2016; WHO, 2013). Given the scale of the problem, governments and international donors are allocating increasing resources to address VAWG, particularly IPV. The United Kingdom and the World Bank alone have invested more than US$400 million in programs for, and research on, the prevention of VAWG and support to survivors in low- and middle-income countries (LMICs; Global Gender-Based Violence Task Force, 2017), and the WHO has issued guidelines on violence prevention programming for policy makers and planners (WHO, 2010).

The scale of the investment and policy focus on VAWG warrants an increased interest in assessing the value for money of VAWG interventions. Evidence on the economic value of violence prevention and development programming is, however, scarce. In addition, assessments of VAWG are methodologically challenging for three reasons. First, it is difficult to account for and value the diverse outcomes of VAWG prevention in areas such as health, education, work, social networks, and access to financial resources. Second, cost estimates are currently only available from small research projects, which may not be representative of costs of implementing at scale. Third, VAWG prevention delivery processes vary by context, and culturally specific gender norms, limiting the generalizability of costs and economic evaluation.

We outline a standardized methodology for the economic evaluation of complex VAWG prevention interventions. We developed this methodology by adapting current general guidance on economic evaluation and costing in global health (Neumann, Sanders, Russell, Siegel, & Ganiats, 2016; Vassall et al., 2017 [https://ghcosting.org/pages/standards/reference_case]; Wilkinson et al., 2016) to the characteristics of VAWG prevention programming. Analysts intending to apply our guidelines should be familiar with the general guidance too. Our aim is to provide further specification around their specific application to VAWG prevention.

“What Works to Prevent Violence Against Women and Girls?” (What Works?) is a multicountry, multiyear research and innovation program that tested the effectiveness and cost-effectiveness of VAWG prevention programs in LMICs. It is funded by the United Kingdom’s Department for International Development (DfID) and it set up an international network of implementers, VAWG prevention professionals, and researchers who co-created and evaluated these interventions. The economic evaluation component was tasked with determining the value for money of six interventions implemented alongside randomized controlled trials. The interventions were implemented in six different countries, five in sub-Saharan Africa, and one in South Asia, and across different platforms: schools, villages/communities, and adult small-group learning settings. We developed a standardized economic evaluation methodology to improve transparency in value for money estimates of VAWG interventions, as well as their comparability, replicability, and generalizability (Evans & Popova, 2016), for What Works? and beyond.

Our structure and approach to economic evaluation methods are in line with the Reference Case approach outlined in the International Decision Support Initiative (iDSI) guidelines (2014), which remain largely within the extra-welfarist framework (Brouwer, Culyer, van Exel, & Rutten, 2008) that is prevalent in economic evaluation in the health sector (Neumann et al., 2016; Vassall et al., 2017; Wilkinson et al., 2016). We do not state here whether a specific method of economic evaluation (cost-effectiveness, cost-utility, cost-consequence, or benefit–cost analysis) should be adopted, because this decision will also depend on the specific problem analysts are faced with, as well as on data availability. For methods related to costing, we also referred to the Global Health Costing Consortium (GHCC) Reference Case (Vassall et al., 2017). Both sets of guidelines are designed for global health and are consistent with the Second U.S. Panel on Cost-Effectiveness in Health and Medicine (Neumann et al., 2016) and the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) guidelines (Husereau et al., 2013).

We also referred to the U.K. DfID Value for Money (VfM) framework for the assessment of the economy, efficiency, effectiveness, and equity of its programs (DfID, 2011). Specifically, we state how our recommendations relate to DfID’s framework and its implementation. The article begins by explaining and discussing the challenges in identifying, measuring, and valuing resources and outcomes. We then list the issues to consider in analysis. We summarize our recommendations for each of the iDSI and GHCC principles in Tables A1 and A2 in the appendix. Copies of the manual and tools to accompany the guidance outlined in this article are available on the What Works? website as a project resource.¹

The Decision Problem

The basis for any economic evaluation of an intervention is the “decision problem” (Vassall et al., 2017 [https://ghcosting.org/pages/standards/reference_case]). The decision problem outlines the investment choice and can be defined by target Population, Intervention, Comparator, and Outcome (PICO) to be achieved in a PICO statement (see Figure 1 for two examples; Richardson, Wilson, Nishikawa, & Hayward, 1995).

Figure 1.

PICO statements from two What Works? interventions.

Population

The typical target populations of VAWG prevention are women and girls. However, due to the complexity of VAWG, costs and outcomes fall on other populations (externalities) and need to be considered for decision makers interested in broader social welfare. For example, male perpetrators of violence may also benefit from decreased use of violence. Children may be adversely affected by mothers’ exposure to violence (Hardt et al., 2015; Smith-Marek et al., 2015) and thus gain from a reduction in violence. Women’s improved access to economic resources may also improve child health and educational outcomes (Duflo, 2003). More generally, changing social norms and gender attitudes through targeting entire communities may result in spillover benefits to those directly and indirectly affected by violence (Stern, Heise, & McLean, 2017).

Intervention

Analysts should describe the intervention, including all activities required for its development and implementation. Broadly, VAWG interventions can be classified as workshop-based interventions (for individuals, families, or couples), economic empowerment interventions (with or without gender-empowerment components), school interventions delivered to classes, psychotherapeutic interventions including self-defense, and gender attitudes and social norms change interventions delivered to communities (often multiple components, for example, including workshops, community action teams, street theater, murals, and community radio). Workshops, gender empowerment, and programs delivered to classes are often administered over a limited number of small-group sessions (Gibbs et al., 2017; Iyengar & Ferrari, 2011). Trained facilitators deliver a curriculum, which includes sessions on gender, power, and violence; teaching a mixture of vocational and communication skills; supporting help-seeking or providing support; and skills of critical reflection. The assumption is that increased information on harmful norms and behaviors and an expanded skillset for challenging these will reduce exposure to, and perpetration of, VAWG. Community-based interventions aimed at reducing VAWG have also shown encouraging impact (Abramsky et al., 2014). They typically entail the identification and training of local volunteers. Some community-based schemes first raise VAWG awareness and then build positive concepts and practices of power in the pool of potential recruits. In turn, the most motivated recipients become community mobilizers and administer the intervention to the rest of the community (Pronyk et al., 2006; Stern & Niyibizi, 2018). Some VAWG prevention interventions are implemented using existing services (such as community financing schemes or schools).

Strengthening local services/platforms at times precedes the intervention (Pronyk et al., 2002; Stern & Niyibizi, 2018) and may incur costs. Analysts should clearly describe the platform, in terms of the existing services delivered, the scale of operation, and where services are delivered jointly. Understanding the platform is required to correctly identify the incremental costs of the investment and generalize results across settings. Specifically, the platform will influence costs and/or outcomes, through synergistic benefits, or economies of scope. Economies of scope occur when costs are reduced through improved efficiency as a result of joint production; for example, where the cost of a combined life skills and microfinance intervention is less than the cost of providing these two interventions separately.

It is also critical to clearly describe any activities required to start the VAWG intervention. Typical start-up activities for complex interventions include curriculum development, curriculum adaptation, frontline workers’ training, and obtaining permission to operate locally. Where feasible, total costs of development should be estimated and included in cost-effectiveness estimates. However, if the interventions were developed over a long period of time through iterative processes, more than five years prior to the study, costs may not be properly documented. This makes it particularly challenging to identify all relevant resources, increasing the likelihood of inaccurate costing and reducing comparability if costs can be determined for some interventions but not for others. In such cases, we recommend reporting total development costs separately, where these can be determined, and refraining from including them in the cost-effectiveness estimates. Where a program is scaling up, a clear description of any activities to scale up from the research setting stage is necessary (Evans & Popova, 2016).

Comparator

In assessing cost-effectiveness, analysts must establish incremental costs and benefits compared with the status quo (i.e., usual care or “do nothing”). In some cases, different investment options may also be compared, as they will be part of the “choice set” of interventions the decision maker is deliberating. For interventions introduced as add-ons to existing programs (Gupta et al., 2013; Iyengar & Ferrari, 2011; Pronyk et al., 2006), the existing program is considered the usual care (control) scenario. Particularly where this may be required to enable scale-up, a do-nothing comparator and the cost of strengthening the status quo should be included.

Relevant comparators that may be accessed by the target population to improve their outcomes should be included, even if not directly connected with the specific Randomised Controlled rial (RCT) or evaluation. Particular difficulties in assessing the status quo do not justify its exclusion from analysis (Sculpher, Claxton, Drummond, & McCabe, 2006). For complex, multicomponent programs already administered at scale and not evaluated with RCTs, analysts may exploit gradual rollout or other econometric or modeling techniques to characterize the comparators or create synthetic controls, rather than exclude the status quo (Jones, 2015).

Outcomes

The primary outcomes typically reported in the evaluation of VAWG prevention are the number of cases of IPV and/or disability-adjusted life years (DALYs) averted (Jan et al., 2011; Michaels-Igbokwe et al., 2016). However, these may only be a partial measure of impact: DALYs capture a subset of IPV health sequelae, and many interventions (particularly those delivered on joint platforms) are likely to affect outcomes beyond VAWG. In addition, outcomes may interact with each other. Therefore, these outcomes may not capture the full direct impact of interventions and their broader societal impact (Neumann et al., 2016).

To address this complexity pragmatically, we recommend analysts first list all relevant outcomes. This “impact inventory” should include all outcomes posited in the intervention’s theory of change (Neumann et al., 2016, p. 101), including impacts that will materialize at a later stage, beyond the immediate effects of the intervention, which may require modeling. Some of these impacts will be captured in the secondary endpoints in trials and evaluations, and will generally be included in the intervention’s theory of change. Testing of multiple outcomes in economic evaluation is handled within a Bayesian framework (Briggs, 1999), which naturally accounts for simultaneous tests (Scott & Berger, 2006). Where there are sectoral decision makers, outcomes should then be assigned to the relevant sector of the economy more likely to benefit from them (Jones, 2015; Remme, Martinez-Alvarez, & Vassall, 2017) or a cost–benefit analysis can be conducted. A list of suggested outcomes, common to the prevention of VAWG, is shown in Table 1.

Table 1.

List of Recommended Outcomes for the Economic Evaluation of Violence Prevention.

Domain	Suggested Outcomes
Partner violence	Self-reported past year physical and/or sexual partner violence, experienced or perpetrated Self-reported past year emotional partner violence, experienced or perpetrated Self-reported past year economic partner violence, experienced or perpetrated Self-reported past year controlling behavior, experienced or perpetrated
Other violence	Violence from/to non-partner (both men and women) Child exposure to intimate partner violence Child abuse
Health	Anxiety/depression Post-traumatic stress Suicidality Hazardous alcohol and/or drug use Physical injuries Preterm and/or low birthweight offspring Miscarriages or abortions	Femicide HIV/STIs Sexual risk taking Nutrition DALYs QALYs Women’s participation in health-related household decisions
Gender norms and roles	Equitable gender attitudes Women’s participation in non-health-related household decisions
Economic and socio-psychological	Individuals’ employment outside the household Individuals’ income Individuals’ ability to retain control over their own income Individuals’ consumption Savings behavior and time preference Poverty Criminality due to lack of money	School performance Absence from school School dropout (including pregnancy-related) Well-being (life satisfaction and flourishing) Locus of control Self-esteem/Self-efficacy Agency
Institutions	Proportion of victims receiving formal or informal support Improved institutional response to cases of VAWG reported Equal opportunities in the workforce

Note. STIs = sexually transmitted infections; DALYs = disability-adjusted life years; QALYs = quality-adjusted life years; VAWG = violence against women and girls.

Measuring exposure to or perpetration of VAWG

At a minimum, outcomes should be reported in natural units, including cases of physical and/or sexual VAWG (experienced or perpetrated) averted. This allows for some comparability with preexisting studies. In addition, analysts should specify the type of VAWG the intervention is tackling: For example, if the intervention tackles IPV, they will specify “cases of IPV exposure (and/or perpetration) averted.” The time period of the outcome (see below) should also be included in the outcome measure, so that, “IPV cases averted in the past year,” is reported as, “IPV-free person years.” In general, past year is an appropriate time frame, because prevention interventions may not prevent further exposure to violence and measuring lifetime exposure would mean that prevention efforts would only be counted among women and girls who have never experienced violence. Furthermore, the use of “past year” aims to strike a balance between recall bias and rarity of the event (Garcia-Moreno, Jansen, Ellsberg, Heise, & Watts, 2006). Finally, a measure of frequency of exposure to violence over the past year permits the estimation of reductions in exposure, even in women who were not free of violence in the previous year, and some studies report costs per past year incident averted (Greco et al., 2018). However, this may be problematic in settings with high rates of violence, for which shorter time periods of reference may be preferred, though this will reduce comparability with studies that measure IPV exposure over the past year.

Health outcomes

Outcomes should be converted to general comparable metrics where feasible, for potential payers in different sectors. For example, if the health sector is likely to be a payer, then estimates of DALYs averted should be derived. Key health sequelae that could be considered will vary according to the age group and social context of the intervention beneficiaries, but mental health measures such as symptoms of post-traumatic stress disorder, anxiety, depression, suicidality, and substance abuse should be included (Devries et al., 2013), as well as injuries. In some populations, sexual and reproductive health outcomes such as preterm births and miscarriages, unwanted pregnancies averted, number of sexual partners, engagement in transactional sex or sex work, condom use, sexually transmitted infections, HIV, or HSV2 (herpes simplex virus type 2, which causes genital herpes) acquisition would be relevant. DALYs can be computed directly on health outcomes using the latest methodological recommendations (Lim et al., 2012; Murray et al., 2012; Salomon et al., 2015), or indirectly with DALY weights attributed to IPV exposure (Lim et al., 2012; Murray et al., 2012; Watts & Cairncross, 2012).

Non-health outcomes

Furthermore, non-health outcomes such as educational attainment among teenagers, participation in household decision-making, controlling behaviors, and social norms on gender relations and the acceptability of violence are associated with VAWG (Fulu & Miedema, 2015; Heise, 1998). They are typically included in theories of change around VAWG, and should therefore be included in the impact inventory, and reported in a cost-consequence analysis at a minimum (Neumann et al., 2016). The set of non-health outcomes will vary across interventions and contexts. For example, an intervention to prevent sexual assault among young teenagers, such as the IMpower intervention in Kenya, could measure exposure to sexual assault as its main outcome, as well as school attendance or drop-out rates, and school grades or children’s cognitive ability as secondary outcomes. In contrast, an intervention designed to prevent IPV in couples that are members of microfinance associations, such as the Indashyikirwa couples’ intervention in Rwanda, will have IPV cases averted as a main outcome and may have participation in household decision-making, social attitudes toward IPV and gender relations, and socially harmful behaviors, such as harmful alcohol and drug consumption, as secondary outcomes.

Outcomes should be converted to general measures where feasible, for potential payers in different sectors. For example, a general measure of education, such as attainment on standard tests or of high school diploma, for the education sector, or net income for the labor sector, or DALYs/QALYs for the health sector.

Recommendations on outcomes reporting

Listing outcomes may help sectoral payers (i.e., policy makers who operate in specific sectors and may commission, and pay for, the intervention to be scaled up) but does not account for the relative value of those outcomes or interaction between outcomes. It also may not fully inform payers interested in overall economic welfare. Analysts may therefore wish to consider composite outcomes, including benefit–cost ratios (equity or non-equity adjusted) and/or using measures of economic well-being. There are several approaches for this, but there has only been limited application of these measures to VAWG prevention to date. One “first step” option is that outcomes could be used to construct “empowerment” indices (Greco, Lorgelly, & Yamabhai, 2016). Empowerment indices require extensive preparation to ensure that domains and weights used in computing the index are locally determined and relevant, while also capturing underlying dimensions of subjective/psychological well-being (Ferrari, 2016).

Another simple approach, which is currently being applied to VAWG evaluation in What Works?, is to collect measures of global well-being to proxy participants’ utility. Measures of flourishing (Diener, 2009) capture a psychological construct of empowerment likely to be affected by complex interventions (Ferrari,2016). Measures of life and health satisfaction are also widely used. These measures partly overlap with health-related quality-of-life measures (Brazier, Ratcliffe, Saloman, & Tsuchiya, 2017; Mukuria & Brazier, 2013). One option being investigated by several research groups is to use these measures to inform a benefit–cost analysis by determining the necessary monetary compensation to keep an individual at the level of well-being corresponding to their permanent income, given their health condition (Jones, 2015; Neumann et al., 2016; Powdthavee & Van Den Berg, 2011; Sunstein, 2016). Such estimates could also be complemented by modeling of costs and benefits to provide more comprehensive estimates of the benefits and costs of these interventions to society.

Perspective

In some circumstances, VAWG prevention interventions require considerable community resourcing. Therefore, a provider perspective, including the costs to the grassroots nongovernmental organization (NGO), the local public payer if public services are used by study participants, and any donor or volunteer contributions, may be appropriate in most cases. Where clients or target populations incur costs to access services, we also recommend a societal perspective (Sanders et al., 2016; Vassall et al., 2017). A societal perspective includes clients’ and households’ costs, including lost productivity from time spent off work due to illness. In some cases, a societal perspective will also include sectors which incur external costs such as police and justice system costs.

Time Horizon

In line with iDSI Principle 6, the time horizon for the evaluation should capture all costs and effects. A lifetime horizon is recommended. Only where shorter time frame is demonstrably adequate, can there be an exception. Two possible reasons would justify a shorter time frame. First, program benefits are only experienced during the intervention. However, VAWG prevention aims for sustainable changes in behavior. Therefore, analysts must use available evidence to inform decisions about the likely persistence of this longer term effect. Consideration should also be given to the expected duration of health effects to determine total DALYs. We recommend a minimum scenario with no effect duration beyond the intervention, and a scenario where duration of effect goes beyond the intervention period, supported by an adequate rationale. In such cases, decision makers should be provided with at least a summary of the evidence on longer term impact of VAWG prevention (Lim et al., 2012; WHO, 2013), even if not directly estimated in their setting (Jones, 2015).

Resource Use and Costing

VAWG interventions are varied and evolving, and policy makers will need to adapt estimates to different settings or as interventions are implemented. Detailed reporting of inputs, allocation criteria, and prices will help users of costs to adjust costs with changing prices and service delivery models and identify scope for improving efficiency with scale-up.

Type of Unit Cost

An economic cost² is required for the economic evaluation, but financial costs, that is, costs that are effectively incurred by an organization, such as expenses or salaries, can provide useful information for decision makers assessing budget impact. In many cases, the decision maker is trying to establish whether to implement VAWG prevention (rather than invest in research) so analysts should estimate “real-world” costs that reflect the reality of implementation. Such costs can include incremental costs of removing constraints on implementation, for example, constraints to recruitment rates imposed by staff numbers (Jan et al., 2011) and staff turnover (Ferrari, Feder, Bailey, Peters, & Hollinghurst, in press).

Scope of Inputs

Many VAWG interventions add to existing services, so analysts will need to determine which costs are incremental. It may be necessary to consider economies of scope: Analysts should empirically estimate the cost of the platform (e.g., microfinance) service provision in both the intervention and the comparator, and the incremental cost obtained from their difference. If this is not feasible, incremental intervention costs should be measured using a transparent allocation procedure for joint costs; determined by the measurement of actual resource use (e.g., human resources should be allocated according to time spent).

A challenge for analysts involved in costing VAWG interventions is likely to be how to cost interventions that employ spare capacity in existing platforms. Even though this use of spare capacity may have no immediate opportunity cost, over time there may be one. For example, in a school-based intervention, VAWG programming may use teacher time. The time spent on the intervention by teachers with spare capacity may have neither immediate nor short-term economic costs. However, in the longer run, there may be an opportunity cost, in that the teacher’s time could be used to support students in other ways. Moreover, as a program is scaled up, spare capacity may not be present (Meyer-Rath et al., 2012). This information is likely to be of use to decision makers (Jones, 2015). We recommend including the cost of such spare capacity in a sensitivity analysis and discussing the extent to which excess capacity was drawn upon in the intervention. For example, in their economic evaluation of the Good School Toolkit in Uganda, Greco et al. (2018) provide evidence that teaching the Toolkit does not displace teachers’ time, and report unit costs that may be used for further analysis.

Methods for Estimating Resource Use

As VAWG prevention is in its early stages and interventions are not standardized, we recommend a micro-costing methodology (i.e., measuring the inputs used to produce a given output) to provide the level of detail requested by GHCC Principle 7 (Table A2). We recommend micro-costing even where some standardized or reference costs (average costs of services computed according to standard methodologies such as in the United Kingdom; Curtis & Burns, 2017) may be available. We recommend distinguishing between bottom-up (where all resources are observed and added up) and top-down estimates (where aggregate costs are used and allocated to different activities and interventions). To aid the costing of VAWG prevention and women’s empowerment interventions, we have developed an Excel™ costing tool and data collection instruments for public use (see https://bit.ly/2vKaGZZ). The tool must be adapted for each intervention and setting but can serve as a starting point to identify all inputs, allocation methods, and methods for calculating unit costs.

Units for the Calculation of Unit Costs

We recommend computing costs for the units listed in Table 2, developed using the GHCC framework. Providing estimates of costs of these standardized units allows for comparability across different interventions and sites.

Table 2.

Standardized Unit Costs for VAWG Interventions.

Mapping Standardized Unit Costs—VAWG Interventions
Unit costs	Quantities	Workshop-Based	Economic Empowerment	School-Based	Community-Based
Above intervention level (general project and overhead costs)
Cost of trainers’ or counselors’ recruitment	Q(trainers/counselors)	√	√	√	√
Cost of training per site^a (includes training of local interventions, such as police, or health clinics)	Q(sites)	√	√	√	√
Cost of manual/curriculum development or adaptation	1	√	√	√	√
Cost of management to support the above and for general organizational needs (e.g., general fundraising for the organization)	Q(sites)	√	√	√	√
Direct intervention
Cost of managing direct intervention support per site	Q(sites)	√	√	√	√
Cost per person session (any from one-to-one, to small group, to community gatherings and activities)	Q(number of person sessions)	√	√	√	√
Cost per screening/initial assessment	Q(screenings/initial assessments)	√
Cost of recruitment	Q(persons recruited)	√	√	√
Cost per treatment completed	Q(completed treatments)	√	√	√	√
Cost per minimum cycle (i.e., minimum number of sessions)	Q(minimum cycles completed)	√	√	√	√
Cost of transport per treatment completed	Q(number of personnel trips for sessions delivery)	√	√	√	√
Cost of community entry	Q(community recruited)	√	√	√	√
Ancillary interventions
Cost of supervision per person/community treated^b	Q(persons/communities treated)	√	√	√	√
Cost of supervision per counselor/facilitator trained	Q(counselors/facilitators)	√	√	√	√
Cost of managing intervention support costs per site	Q(sites)	√	√	√	√

Note. VAWG = violence against women and girls.

Sites are different from communities. There could be more than one site per community.

The cost refers to the total supervision sessions required to support facilitators/counselors to enable them to complete one treatment cycle with one person (a decision will need to be made if this is done for minimum cycles or only for completed treatment).

This framework identifies above-site, direct, and ancillary unit costs of service delivery, and aggregates them as illustrated in Figure 2. Direct service units should be further defined in each setting to capture the outputs that characterize most interventions: teaching delivered to classes, counseling sessions, radio episodes, or community-based activities for the delivery phase.

Figure 2.

Intervention unit costs (UCs) and their components.

Sampling

VAWG interventions are delivered across a range of settings. In some cases, it may be useful to generate nationally representative cost estimates. In others, a focus on specific populations, settings, or interventions may be more appropriate (Evans & Popova, 2016). Sampling design should reflect these different purposes (GHCC Principle 8; Table A2). Costing may require a two-stage sampling of communities and within them specific sites or groups (such as the sampling of community health workers [CHWs]). The sampling frame should contain information on the key cost drivers of VAWG service delivery, such as amount of services delivered per site. If the program is large enough (n > 6 sites per stratum and N > 5 strata), analysts could draw a random sample of delivery sites stratified by the relevant cost driver(s); otherwise, they should sample sites purposively.

Shadow Prices and Valuation

GHCC Principle 14 states that all shadow prices, that is, “the true social opportunity cost[s]” of inputs (McIntosh, 2010, p. 40), be labeled as such and reported alongside the source. The successful piloting and sustainability of scale-up for some VAWG prevention interventions will depend on volunteer labor (Kasteng, Settumba, Källander, Vassall, & inSCALE Study Group, 2016), whose time should be valued appropriately (Jan, Pronyk, & Kim, 2008; Kasteng et al., 2016; Michaels-Igbokwe et al., 2016). Once all tasks in a volunteer’s working day have been identified (e.g., life skills session; administrative report writing, supervision of volunteers), we recommend using replacement price, pricing volunteers’ time at the cost of their closest substitute (e.g., CHW, administrator, CHW supervisor). However, in some cases, replacement prices do not fully represent the cost of foregoing the volunteer’s time on the next best alternative. If programs are scaled up, the increasing demand on community time will mean that opportunity costs may rise, as more productive members of the community are involved.

Uncertainty

We address methodological uncertainty by providing these standardized guidelines (Briggs et al., 2012). We also recommend exploring structural uncertainty, as a minimum. This derives from model assumptions, such as discount factor and conversion rates (Dhaliwal, Duflo, Glennerster, & Tulloch, 2013; Wordsworth & Ludbrook, 2005). We also recommend that, where feasible, uncertainty around economies of scale and scope be examined, particularly when using research setting costs to inform investments in VAWG scale-up (Evans & Popova, 2016). Where empirical analysis is unfeasible, these investigations will include scenario analysis based on the estimation of a cost function to obtain the marginal cost for a given scale or scope of an intervention (Neumann et al., 2016). We recommend presenting a range of scenario analyses that consider both different levels of spare capacity and changes in costs due to different levels of fixed costs across locations as a program expands. If a program is anticipated to have high levels of coverage, then diseconomies of scale, that is, increasing unit costs compared with a situation with lower coverage, should also be explored (Briggs et al., 2012). Table A3 in the appendix reports the sensitivity analyses we recommend.

Conclusion

Donors are increasing resources targeting the prevention of VAWG in LMICs, both by implementing prevention-focused programs and adding VAWG prevention to other interventions. In the face of this increased investment, tools for evidence-based resource allocation should be upgraded.

This article lays out a standardized methodology for the economic evaluation of VAWG prevention interventions, which is also more widely applicable to complex empowerment and behavioral interventions. Such interventions are complex, with multiple components and administered across many service platforms and settings. This poses challenges to the estimation of comparable costs and means that guidance may change over time. We outline criteria for the identification of unit costs that capture core activities likely to be common across interventions. We recommend unit costs for reporting (see Table 2), as well as the measurement and valuation of resource use to improve the comparability of intervention costs across settings and provide tools to support data collection.

In addition, our standard methodology accounts for the diversity of prevention programming recipients. We identify outcomes relevant to women, men, children, and adolescents. The acknowledgment of these distinct groups of recipients, and the use of subgroup-specific outcomes, alongside common outcomes, will yield insights into any heterogeneity in programming cost-effectiveness. This will contribute valuable information for decision-making around the financing of VAWG prevention services that take intersectionality into account.

We do not provide guidance on how to incorporate the availability of health or social-support systems that provide the context for the implementation. Collecting these data would provide information on the impact the intervention may have on downstream services (e.g., a VAWG referral system beyond the intervention being studied, or reduced use of health services due to reduced exposure to VAWG) and help analysts quantify the wider costs or cost savings that the intervention is likely to determine for other services. We think this would be a fruitful area of investigation for the future and encourage efforts in this direction. Many VAWG prevention interventions rely on lay frontline workers. This is especially the case for community-based mobilization interventions. In such cases, little is known about what incentives these workers may require keeping delivering the intervention beyond the lifetime of the initial phase, thereby guaranteeing the sustainability of the intervention. Understanding their motivation for this type of work, and the incentive structure they respond to, is key to guaranteeing the sustainability of community-based interventions beyond VAWG prevention and beyond the health sector. It needs to be better understood to estimate the costs of these interventions. Finally, there remain challenges in identifying standard outcomes that may be compared across interventions, within the health sector and beyond, including as lagged secondary effects, and understanding the duration of the effect.

Our recommendations include the estimation of standardized measures of VAWG and of a synthetic measure of health impact in the DALY. These measures are not comprehensive and other measures can also be used, but as standard minimum data set, they can increase comparability with other VAWG and public health interventions and yield transparent modeling results. We recommend the identification of an “impact inventory,” whose components may be summarized in a multidimensional index, as a measure of impact and of synthetic measures of well-being to capture intervention impact on individuals’ utility. Future research should explore the links between health and non-health dimensions (economic and social outcomes) and violence, to produce estimates for use in the modeling of prevention cost-effectiveness and benefit. Further work is also needed to map and measure all relevant VAWG health sequelae in longitudinal data sets to capture the full long-term health impact of VAWG and, consequently, the health benefits of prevention.

Analysts and practitioners may encounter some obstacles in the implementation of this methodology. First, it is possible that the data are not available at the level of detail required for a micro-costing or impact measurement in line with our recommendations. In such cases, for costing, analysts should adopt a top-down gross costing approach and include the best available outcomes for impact measuring, acknowledging in both instances that they are deviating from the preferred recommendation.

Improved costing and impact measurement to account for the data needs of economic evaluations will be a critical part of the effort to scale-up and mainstream VAWG prevention. We hope that these methods provide a sound and comparable starting point to this effort.

Footnotes

Appendix

Author Contributions

G.F. drafted the manuscript and devised and adapted the methodology under A.V.’s supervision; A.V. edited the first draft of the manuscript; S.T.-R. provided feedback on the methodology; and S.T.-R., C.W., R.J., and C.M.-I. provided feedback on drafts of the paper. All authors reviewed and approved the final version of the manuscript.

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: C.W. is the Chief Scientific Adviser at the Department for International Development (UKAid). This work was conducted as part of her academic role as professor in epidemiology at the London School of Hygiene & Tropical Medicine.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: G.F., S.T.-R., R.J., and A.V. received U.K. aid funding from the U.K. government, via the What Works to Prevent Violence Against Women and Girls Global Program. The views expressed do not necessarily reflect the government’s official policies. The funds were managed by the South African Medical Research Council.

Ethics and Dissemination

Ethical approval for the implementation of this methodology at five sites on the African continent was obtained from the London School of Hygiene and Political Science (12204), and as an amendment to existing ethics approvals at the following institutions: Johns Hopkins University (00006534), Kenya Medical Research Institute (KEMRI; KEMRI/RES/7/3/1), South African Medical Research Council (EC006-2/2015 [South Africa], EC031-9/2015 [Ghana], EC033-10/2015 [Rwanda]), Stanford University (4947), the Rwanda National Ethics Committee (RNEC; 40/RNEC/2015), the University of Ghana (006/15-16), the University of KwaZulu-Natal (BFC043/15), and the University of Zambia Biomedical Research Ethics Committee (UNZAREC; 004-11-15).

ORCID iD

Giulia Ferrari

Notes

Author Biographies

Giulia Ferrari, PhD, is a research fellow in economics at the London School of Hygiene & Tropical Medicine. She works on outcome measurement in economic evaluation, including well-being measures, and uses applied econometrics to investigate the role of economic and psychological factors in mediating the impact of prevention programming on perpetration of and exposure to violence against women and girls (VAWG).

Sergio Torres-Rueda, MSc, is a research fellow at the London School of Hygiene & Tropical Medicine. He specializes in the economic evaluation of health-related interventions, particularly in the areas of HIV and VAWG. He focuses on cost-effectiveness analyses alongside randomized controlled trials in low- and middle-income countries (LMICs).

Christine Michaels-Igbokwe, PhD, is a postdoctoral fellow at the University of Calgary. Her research interests include economic evaluation and stated preferences for health care services and policy interventions. Her work includes VAWG prevention and, more recently, on elicitation of individual preferences and economic evaluations of interventions for child and adolescent health in Canada.

Charlotte Watts, PhD FMedSci, is chief scientific adviser at the Department for International Development seconded from the London School of Hygiene & Tropical Medicine, where she is professor of social and mathematical epidemiology. She has advised the World Health Organization (WHO); led a systematic review of the global health burden of violence, and trials showing that violence is preventable; and published more than 200 academic articles.

Rachel Jewkes, MD, is the executive scientist for research strategy in the Office of the South African Medical Research Council (SAMRC) President, and an Honorary Professor at the University of the Witwatersrand School of Public Health, South Africa. She has led research on the intersection of gender inequity, gender-based violence, and health, publishing 300+ academic articles.

Anna Vassall, PhD, is professor of health economics at the London School of Hygiene and Tropical Medicine. She specializes in the economics of HIV, sexual and reproductive health, VAWG prevention, and tuberculosis in LMICs. She is leading the first large multicountry study on the cost-effectiveness of VAWG prevention programming.

References

Abramsky

Devries

Kiss

Nakuti

Kyegombe

Starmann

. . . Musuya

(2014). Findings from the SASA! Study: A cluster randomized controlled trial to assess the impact of a community mobilization intervention to prevent violence against women and reduce HIV risk in Kampala, Uganda. BMC Medicine, 12, Article 122.

Brazier

Ratcliffe

Saloman

Tsuchiya

(2017). Measuring and valuing health benefits for economic evaluation. Oxford, UK: Oxford University Press.

Briggs

A. H.

(1999). A Bayesian approach to stochastic cost-effectiveness analysis. Health Economics, 8, 257-261.

Briggs

A. H.

Weinstein

M. C.

Fenwick

E. A.

Karnon

Sculpher

M. J.

Paltiel

A. D.

Force

I.-S. M. G. R. P. T

. (2012). Model parameter estimation and uncertainty: A report of the ISPOR-SMDM modeling good research practices task force–6. Value Health, 15, 835-842. doi:10.1016/j.jval.2012.04.014

Brouwer

W. B. F.

Culyer

A. J.

van Exel

N. J. A.

Rutten

F. F. H.

(2008). Welfarism vs. extra-welfarism. Journal of Health Economics, 27, 325-338. doi:10.1016/j.jhealeco.2007.07.003

Curtis

Burns

(2017). Unit costs of health and social care 2017. Retrieved from https://www.pssru.ac.uk/project-pages/unit-costs/unit-costs-2017/

Department for International Development. (2011). DfID’s approach to Value for Money (VfM). Retrieved from https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/49551/DFID-approach-value-money.pdf

Devries

K. M.

Mak

J. Y.

García-Moreno

Petzold

Child

J. C.

Falder

. . . Watts

C. H.

(2013). Global health: The global prevalence of intimate partner violence against women. Science, 340, 1527-1528. doi:10.1126/science.1240937

Dhaliwal

Duflo

Glennerster

Tulloch

(2013). Comparative cost-effectiveness analysis to inform policy in developing countries: A general framework with applications for education. In Glewwe

(Ed.), Education policy in developing countries (pp. 285-338). Chicago, IL: The University of Chicago Press.

10.

Diener

(Ed.). (2009). Assessing well-being: The collected works of Ed Diener (Kindle ed., Vol. 39). New York, NY: Springer.

11.

Drummond

Sculpher

M. J.

Claxton

Stoddart

G. L.

Torrance

G. W.

Methods for the Economic Evaluation of Health Care Programmes Oxford Medical Publications: Oxford University Press Duflo, E. (2003). Grandmothers and granddaughters: Old-age pensions and intrahousehold allocation in South Africa. The World Bank Economic Review, 17, 1-25.

12.

Evans

D. K.

Popova

(2016). Cost-effectiveness analysis in development: Accounting for local costs and noisy impacts. World Development, 77, 262-276. doi:10.1016/j.worlddev.2015.08.020

13.

Ferrari

(2016). Economic evaluation of gender empowerment programmes with a violence prevention focus: objective empowerment and subjective wellbeing. PhD thesis, mimeo The London School of Economics and Political Science (LSE)..

14.

Ferrari

Agnew-Davies

Bailey

Howard

Howarth

Peters

T. J.

. . . Feder

G. S.

(2016). Domestic violence and mental health: A cross-sectional survey of women seeking help from domestic violence support services. Glob Health Action, 9, 29890. doi:10.3402/gha.v9.29890

15.

Ferrari

Feder

Bailey

J. E.

Peters

T. J.

Hollinghurst

(2018). Cost-effectiveness of a psychological intervention for survivors of domestic violence and abuse survivors: Economic evaluation of the Psychological Advocacy Towards Healing (PATH) Trial.

16.

Fulu

Miedema

(2015). Violence against women: Globalizing the integrated ecological model. Violence Against Women, 21, 1431-1455. doi:10.1177/1077801215596244

17.

Garcia-Moreno

Jansen

H. A.

Ellsberg

Heise

Watts

C. H.

(2006). Prevalence of intimate partner violence: Findings from the WHO multi-country study on women’s health and domestic violence. The Lancet, 368, 1260-1269.

18.

Gibbs

Washington

Willan

Ntini

Khumalo

Mbatha

. . . Strauss

(2017). The stepping stones and creating futures intervention to prevent intimate partner violence and HIV-risk behaviours in Durban, South Africa: Study protocol for a cluster randomized control trial, and baseline characteristics. BMC Public Health, 17, Article 336.

19.

Global Gender-Based Violence Task Force. (2017). Working together to prevent sexual exploitation and abuse: Recommendations for World Bank investment projects. Washington, DC: Author.

20.

Greco

Knight

Ssekadde

Namy

Naker

Devries

(2018). Economic evaluation of the Good School Toolkit: An intervention for reducing violence in primary schools in Uganda. BMJ Global Health, 3(2), e000526. doi:10.1136/bmjgh-2017-000526

21.

Greco

Lorgelly

Yamabhai

(2016). Outcomes in economic evaluations of public health interventions in low- and middle-income countries: Health, capabilities and subjective wellbeing. Health Economics, 25, 83-94.

22.

Gupta

Falb

K. L.

Lehmann

Kpebo

Xuan

Hossain

. . . Annan

(2013). Gender norms and economic empowerment intervention to reduce intimate partner violence against women in rural Côte d’Ivoire: A randomized controlled research setting study. BMC International Health and Human Rights, 13, Article 46.

23.

Hardt

Bernert

Matschinger

Angermeier

M. C.

Vilagut

Bruffaerts

. . . Alonso

(2015). Suicidality and its relationship with depression, alcohol disorders and childhood experiences of violence: Results from the ESEMeD study. Journal of Affective Disorders, 175(Suppl. C), 168-174. doi:10.1016/j.jad.2014.12.044

24.

Heise

L. L.

(1998). Violence against women: An integrated, ecological framework. Violence Against Women, 4, 262-290. doi:10.1177/1077801298004003002

25.

Husereau

Drummond

Petrou

Carswell

Moher

Greenberg

. . . Loder

(2013). Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. Cost Effectiveness and Resource Allocation, 11, 6.

26.

Iyengar

Ferrari

(2011). Comparing economic and social interventions to reduce intimate partner violence: Evidence from central and Southern Africa. Retrieved from https://www.nber.org/papers/w16902

27.

Jan

Ferrari

Watts

C. H.

Hargreaves

J. R.

Kim

J. C.

Phetla

. . . Pronyk

P. M.

(2011). Economic evaluation of a combined microfinance and gender training intervention for the prevention of intimate partner violence in rural South Africa. Health Policy and Planning, 26, 366-372.

28.

Jan

Pronyk

Kim

(2008). Accounting for institutional change in health economic evaluation: A program to tackle HIV/AIDS and gender violence in Southern Africa. Social Science & Medicine, 66, 922-932. doi:10.1016/j.socscimed.2007.11.010

29.

Jones

(2015). Integration of iDSI’s Reference Case principles for economic evaluation and DFID’s approach to value for money analysis. Opportunities and challenges. Retrieved from https://f1000research.com/documents/6-239

30.

Kasteng

Settumba

Källander

Vassall

, & inSCALE Study Group. (2016). Valuing the work of unpaid community health workers and exploring the incentives to volunteering in rural Africa. Health Policy and Planning, 31, 205-216.

31.

Lim

S. S.

Vos

Flaxman

A. D.

Danaei

Shibuya

Adair-Rohani

Memish

Z. A.

(2012). A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990-2010: A systematic analysis for the Global Burden of Disease Study 2010. Lancet, 380, 2224-2260. doi:10.1016/S0140-6736(12)61766-8

32.

McIntosh

(2010). Applied methods of cost-benefit analysis in health care. New York, NY: Oxford University Press.

33.

Meyer-Rath

Schnippel

Long

MacLeod

Sanne

Stevens

. . . Rosen

(2012). The impact and cost of scaling up GeneXpert MTB/RIF in South Africa. PLoS ONE, 7(5), e36966. doi:10.1371/journal.pone.0036966

34.

Michaels-Igbokwe

Abramsky

Devries

Michau

Musuya

Watts

(2016). Cost and cost-effectiveness analysis of a community mobilisation intervention to reduce intimate partner violence in Kampala, Uganda. BMC Public Health, 16, Article 196.

35.

Mukuria

Brazier

(2013). Valuing the EQ-5D and the SF-6D health states using subjective well-being: A secondary analysis of patient data. Social Science & Medicine, 77(Suppl. C), 97-105. doi:10.1016/j.socscimed.2012.11.012

36.

Murray

C. J.

Vos

Lozano

Naghavi

Flaxman

A. D.

Michaud

. . . Memish

Z. A.

(2012). Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990-2010: A systematic analysis for the Global Burden of Disease Study 2010. Lancet, 380, 2197-2223. doi:10.1016/S0140-6736(12)61689-4

37.

Neumann

P. J.

Sanders

G. D.

Russell

L. B.

Siegel

J. E.

Ganiats

T. G.

(2016). Cost-effectiveness in health and medicine. Oxford, UK: Oxford University Press.

38.

Powdthavee

Van Den Berg

(2011). Putting different price tags on the same health condition: Re-evaluating the well-being valuation approach. Journal of Health Economics, 30, 1032-1043.

39.

Pronyk

P. M.

Hargreaves

J. R.

Kim

J. C.

Morison

L. A.

Phetla

Watts

. . . Porter

J. D. H

. (2006). Effect of a structural intervention for the prevention of intimate-partner violence and HIV in rural South Africa: A cluster randomised trial. The Lancet, 368, 1973-1983. doi:10.1016/S0140-6736(06)69744-4

40.

Pronyk

P. M.

Kim

Makhubele

Hargreaves

Mohlala

Hausler

(2002). Introduction of voluntary counselling and rapid testing for HIV in rural South Africa: From theory to practice. AIDS Care, 14, 859-895.

41.

Remme

Martinez-Alvarez

Vassall

(2017). Cost-effectiveness thresholds in global health: Taking a multisectoral perspective. Value in Health, 20, 699-704.

42.

Richardson

W. S.

Wilson

M. C.

Nishikawa

Hayward

R. S.

(1995). The well-built clinical question: A key to evidence-based decisions. ACP Journal Club, 123, A12.

43.

Salomon

J. A.

Haagsma

J. A.

Davis

de Noordhout

C. M.

Polinder

Havelaar

A. H.

. . . Vos

. (2015). Disability weights for the Global Burden of Disease 2013 study. The Lancet Global Health, 3(11), e712-723. doi:10.1016/S2214-109X(15)00069-8

44.

Sanders

G. D.

Neumann

P. J.

Basu

Brock

D. W.

Feeny

Krahn

. . . Prosser

L. A.

(2016). Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: Second panel on cost-effectiveness in health and medicine. Journal of the American Medical Association, 316, 1093-1103.

45.

Scott

J. G.

Berger

J. O.

(2006). An exploration of aspects of Bayesian multiple testing. Journal of Statistical Planning and Inference, 136, 2144-2162.

46.

Sculpher

M. J.

Claxton

Drummond

McCabe

(2006). Whither trial-based economic evaluation for health care decision making? Health Economics, 15, 677-687. doi:10.1002/hec.1093

47.

Smith-Marek

E. N.

Cafferky

Dharnidharka

Mallory

A. B.

Dominguez

High

. . . Mendez

(2015). Effects of childhood experiences of family violence on adult partner violence: A meta-analytic review. Journal of Family Theory & Review, 7, 498-519.

48.

Stern

Heise

McLean

(2018). The doing and undoing of male household decision-making and economic authority in Rwanda and its implications for gender transformative programming. Culture, Health & Sexuality, 20, 979-991.

49.

Stern

Niyibizi

L. L.

(2018). Shifting perceptions of consequences of IPV among beneficiaries of Indashyikirwa: An IPV prevention program in Rwanda. Journal of Interpersonal Violence, 33, 1778-1804.

50.

Sunstein

C. R.

(2016). Cost-benefit analysis, who’s your daddy? Journal of Benefit-Cost Analysis, 7, 107-120.

51.

Vassall

Sweeney

Kahn

J. G.

Gomez

Bollinger

Marseille

. . . Levin

. (2017). Retrieved from https://ghcosting.org/pages/standards/reference_case.

52.

Watts

Cairncross

(2012). Should the GBD risk factor rankings be used to guide policy? The Lancet, 380, 2060-2061. doi:10.1016/S0140-6736(12)62121-7

53.

Wilkinson

Sculpher

M. J.

Claxton

Revill

Briggs

Cairns

J. A.

. . . Walker

D. G

. (2016). The International Decision Support Initiative reference case for economic evaluation: An aid to thought. Value in Health, 19, 921-928. doi:10.1016/j.jval.2016.04.015

54.

Wordsworth

Ludbrook

(2005). Comparing costing results in across country economic evaluations: The use of technology specific purchasing power parities. Health Economics, 14, 93-99.

55.

World Health Organization. (2010). Preventing intimate partner and sexual violence against women: Taking action and generating evidence. Geneva, Switzerland: Author.

56.

World Health Organization. (2013). Global and regional estimates of violence against women: Prevalence and health effects of intimate partner violence and non-partner sexual violence. Geneva, Switzerland: Author.