Intracluster Correlation Coefficients of Household Economic and Agricultural Outcomes in Mozambique

Abstract

Background:

This article offers important statistics to evaluators planning future evaluations in southeast Africa. There are little to no published statistics describing the variance of southeast African agricultural and household indicators.

Objective:

We seek to publish the standard deviations, intracluster correlation coefficients (ICCs), and R ²s from outcomes and covariates used in a 2014 quasi-experimental evaluation of the Millennium Challenge Corporation’s Mozambique Farmer Income Support Project (FISP) and thus guide researchers in their calculation of design effects relevant to future evaluations in the region.

Method:

We summarize data from a roughly 168-item farmer survey conducted in 1,227 households during June–July 2014 in coconut-farming regions of the Zambezia province in Mozambique. We report descriptive statistics, estimates of ICC, and R ²s obtained from linear regression models with cluster random effects. We consider three different cluster definitions.

Results:

We report ICCs for a range of different specifications. For the FISP evaluation, the average design effect for education outcomes is 1.16. Average design effects for wealth measures based on consumption are 1.23. For agricultural-related outcomes, 1.05 is the average design effects for income measures, 1.47 for knowledge, and 1.64 for sales of specific crops.

Conclusion:

We offer a detailed picture of the variance structure of agricultural and other outcomes in Mozambique. Our results indicate that the design effect associated with these outcomes is less than the rule-of-thumb design effect (2.0) used in nutrition studies which are commonly cited in the studies of this region.

Keywords

Mozambique power analysis evaluation intracluster correlation coefficient (ICC)design effect standard deviation survey research agriculture Millennium Challenge Corporation Africa household survey

While designing the impact evaluation of the Millennium Challenge Corporation’s Farmer Income Support Project (FISP) in Mozambique, we found almost no guidance or reliable economic data to help us plan the necessary sample sizes for data collection, except that a rule-of-thumb design effect in the nutrition literature is 2.0 (Magnini, 1997).¹ Sample size selection is a critical step in an impact evaluation seeking to attribute causal impacts to a particular program or intervention. Usually, observations are not independent because they are driven by common, or shared, characteristics. In the evaluation of FISP in Mozambique, we assumed that household-level outcomes of interest such as income and agricultural practice would be correlated with the outcomes of nearby households’ outcomes, due to the fact that “nearby” households’ outcomes are similarly affected by factors such as weather, soil conditions, prices in the local agricultural market, cultural similarities that affect agricultural practices, and similarity of agricultural pest and disease burden. As a step toward improving resources to future researchers on sample size calculations for agricultural, wealth, and education impact evaluations in Mozambique and other southeastern African countries, we report key statistics from a survey of 1,227 rural households conducted in August, 2014, in the coconut belt of Mozambique.²

Agriculture employs 80% of the workforce in Mozambique (U.S. Agency for International Development, 2015); in 2011, coconut oil was one of the top 10 exports along with tobacco, sugar products, cashew nuts, cotton lint, sesame seeds, wheat, and bananas (Food and Agriculture Organization Corporate Statistical Database, 2011). The two provinces where FISP was implemented are the two most populous, representing a combined 38% of the country’s population. In all of Mozambique, 69% of the population lives in rural areas, whereas 100% of our sample was drawn from households in rural areas.³ The heads of household that we interviewed had similar literacy rates compared to the national population (50% vs. 51%, respectfully), although they had lower average years of school (3.62 years vs. 5 years).⁴ We believe the intracluster correlation coefficients (ICCs) reported in this article can provide researchers with valuable information when working in Mozambique, especially in agricultural settings. To the extent that correlated outcomes are driven by social structures and these social structures are similar in other rural areas of southeast African countries, our data may provide further guidance on planning sample sizes for evaluations in these other areas.

The next section of this article describes the program and data that we evaluated. Third section presents our methodology. Fourth section describes our results and fifth section concludes.

Data

FISP was a US$20.8 million project implemented by the Millennium Challenge Authority Mozambique that intended to “improve the productivity of coconut products and encourage diversification into other cash crop production” within project zones in the provinces of Zambézia and Nampula Provinces, specifically in the Inhassunge, Chinde, Nicoadala, Namacurra, and Maganja da Costa districts (see Figure 1). The household survey was administered to a sample of rural households in these five districts that were currently producing coconuts or had done so in the past 5 years. The survey was fielded by a local survey firm, Polaris, Ltd., that used paper-and-pencil interviewing to collect the data. All the data were entered into electronic format twice (independently), and all differences were reconciled using the hard copies of the surveys. The survey instrument consisted of roughly 168 questions, covering topics including demographic information, education, wealth, income levels and sources, and crop-specific input costs and sales revenue.⁵

Figure 1.

Map of Mozambique. Left: Provinces where the intervention took place are colored yellow, Nampula and Zambezia. Right: Specific districts where the intervention took place are colored yellow, Inhassunge, Chinde, Nicoadala, Namacurra and Maganja da Costa. Source. Narayan, Davis, and Geyer (2014).

The data collection instrument used during this survey was adapted from the Trabalho Inquerito Agricola (TIA; Michigan State University, 2008; Ministry of Agriculture of Mozambique, 2008). The TIA is the standard agricultural survey in Mozambique, and the current instrument has been used for several consecutive surveys (once every 2–3 years). We adapted our instrument from the TIA to ensure both the questions we used were already verified as effective from repeated use in the field and our measures would be comparable to data collected from previous surveys. We pretested the adapted instrument before going to the field in order to see if any items on the questionnaire were difficult to understand or answer; where challenges arose, the instrument was edited and improved. After fielding the survey, the survey firm reinterviewed 10% of sampled households to test whether the farmers, questions, and enumerators were providing consistent and reliable answers. From analyzing the reinterviews, we have no reason to believe the survey was implemented in a biased or otherwise incorrect manner.

The primary sampling unit (PSU) for the survey was an enumeration area (EA), which is defined and demarcated by the National Statistics Institute as an area of land with approximately 100 households. EAs are often, but not always, defined by road and river boundaries; their average size is 2.23 km² (standard deviation of 2.89 km²). The reader is referred to Narayan, Davis, and Geyer (2014) who describe the selection of EAs for inclusion in the sample; worth mentioning here, the selection of EAs was stratified by district in proportion to district population sizes. The selection of EAs was otherwise not random, rather it was based on both the 2008 prevalence of coconut lethal yellowing disease (CLYD) in area coconut palms and the implementation details of FISP. Within each selected EA, interviewers randomly selected eight farmers to interview. The sample size is 1,227 households, spanning 176 EAs. Table 1 provides additional detail about the sample size within each district. As we will discuss in the next section, there are several levels of social organization. The EAs were located across 44 villages and across 12 administrative posts.

Table 1.

Number of Clusters and Households Sampled in Each District.

	Number of EAs Sampled	Average Number of Households Sampled Per EA	Number of Villages sampled	Average Number of Households Sampled Per Village	Number of Administrative Posts Sampled	Average Number of Households Sampled Per Administrative Post	Number of Households Sampled
	(1)	(2)	(3)	(4)	(5)	(6)	(7)
Chinde	17	6.88	6	19.5	2	58.5	117
Namacurra	46	6.83	8	39.3	1	314	314
Nicoadala	42	6.71	12	23.5	3	94	282
Inhassunge	52	7.19	11	34.0	2	187	374
Maganja da Costa	20	7.00	7	20.0	3	46.7	140
Total	176	6.92	44	27.3	12	140	1,227

Note. EA = enumeration area.

Our survey asked many questions about the types of agricultural outcomes including crops grown, crop yield, crop sales, and knowledge about CLYD. Farmers were also asked to recall their farm activities from 5 years prior, and we used this information as “baseline” covariates in our study of agricultural outcomes.

Although our study focused on farm production, our survey was very broad and can thus reveal information about the data generation process for education and wealth outcomes as well. There are many interventions that focus on education or wealth generation instead of agriculture, and thus we provide descriptive statistics and ICC estimates for these measures as well as agricultural outcomes. We have data on education level, literacy, and nonfarm income. We also asked the farmers to recall their ownership of durable goods (mobile phones, bicycles, etc.—see Tables 1 and 2) and the construction material of their houses and barns from 5 years prior.

Table 2.

Means, Standard Deviations, Enumeration Area (EA) ICCs, With and Without Covariates.

Outcomes	Mean	Standard Deviation	ICC, No Covariates	ICC, With Covariates	ICC, With Covariates and L2 Aggregates	GEE ICC, With Covariates and L2 Aggregates
Outcomes	(1)	(2)	(3)	(4)	(5)	(6)
Education^a
Years of education	3.62	3.73	.060	.048	.044	NA
Can read and write	0.51	0.50	.044	.025	.021	.013
Completed some primary school	0.52	0.50	.012	.006	.001	.006
Wealth^b
House has improved walls	0.02	0.14	.145	.129	.118	.047
House has improved roof	0.14	0.34	.071	.054	.047	.030
Number of improved barns owned in 2009	0.13	0.33	.203	.210	.143	.099
Number of radios owned in 2009	0.40	0.49	.047	.034	.025	.018
Number of mobile phones owned in 2009	0.09	0.29	.042	.039	.033	.025
Number of motorbikes owned in 2009	0.02	0.12	.046	.050	.038	.013
Agricultural Outcomes^c
Annual income (MZN)	5,786	13,506	.028	.012	.003	NA
Annual nonfarm income (MZN)	5,189	13,183	.029	.015	.008	NA
Annual farm/agricultural income (MZN)	628	2,414	.030	.008	.000	NA
Whether farmer is currently economically “better off” than s/he was 5 years ago	0.27	0.45	.007	.001	.000	.004
Annual sales from coconut derivatives (MZN)	90.35	739.65	.132	.109	.085	NA
Annual sales of FISP- promoted crops (MZN)	5.03	24.85	.217	.217	.171	NA
Annual groundnuts sales	4.02	71.00	.029	.028	.015	NA
Annual nhemba bean sales	6.38	67.53	.076	.081	.062	NA
Annual coconut production (kg)	46.31	273.24	.046	.037	.023	NA
Number of coconut trees in production in the last 12 months	13.97	31.00	.201	.173	.134	NA
Score from income diversity metric^d	5.27	30.06	.070	.064	.050	NA
Percentage of surviving saplings from all intervention saplings planted	0.09	0.27	.093	.095	.065	NA
Number of non-FISP crops planted by the household	3.16	1.22	.243	.170	.061	N/A
Whether the farmer knows that cutting and burning is the best way to control CLYD	0.36	0.48	.120	.112	.098	.050
Whether farmer is able to correctly identify CLYD symptoms	0.66	0.48	.037	.025	.000	.015
Whether the household knows that the best CLYD mitigation is to cut trees and then burn stumps/roots	0.07	0.25	.125	.096	.071	.033
Whether the household knows that tree cutting is a good way to mitigate CLYD spread	0.91	0.29	.095	.085	.051	.039

Note. N = 1,227. ICC = intracluster correlation coefficient; MZN = Mozambican Metical; FISP = Farmer Income Support Project; CLYD = coconut lethal yellowing disease.

^aRegression includes household demographic data (head of household age, number of people in the household, and marital status of head of household) and basic cluster-level geographic data (average rainfall, distance to coast, and distance to program geographic boundary).

^bRegression includes household demographic data, two of the covariates listed under “education” (can read/write, and some primary), and basic cluster-level geographic data (average rainfall, distance to coast, and distance to program geographic boundary).

^cRegression includes household demographic data, two of the covariates listed under education (can read/write, and some primary), baseline farm characteristics (number of coconut trees in production in 2009; binary variables for whether or not the household produced coconut wood, copra, coconuts, coconut alcohol, or fabric in 2009; and estimated total farmland value in 2009 and in meticais), and basic cluster-level geographic data (average rainfall, distance to coast, and distance to program geographic boundary).

^dThe income diversity score is calculated following Ersado (2003). Sk = (Yk/Y) and D = $\sum_{k = 0}^{n} (1 / S_{k})$ , where Yk is the total income from source k (n = number of sources) and Y is the total household income. Using this technique, D will be the diversification “score,” and the higher the number, the more “diverse” the household income.

Method

Clustering

Our PSU was the EA. However, future interventions may randomize households or individuals at other levels of social organization. Therefore, we consider three possible clustering definitions: EAs, villages, and administrative posts. In the remainder of this subsection, we define each cluster type and describe why interventions may randomize at that level.

EAs, as discussed above, are defined by a central government entity, and like a U.S. census tract, it may have logical or visible boundaries, depending on the population density. The EAs are typically smaller than the smallest level of community organization—villages. However, clustering at this level may be a useful option because EAs are the lowest level at which the central government gathers data. Since the EAs are typically smaller than villages, we expect a high level of homogeneity at this level.

We also consider that clustering may occur at the village level. Villages are generally “larger” entities than EAs. Rural villages in Mozambique are not easily recognizable by external parties using maps or any centralized list of geographies, but there is clarity among community members on the village boundaries. Communities typically organize themselves into a natural hierarchy and tend to share information, cultural practices, and religion. Randomization at this level is often logical if the intervention hinges on the approval or involvement of social or village structures. These structures could potentially lead to clustering, and a priori, we expect to see a high level of homogeneity at this level due to the fact that most households in the study area are coconut farmers, speak the same language, and have the same religion.

Villages, in turn, are clustered at the administrative post level. Randomization at this level makes sense if the intervention hinges on approval or involvement of government officials, since each administrative post is governed by different local officials who are responsible for creating and implementing their own agricultural and economic policies. Farmers affected by the same local agricultural and economic policies likely have similar incentives or experience similar efficiencies in delivery of extension programs. We examine and compare ICCs at the EA, village, and administrative post levels. For simplicity and due to limited sample size, we did not investigate a three-level model, for example, EAs clustered within the administrative posts.

Statistical Approach

ICCs arise because measures of household characteristics are assumed to have two sources of variance: a cluster-level random effect and a household-specific random effect. Equation 1 illustrates this assumption, where household i in cluster j has characteristic y_ij which has population mean a, a cluster-level random effect γ _j , and a household-specific random effect ∊ _ij .

y_{i j} = a + γ_{j} + ∊_{i j} .

The ICC is the percentage of variation attributable to the cluster-level random effect. The ICC (ρ) is defined as:

ρ = \frac{Var (γ_{j})}{Var (γ_{j}) + Var (∊_{i j})} .

When Equation 1 is expanded to include a treatment indicator variable on the right-hand side, the standard error of the estimated treatment effect is affected by the cluster-level random coefficient (γ _j ). The sample size needed to detect statistical significance of the treatment effect is inflated by the design effect, which is defined in Equation 3 and is greater than or equal to 1. The design effect (D _eff) depends on the level of clustering and the sample size within each cluster. Since we study three different levels of clusters, and intervention designs will choose various cluster-level sample sizes (m), we focus only on reporting the ICC (ρ) required to compute the design effect. We do not report the design effect associated with each ICC estimate, but in the text, we report the design effect relevant to the Mozambique FISP evaluation.

D_{eff} = 1 + (m - 1) ρ .

We report the standard deviation and ICC for education, wealth, and agricultural measures. We also study the influence of household-level (Level 1) covariates and cluster-level (Level 2) covariates and report the proportion of variance in the outcome variable explained by the household-level covariates (R ₁) and the proportion of variance explained by the cluster-level covariates (R ₂). To estimate these proportions of variance (R ²s), we run the regression model specified in Equation 4, which is based on Equation 1 but includes household-level covariates X_ij and cluster-level covariates Z_j .

y_{i j} = a^{*} + β X_{i j} + ϑ Z_{j} + γ_{j}^{*} + ∊_{i j}^{*} .

The household-level and cluster-level R ²s are found by comparing the variances of the random effects in Equation 4 with the variances of the random effects in Equation 1 using the relationships illustrated in Equations 5 and 6.

Var (γ_{j}^{*}) = (1 - R_{2}) Var (γ_{j}) .

Var (∊_{i j}^{*}) = (1 - R_{1}) Var (∊_{i j}) .

We estimate the regression models using the mixed routine in Stata 13, which uses a maximum likelihood routine (Stata Manual).

When the outcome variable (y_ij ) is binary, Equations 1 and 4 are linear probability models. For our evaluation, we used linear models because we focused on the impacts on proportions rather than log odds. Other popular methods for studying binary outcomes are to estimate nonlinear regression models, for example, logit or probit models. There is not a consensus in the literature about whether to use linear or nonlinear regression models for binary variables and, moreover, how to consider ICC of binary variables. Unlike continuous variables, variation in binary variables is directly tied to their mean value, and thus their ICCs will vary based on their means. We recognize that researchers take various approaches, and therefore we report ICCs for binary outcomes using two methods: the standard linear model and a generalized estimating equation (GEE) approach that obtains impact estimates by differencing two logit equations. The GEE approach is described in Schochet 2013, and the relevant ICC formula (Equation 7) averages the pairwise correlations of model residuals for observations in the same cluster.

ρ = \frac{1}{N - 2} \sum_{j} \sum_{i} \sum_{k \neq i} \frac{(y_{i j} - \hat{y_{i j}}) (y_{k j} - \hat{y_{k j}})}{\sqrt{\hat{y_{i j}} (1 - \hat{y_{i j}})} \sqrt{y_{k j} (1 - \hat{y_{k j}})}} .

Wu, Crespi, and Wong (2012) compare five methods of calculating ICCs for binary variables and find that the GEE approach may be preferred in cases where outcome probabilities are very different between study arms and in cases where the true ICC is negative.

Outcome and Covariate Specification

We report the mean, standard deviation, and ICC for measures in three domains: education, wealth, farm production. We estimate ICCs (in each of these domains) in three ways: (a) without any covariates; (b) with only individual-level covariates and basic cluster-level geographic covariates; and (c) with individual-level covariates, cluster-level means of individual-level covariates, and basic cluster-level geographic covariates. In the remainder of this subsection, we describe our covariate selection when considering how to specify (b) and (c).

To study farm production outcomes, it is typical for covariates to include demographic data, education, baseline wealth information, and baseline farm characteristics such as farm size and whether or not the farm has already invested in certain crop types. All of these covariates may explain a farmer’s efficiency, capacity, and available inputs.

To study wealth, it is typical for covariates to include demographic data and education data. Interventions targeting wealth may involve farmers but may also involve nonfarmers, so we exclude baseline farm characteristics when describing how covariates affect the ICCs for wealth outcomes.

To study educational outcomes, it is typical to include demographic data in the set of covariates. We do not include baseline farm characteristics because interventions targeting educational achievement may not target farmers or may not have the sufficient cause to collect the in-depth baseline farm characteristics that we have.

Whenever we include any household-level covariates, we also include a basic set of cluster-level covariates that describe the climate of the cluster: average rainfall, distance to coast, and distance to the physical boundary separating the treatment and comparison areas.

Some evaluators include cluster-level means of individual-level covariates in their estimation in order to improve precision. Other evaluators prefer not to include cluster-level aggregate statistics because their inclusion makes it difficult to interpret the marginal effects of specific covariates.

Results

We focus our reporting using the EA cluster level because we believe other evaluators will agree that it is more feasible to build a design and sampling frame based on EAs rather than villages or administrative posts because EAs are available in public data sets, whereas lists of villages and/or administrative posts are much harder to come by. Table 2 shows the means, standard deviations, and EA-level ICCs, with and without covariates, for each variable within the three domains: education, wealth, and agricultural outcomes. We first discuss all of the findings by domain. Next, we discuss whether the GEE-based ICC comport with the ICC estimates of binary variables using the standard linear model, and how the ICC estimates change by cluster definition (EA vs. village vs. administrative post).

Education

Within the education domain, roughly half (51%) of the sampled farmers could read and write, and almost the same portion (52%) had completed some primary school (Table 2, column 1). The ICC estimates within the education domain are small, all below .06 without covariates (Table 2, column 3). Individual-level demographic characteristics are strong predictors of educational achievement. Column 4 of Table 3 shows that individual-level covariates explain a high proportion of the variance in education outcomes, ranging from 30% (years of education) to 47–49% (can read and write, completed some primary school). When including individual-level covariates, the ICC estimates are significantly reduced (Table 2, columns 4 and 5). Based on our estimates, the average design effect for education outcomes in the Mozambique FISP evaluation is 1.16 (including individual-level covariates and clustering at the EA level).

Table 3.

ICCs at the Enumeration Area (EA), Village, and Administrative Post Levels and at R ²s.

Outcomes	ICC, EA	ICC, Village	ICC, Administrative Post	Level 1 R ²	Level 2 R ²
Outcomes	(1)	(2)	(3)	(4)	(5)
Education^a
Years of education	.048	.033	.010	.307	.114
Can read and write	.025	.004	.005	.467	.066
Completed some primary school	.006	.000	.000	.489	.043
Wealth^b
House has improved walls	.129	.064	.112	.134	.009
House has improved roof	.054	.018	.026	.258	.021
Number of improved barns owned in 2009	.210	.116	.072	−.034	.015
Number of radios owned in 2009	.034	.019	.016	.325	.048
Number of mobile phones owned in 2009	.039	.000	.000	.110	.046
Number of motorbikes owned in 2009	.050	.000	.000	−.067	.018
Agricultural outcomes^c
Annual income (MZN)	.012	.011	.000	.641	.116
Annual nonfarm income (MZN)	.015	.014	.000	.538	.110
Annual farm/agricultural income (MZN)	.008	.000	.002	.773	.078
Whether farmer is currently economically “better off” than s/he was 5 years ago	.001	.009	.012	.839	.040
Annual sales from coconut derivatives (MZN)	.109	.106	.048	.205	.015
Annual sales of FISP-promoted crops (MZN)	.217	.093	.059	.042	.043
Annual groundnuts sales	.028	.002	.000	.068	.011
Annual nhemba bean sales	.081	.011	.000	−.052	.018
Annual coconut production (kg)	.037	.015	.003	.226	.037
Number of coconut trees in production in the last 12 months	.170	.065	.003	.380	.033
Score from income diversity metric^d	.064	.000	.000	.115	.019
Percentage of surviving saplings from all intervention saplings planted	.095	.060	.080	.012	.035
Number of non-FISP crops planted by the household	.170	.065	.003	.380	.033
Whether the farmer knows that cutting and burning is the best way to control CLYD	.112	.055	.017	.139	.063
Whether farmer is able to correctly identify CLYD symptoms	.025	.000	.000	.357	.040
Whether the household knows that the best CLYD mitigation is to cut trees and then burn stumps/roots	.096	.054	.015	.264	.010
Whether the household knows that tree cutting is a good way to mitigate CLYD spread	.085	.051	.016	.141	.026

Note. N = 1,227. The R ² estimates stem from linear models (Equation 4) at the EA cluster level. ICC = intracluster correlation coefficient; MZN = Mozambican Metical; FISP = farmer income support project; CLYD = coconut lethal yellowing disease.

^cRegression includes household demographic data, two of the covariates listed under “education” (can read/write, and some primary), baseline farm characteristics (number of coconut trees in production in 2009; binary variables for whether or not the household produced coconut wood, copra, coconuts, coconut alcohol, or fabric in 2009; and estimated total farmland value in 2009 and in meticais), and basic cluster-level geographic data (average rainfall, distance to coast, and distance to program geographic boundary).

Wealth

All of the variables in the wealth domain are binary variables, and we find that the estimates from the GEE-based ICC calculation are more stable than the ICCs from the linear probability model, so we highlight these in this text (Table 2, column 6). The average farmer we sampled is very poor. Roughly 14% live in structures with waterproof roofs. Roughly 13% lived in structures with wood, stone, or concrete brick walls as opposed to grass, clay and stakes, or mud bricks. The GEE-based ICC calculation revealed ICCs for dwelling type that range from .030 to .047. The ICC for whether or not the farmer owned an improved barn is much higher at .099. ICCs for ownership of durables were a bit lower ranging from .025 to .18. Table 3 (column 5) shows that the cluster-level aggregated covariates explain a higher proportion in the ownership of durables (almost 5%) than they explain of dwelling structure (1–2%), which is likely why the ICCs for ownership of durables are lower than the ICCs for dwelling type. Based on our estimates, the average design effect for wealth outcomes in the Mozambique FISP evaluation is 1.23 (including individual-level covariates and clustering at the EA level).

Agricultural Outcomes

The variance of reported income and sales amounts is very large. For these measures, the standard deviation always exceeds 3 times the mean. Annual income, annual farm income, and annual nonfarm income have ICCs ranging from .028 to .030 before the use of covariates. Covariates are significantly lower the ICC because individual-level covariates explain between 53% and 77% of the variance (Table 3, column 4) and cluster-level covariates explain an additional 8–12%. Roughly 27% of farmers reported that they are better off than they were in 2009, but 84% of the variation in their responses can be explained by the individual-level covariates (Table 3, column 4), so the ICC estimate is small (.004, GEE based). Based on our estimates, the average design effect for income measures in the Mozambique FISP evaluation is 1.05 (including individual-level covariates and clustering at the EA level).

Compared to ICCs for income, ICCs for levels of sales tend to be low (.037 to .109), with the exception of sales for FISP-promoted crops, which has a high ICC of .217. Variables describing farmer inputs (number of coconut trees, number of non-FISP crops) have high ICCs (.173 to .243), while variables describing production (production, percentage of surviving seedlings) have lower ICCs with a wide range of ICCs (.046 to .093). Knowledge about CLYD also has lower ICCs (GEE based, .015 to .050). Compared to sales and income, individual-level covariates explain a smaller proportion of the variance in agricultural inputs, production, and knowledge. Based on our estimates, the average design effect for agricultural sales outcomes in the Mozambique FISP evaluation is 1.64 (including individual-level covariates and clustering at the EA level), and the average design effect for knowledge outcomes is 1.47.

GEE-Based ICCs

In all domains, the GEE-based calculations of the binary variables yielded more stable ICC estimates within subcategories of the domain: wealth-dwelling structure, wealth-ownership of durables, agricultural outcomes knowledge. The reader is referred to Schochet (2013) and should bear in mind that these ICCs are relevant for a different kind of impact estimation routine than a simple linear regression or even a simple logit model. The impact estimation proceeds by estimating a logit model and then taking the average difference in the log odds of (y = 1) if the treatment indicator is equal to 1 and the log odds of (y = 1) if the treatment indicator is equal to 0. The population-level R ²s are not well defined for the logit model, however, and the user of these statistics should bear in mind that the R ²s that we report stem from the linear probability models.

Cluster Levels

Table 3 compares the ICCs we obtain by clustering the data at the EA level, with ICCs obtained by clustering at the village and administrative post levels. For this exercise, we only used the linear model—even for binary variables. As one would expect, we observe that ICCs tend to be slightly higher at the EA and village levels. ICCs are higher at the EA level (compared to both village and administrative post levels) for all but one of the variables analyzed; the one exception is a binary variable. Similarly, the ICCs at the village level are higher than administrative post level ICCs for all but four variables, three of which are binary variables.

Conclusions

We studied education, wealth, and agricultural data from a large survey of rural coconut farmers in Mozambique and report ICCs for various measures within each of these domains. Household-level covariates explain large portions of variance in outcome data, particularly for income and education outcomes, and thus are helpful at reducing the design effect. ICCs decrease as the geographic area of the cluster increases and decrease as covariates are added to the regression equation. For binary variables, GEE-based ICC calculations provided estimates of ICC that were less variable within outcome domain than ICC calculations based on a linear probability model.

Homogeneity of samples within clusters has implications for the design of data collection efforts. When we began our evaluation of the FISP agricultural intervention in Mozambique, we did find adequate information or data on which to base our power analysis. The statistics reported in this article offer a glimpse at the expected behavior of agricultural outcomes in Mozambique and may be applicable to other agricultural studies in Africa.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

References

Ersado

(2003). Income diversification in Zimbabwe: Welfare implications from urban and rural areas (Food, Consumption and Nutrition Division Discussion Paper, Policy Research Working Paper No. 3964). Washington, DC: International Food Policy Research Institute.

Food and Agriculture Organization Corporate Statistical Database. (2011). Food and Agricultural Organization of the United Nations, National Export Statistics. Retrieved March 8, 2016, from http://faostat.fao.org/desktopdefault.aspx?pageid=342&lang=en&country=144

Liverpool-Tasie. (2012). Farmer groups, input access, and intragroup dynamics: A case study of targeted subsidies in Nigeria (Discussion Paper 01197). Washington, DC: International Food Policy Research Institute.

Duarte

Macaringue

Khossa

Ismael

Nassabe e Nomen Traquino

(2012). Estadisticas e Indicadores Socias 2012-2013. Mozambique: Instituto Nacional de Estadistica.

Magnini

(1997). Sampling guide. Washington DC: Food and Nutrition Technical Assistance (FANTA).

Megill

(2000). Review of sample design for Zambia post-harvest survey and recommendations for improving the sampling strategy and estimation procedures (Food Security Research Project, Working Paper No. 2). Retrieved from http://purl.umn.edu/54457

Michigan State University. (2008). Trabalho de Inquerito Agricola (TIA). Joint work with Mozambique Ministerio da Agricultura and USAID. Retrieved March 1, 2014, from http://fsg.afre.msu.edu/mozambique/policypres/TIA2007_Dissemination.pdf

Ministry of Agriculture of Mozambique. (2008). National Agricultural Household Survey. Retrieved from http://fsg.afre.msu.edu/Mozambique/survey/index.htm

Narayan

Davis

Geyer

(2014). Performance evaluation: Mozambique farmer income support project: Revised final evaluation design. Prepared for the Millennium Challenge Corporation. Contract MCC-13-BPA-0036. Retrieved from https://data.mcc.gov/evaluations/index.php/catalog/131/download/569

10.

Practical Analysis of Nutritional Data. (2012). Food security, analysis example training module. Retrieved October 12, 2012, from http://www.tulane.edu/∼panda3

11.

Schochet

(2013). Statistical power for school-based RCTs with binary outcomes. Journal of Research on Educational Effectiveness, 6, 263–294.

12.

U.S. Agency for International Development. (2015). Agriculture and food security, Mozambique. Retrieved March 8, 2016, from https://www.usaid.gov/mozambique/agriculture-and-food-security

13.

United Nations Educational, Scientific and Cultural Organization Institute for Statistics Data Center. (2009). Web. Retrieved February 1, 2016, from http://data.uis.unesco.org/

14.

Crespi

C. M.

Wong

W. K.

(2012). Comparison of methods for estimating the intraclass correlation coefficient for binary responses in cancer prevention cluster randomized trials. Contemporary Clinical Trials, 33, 869–880.