Abstract
This article develops a method to estimate the impact of change in a particular social setting, the residential neighborhood, that is designed to address nonrandom selection into a neighborhood and nonrandom selection out of a neighborhood. Utilizing matching to confront selection into neighborhood environments and instrumental variables to confront selection out of changing neighborhoods, the method is applied to assess the effect of a decline in neighborhood concentrated disadvantage on the economic fortunes of African American children living within changing neighborhoods. Substantive findings indicate that a decline in neighborhood concentrated disadvantage during childhood leads to increases in adult earnings and income, but has no effects on educational attainment or other social outcomes.
Over the last few decades, the effects of social settings on individual outcomes has emerged as a central focus of sociology, with newly developed methods, specialized software, and thousands of studies devoted to the topic (Bryk and Raudenbush 1992; Raudenbush and Wilms 1995; Sampson, Morenoff, and Gannon-Rowley 2002). Hanging like a cloud over much of this research is the problem of selection bias, or the possibility that unobserved characteristics of individuals and families may jointly predict selection into a specific type of social setting and the outcome of interest. The observational literature on neighborhood effects, in particular, has been challenged repeatedly for failing to deal with selection bias adequately (Jencks and Mayer 1990; Ludwig et al. 2008).
This article develops a new approach designed to address selection into and out of social settings, with a specific focus on the impact of neighborhood change. The method makes comparisons among families that have selected virtually identical neighborhoods, but neighborhoods that change in different ways after selection has taken place. The central idea underlying the approach is that, under certain conditions that will be specified, it may be possible to think about the future of a family’s neighborhood as a type of natural experiment, in the sense that families have a choice about where they would like to live, but often have little choice about how their neighborhood environment will change once they are there. The phenomenon of neighborhood change occurring around individuals thus presents an opportunity to understand how neighborhoods influence residents that confronts the selection issue directly. However, this opportunity comes with additional challenges—most notably, it is necessary to address not only the problem of selection into a neighborhood but also selection out of that neighborhood as it undergoes change. To deal with selection out of changing neighborhoods, an instrumental variable technique is used that mimics the use of instrumental variables in the experimental context to address noncompliance. The proposed method has widespread applicability for the study of change in any social setting; here it is used to estimate the effects of neighborhood change during childhood on adult economic outcomes, a topic of central importance for both urban sociology and public policy.
Analyzing Neighborhood Change
Motivation for the Study of Neighborhood Change
One could argue that much of the resurgent interest in “neighborhood effects” can be traced back to a single demographic trend: the rise of concentrated poverty in the 1970s and 1980s, as first described and analyzed by William Julius Wilson in The Truly Disadvantaged (1987). In the time since Wilson wrote, a mountain of empirical research has considered the consequences of living in high-poverty neighborhoods, and theory on the ways in which neighborhoods influence the lives and the life chances of residents has been elaborated and refined in numerous original studies and research reviews (e.g., Ellen and Turner 2003; Jargowsky 1997; Jencks and Mayer 1990; Massey and Denton 1993; Sampson, Morenoff, and Gannon-Rowley 2002; Sampson, Raudenbush, and Earls 1997; Small and Newman 2001; Wilson 1996). Over the same time frame, however, neighborhoods in America’s cities have begun to change in ways that are very different from the past. After the sharp rise in concentrated poverty, the 1990s saw a decline in the prevalence of extreme poverty in central-city neighborhoods (Ellen and O’Regan 2008; Jargowsky 2003). The continued growth of immigration has transformed the ethnic composition of urban neighborhoods, creating the need for new understandings of neighborhood dynamics in multiethnic settings (Denton and Massey 1991; Fong and Shibuya 2005).
These developments suggest that a new direction for research on neighborhood effects is necessary. Whereas much of the literature to this point was designed to assess the consequences of concentrated poverty, it is now critical to expand this focus and to consider the impact of new forms of neighborhood change. 1 At the heart of this expanded agenda is a question that has generated little empirical attention: How are individuals and families affected when concentrated disadvantage declines—that is, when a neighborhood undergoes a partial transformation from an area of racial and economic segregation to racial/ethnic and economic diversity?
The vast majority of research examining the consequences of neighborhood change has focused either on neighborhood deterioration, as in Wilson (1987), or on the process of “gentrification.” Gentrification is a poorly defined process, but the term is frequently used to describe a transition in the racial or class composition of a neighborhood from an area that is composed primarily of racial or ethnic minorities and/or lower-income families to one with a substantial representation of Whites and/or well-educated or upper-income families. Quantitative research examining the consequences of this type of process for original residents focuses on outcomes related to housing, such as displacement or housing costs. Despite the negative connotations associated with the term, several recent studies find that entry into a neighborhood of residents with more schooling or higher income has minimal or no effects on the housing outcomes of original residents (Freeman 2005, 2006; Freeman and Braconi 2004; McKinnish, Walsh, and White 2008; Vigdor 2002). 2
Few studies have attempted to assess the effects of gentrification, or related processes of neighborhood transformation, on other aspects of families’ lives or on children’s trajectories (Atkinson 2004). Some exceptions are Pattillo’s (2007) research in Chicago and Freeman’s (2006) research in New York, studies that reveal the conflicting ways that neighborhood change impacts residents. While the ethnographic research that has been produced on the topic is extremely well suited to examine the complex process of neighborhood transformation and the mechanisms mediating its influence, the difficulty of following people over long periods of time and the relatively small sample sizes in many such studies make it difficult to examine the long-term trajectories of original residents living in changing neighborhoods. By contrast, the data set and the empirical strategy utilized for the present analysis are uniquely suited to provide evidence on the basic question driving the analysis: What impact does a decline in concentrated disadvantage have on the economic and social trajectories of children?
Methodological Approaches to the Identification of Neighborhood Effects
The central challenge in identifying the effect of the neighborhood environment on individual outcomes is the problem of selection bias. Several observational approaches have been used to confront selection bias in the neighborhood effects literature, including: (a) collecting rich data on previously unobserved covariates and developing more elaborate models for selection into advantaged and disadvantaged neighborhoods (Sampson, Sharkey, and Raudenbush 2008); (b) using sensitivity analysis to assess how robust estimates are in the presence of potential unobserved factors with varying associations with the causal treatment and the dependent variables (Harding 2003; Sharkey and Elwert, in press); (c) exploiting variation in neighborhood conditions among siblings to control for fixed characteristics of families (Plotnick and Hoffman 1999; Vartanian and Buck 2005). While all of these approaches represent advances over traditional regression analysis, they all rely on variation among individuals or families that have selected different neighborhoods, or on variation in the neighborhoods that a single family selects at different times. The central assumption of such approaches is that selection into different neighborhood environments is ignorable, meaning there are no systematic differences in the “potential outcomes” of individuals in different types of neighborhoods (Morgan and Winship 2007). Without complete knowledge of the specific factors that lead families to select different environments, the selection bias critique remains unresolved.
Findings from observational studies examining the relationship between characteristics of children’s residential environments and their adult economic and social outcomes typically find some association between measures of neighborhood economic status during childhood and adult economic status, although the strength of the association varies widely depending on the methods used, the specific neighborhood measures considered in the analysis, the outcome under study, and the subpopulations examined. Several studies using ordinary least squares or related regression methods find that neighborhood poverty (or associated neighborhood characteristics) has negative impacts on adult outcomes (Datcher 1982; Corcoran and Adams 1992; Corcoran et al. 1992; Vartanian 1999), while studies using family fixed effects methods or sibling and neighbor correlations report inconsistent results (e.g., compare Plotnick and Hoffman [1999] with Aaronson [1997], Page and Solon [2003], and Vartanian and Buck [2005]).
A second approach to the study of neighborhood effects exploits variation in neighborhood environments arising from quasi-experimental or experimental residential mobility programs (Briggs 1997; DeLuca and Drayton 2009; Goering and Feins 2003; Ludwig et al. 2010; Rubinowitz and Rosenbaum 2000). The experimental approach, as exemplified by the Moving to Opportunity (MTO) program, involves randomly assigning families to a control group or an experimental group, the latter of which receives vouchers that allow the family to move to low-poverty neighborhoods in the case of MTO. 3 Because of randomization, the potential outcomes of the experimental group and the treatment group will be the same in expectation. The experimental approach is not without its own assumptions (e.g., see Sobel 2006), but it does confront the central challenge of selection bias. However, evidence from experimental studies like MTO provides information on a somewhat narrow question: Does moving from a public housing complex to a new neighborhood with lower poverty affect social and economic outcomes? While this is an important question for the development of policy, estimates from MTO are less helpful in understanding the impact of neighborhoods because they conflate the effect of moving itself with the effect of a change in neighborhood environment. Considering the extensive literature suggesting negative effects of residential mobility on developmental outcomes (Coleman 1988; Hagan, MacMillan, and Wheaton 1996; Haynie and South 2005; Pribesh and Downey 1999), this is an important limitation.
Among several experimental and quasi-experimental mobility programs that have been studied in the literature, the two most prominent examples are the Gautreaux program in Chicago and the more recent MTO experiment, which was conducted in five US cities. In Gautreaux, low-income Chicago families were provided housing subsidies and other forms of assistance to move out of segregated neighborhoods and into more racially and economically diverse neighborhoods across the metropolitan area (Rubinowitz and Rosenbaum 2000). Much of the research from Gautreaux focuses particular attention on differences in outcomes among families that remained within the city and those that moved to the suburbs, and finds that children in Gautreaux families that moved to surburban neighborhoods had higher rates of high school completion, college attendance, and labor force participation in early adulthood (Kaufman and Rosenbaum 1992; Rubinowitz and Rosenbaum 2000).
The MTO program has been running for a much shorter duration, and so far the results have been very different. Several years after the program started, Kling, Liebman, and Katz (2007) find that the effects of residential mobility on several children’s outcomes appear to vary by gender, with girls showing positive effects across several developmental outcomes and boys showing null or negative effects. Recent research has shown that the impacts of the experiment on academic test scores vary markedly by city (Burdick-Will et al. in press). The experiment has not been running long enough to evaluate the adult economic outcomes of children in families that received housing vouchers. While the designs of both Gautreaux and MTO have limitations and potential flaws (for descriptions of the limitations of each study, see Keels et al. 2005; Sampson 2008; Sobel 2006; and Votruba and Kling 2008), findings from both programs are useful for understanding the impact of a change in the neighborhood environment arising from a residential move out of high-poverty public housing. With few exceptions, a similar body of evidence does not exist to evaluate the impacts of change occurring around individuals or families.
An alternative Approach: Exploiting Change in the Neighborhood
This article develops an alternative method to estimate the effect of neighborhood change. The central idea underlying the method is that, under certain conditions that will be specified, it is possible to think about the future of a family’s neighborhood as a type of natural experiment, in the sense that families have a choice about where they would like to live, but have little choice about how their neighborhood environment will change once they are there. 4 Although the analytic framework is presented in more detail below, here I address two central challenges that arise in attempting to identify the effects of neighborhood change. The first challenge is that neighborhood change does not occur in a random manner—some neighborhoods are more likely than others to deteriorate or to improve quickly, and some individuals are likely better able to predict how a neighborhood will change in the future. The analysis confronts this problem by matching individuals on the characteristics of their neighborhoods and the trend of change in the neighborhood in the period before the matching takes place. Thus, matched pairs are selected among families that live in extremely similar neighborhoods that have experienced the same changes in racial and economic composition in the prior period. The difference between the matched families is in the change that occurs in their neighborhoods subsequent to the matching. The matching procedure does not eliminate all potential sources of bias. It is possible that some individuals or families are better able to predict the future of their neighborhoods than others, even if they live in places that have experienced similar trajectories of change in the recent past. To the extent that families have unobserved characteristics that allow them to make “better bets” on the future of their neighborhoods, and that also predict their future economic outcomes, the method will generate biased results. Below, I describe an empirical test that provides suggestive evidence on whether this is likely to be a source of bias in the current application.
A second challenge is that families often move out of a neighborhood as it begins to change around them, and thus may not actually experience any change in the neighborhood environment even if they begin in a neighborhood that undergoes rapid demographic or economic transformation. This issue can be thought of as essentially equivalent to the issue of noncompliance in the experimental context. It is possible to estimate effects of neighborhood change even if all families do not actually experience the treatment of interest. In the MTO Experiment, for instance, researchers have used the offer of a housing voucher, which is randomly distributed but highly correlated with actual use, as an instrument for utilization of a voucher in order to estimate the effects of residential mobility among “compliers” (Kling, Liebman, and Katz 2007). In the present case, a dichotomous indicator for whether individuals live in a neighborhood that is on the verge of undergoing a change in economic and demographic composition can be used to instrument for the degree of change actually experienced over the decade. Conditional on the variables used in the matching procedure, the key assumption is that individuals in neighborhoods that undergo change would have had similar outcomes as their matched counterparts in the absence of the changing neighborhood environment, or that there are no systematic differences in the “potential outcomes” of treatment and control group members (Morgan and Winship 2007). This assumption is impossible to test, but it is possible to produce evidence that bolsters or weakens the case for considering treatment status to be ignorable.
Data
The analysis utilizes data from the Panel Study of Income Dynamics (PSID), an ongoing longitudinal survey begun in 1968 with a nationally representative sample of about 4,800 families (Hill and Morgan 1992). The PSID has attempted to follow all family members of the original sample as they “splitoff” from the sample family, making it possible to observe individuals’ childhood neighborhood environments as well as their adult social and economic status. The PSID contains an oversample of low-income households, which allows for reliable comparisons of Blacks and Whites.5,6 The PSID makes available a restricted-use geocode file that contains census tract identifiers for sample families, from 1968 through 2007. 7 Tract identifiers are used to merge the data from the PSID with data from all census tracts in the United States available from the Neighborhood Change Database (GeoLytics 2003) for Census years 1970, 1980, 1990, and 2000. For the present analysis, a sample is selected comprising individuals in the PSID who meet two primary criteria: (1) they must be observed as children, between the age of 5 and 15, in PSID households in 1980, when treatment neighborhoods are identified; (2) they must be observed again as household “heads” or “wives” (the label given to the spouse or domestic partner of household heads in PSID households) in at least one survey year from 1990 to 2007. All outcomes are measured over the period from 1990 to 2007, and thus individuals who leave the survey before the 1990 survey are excluded from the analysis. The sample of individuals who meet these criteria is composed of 1,274 African Americans and 1,196 Whites.
Drawing on previous research that emphasizes the multiple dimensions of spatial disadvantage (Sampson, Raudenbush, and Earls 1997), the measure used to define treatment status is a composite scale of neighborhood concentrated disadvantage. The normalized scale is generated from a principal component analysis of six census tract characteristics found in previous research to load on a single component which is referred to as “concentrated disadvantage”: welfare receipt, poverty, unemployment, female-headed households, racial composition (percentage Black), and density of children (percentage of residents under 18). The measure of concentrated disadvantage is based on all U.S. census tracts and is constructed separately in census years 1970, 1980, 1990, and 2000. 8 Measures of change in concentrated disadvantage from one decade to the next represent change in the relative position of the census tract in comparison with all other tracts. The treatment is defined as living in a neighborhood that undergoes any decline in concentrated disadvantage from 1980 to 1990, compared to living in a neighborhood that undergoes no change or an increase in concentrated disadvantage over the same time period.
An initial analysis of neighborhoods where concentrated disadvantage declined over the 1980s reveals very different patterns for Whites and Blacks. 9 For African Americans, positive neighborhood change during the 1980s typically meant a transformation from a severely disadvantaged social environment to one that became more ethnically diverse, with a higher prevalence of Latinos and immigrants, and with an improved economic environment. For Whites, the more common pattern of positive change involved a transition from an advantaged environment to a more advantaged environment. Most Whites who lived in neighborhoods that underwent a rise in status in the 1980s were not living in disadvantaged settings. Because the primary focus of the analysis is in the impact of a decline in concentrated disadvantage, the analysis focuses on the effects of neighborhood change for African Americans. A parallel analysis of neighborhood change for Whites is available from the author.
The primary dependent variables in the study measure three dimensions of adult economic status: individual earnings, family income, and household wealth, all inflated to 2008 dollars. To adjust for age and year effects, the measures are constructed by first regressing each outcome on age and survey year using the full PSID sample over all years of the survey (Solon et al. 1991). Residuals from the regression are then added to the predicted values for individuals who are 30 years old, in the year 2007, and the adjusted measures are used as the dependent variables in the analysis. This approach adjusts for the fact that outcomes are measured at different ages and over different years of the survey—the method accounts for changes in earnings profiles over the life course, and for possible trends in earnings due to fluctuations in the business cycle. All dependent variables are measured as the average over all survey years from 1990 to 2007 in which the individual is between the age of 21 and 32 and is observed as a household head or the spouse of a household head. Averages over multiple years are used in order to reduce measurement error and create approximate measures of average adult social and economic status. In subsequent analysis of the stability of results, I have found that results are somewhat sensitive to the age range at which outcomes are measured. Specifically, if outcomes are measured over the age range of 26–32, the effects of neighborhood change are in the same direction but are slightly weaker and have wider confidence intervals. Although the results presented are preferable because they are more precise, these exploratory results do suggest that the effects of neighborhood change may be stronger for economic outcomes measured in early adulthood.
Individual earnings includes all income derived from the individual’s labor market activity in the year prior to the survey. If the individual did not work in a given year, the measure is coded as “0”. To measure individual earnings (and all other dependent variables), the measure of earnings is created for each survey year from 1990 to 2007 in which the individual is between the age of 21 and 32 and is identified as a household head or spouse. The measures for each survey year are then averaged together to create the final dependent variable used in the analysis. The same basic procedures are used for all outcome variables (with the exception of marital status, health, and welfare receipt, which are coded as dichotomous as described below). Family income measures income from all sources in the family, providing a complementary measure of economic success that considers the individual’s fortunes in the labor market as well as the fortunes of other family members, particularly the spouse or domestic partner. The last measure of adult economic status is family wealth, which represents the total value of all assets (including real estate) less all debts held by the family. The wealth measure is included based on research demonstrating the severe racial gaps in household wealth, which are more pronounced than racial gaps in income or earnings (Conley 1999; Oliver and Shapiro 1995). Data on wealth were collected in the 1984, 1989, 1994, 1999, 2001, 2003, 2005, and 2007 surveys. Because the data were not collected in every survey year from 1990 to 2007, some individuals have missing data on wealth and the sample is smaller for analyses examining wealth as the dependent variable.
The analysis considers various potential mechanisms by which neighborhood change may be linked with adult economic status, including educational attainment, annual hours worked, welfare receipt, health, and marital status. The analysis of mechanisms is exploratory in nature and is not exhaustive, but is designed to assess several common theories about why neighborhoods might matter for children’s economic fortunes, which often focus on institutions such as the schools, family structure, health, or participation in the labor market (e.g., see Ellen and Turner 2003; Jencks and Mayer 1990; Wilson 1987). Educational attainment is measured as the individual’s total years of schooling. Annual hours worked represents the self-reported hours worked on all jobs in the year prior to the interview, including jobs not identified as the “main job”. Annual hours worked, like all other outcomes, is averaged over all years in which the individual is a household head or spouse and is between 21 and 32 years old. This measure is based on the average annual hours worked in years when the individual works. Hourly wages is measured as the calculated hourly wage of the individual for the main job in the year prior to the survey, even if the individual is paid on a salary basis. Similar to the measures of earnings, income, and wealth, this measure is adjusted for year and age and is inflated to represent year 2008 dollars. Like annual hours worked, the measure of hourly wages is coded as missing in years in which the individual is not working.
Whereas all of the prior outcomes are measured as averages over multiple survey waves, the measures of health, welfare receipt, and marital status are also constructed in each survey year but then are transformed into dichotomous dependent variables. Health is a dichotomous measure indicating if the individual’s health is classified as “fair” or “poor” at any point over the same period. This measure is based on questions about self-reported health using a scale of 1 to 5, with 1 indicating excellent health, 4 indicating fair health, and 5 indicating poor health. If the self-reported value is at least “4” in any year in which the individual is at least 21 years old and is the head of household or spouse, the individual is classified as in “poor health”. The self-reported scale of health is widely used in epidemiological studies and is strongly predictive of morbidity and mortality (Idler and Angel 1990; Kaplan and Camacho 1983; Miilunpalo et al. 1997). Welfare receipt is a dichotomous indicator for whether the individual, or his or her partner, ever reports receiving any income from programs typically referred to as “welfare,” including Aid to Dependent Children (ADC), Aid to Families with Dependent Children (AFDC), or Temporary Assistance for Needy Families (TANF), over the period from 1990 to 2007 in which the individual is between 21 and 32 years old and is a household head or spouse. Ever married is a dichotomous measure indicating if the individual reports being married at any time over the period from 1990 to 2007 in which the individual is between 21 and 32 years old and is a household head or spouse.
All estimates of treatment effects adjust for covariates at the level of the census tract, the metropolitan area, and the family. At the level of the census tract, the following measures are included in regression models: concentrated disadvantage in 1980, the poverty rate, and % Black in 1980, along with the degree of change in concentrated disadvantage, poverty, and % Black from 1970 to 1980. At the level of the metropolitan area, the following measures are included: the poverty rate and % Black in 1980, along with measures of change in poverty and % Black from 1970 to 1980. At the level of the family, a set of measures are included in regression models that are designed to capture demographic characteristics of the child and his or her family and aspects of family background available in the PSID. All of these variables are measured as of 1980, prior to the measurement of the treatment. 10 They include, first, several measures of family social and economic status: the household head’s labor earnings and total family income in 1980 (each measured using the same methods as are used to construct the dependent variables, described above), the household head’s average annual hours of work (measured categorically as less than 250, 250–2,000, or more than 2,000 hours), welfare receipt among any member of the household, and the educational attainment of the household head (less than high school, high school graduate, or at least some college). The household head’s occupational status is the average status of the main jobs held by the individual, and is based on the socioeconomic index of all occupations, based on occupational categories from the 1970 Census (Stevens and Featherman 1981)—this measure is recoded into equally sized categories (based on the full distribution of sample members) representing high, medium, and low status, along with a category for missing status due to unemployment. Second, several demographic characteristics and life-cycle measures are included: the age, age squared, and gender of the child, the number of children in the child’s family, an indicator for the household head having a work-limiting disability, an indicator for whether the household head was married in 1980, and an indicator for whether the family owned its home in 1980. Descriptive statistics for all variables among the sample of 1,274 African Americans, prior to matching, are shown in Table 1. The temporal sequence of the various components of the analysis is displayed in Figure 1.
Descriptive Statistics for Key Variables
Mean excludes two outliers with over $2,000,000 wealth. bThe scale of concentrated disadvantage is normalized to have mean = 0 and SD = 1 across all U.S. census tracts.
Timeline of measurement for key components of analysis
Methods
In this analysis, change in the neighborhood environment is analyzed as the causal treatment of interest. Specifically, the treatment is defined as living in a neighborhood in 1980 that undergoes any decline in concentrated disadvantage (i.e., an “improvement” in neighborhood status) from 1980 to 1990. Formally, for individuals i, T i is an indicator variable denoting whether the individual is a member of the treatment group (T i = 1) or the control group (T i = 0), and Y i 1 is the potential outcome for individual i in the treatment state and Y i 0 is the potential outcome for individual i in the control state (Morgan and Winship 2007). The individual causal effect, which can never be observed in the data, is defined as the difference in the potential outcomes in the treatment and control states, δ i = Y i 1− Y i 0. The average treatment effect for the treated is defined as:
where N T is the number of individuals in the treatment group.
In this application, matching is used to adjust for nonrandom assignment to treatment. Matching is conducted based on neighborhood characteristics in 1980 and changes in neighborhood characteristics from 1970 to 1980. The core assumption of the method is that assignment to the treatment group is “ignorable” conditional on the variables used in the matching procedure (Morgan and Winship 2007; Rosenbaum 2002; Rosenbaum and Rubin 1983). In other words, conditional on 1980 neighborhood characteristics and changes in neighborhood characteristics from 1970 to 1980 (denoted by X i ), membership in the treatment group is assumed to be independent of the potential outcomes, Y i T :
The assumption of ignorable treatment assignment could be challenged if one believes that families are differentially equipped to predict the future of their neighborhood, even with the same information on trends of change in neighborhood racial and economic composition in the years prior to measurement of treatment. The qualities that might enable one family to choose a neighborhood that is on the verge of a decline in concentrated disadvantage could also predict the adult economic outcomes of children in the family. While suggestive evidence can be generated to assess this possibility, the ignorability assumption is not possible to test definitively.
To conduct the matching, the analysis uses a newly developed matching procedure called coarsened exact matching (CEM; Blackwell et al. 2009; Iacus, King, and Porro 2009). Part of a class of methods labeled “Monotonic Imbalance Bounding,” the key feature of CEM is that it sets bounds, prior to the matching, on the maximum allowable imbalance between the matched treatment and control groups on all variables used in the matching procedure. While commonly used matching procedures, such as propensity score matching, generate balance on all variables used to predict selection into treatment in expectation, CEM ensures balance on all variables used in the matching procedure within the sample.
As described in Iacus, King, and Porro (2009), CEM involves three steps. First, matching variables are “coarsened” into intervals. In the present application, the two primary matching variables are the degree of concentrated disadvantage in 1980 and the change in concentrated disadvantage from 1970 to 1980. For each measure, all U.S. census tracts are ranked and split into deciles on each measure, and matches are selected within deciles. Other dimensions of the “selected” neighborhood also are incorporated into the matching in order to ensure that treatment and control group members live in neighborhoods with similar racial and economic composition, and matches are made only within Census region/divisions. Second, all observations are placed within strata of the matching variables, and matches are made within strata of the coarsened variables. Third, strata that do not include members of both the treatment and the control groups are discarded. The resulting estimates produced after eliminating strata that do not include members of both groups are referred to as “local sample average treatment effects on the treated” by Iacus, King, and Porro (2009). This label is used to emphasize the fact that estimates do not generalize to a wider population, nor do they apply to the entire sample. Instead, estimates apply to the segment of the sample that is successfully matched—information on this portion of the sample is described in the text and in Table 2.
Comparison of “Matched” and “Unmatched” African American Sample Members
The procedure differs from more common matching techniques, such as optimal matching or propensity score matching (Gu and Rosenbaum 1993; Morgan and Harding 2006; Rosenbaum 2002). CEM on a set of variables representing the “selected neighborhood” is chosen because it provides a simple, intuitive, and effective way to compare sample members who have chosen extremely similar neighborhoods in which to live, and neighborhoods that showed similar trends of change in the 1970s. Whereas a propensity score matching procedure would create matched pairs based on the estimated probability of selecting into a given treatment, the present analysis addresses the central critique of neighborhood effects research more directly, by matching on the neighborhoods that families have already selected. Thus, it is an approach that explicitly accounts for selection bias as commonly thought of in the neighborhood effects literature.
An added benefit of this approach is that it allows for a suggestive test of the central assumptions of the matching method. Because the matching is carried out on only a few key variables describing the selected neighborhood environment, there are numerous measures of family background that are not included in the matching procedure. The advantage of this aspect of the analysis is that, after conducting the matching, it is possible to compare treatment and control group members on all of the observed variables that were not included in the matching procedure. The comparison on observable measures of family background that were not included in the matching procedure provides some indication of whether there is likely to be balance on unobservable characteristics of treatment and control group members, and thus provides evidence to bolster or else weaken the case for considering treatment status to be ignorable. It is not possible to provide definitive evidence on whether the treatment and control group are balanced on unobservables. For instance, if unobservable characteristics of families that are important in allowing them to predict the future of their neighborhood are not highly correlated with observable measures of family background, then comparisons of observable characteristics are not helpful in assessing the plausibility of the ignorability assumption.
Further, while the use of CEM has the advantage of being intuitive and confronting the major critique of much neighborhood effects research, it is less effective in minimizing imbalance between treatment and control group members on the full range of pretreatment background variables that are not included in the matching. For this reason, additional results are presented, for comparison, that use matching on the propensity score.
CEM results in a subsample of individuals within strata that contain members of both the treatment and the control groups. Within this subsample, I generate two sets of estimates. The first set of estimates represents the effect of membership in the treatment group, hereafter referred to as “treatment group effects”. Treatment group effects represent the effect of living in a neighborhood in 1980 that is on the verge of undergoing a decline in concentrated disadvantage from 1980 to 1990. This set of estimates is derived from equation (3):
In equation (3), the outcome Y i , representing a given measure of adult economic status, is regressed on the indicator for treatment status, T i , and a set of covariates, X i , including characteristics of the neighborhood and the metropolitan area, as well as family background characteristics. The inclusion of neighborhood, metropolitan area, and family-level covariates is designed to adjust for any imbalance on observable characteristics among the matched subsample of treatment and control group members. The coefficient π is an unbiased estimate of the effect of treatment status on adult outcomes under the assumption of ignorable treatment assignment within strata of the matching variables.
This first set of estimates does not reveal how living in a changing neighborhood affects adult economic outcomes, because some families living in such neighborhoods will exit the neighborhood. For this reason, a second set of estimates is generated that represents the effect of actually experiencing a change in neighborhood concentrated disadvantage on adult economic and social outcomes. To generate the second set of estimates, the dichotomous measure of treatment status, T i , is used as an instrument for the actual change in the level of concentrated disadvantage in the individual’s own neighborhood from 1980 to 1990, Δ i . Two-stage least squares estimates are generated—in the first stage equation (4), change in neighborhood concentrated disadvantage from 1980 to 1990 is regressed on the indicator for treatment status along with the full set of controls, X i :
Note that the effect of treatment status in this model (π) reveals the degree to which treatment group status leads to actual change in individuals’ neighborhood environments. If this effect is minimal, it would suggest that individuals living in neighborhoods that are about to experience a decline in concentrated disadvantage do not actually experience much of that change; if the effect is large, it suggests that members of the treatment group experience a substantial amount of change. In results described below, I find that treatment status leads to a 1.04 standard deviation decline in concentrated disadvantage, indicating that children in the treatment group experience a substantial amount of change in their neighborhood. In the second stage equation (5), the set of adult outcomes is regressed on the predicted level of change in concentrated disadvantage (Δ i *) and the same controls:
This second set of estimates represents the effect of neighborhood change experienced over the 1980s on adult outcomes, measured from 1990 to 2007—I refer to this set of estimates as “neighborhood change effects”. The measure of concentrated disadvantage is measured in standard deviations, and thus estimates of neighborhood change effects can be interpreted as the effect of a one standard deviation decline in concentrated disadvantage on adult outcomes. Because a one standard deviation change is roughly the average decline in concentrated disadvantage experienced by the treatment group, estimates of “treatment group effects” and “neighborhood change effects” are very similar—this does not have to be the case, and is only true because membership in the treatment group led to roughly a one standard deviation decline in concentrated disadvantage. The reader should note that the similarity between “treatment group effects” and “neighborhood change effects” reveals nothing about the “strength” of the treatment. As noted above, information about the strength of the treatment is contained in the first stage regression results, and in this application the results show that the change induced by membership in the treatment group is quite strong.
Analysis
Generating Treatment and Control Groups
The “treatment” group is composed of individuals who live in a neighborhood in 1980 that undergoes a decline in concentrated disadvantage from 1980 to 1990. Specifically, a measure of concentrated disadvantage is constructed for all U.S. census tracts in both 1980 and 1990. Individuals living in 1980 in tracts that experience a decline in the scale of concentrated disadvantage compose the treatment group, and individuals in tracts that experience no change or an increase in concentrated disadvantage compose the control group.
Based on the criteria for sample selection and these definitions, there are 561 African Americans in treatment neighborhoods and 713 in control neighborhoods. Figure 2 displays the average change from 1980 to 1990 in the neighborhoods of African Americans in the treatment group. African Americans in “treatment” neighborhoods lived in extremely disadvantaged neighborhoods in 1980 that became more ethnically diverse and saw improvements in economic status over the decade. In 1980, this group lived in neighborhoods with an average poverty rate of 28 percent and an unemployment rate of 12 percent, with average racial/ethnic composition of 70 percent Black, 27 percent White, 4 percent Latino, and 4 percent foreign born. Over the 1980s, the poverty rate in treatment group members’ neighborhoods dropped to 26 percent, on average, and the unemployment rate dropped to 11 percent. The average percentage of Black residents dropped slightly to 67 percent, and the percentage of White residents did not change, while the percentage of Latino residents and foreign born residents rose to 6 percent. Thus, the neighborhoods that experienced declines in concentrated disadvantage over this decade were not neighborhoods that attracted an influx of Whites, but were instead neighborhoods that saw an increase in ethnic diversity due to growing numbers of Latinos and immigrants.
Neighborhood characteristics in 1980 and 1990 in neighborhoods of African Americans where concentrated disadvantage declined
Matching Treatment and Control Group Members
The primary variables used to match treatment and control group members are the level of concentrated disadvantage in the individual’s neighborhood as of 1980, and the trend of change in concentrated disadvantage in the decade prior to the treatment; that is, from 1970 to 1980. For each measure, all U.S. census tracts are ranked and coarsened into deciles. To further reduce imbalance on the selected neighborhood environment, measures of neighborhood racial composition and economic composition are incorporated into the matching. Specifically, measures of the percentage of African American residents in the Census tract in 1980 and 1970 are coarsened into the following categories: less than 33 percent, 33–66 percent, and 67–100 percent. While these are broad categories, the intent is simply to ensure that matches are not made among individuals living in neighborhoods with similar levels of concentrated disadvantage but large discrepancies in racial composition. Measures of the neighborhood poverty rate are coarsened into the following categories: less than 5 percent poor, 5–14.99 percent poor, 15–24.99 percent poor, and 25 percent or more poor. Finally, exact matching is required for the individual’s Census region, based on recent research suggesting that matching procedures are more effective in reproducing treatment effect estimates from experimental data if the matches are made among individuals close in geography (Cook, Shadish, and Wong 2009). 11
The matching results in 214 strata that have more than one sample member, 59 of which contain members of both the treatment and control groups. Of the 561 members of the treatment group, 306 are matched (55 percent). Individuals in the remaining strata are dropped from the analysis. Table 2 compares sample members who are successfully matched to those who are not matched. The first group of rows reveals that sample members in strata that contain both treatment and control group members live in neighborhoods with substantially higher levels of poverty and higher percentages of African American residents than unmatched sample members. For instance, the poverty rate in matched sample members’ neighborhoods in 1980 is 33 percent, compared to 20 percent for unmatched sample members. The second group of rows indicates that matched sample members live in metropolitan areas that are quite similar to unmatched sample members, with only slightly higher prevalence of African Americans. The third group of rows indicates that African Americans who are successfully matched are more disadvantaged than those who are not matched, with lower income, fewer hours worked, lower educational attainment, and lower occupational status. These comparisons indicate that African Americans who are successfully matched are more disadvantaged and come from more disadvantaged neighborhoods than unmatched sample members. The estimates derived from the analysis should thus be thought of as estimates of the impact of declines in concentrated disadvantage among the most disadvantaged segment of African Americans.
Based on the matching criteria, treatment and control group members are perfectly balanced on race and on Census region, but how well are they balanced on their selected neighborhoods as of 1980? Table 3 displays differences between the matched treatment and control groups in the values of several variables as measured at the mean, at the 25th percentile, the 50th percentile, and the 75th percentile. For instance, the figures in the row labeled “Concentrated disadvantage, 1980” indicate that treatment group members have levels of concentrated disadvantage that are .13 standard deviations higher than control group members at the means of each distribution, .17 standard deviations higher at the 25th percentiles of each distribution, .01 standard deviations lower at the median, and .35 standard deviations higher at the 75th percentile. All differences are measured in the units of the variable. Balance is shown for neighborhood characteristics in 1980 and 1970, followed by Metropolitan Statistical Area (MSA) characteristics in 1980 and 1970, and finally several family-level characteristics measured in the 1980 PSID survey.
Differences between Matched Treatment and Control Groups on Neighborhood, MSA, and Family Background Characteristics at the Mean, 25th, 50th, and 75th Percentiles, African Americans
Table 3 suggests that the matching procedure produces strong balance on characteristics of the selected neighborhoods of treatment and control group members, with slight imbalance at specific points in the distribution. Treatment group members live in neighborhoods with slightly higher levels of concentrated disadvantage in 1980 and a decade earlier in 1970. The neighborhoods of treatment and control group members have close to identical racial composition and poverty rates in 1980 and in 1970, with the exception of slight imbalance in neighborhood percentage Black. The metro areas in which they live also look extremely similar in 1980 and 1970. Metropolitan area poverty rates are similar across the entire distribution, and treatment group members live in metropolitan areas with a slightly higher presence of African Americans. Overall, the matching produces strong balance on the selected environment, and the imbalance that is present suggests that treatment group members live in neighborhoods and metropolitan areas that are slightly more racially segregated and have slightly higher levels of concentrated disadvantage.
The measures in the final group of rows of Table 3 represent various dimensions of family background and demographic characteristics. These variables are not included in the matching process, and thus we should not expect the same degree of balance. The table shows that members of the treatment group are from families with slightly lower family income, families that are more likely to receive welfare, and families in which the household head works fewer hours per year. However, treatment group members are from families with slightly higher educational attainment and occupational status. Overall, the evidence available suggests that African Americans living in neighborhoods that improved in the 1980s do not appear to have any systematic advantages that would give them an edge in selecting neighborhoods on the verge of positive change. To adjust for the imbalance on neighborhood characteristics, metropolitan area characteristics, and family background, all of the variables displayed in Table 3 are included as regressors in the final specifications estimating the effects of neighborhood change.
The Effect of a Decline in Concentrated Disadvantage on Adult Outcomes
The first step in assessing the impact of neighborhood change is to consider the effect of living in a treatment neighborhood in 1980 on actual change in the level of concentrated disadvantage from 1980 to 1990. This estimate reveals the strength of the relationship between treatment status and the amount of change in individuals’ actual neighborhoods over the decade. If treatment status is only weakly related to actual neighborhood change, it would suggest that members of the treatment group typically move out of improving neighborhoods, or are displaced, and do not experience any potential benefits of a rise in neighborhood status. If there is a strong relationship, there is reason to believe that treatment group members do, in fact, experience any benefits arising from life in an improving neighborhood.
Table 4 shows results from the specification estimating the effect of membership in the treatment group on the degree of change in concentrated disadvantage experienced over the 1980s—the full results are shown with all control variables, although the coefficient of interest is the effect of membership in the treatment group in the first row. The results in Table 4 are equivalent to the first stage regression results from the two-stage least squares analyses, as represented in equation (4). African Americans in the treatment group experienced a decline in concentrated disadvantage that was 1.04 standard deviations greater than matched members of the control group. To provide a point of reference, the average change among all neighborhoods in the quarter of U.S. neighborhoods that experienced the largest declines in concentrated disadvantage from 1980 to 1990 was roughly one standard deviation on the same scale. A likelihood ratio test comparing the fit of the specification shown in Table 4 to the fit of the identical specification that excludes the variable for membership in the treatment group is highly significant (p < .0001). The large effect of treatment status on actual change in neighborhood conditions means that African Americans living in treatment neighborhoods experienced much of the change occurring in these neighborhoods.
Estimated Effects of Treatment Group Status on Change in Concentrated Disadvantage over the 1980s, African Americans
How does change in an individual’s neighborhood environment affect adult outcomes? Two sets of estimates are generated to address this question, the first set representing the effect of living in a neighborhood that subsequently changes (treatment group effects), the second set representing the effect of actually experiencing a positive change in neighborhood conditions (neighborhood change effects). Because the latter are arguably more relevant for understanding the impact of neighborhood change, interpretation of results is focused on neighborhood change effects, as displayed in the second column of Table 5. For these analyses, the measure of change in concentrated disadvantage over the 1980s is reverse coded, so that a positive coefficient indicates a positive change, or a decline in concentrated disadvantage. Neighborhood change effects can be interpreted as the effect of a one standard deviation decline in concentrated disadvantage on adult outcomes. More specifically, the estimates represent the effect of change in the neighborhood environment arising for those who actually experience some change (Imbens and Angrist 1994). The estimates do not apply to individuals who immediately left an improving neighborhood and therefore experienced no change in their own neighborhood environment. For instance, if families are priced out of a neighborhood that is rapidly gentrifying, that family might not experience any potential benefits of living in a neighborhood where poverty is declining. The estimates from this analysis would not apply to such families. 12
Estimated Effects of a Decline in Concentrated Disadvantage on Adult Economic Outcomes, African Americans
Note: a“Treatment group effects” represent the effect of being in the treatment group relative to the control group. b“Neighborhood change effects” represent the effect of a one standard deviation decline in concentrated disadvantage. cThe untransformed measure of wealth excludes two outliers with over $2,000,000 wealth.
Standard errors in parentheses. *p < .10. **p < .05. ***p < .01.
As shown in the second column of results in Table 5, a one standard deviation decline in concentrated disadvantage is found to increase the adult earnings of African Americans by $3,670 and family income by $5,948. The effect on wealth is also substantively large, but is not significant due to imprecision in the estimate. These initial results provide evidence that positive neighborhood change has a substantial positive effect on adult earnings and family income. The finding that the effect on family income is larger than the effect on individual earnings may indicate that neighborhood change improves individuals’ prospects in the marriage market as well as in the labor market. That is, individuals in improving neighborhoods may be more likely to marry stably employed partners, thus amplifying the positive impact of neighborhood change.
To test this idea and other potential mechanisms, Table 6 considers the effects of neighborhood change on various additional outcomes that may shed light on why a decline in concentrated disadvantage has such beneficial economic effects for individuals. None of the “neighborhood change effect” estimates in Table 6 approach statistical significance, meaning the table provides very little information on why it is that a decline in disadvantage leads to improved economic outcomes as adults. Perhaps most interesting is the finding that improvements in educational attainment do not seem to be the primary explanation. However, it is possible that positive neighborhood change impacts the quality of children’s education through additional resources in the school system—unfortunately it is not possible to test this potential mechanism.
Estimated Effects of a Decline in Concentrated Disadvantage on Various Outcomes, African Americans
Note: a“Treatment group effects” represent the effect of being in the treatment group relative to the control group. b“Neighborhood change effects” represent the effect of a one standard deviation decline in concentrated disadvantage. cMeasures of welfare receipt, “ever married,” and “ever in poor health” are dichotomous; estimated coefficients are from probit regressions.
Standard errors in parentheses. *p < .10. **p < .05. ***p < .01.
To gauge the sensitivity of the results to the selected matching procedure, Table 7 shows results from an additional analysis using a more traditional propensity score matching approach. Models predicting treatment status are estimated using the same set of covariates shown in Table 1, and matches of treatment group members to control group members are accepted within a caliper of .01 on the estimated propensity score. Note that the differences in the matching procedure mean that the estimates are based on a different sample, and thus one should not expect identical results even if both methods produce unbiased estimates. Nevertheless, the pattern of results for economic outcomes is quite similar to that found in Table 4. The primary difference is that the magnitude of the estimated effects of neighborhood change on economic outcomes is smaller when using propensity score matching. Effects on earnings are substantially smaller in magnitude, while estimated effects on family income are quite similar using the propensity score approach as compared with CEM on the selected environment.
Propensity Score Estimates of the Effect a Decline in Concentrated Disadvantage on Various Outcomes, African Americans
Note: Propensity score models use radius matching within a caliper of .01 and common support. aThe untransformed measure of wealth excludes two outliers with over $2,000,000 wealth.
p < .10. **p < .05. ***p < .01.
Replication: Neighborhood Change in the 1990s
The substantive findings from the analysis suggest that declines in concentrated disadvantage may have important effects on economic outcomes, but minimal effects on other social outcomes. The final analysis in this article examines how stable these findings are across time periods by examining the impact of neighborhood change a decade later, in the 1990s. To conduct the replication, every aspect of the analysis is carried out in the same manner, except that the timing of measurement of all variables is pushed back by a decade. The sample thus consists of individuals who were children in PSID households, between the ages of 5 and 15, in 1990, and were observed as household heads or spouses between 2000 and 2007. Because the last survey year available is 2007, the sample for the 1990s replication is smaller, and much younger, than the main analysis examining the impact of change in the 1980s. The treatment under study is living in a neighborhood in 1990 in which concentrated disadvantage declines over the 1990s.
Full results of the 1990s replication are available from the author; here I report only major differences between the 1990s analysis and the 1980s analysis, along with main results on economic outcomes from the replication. Of the 622 African Americans in the treatment group, only 225 (36 percent) are matched successfully. The matched sample is more disadvantaged than the unmatched portion of the sample, as is true in the main analysis. The matched sample is also balanced on neighborhood and MSA characteristics in 1990 and 1980, and there is no evidence of systematic heterogeneity in family background characteristics among members of the treatment group relative to the control group. Differences between the treatment group and control group suggest that, if there is any difference, the treatment group again looks slightly disadvantaged relative to the control group.
In the main analysis, membership in the treatment group leads to an average decline in concentrated disadvantage of 1.04 standard deviations—a substantial change. In the analysis of change in the 1990s, membership in the treatment group leads to an average decline in concentrated disadvantage of .79 standard deviations. This suggests that treatment group members did not experience as much change in their own neighborhood environments as a result of living in an improving neighborhood in 1990. In other words, the treatment is weaker in the 1990s analysis. This finding may mean that displacement of original residents was more prevalent in neighborhoods undergoing economic and demographic change in the 1990s as compared with the 1980s.
Results for economic outcomes are shown in Table 8—again, I focus attention on the “neighborhood change effects” displayed in the second column. Estimated effects of neighborhood change in the 1990s on economic outcomes are in the same direction as effects of change in the 1980s, but are larger in magnitude. A one standard deviation decline in concentrated disadvantage in the 1990s is found to increase the adult earnings of African Americans by $6,325 and family income by $12,528. The effect on wealth is large in magnitude but imprecisely estimated. Not shown in the table are results for other social outcomes. Similar to the analysis of change in the 1980s, I find no strong evidence of neighborhood change effects on education, marriage, or employment. A decline in concentrated disadvantage does have a positive impact on self-reported health in the 1990s, whereas there was no effect in the 1980s
In interpreting the results from the analyses of economic outcomes, the reader should note the wide confidence intervals. Considering the imprecision of the estimates and the discrepancies between the estimates derived from different matching procedures, it is a mistake to place too much emphasis on the magnitude of the effects reported in Table 8. The main conclusion to be taken from these estimates is that, similar to the main analysis, declines in concentrated disadvantage during the 1990s appear to have substantively important effects on adult economic outcomes.
Replication: Estimated Effects of a Decline in Concentrated Disadvantage over the 1990s on Adult Economic Outcomes, African Americans
Note: a“Treatment group effects” represent the effect of being in the treatment group relative to the control group. b“Neighborhood change effects” represent the effect of a one standard deviation decline in concentrated disadvantage.
Standard errors in parentheses. *p < .10. **p < .05. ***p < .01.
Discussion
This article develops a method for estimating the effect of neighborhood change that makes three contributions that can be applied to a wide range of studies examining the impact of social settings on individual outcomes. First, by matching treatment and control group members on neighborhood characteristics and trends of change in the neighborhood prior to the treatment, the method makes it possible to estimate the effects of changes in the neighborhood environment after selection has taken place. While this approach requires its own assumptions, it confronts the primary challenge to observational studies of neighborhood effects, the problem of selection bias. Conditional on the types of neighborhoods in which families live and how those neighborhoods have changed in the recent past, the method relies on the assumption that change occurring after selection has taken place can be considered exogenous. This is a much weaker assumption than that made in most of the observational literature on neighborhood effects, which assumes that neighborhood selection is exogenous conditional on observables.
The second contribution of the method is that, unlike other matching methods, it provides a test that offers suggestive evidence to either weaken or bolster confidence in the assumption of “ignorability”. Because matching is conducted only on neighborhood characteristics, comparisons of family background characteristics of treatment and control group members provide suggestive evidence on whether there is likely to be systematic heterogeneity on a range of observed and unobserved characteristics that might be thought to influence adult outcomes. Third, the method incorporates an instrumental variable strategy that addresses the problem of nonrandom selection out of changing neighborhoods by extending methods for noncompliance utilized in the experimental literature. Under the assumption of ignorable treatment assignment, treatment status serves as a valid instrument for actual change in neighborhood conditions experienced over the decade. Thus, the method allows for estimates of the effect of actual change in individuals’ neighborhood environments regardless of whether the individual remains in the origin neighborhood or leaves.
Under the assumptions outlined above, the method allows for estimates of the impact of neighborhood change using observational data. Considering the limitations of experimental data from residential mobility programs and the difficulty of implementing randomized neighborhood-level interventions, the method has widespread applicability for using observational data to assess how changing communities alter the trajectories of residents within them. But the method also can be applied to study the impact of change in any social setting, including cities or regions, schools or classrooms, residential apartment complexes, or even floors of an apartment complex.
To provide another example of how this method might be used to study change in a different setting, I will describe in nine steps a hypothetical study examining the effects of change in the composition of youth living in a given floor of an apartment complex on original children’s academic test score performance. 13 Specifically, the hypothetical study will analyze whether the school performance of young adults living on a given floor of an apartment complex is affected when a young adult with a criminal record moves into an apartment on the same floor. This study might be seen as a test of the impact of peers in a child’s immediate residential environments. For the sake of simplicity, assume that the study could be conducted with data from a single set of high-rise public housing projects with a relatively high concentration of youth with criminal records. The selection of a sample where there is nontrivial change in the social setting of interest is the first step of the analysis.
The second step is to identify the “treatment” of interest—that is, what type of change in the setting is of most interest—and split the sample into a treatment group located within settings that experienced change and a control group located within settings that did not. The treatment group in the hypothetical study consists of youth in building floors in which a young adult with a criminal record moved in to an apartment on the same floor within the period under study. The control group is composed of youth in the same complex who did not have a youth with a criminal record move to the same floor.
Third, identify a set of key measures of the “selected” setting on which the matching is based and determine the amount of tolerable imbalance between the treatment and control groups on these key measures. Matching in this example might be based on the characteristics of individuals in the treatment group and control group (age of the child, race/ethnicity, economic status), as well as the characteristics of all youth on the individual child’s floor as of the baseline time point, such as the number with a criminal record, the economic and racial composition, and the size of the floor. If available, trends of change in the number of youth on the floor with a criminal record may also be used as a matching variable to indicate whether signs of change on the floor are apparent to residents. It may be reasonable, in this example, to assume that change in the apartment floor is exogenous conditional on the youth’s characteristics and the initial composition of the floor. If the sample was selected from multiple housing complexes across a city, it would be necessary to match based on location and other characteristics of the housing complex as well.
Fourth, conduct the matching and assess whether there are a sufficient number of strata containing members of both the treatment and the control groups—if not, it is unlikely that the method can be applied to study the specific type of change. In this example, the matched sample would consist of individuals in floors that look similar in terms of the number of youth with a criminal record at baseline, the racial and economic composition of the floor, and the characteristics of youth in the treatment and the control groups. Fifth, after matching, assess balance between matched treatment and control group members on variables used in the matching analysis as well as any variables not used in the analysis. The assessment of balance between treatment and control group members on variables in the matching analysis simply tells the analyst how much balance has improved due to matching. The assessment of balance on variables not included in the matching provides a sense of whether there is systematic heterogeneity between members of the matched treatment and control groups. If so, it may reveal that the central assumptions of the matching method are invalid. For instance, if youth in the treatment group had lower test scores, were from poorer families, or were more likely to have a criminal record at baseline than youth in the control group, one might suspect that the assumption of ignorable treatment assignment is not valid. It could be that the entrance of youth with a criminal record to a given floor of an apartment complex is not random, but instead is driven by social networks within the complex or other unobserved factors. In this case, the central assumptions of the method would be violated and it is unlikely that the study could be carried out in a convincing manner.
The sixth step of the analysis is to assess any differences between sample members who are successfully matched versus those who are not. The results from this assessment allow for a more refined characterization of the sample or population to whom the study’s results pertain. For example, there may be certain buildings within a complex of high-rises in which a large majority of youth with criminal records reside. If this is the case, the comparison of matched to unmatched sample members would suggest that the results pertain only to the residents of this subset of building within the wider complex.
Seventh, analyze the effects of membership in the treatment group on a given outcome of interest, measured after the change has taken place. This step can be done while correcting for remaining imbalance between the treatment and control groups if necessary. In this application, one might regress individuals’ standardized test scores at the end of the study on treatment group status, while controlling for baseline characteristics of the child, the family, the floor, and the building. This analysis would not consider whether the individual child remained living in the floor during the period in which a young adult with a criminal record entered. The eighth step, if appropriate, is to conduct a two-stage least squares analysis using treatment group status as an instrument for experienced change in order to estimate “setting change effects”. Again, this step will depend on whether a nontrivial number of sample members have the chance to leave the setting of interest, and whether the assumptions for a valid instrument are met. In this example, membership in the treatment group would be used to instrument for actually remaining in the apartment floor in which a young adult with a criminal record entered. The ninth and final step of the analysis would be to consider potential violations of the central assumptions of the method and interpret results.
This example outlines the set of suggested steps necessary to carry out an analysis of change in a given social setting. Applying the method to study the impact of neighborhood change in the 1980s, the central substantive finding from the analysis is that living in a neighborhood that undergoes a decline in concentrated disadvantage has positive effects on multiple measures of adult economic status among African Americans, suggesting a link between positive neighborhood change and economic mobility. A one standard deviation decline in concentrated disadvantage, which is roughly the average amount of change experienced in the quarter of U.S. neighborhoods experiencing the most change over the decade, is found to increase adult earnings by $3,700 and adult family income by almost $6,000. The effects on other adult outcomes, such as education and marriage, are estimated less precisely and do not provide a clear explanation of the mechanisms leading to improved economic status. Effects on economic outcomes are similar in a replication analyzing neighborhood change in the 1990s.
The major limitation of the analysis is that it provides no information on the mechanisms at the neighborhood level that bring about such positive results. Data available on census tract composition tell us nothing about how the demand side of the labor market changed over the decade, how institutions such as the schools and the police departments changed, or about other important changes in the neighborhood. Because it is not possible to follow individuals across censuses, the study cannot distinguish between change in neighborhood composition due to in-migration from change due to the economic circumstances of the original neighborhood residents.
Without knowledge of the specific trends and forces within each neighborhood across the country, one can only speculate that the “treatment” under study is likely a mix of a diverse set of shocks that alter neighborhood conditions. Most change is likely due to either changes in neighborhood composition—such as the movement of immigrants or higher-income residents into urban neighborhoods—or to changes in local economic conditions arising from either specific shocks, such as the entrance of a large employer or a number of smaller employers into the area, or a more general downturn or upturn in the labor market. However, there are undoubtedly other, more specific shocks that affect individual neighborhoods. For instance, one can imagine a scenario where an elite charter school moves into a child’s neighborhood, improving the child’s own education and also causing affluent people to move into the neighborhood. 14 The treatment effects estimated here would include effects arising from this type of shock, despite the fact that the key mechanism is the new school as opposed to the change in the neighborhood. The more general limitation is that the treatment effect estimates should be thought of as pooling the effects of a range of different shocks, all of which alter neighborhood conditions in broadly similar ways.
In addition to the absence of information on mechanisms, the analysis relies on the crucial assumption that the potential outcomes of individuals in neighborhoods where concentrated disadvantage did and did not decline are the same, conditional on neighborhood characteristics and trends of change in the neighborhood. Intuitively, one must assume that individuals are not differentially equipped to pick up subtle cues about impending change in the neighborhood or to make better bets about how their environment is likely to change in the future. The evidence available suggests that individuals in neighborhoods where concentrated disadvantage declined were not advantaged in any observable way. However, it is possible that individuals may have unobservable advantages, such as better-informed social networks or a more refined understanding of the housing market, and these advantages may predict both whether the individual lives in a neighborhood that experiences improvement and individual economic and social outcomes. This is a threat to the central assumptions of the analysis that is impossible to test definitively.
With these limitations in mind, the method developed has potentially broad applicability for studying the effect of changes in any social setting on individuals within that setting. A similar approach could be used to examine any of the following questions: How do changes in a school’s racial composition affect the educational trajectories of children from different racial and ethnic groups? How did deindustrialization in rustbelt cities affect working families living within them? How does a shift in the gender composition of a workplace influence the trajectories of male and female employees? By combining matching with the instrumental variable strategy, the method provides a guide for responding to these and other similar questions while confronting the distinct problems of bias arising from nonrandom selection into and out of each social setting.
Footnotes
Acknowledgements
I would like to thank Larry Aber, Dalton Conley, Ingrid Gould Ellen, Bryan Graham, Jennifer Hill, Christopher Jencks, Dohoon Lee, Stephen Morgan, Kathy O’Regan, Robert Sampson, Marc Scott, Florencia Torche, William Julius Wilson, Chris Winship, Scott Winship, and anonymous reviewers for helpful comments and feedback. I am grateful to Donna Nordquist for assistance with the geocode files from the PSID.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Part of the research conducted for this study was supported by a grant from Harvard’s Institute for Quantitative Social Science.