Abstract
Despite the importance of social capital to political science research, conventional means of measuring it are subject to a range of problems, including nonresponse bias, declining validity over time, and/or a lack of conceptual coherence. We argue that, in the case of the United States, rates of response to the decennial census represent a powerful yet overlooked measure for aggregate social capital. In this research note, we elaborate a theoretical rationale for the measure and empirically validate it, showing across multiple data sets and levels of geographic aggregation that census response rates (CRR) strongly predict various dimensions of social capital. Our findings highlight an important opportunity for social capital scholars to use existing governmental data to better measure geospatial variation in a key social science construct.
In this special edition research note, we explore the use of census response rate (CRR) data collected by the U.S. Census Bureau as a measure of aggregate social capital. We select this topic because the rate of response to the Census is currently understudied; yet, as our analysis will demonstrate, it is a particularly powerful proxy measure of community norms of social trust and reciprocity, as well as engagement in community social and organizational life. Beyond the virtue of being readily available and free to use, census response is a direct behavioral measure of these concepts, rather than a self-reported measure subject to nonresponse bias, social desirability, or other forms of bias. Likewise, use of this single measure is much more parsimonious in capturing aggregate levels of social capital than prior efforts that measure the phenomenon through multiple contextual indicators of known relevance to trust (e.g., residential mobility rates, commuting times, education levels, racial diversity, etc.). Furthermore, as CRR data are derived from a census, there is no sampling error; this allows for precision at very low levels of aggregation. For all of these reasons, we believe CRRs represent a boon to scholars of social capital and merit closer attention.
In the following sections, we provide a brief theoretical foundation for the correspondence between the phenomena of social trust and census response, including a review of prior work employing the measure. Next, we elaborate on the methodological virtues of this measure over others. We follow this section with an analysis that proceeds in three parts. First, utilizing the 2000 Social Capital Benchmark Survey (SCBS) and the 2006 Social Capital Community Survey (SCCS), we demonstrate that CRRs calculated at the census tract level significantly predict tract-level estimates of social capital. Second, we corroborate these results with the 1994-1995 Project on Human Development in Chicago Neighborhoods (PHDCN) Community Survey, where we demonstrate that CRRs calculated at the neighborhood level significantly predict various measures of neighborhood social capital. Last, we demonstrate that state-level estimates of CRR strongly correlate with established measures of state-level social capital. We conclude by discussing possible applications for this measure across various areas of research in political science.
Capturing Social Capital With Census Response
At its core, research on social capital is premised on the idea that healthy democratic societies rest on a bedrock culture of civic and social engagement. Putnam defines social capital in terms of a community’s interconnected social networks, as well as community-wide norms of reciprocity and trustworthiness (Putnam, 1995). The more interconnected the community’s social networks, the more its citizens are able to overcome problems of collective action and effectively lobby government for their interests. Of the many contributions offered by the social capital literature, one has been the measurement of aggregate levels of social capital. Such measures enable researchers to compare social capital across neighborhoods, cities, states, nations, and so on; to evaluate the antecedents of aggregate social capital (Keele, 2005); and perhaps most importantly, to employ such measures in explaining spatial variation in a variety of outcomes of interest, such as political trust (Keele, 2007), racial inequalities in education, incarceration, and political participation rates (Hero, 2003), economic development (Grootaert & Van Bastelaer, 2002), and government performance (Andrews, 2011; Knack, 2002; Knack & Keefer, 1997; Putnam, Leonardi, & Nanetti, 1993).
Despite its conceptual importance, the means used to measure the concept have drawn significant criticism. Aggregate measures of social capital are typically constructed from individual-level data. For example, an initial measure offered by Putnam et al. (1993) in his landmark study of Italian regions aggregates survey respondents’ self-report of their involvement with social or civic organizations as a measure of the level of civic engagement in a community. One criticism of this type of measure, however, is that it fails to capture an important generational change in the mode of engagement, and thus misrepresents “civic decline,” as younger cohorts of citizens in the 21st century have actually remained engaged in civic life, but engage in less formal organizations and participate through less conventional mediums (see Dalton, 2008). Thus, this type of approach to measuring aggregate social capital may no longer capture the full breadth of civic life. As an alternative, scholars have employed measures based on aggregation of individually reported levels of social trust, which may share similar weaknesses (e.g., Keele, 2005, 2007).
An even deeper problem with both of these aforementioned measures of social capital is that, given their reliance on individual-level survey data, the measures are subject to nonresponse bias. Roughly concurrent with the decline in social capital and trust is a decline in survey response rates (Curtin, Presser, & Singer, 2005). Indeed, the individual-level decision to respond to surveys has been shown to be associated with precisely the sort of prosocial attitudes and behaviors that are meant to be captured by survey items designed to measure social capital. For example, Keeter, Kennedy, Dimock, Best, and Craighill (2006) find evidence that a survey with comparatively higher levels of nonresponse leads to overestimates of social trust, the strength of partisanship, voter turnout, and other items indicating political interest and participation.
The potential problem of nonresponse bias, however, can be turned into an asset; if the likelihood of response is associated with prosocial, trusting, civic-oriented behavior, then measuring variation in aggregate response rates to surveys themselves may be an optimal strategy for capturing social trust and reciprocity. Given the scope and sheer number of respondents, we argue that the U.S. Census represents an ideal source of information about aggregate levels of survey response and thus a measure of social capital. Ours is not the first study to suggest a relationship between census response and social capital. A study of the 1990 census found that individual census response was associated with belonging to a civic organization (Couper, Singer, & Kulka, 1998). Prior work has argued that the Census is a type of public good, and thus census response is a form of social cooperation in the production of this good (Thompson, 1991); as such, census response and spatial variation in response rates can be viewed as capturing variation in “civic” or “cooperative” norms in a given community (Knack, 2002; Knack & Kropf, 1998).
Past work has attempted to validate this line of reasoning empirically by demonstrating that county-level CRR serves as a significant predictor of individual-level generalized social trust, as well as individual voter turnout (Knack & Kropf, 1998). This earlier work, however, leaves significant opportunity for more thorough validation. For one, as more recent research on social context has shown, the U.S. county is too large, and thus too heterogeneous, a geographic unit to effectively measure local community or neighborhood context (e.g., Oliver & Mendelberg, 2000; Oliver & Wong, 2003). Thus, the attempt of Knack and Kropf (1998) to validate the use of CRR as a measure of social capital via its use as a contextual predictor of individual attitudes and behavior leaves a significant degree of uncertainty about their results given their use of county-level data. To be sure, the existence of intracounty variation in census response at the tract level 1 leaves open to question whether individual respondents actually reside in neighborhoods with response rates on par with those of their county as a whole, and thus, whether county-level correlations between response rates and outcomes of interest are driven by an omitted variable.
In short, one definite improvement over past research, and a step forward in the validation of the use of CRRs as a measure of aggregate social capital, is to evaluate census response at a smaller level of geographic aggregation approximating a “neighborhood,” such as the census tract. 2 Indeed, such a study could determine whether the observed relationship found by Knack and Kropf (1998) between generalized social trust and county CRRs hold when using tract-level response rates. An additional concern is that Knack and Kropf rely on a measure of generalized social trust from the 1992 National Election Study (NES), which taps trust in “people in general.” With the release of data such as the 1994-1995 PHDCN Community Survey and the 2000 and 2006 Social Capital Surveys, however, scholars were provided with large-N data sets containing a wealth of finer grained attitudinal and behavioral measures, such as trust specifically in one’s neighbors, self-reported level of interaction with one’s neighbors, and perhaps most importantly, perceived community-level civic cooperation. Usage of such finer grained items in validity tests represents yet another avenue for advancing our understanding of the validity of CRRs as a measure of community social capital, particularly if relationships are empirically assessed between such measures and tract-level response rate data.
Social Capital Surveys
As an initial validation effort, we drew on the 2000 SCBS and the 2006 SCCS 3 to generate census tract-level estimates for each survey of several constructs theorized to capture social capital. One benefit of using these surveys is that they contain identical measures of our variables of interest and thus afford a unique opportunity for replication tests. In line with prior theorizing about the core dimensions of social capital, we focus on neighborhood-specific measures of interpersonal trust, informal sociability, and community organizational life (Putnam, 2000), as well as collective efficacy (Sampson, Raudenbush, & Earls, 1997). Furthermore, as Putnam et al. (1993) conceptualization of neighborhood social capital placed importance on the existence of interpersonal trust that “facilitates coordination and cooperation for mutual benefit,” we also measured perceived neighborhood social cooperation. 4 For each of these variables, we generated tract-level estimates from individual survey response data using multilevel regression and poststratification (MRP) 5 (Park, Gelman, & Bafumi, 2004; Warshaw & Rodden, 2012). For each of the n = 9,981 census tracts in the 2000 SCBS and the n = 4,638 census tracts in the 2006 SCCS, the MRP procedure yielded estimates of the percentage of residents within each tract that trust their neighbors (Trust in Neighbors), talk to or visit with their neighbors several times a month or more (Interaction With Neighbors), perceive themselves as having an impact in making their community a better place to live (Collective Efficacy), perceive it as likely that people in their community would cooperate to save water or electricity in the midst of an emergency (Perceived Cooperation), and participate in a neighborhood/block/homeowner association or crime watch group (Neighborhood Association). In addition, for each survey, we created a composite measure of social capital at the tract level comprised of tract-level estimates of neighborhood trust, interaction, collective efficacy, and perceived cooperation. 6
As estimates yielded from the MRP procedure are percentage point estimates (e.g., the percentage of individuals within a tract that trust their neighbors), we estimated regression models with beta-distributed dependent variables. Tables 1 and 2 present the results from bivariate and multivariate regression analyses of the effect of tract-level CRR obtained from the 2000 Decennial Census 7 on our tract-level social capital estimates. For ease of interpretation, CRR and all other tract-level controls were recoded to range from 0 to 1. In each table, for each dependent variable, the first column presents the beta coefficient for CRR from a bivariate regression, whereas the second column presents beta coefficients from multivariate analyses including a host of contextual control variables of established importance in predicting social capital (e.g. Putnam, 2007). As information about median household income, education, gender, and race (e.g., White, Black, Asian) were incorporated into our MRP model estimating each aggregate social capital outcome measure, we exclude measures of these as controls in the multivariate regression analyses presented here.
The Impact of Tract-Level Census Response Rate on Tract-Level Estimates of Social Capital (2000 SCBS).
Note. Entries in columns 1 to 5 are coefficients and standard errors from bivariate and multivariate maximum likelihood regression models with beta-distributed dependent variables. Entries in column 6 are coefficients and robust standard errors from bivariate and multivariate general linear models (GLM) with logit link functions. Δ pred (Y) indicates the change in the predicted value of each dependent variable associated with a one unit change in CRR. SCBS = Social Capital Benchmark Survey.
p < .05. **p < .01. ***p < .001, based on two-tailed hypothesis tests.
The Impact of Tract-Level Census Response Rate on Tract-Level Estimates of Social Capital (2006 SCCS).
Note. Entries in columns 1 to 5 are coefficients and standard errors from bivariate and multivariate maximum likelihood regression models with beta-distributed dependent variables. Entries in column 6 are coefficients and robust standard errors from bivariate and multivariate general linear models (GLM) with logit link functions. Δ pred (Y) indicates the change in the predicted value of each dependent variable associated with a one unit change in CRR. SCCS = Social Capital Community Survey.
p < .05. **p < .01. ***p < .001, based on two-tailed hypothesis tests.
The results across Tables 1 and 2 reveal that an increase in CRR is associated with a significant increase in all measured dimensions of tract-level social capital. The bottom row of each table presents the magnitude of the effect of CRR on each variable; as can be seen, CRR exerts substantively meaningful effects, though the magnitude of these effects are diminished with the addition of controls. Across both data sets, CRR exerts the largest effect on neighborhood trust; indeed, after controlling for many tract-level characteristics, a 0 to 1 increase in CRR is associated with 21 and 15 percentage point increases in trust. Furthermore, in looking at the effect of CRR on Social Capital Scores, we see that CRR exerts extremely large effects in the bivariate models and moderately large effects in the multivariate models. These initial analyses provide unprecedented evidence of the connection between CRR and neighborhood social capital. Across two separate surveys and several measures capturing distinct theoretical dimensions of social capital, we find that CRR is strongly predictive of social capital. What is important to reiterate is that the effect of CRR is observed after controlling for a substantial range of relevant contextual variables, thus mitigating concern that response rates might be capturing the effect of an omitted variable. 8
1994-1995 PHDCN Community Survey
To provide additional validation for the use of CRR as a measure of social capital, we draw on the 1994-1995 PHDCN Community Survey to assess whether CRRs serve as a significant predictor of neighborhood social capital. The 1994-1995 Chicago Community Survey is comprised of a representative sample of n = 8,782 Chicago city residents residing across n = 343 “neighborhood clusters” (each of which may contain multiple census tracts). 9 Within each neighborhood cluster, the principal investigators of this survey aggregated individual responses to various social capital–related questions to generate various measures of social capital at the neighborhood cluster level. In addition to providing a distinct means to replicate our findings, these data have other desirable qualities that strengthen the validity of our findings. While our prior analysis drew on data gathered from large but more sparsely sampled surveys that required MRP to get meaningful estimates at the appropriate level of aggregation, these estimates are based on responses to a geographically focused survey sampled so as to be representative at a low level of aggregation (i.e., the “neighborhood cluster”). This means they have the benefit of a larger average number of respondents per cluster (n = 25) than the average number of respondents per tract in our prior analyses.
We utilize six separate measures constructed by Earls, Brooks-Gunn, Raudenbush, and Sampson (1994-1995) to capture neighborhood civic life: (a) Social Capital, (b) Social Ties, (c) Social Cohesion, (d) Social Control, (e) Organizational Participation, and (f) Neighborhood Activism. In addition, to conduct a discriminant validity test, we also draw on a constructed aggregate measure of neighborhood (a) Social Disorder and (b) Social Anomie. Each measure is a scale comprised of selected survey questions, and individual scale values were aggregated up to the neighborhood cluster level to generate cluster estimates of these measures. Social Capital, as constructed by the survey authors, is comprised of items tapping how acquainted adults are with children in the neighborhood, as well as the extent to which adults cooperate to look after neighborhood children. Social Ties measures the extent of informal social interaction and in-home visitation within a neighborhood, and Social Cohesion measures the intimacy of neighborly relations and levels of trust and shared values among people in a neighborhood. Social Control taps into the extent of collective action within a neighborhood to exert control over children, deviant behaviors, and to maintain community public goods, such as a fire station. Organizational Participation measures the extent of organizational activities (e.g., religious, neighborhood watch, civic, Ward meetings, etc.) in the neighborhood, and Neighborhood Activism measures the extent to which people in a neighborhood report contact with local politicians and participated in efforts to address problems in the neighborhood. Social Disorder captures the extent to which residents in a neighborhood report litter, broken bottles, graffiti, deserted houses and storefronts, and drug and alcohol use as a problem in the neighborhood. Last, Social Anomie measures the extent to which residents in a neighborhood report a lack of concern for law and order, toleration of disorderly conduct, and endorsement of hedonistic values. 10
It is important to reiterate that these measures were constructed by the authors of the Chicago Community Survey; for our present purpose, we use them as “convenience measures” enabling us to perform additional tests of whether CRRs positively correlate to various dimensions of neighborhood social capital, as well as whether CRRs negatively correlate with aggregate perceived disorder or reported social anomie. Each scale is continuous and was recoded to range from 0 to 1. To assess the impact of CRRs on these various measures, we estimated the 1990 CRR of each neighborhood cluster by taking the average of the CRR for each census tract included in each cluster and weighting each tract CRR estimates by the total population in that specific tract relative to the total population in the neighborhood cluster. The resultant neighborhood cluster-level measure of CRR is the weighted average CRR for the tracts in each neighborhood cluster; for ease of interpretation, this variable, along with all other control variables, was recoded to range from 0 to 1. We estimated eight separate models—one for each dependent variable—and present the results in Figure 1. Each of the eight models included cluster-level controls for median income, education levels, unemployment, percent Black, percent foreign born, homeownership rates, residential mobility and long distance commuting, reported crime victimization, and total population. 11
Census response rate and social capital in Chicago neighborhoods (1994-1995 Chicago Community Survey).
The results in Figure 1 reveal that an increase in CRR in a neighborhood is associated with a significant increase in all measured dimensions of neighborhood social capital. 12 In addition, the results reveal that an increase in CRR is associated with a significant decrease in aggregate perceptions of social disorder and reported levels of social anomie. These findings serve as a source of convergent and discriminant validity for the use of CRR as an indicator of social capital. These results also serve to corroborate those from the Social Capital Surveys using a well-known survey in the social capital literature. Moreover, they demonstrate that the systematic relationship of CRR to social capital holds when using different measures of social capital, different units of geographic analysis, and in both nationally sampled data sets and a more focused analysis of a specific urban area.
State-Level CRR and Current State Social Capital Indices
As a final effort to validate the use of CRR as a measure of social capital, we assess the correlation between CRR at the state level and existing measures of state-level social capital. While the previous analyses demonstrate the link between CRR and social capital at the neighborhood level, this analysis will test whether tract-level CRR data, when aggregated up to the state level, relate to existing social capital measures at the state level. Although one of the strengths of using CRR is its low level of aggregation, allowing for fine-grained measurement and a much higher degree of observable variation, it is worth demonstrating the robustness of this measure at higher levels of aggregation by validating it against prior measures of state social capital. For this analysis, we rely on Putnam’s (2000) state-level measure of social capital in 2000, as well as a more recent state-level measure offered by Hawes et al. (2013). Figure 2 presents the correlation between state CRR and each of these state-level measures. Starting with Putnam’s measure (Panel A), we observe a very strong correlation (r = .73). Turning to Panel B, we observe a more modest, yet moderately high correlation between CRR and state social capital (r = .54). One possible explanation for these differences lies in the divergence between Putnam’s and Hawes and colleagues’ measures. For example, Putnam’s measure includes five subcomponents of social capital, including social trust, whereas Hawes and colleagues’ measure includes only three of these five components, and perhaps most important, lacks measures of social trust and informal sociability. Indeed, the correlation between the measures offered by Hawes and colleagues and that by Putnam in the year 2000 correlate at .727. In the end, arbitrating between these two existing measures is beyond the scope of our analysis; rather, our core prerogative is to assess how well state-level CRR relates to established measures. The correlations presented in Figure 2 demonstrate a moderate to very strong relationships between CRR and social capital at the state level, thus illustrating the usefulness of CRR as a measure of social capital available for analyses of state-level outcomes of interest to social and political scientists.
Correlation between state-level census response rate and existing social capital measures in year 2000.
Discussion
In summary, we argue that CRR represents an untapped resource for researchers to use as a measure of social capital. The results of this article provide strong evidence for the validity of the measure. We show that census response is associated with aggregated measures of many different self-reported behaviors and attitudes that directly indicate social capital in a community. We replicate this finding across three large data sets, and a broad variety of communities throughout the United States using the best available means of aggregating these survey responses to allow for accurate validation at the neighborhood level. We further show that CRR is a valid measure of social capital at the state level. Of course, CRR data would be far less interesting if its properties were simply as good as prior measures. We contend that CRR represents a vast improvement on the state of the art. As a direct measure of behavior, it is not susceptible to nonresponse bias, thus separating it from existing aggregate measures based on individual responses to surveys. As the product of a government census, CRR counts are not sampled and thus not susceptible to sampling error like prior measures based on individual-level survey responses. And most important, CRR allows for precise measurement at a significantly lower level of aggregation than prior measures, without the need to independently mount an exceedingly costly social survey project: The data are already being created (and updated, presumably in perpetuity) as a by-product of a constitutional obligation of the U.S. government to enumerate its citizens. One distinct benefit of this is that scholars interested in analyses of outcomes occurring at the local level (e.g., neighborhood, city, county) who wish to analyze social capital as either a predictor or outcome have a geographically complete and precise measure. This allows for scholars to avoid the problem of having missing data for geographic cases present in their own data but absent in survey data on which social capital estimates are derived. Furthermore, although only collected every 10 years, CRRs are quite stable over time, enabling scholars to interpolate values for analyses with data collected between decennial census years. 13
Applications in Future Research
Having made significant advances in the validation of CRRs as a measure of aggregate social capital, we see several possible applications for these data in future research. For example, in the area of political participation, the availability of response rate data at varying levels of locality could be used to help clarify the role of cooperative context in shaping the level and varying types of participation (electoral, extraelectoral, protest) observed across communities. Response rate data could also be employed in explorations of the linkage between diversity and social capital. To be sure, a staple finding in past research is that diversity erodes social capital; the availability of response rate data at varying geographic levels, however, enables scholars to retest such effects nationally using tract and county levels of aggregation, and to assess at the individual level whether “cooperative norms” at the community level condition the impact of racial diversification. Lastly, scholars could employ response rates as moderating variables in the exploration of the effects of various exogenous shocks to society, such as natural disasters (e.g., Hurricane Katrina or Sandy) or more recent, government failures (e.g., the 2013 Government Shutdown). The potential negative effects of events such as these, that disrupt the normal operation of social institutions, may be shaped by variation across communities in social capital.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.