The Trouble With Types: A Partial Test of the Validity of Membership Association Content as a Proxy for Structure

Literature Review

Inconsistent Findings From Proxy Analyses

Although reasonable justifications are made for each proxy variable, findings are not cumulating across studies. A few content types, such as unions and political parties, consistently predict political participation. Many others have effects that vary dramatically across studies. Table 1 lists content types with inconsistent effects on political participation across studies (for a full listing of types and effects, see online appendix Table O2). Ten different types sometimes have positive effects on participation and sometimes have no effect (animal rights, environment, handicapped advocacy, hobbies, music, parents, religion, politics, sports, and women)—and two of those sometimes have negative effects as well (sports, religion). Any pairing of inconsistent findings might be explained by survey locations, sampling frames, or other technical matters, but the number of inconsistencies suggests a broader concern.

OPEN IN VIEWER

Table 1.

Association Types With Inconsistent Effects on Political Participation by Study.

Study & sample characteristics	Political participation is encouraged by. . .	Political participation is not impacted by. . .	Political participation is reduced by. . .
McFarland and Thomas (2006), United States, adults (associations from high school, NELS)	Music, Religious	Environment, Political, Sports
McFarland and Thomas (2006),United States, adults (associations from high school, Add Health)	Music	Music, Sports	Sports
Quintelier (2008),Belgium, youth	Environment, Music, Religious	Hobby, Sports
Schultz and Bailer (2012),Switzerland, adults	Animal Rights, Environment, Parents, Women	Parents, Religious
Stolle and Rochon (1998),United States, Germany, Sweden, adults	Environmental, Hobby, Music, Religious, Parents, Political, Sports	Animal Rights, Handicapped, Women
van der Meer and van Ingen (2009),17 European countries, adults	Environment, Sports
Wollebaek and Selle (2002),Norway, adults	Handicapped, Hobby, Music, Sports		Religious

Note. Types in the table are only those that have inconsistent effects across studies (i.e., appear in multiple columns of the table). Types with consistent effects (i.e., would appear in only one column) are omitted. A table with the full listing appears in the supplemental materials online. Types that appear in all three columns are in boldface type. McFarland and Thomas (2006) conduct similar analyses on two different data sets: the National Educational Longitudinal Study (NELS) and the National Longitudinal Study of Adolescent Health (Add Health). The two different data sources have two different typologies, so the results are listed separately here.

Inconsistencies in the literature are at least partly a function of the proliferation of typologies with differing placements of particular types (see online appendix Table O3 for all typologies by study). For example, consider hobby groups (such as knitting circles). Hooghe and Quintelier (2013) use hobby groups as a standalone category. Van Deth (2006) does the same, but his survey asked about hobby groups along with “special interest clubs” such as fan clubs. Quintelier (2008) creates a secondary typology combining hobby groups with sports associations (calling them “expressive” organizations). Schulz and Bailer (2012) combine hobby groups with sports, youth, clubs, culture, and music associations (dubbed “activation” organizations). Wollebaek and Selle (2002) include hobby groups with sports, music, and fraternities as “nonpolitical culture and recreation.” Each choice is reasonable, but the changing placement makes it impossible to look across studies for the effects of hobby groups.

Religious groups are a rare point of partial consistency. Most studies distinguish religious organizations from all others—although sometimes religion includes congregations and sometimes only includes noncongregation religious associations. Nevertheless, the effects of religious associations can be roughly compared across several studies—and the findings are mixed. Some find that religious organizations foster political engagement (McFarland & Thomas, 2006; Quintelier, 2008; van Deth, 2006; Verba et al., 1995); others find no effect (Hooghe & Quintelier, 2013; Maloney et al., 2008; Schulz & Bailer, 2012); and others find negative effects (van Deth, 2006; Wollebaek & Selle, 2002). Despite the repeated appearance of religious groups, it is unclear what effect they have on citizenship socialization.

Why are findings inconsistent, even when types are similar? We suspect the problem is the assumption underlying content proxies: that associations pursuing similar content have similar structures—or at least that there is more structural variation between content types than within them (e.g., the average political group will differ structurally from the average hobby group). As noted above, organization-level studies suggest that within-type variability might actually be quite high—potentially higher than between types (Andrews & Edwards, 2005; Andrews et al., 2010; Baggetta, 2009; Binder & Wood, 2013; Chaves, 2004; B. Edwards & Foley, 2003; McCarthy & Wolfson, 1996). If that is true, then content typology categories will contain different mixes of structures with each new survey sample—and results from analyses connecting content types to political participation outcomes will vary randomly across studies.

So, are assumptions of within-type similarity warranted? How much do content types really tell us about structures? We turn now to an empirical analysis to address these questions.

Data and Measures

To assess the validity of content as a structural proxy, data are needed on socialization structures in a large sample of organizations from a wide range of content types. Although many organization-level studies exist, none capture a broad enough pool with sufficient depth. Some select organizations from only a few content types (e.g., Andrews & Edwards, 2005; Baggetta, 2009; Knoke, 1990; Stolle, 2001). Others limit themselves to very large organizations, national organizations, or organizations active in Washington, DC, overlooking smaller, purely local, and nonpolitical groups (e.g., Berry, 1977; Schlozman & Tierney, 1986; Skocpol, 2003; J. L. Walker, 1991). Data from Internal Revenue Service 990 forms filed by U.S. nonprofit organizations or Encyclopedia of Associations entries capture many organizations from many types, but have little data on structures—especially for local subunits of national or regional federations, whose details are subsumed under the national organization’s files (e.g., Minkoff, 1995). The most representative sample assembled, the Nebraska community study (see McPherson & Smith-Lovin, 1986), collected data on member demographics and not on socialization structures.

In the absence of a suitable organization-level data set, we instead leverage data collected at the individual level. We reexamine data from the ACPS (Verba, Schlozman, Brady, & Nie, 1990)—the only large-scale survey we have identified that includes measures of content and structure for a wide array of associations. Administered in 1990, the ACPS was conducted with a nationally representative sample of individuals, but its data can be reorganized to study organizations. ACPS respondents were asked if they were a member of each of 20 different content-based types of associations. For any type where a respondent said they were a member, interviewers asked for the name of the association (respondents with multiple memberships in a type named the association they were most involved with). For each named organization, respondents were asked seven questions revealing something about its citizenship socialization structure. Questions establish whether an organization creates opportunities for

Although not exhaustive, the concepts reflect a range of structures identified in the associations literature (see Baggetta, 2009; Fung, 2003; Warren, 2001).

The association questions produce a data matrix where individuals are rows and organization structures are columns. ³ A respondent could answer all seven items for one organization in each of the 20 types. We reorganize this 140-variable battery to treat each organization reported on as a case (N = 5,371). In our data matrix, rows are organizations and columns are the seven structures. In effect, we treat respondents as key informants reporting on their organizations. A member of all 20 types of organizations would contribute 20 cases to our data set; a respondent with no memberships contributes none. The actual number of organizations per respondent ranges from 0 to 15, with 0 most common (28%) followed by 1 (20%). Only 20% of respondents reported on more than four groups.

Our approach treats the ACPS as if it were a hypernetwork sample of organizations (McPherson, 1982). Hypernetwork sampling identifies random samples of organizations by asking randomly sampled individuals about their associations (e.g., Chaves, 2004). Reorganizing the ACPS data does not produce a true hypernetwork sample because the ACPS oversampled racial/ethnic minorities and political activists (for details, see Verba et al., 1995, Appendix A). This means our reorganized data set will not provide a perfect picture of the relative prevalence of content types in the United States. It will likely include more organizations, more politically engaged organizations, and more groups in the “nationality, ethnic, or racial group” category than a true random sample (see appendix Table A1 for the observed distribution by type).

Oversampling organizations in certain content types is of limited analytic concern; our estimates of average structures in those types should be marginally more precise. Oversamples of organizations with certain structures would be more problematic, and the fact that respondents reported on the organization they were most involved with in each type raises this concern. It is likely that in multimembership types, the organization on which the respondent reported had more-participatory structures than the others. If many respondents were in multiple organizations in each type, this would produce an oversample of more-participatory organizations. They could potentially increase the apparent variability of types that, in reality, have quite low levels of member participation, as extra participatory groups are added to those types. ⁴ Fortunately, rates of multiple membership within types is quite low in the survey (see appendix Table A2). Of the 2,517 respondents, fewer than 100 had multiple memberships in 12 of the 20 types and fewer than 200 had multiple memberships in 18 of the 20 types. Only business/professional associations (251) and social service organizations (602) have more than 200 respondents with multiple memberships. All types have average memberships per respondent between 1.1 and 1.8, with the exception of social service organizations (2.4). These patterns suggest that the vast majority of the responses we have come from respondents’ only organizational affiliation within a category. Any bias toward participatory forms at the organization level due to oversampling is likely to be small.

In general, then, the sampling strategy should not have substantially over- or undersampled organizations with certain structures within each content type, despite likely oversampling certain content types. No type has fewer than 32 cases within it and most have well over 100, giving us confidence that our analyses provide reasonable estimates of within- and between-type structural variability.

Although the ACPS can be reorganized, it was not designed for organization-level analysis. Because of this, items vary in the appropriateness of question wording and the number of respondents asked. Two questions have appropriate wording and were asked of all respondents:

Positive responses indicate that the organization offers civic skill development opportunities (members serving on committees, working on projects, or organizing meetings) and has member meetings that the respondent has attended. A negative response could mean the organization does not offer these civic skill opportunities, or it does, but the respondent has not done them; missing data indicates the group does not have member meetings or the respondent has not attended one. Our data will undercount this structure.

These undercounts are frustrating realities of repurposing the ACPS. The question formats likely prevent us from deriving precise estimates of the prevalence of socialization structures in U.S. associations in 1990. They do not, however, preclude us assessing the validity of types as structural proxies because our analysis examines structural variability within types. The proxy approach assumes low levels of within-type structural variation. If undercounting produces artificially inflated variability within types, our analyses would run the risk of understating the structural uniqueness of content types. To determine if and how undercounting was likely to bias observed levels of within-type variation, we ran 100 Monte Carlo simulations for each type, adding in randomly generated additional structures to organizations in each round, and then averaging across simulations (see Appendix A3 for details). We then compared the variances of the average simulated types with our observed types (see Appendix Table A3), finding that all but three types (cultural, neighborhood, and political) were statistically indistinguishable—and the three distinguishable types all had higher variances in the Monte Carlo simulations than in our observed data (i.e., undercounting in the observed types likely reduced within-type structural variability for those types). In only one of the 20 types was the variance estimate larger in the observed data (service/fraternal), but the difference was very small and statistically indistinguishable from the Monte Carlo estimate. These analyses suggest that if the undercounting of structures in the ACPS data is having any impact on our results, it is likely to reduce within-type variance, biasing our analysis toward finding the proxy valid. If, despite the possible downward bias, we still find substantial within-type variability, and less between-type variability, we will have reasonable evidence that the assumption underlying the proxy is not met.

Beyond undercounting, three of the items (1, 5, and 6) use the vague quantifier “sometimes” in the item text. Although vague quantifiers are common, researchers have long known that respondents’ interpretations of them vary (Hakel, 1968; Simpson, 1944). Recent work suggests that the meaning of vague quantifiers depends on context (Nelson Laird, Korkmaz, & Chen, 2008). Respondents use knowledge of what they think the survey population is to anchor their answer (Wanke, 2002). Situations where respondents likely had population reference points increase confidence in the data collected using vague quantifiers.

Fortunately, ACPS respondents are likely to have known members of other associations and therefore had a sense of norms of associational participation. Association membership in the United States was common and increasingly prevalent around the time of the ACPS survey (Rotolo, 1999). In 1990, 79% of Americans were involved with at least one association and 41% were affiliated with four or more (Verba et al., 1995). Among joiners, 65% had attended a meeting in the last year and 42% considered themselves “active” members of at least one group (Verba et al., 1995). As of 1985, the average U.S. adult spent more than 1 hr per week engaged in organized, nonreligious associational activity (Robinson & Godbey, 1999). Beyond associational involvement, the average American in 1985 also spent nearly 7 hr per week socializing (Robinson & Godbey, 1999). Although we cannot know for certain, given high rates of associational involvement and frequent socializing, it is likely that ACPS respondents had talked with many members of other associations about participation, providing a rough population norm to anchor understandings of “sometimes” in survey questions. ⁵

Apart from the vague quantifiers, the items are worded in ways that should generate valid organizational assessments. Informants are generally good at reporting on “directly observable organizational characteristics” (Frenk, Anderson, Chaves, & Martin, 2011, p. 78) and reports of such features are similar for both officials and ordinary members of the same organizations (Chaves, Konieczny, Beyerlein, & Barman, 1999; McPherson & Rotolo, 1995). Given that the ACPS items we use ask about specific, observable organizational features, respondents should have reported accurately.

We marshal these items in two ways. First, we look for variation within types item-by-item (as dichotomous variables where 1 means an organization has the structure). Second, we create two composite measures. One is the total of all seven structures and the other is the total of the four variables that ask about structures independent of a member’s participation in an activity (takes stands on issues, provides charitable services, political discussions on agendas, and chat about politics). Because both composites include items that were only asked of those who attended meetings, they narrow our sample to 2,986 usable observations. Again, we suspect this will bias our results in ways that make types seem more internally consistent making for a conservative assessment of content as a structural proxy.

Results

How much does socialization structure vary within and between content types? Our analytic approach is straightforward. We present univariate statistics for structure measures (first by item, then as composites) by content type, looking for patterns of within-type consistency or divergence. We then conduct a variance components analysis that decomposes the amount of variance within and between types to confirm the descriptive patterns.

Table 2 arrays mean values for each structure by type (sorted by Takes Political Stands). This shows the proportion of organizations within the type that have the structure. If type is a good proxy for structures, we should see values close to 0 or close to 1, suggesting that most organizations within the type do or do not have that structure. We highlight situations of consistency by bolding values below 0.11 and above 0.89 (14 of 140; 10%). Values between 0.39 and 0.60 (25 of 140; 18%), in italics, are situations of extreme inconsistency; the type is evenly split between organizations that do and do not have the structure.

OPEN IN VIEWER

Table 2.

Proportions of Groups With Civic Structure Characteristics.

Content type	Takes political stands	Provides charitable service	Hold member meetings	Members on governing board	Political topics on agendas	Informal political discussion	Have active members
Political	0.95	0.50	0.36	0.08	0.96	0.98	0.68
Active in elections	0.95	0.29	0.55	0.20	1.00	0.98	0.75
Liberal or conservative	0.90	0.48	0.35	0.10	1.00	1.00	0.82
Women	0.79	0.81	0.54	0.14	0.79	0.89	0.64
Ethnic	0.77	0.82	0.60	0.16	0.76	0.81	0.75
Veterans	0.72	0.73	0.37	0.11	0.64	0.88	0.57
Labor union	0.72	0.49	0.59	0.14	0.72	0.85	0.49
Nonpartisan civic	0.69	0.60	0.65	0.41	0.74	0.92	0.86
Senior citizen	0.67	0.35	0.30	0.14	0.50	0.72	0.64
Business/professional	0.63	0.40	0.68	0.19	0.59	0.82	0.58
Neighborhood	0.54	0.26	0.70	0.18	0.55	0.73	0.61
Educational	0.46	0.59	0.65	0.20	0.40	0.70	0.63
Other	0.38	0.49	0.61	0.34	0.33	0.69	0.76
Service/fraternal	0.38	0.86	0.69	0.36	0.42	0.76	0.72
Religious	0.34	0.85	0.82	0.35	0.32	0.64	0.66
Social service	0.25	0.90	0.39	0.18	0.28	0.58	0.76
Youth	0.23	0.69	0.80	0.36	0.15	0.40	0.68
Cultural	0.22	0.51	0.34	0.10	0.23	0.56	0.62
Literary/art/discussion	0.20	0.37	0.80	0.24	0.36	0.66	0.69
Hobby/sports	0.20	0.29	0.57	0.21	0.21	0.55	0.65
Total	0.50	0.55	0.59	0.21	0.48	0.72	0.65

Source. Authors’ analysis of American Citizen Participation Study (Verba, Schlozman, Brady, & Nie, 1990).

The general pattern is of within-type inconsistency. High and low values (suggesting consistency) appear primarily in types concerned with partisan politics (political, active in elections, and liberal/conservative) and the consistent structures are the politically relevant ones (taking political stands, including political topics on meeting agendas, and having informal political discussions). Other dimensions, such as providing charitable services or holding member meetings, appear in the middle range, indicating variation across politically oriented organizations in these structures. Beyond partisan groups, little consistency is found. Looking across rows, no type has a pattern of similar values across measures, suggesting there are not one or two but many structural combinations within types.

Table 3 displays univariate statistics by type using the seven-item composite (sorted by mean). This provides a more inclusive picture of within-type variation, although only for organizations that hold meetings (N = 2,961). Standard deviation is the key statistic here. Means indicate how many structures appear in the average organization in a type. A type with structurally similar organizations should have small standard deviations regardless of the mean value, because organizations would share the same structures and not include others.

OPEN IN VIEWER

Table 3.

Variation in Civic Structure Characteristics by Type (Seven-Item Composite).

Content type	Minimum	M	Med.	Maximum	SD
Political	3	5.4	5	7	1.0
Active in elections	3	5.4	5	7	1.0
Liberal or conservative	4	5.3	5	7	0.9
Nonpartisan civic	2	5.2	5	7	1.2
Women	2	5.2	5	7	1.2
Ethnic	1	5.1	6	7	1.6
Veterans	2	4.9	5	7	1.3
Service/fraternal	1	4.7	5	7	1.4
Labor union	1	4.6	5	7	1.6
Senior citizens	1	4.4	5	7	1.8
Business/professional	1	4.3	5	7	1.7
Social Service	1	4.3	4	7	1.6
Other	1	4.3	5	7	1.7
Religious	1	4.2	4	7	1.5
Educational	1	4.2	4	7	1.7
Neighborhood	1	4.0	4	7	1.6
Literary/art/discussion	1	3.6	3	7	1.7
Youth	1	3.6	3	7	1.5
Cultural	1	3.6	4	7	1.7
Hobby/sports	1	3.4	3	7	1.7
Total	1	4.3	4	7	1.7

Source. Authors’ analysis of American Citizen Participation Study (Verba, Schlozman, Brady, & Nie, 1990).

For example, say a type was internally consistent with all organizations taking stands on political issues, having members as officers, and having members take active roles. The type mean would be near 3.0. Some organizations in the type would only include those three structures; most others would only deviate from the core by one structure. The standard deviation of structures would be at or below 1.0 (i.e., most organizations in the type would have 2-4 structures). If most others deviated from the core by two or more structures, the standard deviations would be above 1.0 (i.e., most organizations in the type would have 1-5 structures).

This analysis is conservative because a type will tend to appear more internally consistent than it actually is. Types with means near 3.0 could have a single core structure as described above, but could also have a common core of two structures (takes stands and member officers) and then two other structures that regularly appear but never together (i.e., takes stands + member officers + active members or takes stands + member officers + charitable services). Two different three-item structural cores within one type would still produce a mean of about 3.0. If few organizations include structures beyond those four, the type would also have a small standard deviation, despite the actually higher level of structural variation. Table 3 does not distinguish these two scenarios (or the other combinations that could occur), biasing types toward consistency. Despite the consistency bias, the only types with low standard deviations are the partisan political groups. Most types have standard deviations of 1.5 or higher, demonstrating that organizations within types include many different numbers of socialization structures—and it is likely that those numbers are hiding many more combinations of structures.

As a final investigation, we ran two variance components models. The analyses use the composite measures as dependent variables in hierarchical linear regression models where organizations are the first-level variable clustered within types, but include no covariates. The analyses decompose the amount of variation in the dependent variable due to organizations versus the amount due to types. When using the seven-item composite, we find the within-type standard deviation (1.56) is more than twice the between-type standard deviation (.65) and that types explain only about 14% of the variance in structure; the rest is determined by organizations. The four-item composite results are similar. The within-type standard deviation (1.2) is twice the between-type standard deviation (.66) and types explain 22% of the variance in structure. These results suggest that the socialization structures of organizations vary substantially within types—and much more than they vary between types.

Discussion

Although our analyses point to the inadequacy of content as a proxy for structure, our results are qualified because our data are imperfect. First, the sample is bounded by time. Data were collected in 1990 and U.S. associations have long trended toward centralization and professionalization (Skocpol, 2003). If structure and content correspond more closely now, then a replication study today might reveal less within-type variability. Still, given the amount of variation we found in data that likely understates the amount of within-type variability, we expect that contemporary levels of within-type variability still far outpace the differences between types. That said, if structural variability has changed since 1990, that is an excellent reason to collect new, better data on association structures (which we discuss below).

Second, the ACPS was not intended for this purpose. Structure questions were asked in nonideal ways and, even if they were worded better, the number of items is small. There are far more dimensions of structural variation we would like to assess, like the extent of member interactions across lines of demographic difference, the breadth of deliberation about political topics, or the frequency of leadership skill practice.

Because of these limitations, our findings offer a partial assessment of the validity of content as a structural proxy. We hope that scholars respond to this study, and others like it (e.g., Firat & Glanville, 2017), by identifying or collecting other data suitable for a more rigorous assessment. One approach would be to use hypernetwork sampling (McPherson, 1982) to assemble a large, representative sample of membership associations. Detailed organization-level surveys asking about content and structure could be conducted either with the originally sampled members or with key informants (Chaves et al., 1999). Such data would allow for more thorough examinations of within- and between-type structural variability.

Despite limitations, our results and our review of the literature suggest that analyses of new, better data will still find more structural variability within content types than between them. High levels of within-type variability can explain why findings are inconsistent across proxy-based studies. Because structures differ, some organizations within a type are having positive impacts on individual political participation while other organizations within the same type are having negative impacts. These cross-cutting organization-level effects could make a type appear to have little or no effect, and random variations could lead to statistically significant correlations between types and outcomes that are positive in one sample and negative in another.

Paths Forward

If content is not a valid proxy for structure, what should be done? Researchers in this field need to identify large, representative samples of organizations, collect accurate data on content and structure, and connect those organizational characteristics to the attitudes and behaviors of individual participants. Content data remain important. Beyond the value of aided-recall questioning, content types provide insight into selection dynamics (Quintelier, 2013; Quintelier & Hooghe, 2012); content is, after all, what people join for. New data should include content types and also capture an array of structural measures. This will allow for analyses that can identify the relative effects of each.

Two hypernetwork sampling strategies offer good starting points (McPherson, 1982). In a “domain based” strategy, researchers randomly sample individuals and ask respondents to report where they encounter certain structured opportunities (e.g., “where do you talk about politics?” or “where do you lead teams?”; Tran, Graif, Jones, Small, & Winship, 2013). This starts with a respondent’s experience of a structure and works back to the organizational setting where it occurred. The questioning captures some structure data while it develops the organizational sample. In an alternative aided-recall strategy, researchers again randomly sample individuals and then use traditional content questions about associational affiliations. Both techniques should produce representative samples of associations (with the domain approach capturing other organizations too).

After sampling, data are needed on organizational characteristics. The initial sample of individuals could be used as informants on all of their associations, providing content and structure details for each. Content categories would ideally be synchronized with other studies to enhance comparability. For structure, versions of the seven ACPS items could be a starting point, but a wider array of structures should eventually be captured. Warren (2001), for example, theorized at least 34 possible structure types based on features like ease of exit and the goods created by associations, and Baggetta (2009) measured 26 structures that offered opportunities for interpersonal interactions, governance experiences, and institutional relationships. When measuring any such structures, it will be essential to separate questions about the existence of structures (e.g., “how often does your association hold member meetings?”) from the extent of the respondent’s engagement (“how often do you attend member meetings?”). Such a format may be unwieldy for joiners of many organizations, but the ACPS data suggest most individuals join at modest rates. Cultivated panels such as those at Knowledge Networks (www.knowledgenetworks.com/ganp/) have respondents who may be more willing to complete lengthy surveys.

To avoid overburdening respondents, other data collection techniques could be used to backfill organization-level data such as surveys of key organization informants or coding of organizational documents. More ambitiously, a team of researchers could use systematic social observation (Reiss, 1971) to scale up the approach of multisite ethnographies (e.g., Baiocchi, Bennett, Cordner, Klein, & Savell, 2013) by attending and coding meetings of sampled organizations. Organization-level data could then be merged to individual-level political participation data collected from the originally sampled individuals.

We suspect collecting representative, multilevel data on individuals and associations will be most feasible in city-level studies. Medium-sized cities should have enough associations to ensure variability in content and structure, but a small enough number that organizational data collection would eventually approach saturation (i.e., nearly every association has been coded). This would ease the time and resource burden as a study plays out. New waves of individuals could be sampled to answer individual-level questions on memberships, experiences, and political participation, with the organization-level data merged to their records. By repeating the study design across cities (like the Social Capital Benchmark Surveys), a data set will be developed that reveals national patterns in association characteristics and effects on individual outcomes.

While city-based, multilevel studies will be important, large-scale omnibus surveys of individuals (e.g., General Social Survey, World Values Survey) will continue. Modest additions could improve data gathered from these surveys. After using a content-based, aided-recall battery to generate memberships by type, interviewers could ask respondents for the three organizations in which they are most involved across all stated types. Respondents could then report on structures (and their engagement in them) for those three organizations. The data would be less comprehensive and biased toward organizations with participatory structures, making them less useful for identifying distributions of associational structures in society, but would be better for analyzing the effects of associations on individuals.

Conclusion

Researchers studying the effects of associations on individuals often ask survey respondents about their affiliations using aided-recall lists of content and then use that variation for explanatory leverage. This assumes that organizations within content types are structurally similar. Is that assumption warranted? Is content a reasonable proxy for structure? Based on our analyses, the answer is a qualified no. Our data suggest that organizations involved in partisan politics are somewhat similar; they are likely to take stands on political issues, formally and informally discuss politics, and not have member-leaders. Such groups still vary substantially in holding member meetings, having active members, and providing charitable services. Most other types, from labor unions to arts groups, are so internally varied that knowing the type gives little insight into what structures members might encounter in a particular organization.

Our study, therefore, draws attention to the need for better data on associations. Our argument is not an attack on researchers who have used content types as proxies. The studies we have cited are the work of thoughtful scholars doing the best they can with the data they have. We simply highlight that if our field wants better answers to important questions, we need to develop strategies for getting the right data to do it.

Supplemental Material