Abstract
Scholars of civil society organizations (CSOs) have long been concerned with face-to-face interactions among participants at meetings, events, and activities—what we collectively call “convenings.” Small-N ethnographies have uncovered substantial differences in the dynamics of convenings within and across CSOs. Large-N quantitative techniques, however, capture little data on convening dynamics; instead, they rely on questionable proxy measures. How much do dynamics vary across CSO convenings? New research tools are needed to answer this question. We adapt systematic social observation (SSO) to the study of CSO convenings as one such tool. SSO uses trained, independent observers and carefully constructed protocols and forms to collect detailed, standardized, quantitative data from many settings. Here, we illustrate what SSO can capture beyond conventional quantitative approaches using data from a pilot study of collegiate CSO convenings. We argue that SSO data can improve on prior quantitative findings about CSOs and extend research in new directions.
Scholars and observers from Alexis de Tocqueville ([1835] 1969) onward have considered face-to-face interactions in civil society organizations (CSOs) to be essential for the functioning of democracy. Ethnographers have long observed such interactions at CSO meetings, events, and activities—what we collectively call “convenings.” They have uncovered substantial differences in internal dynamics within and across CSOs (Lichterman and Eliasoph 2014). Even the most ambitious ethnographies, however, only study small numbers of CSOs (Baiocchi et al. 2013; Blee 2012). Quantitative approaches to studying CSOs, like surveys of organizers or participants, collect data from large numbers of organizations (e.g., Knoke 1990) or individuals (e.g., Verba, Schlozman, and Brady 1995) but capture little about the dynamics of convenings. Instead, they rely on proxy measures that may not accurately capture convening dynamics (Firat and Glanville 2017). The study of CSOs, therefore, faces a fundamental, unanswered empirical question: How much do dynamics vary across CSO convenings?
To answer this question, new methodological tools are needed that capture detailed, quantitative data from large numbers of CSOs and the multiple interaction contexts within them. We argue for the adaptation of systematic social observation (SSO) to the study CSOs as one such tool. SSO uses trained, independent observers and carefully constructed protocols and forms to collect detailed, standardized, quantitative data from many settings. We begin our argument by summarizing traditional quantitative methods used to study CSOs, noting what they can capture and what they miss. We then highlight two streams of research that would benefit from better data on internal CSO dynamics—one well established and one newly emerging—and identify illustrative distributional questions from each. We introduce SSO methods and describe our adaptation to CSOs as a strategy for answering such questions. We then demonstrate the viability of the approach by reporting statistics addressing our illustrative questions from a pilot study of collegiate CSO convenings.
Background: The Quantitative Study of CSOs
How much do internal dynamics vary across CSOs? To answer this question, scholars would ideally obtain accurate, consistent, comparable data from convenings of many CSOs. To the best of our knowledge, no such data exist. Instead, scholars make do with the best approximations available.
Current quantitative methods for studying internal CSO dynamics vary along two important dimensions: the vantage point of the data source and the scale of the organizational context. Vantage point refers to the position of the person whose report about an organization becomes a data record. Some data reporters, like journalists, are external to the organizations they describe; others, like group leaders, are elites within those organizations; still others are ordinary organizational participants—the members, staff, visitors, and passersby who participate in a given activity. 1 Context scale refers to the granularity of data records. Some data sources report on an organization as a whole—providing a description of the shell within which interactions occur. Other sources depict internal organizational contexts but in general ways that homogenize internal variation. Others describe specific convenings, the discrete settings and events where interactions occur.
Table 1 presents the intersections of these two dimensions. Cells in the table list commonly used data sources reflecting each combination of vantage point and context scale. We do not attempt a comprehensive review of data sources and studies here but rather provide an overview that highlights situations where analysts are forced to assume certain internal CSO dynamics rather than observing them directly.
Characteristics of Quantitative Data Sources for Civil Society Organizations.
We begin with data that describe whole organizations and are reported by external observers. Administrative records fit this mold. For example, several scholars have made interesting use of city directories—lists of organizations produced by some U.S. city governments dating back to the 1800s. By coding names into categories, such data can reveal the distribution of CSOs across types, across cities, and over time (e.g., Gamm and Putnam 1999; Skocpol, Ganz, and Munson 2000). Scholars wishing to assess the impacts of such associations on members or communities, however, must assume that the name of the organization is a suitable proxy for the experiences members had within them (e.g., Kaufman 2002).
For more recent time periods, data are available that describe whole organizations and are reported by organization elites. Private companies solicit information from CSO leaders to compile databases. These elite-populated data records provide data on CSOs overall and can also be used to track distributions of organizations by type over time. For example, Minkoff (1995) used data from the Encyclopedia of Associations (collected by the Gale Research Company) to study field-level competition among membership associations with women or racial/ethnic minorities as constituents. Another strategy is to code histories of CSOs—whether published by historians or CSOs themselves—for details about organizational structures and activities (e.g., Gamson 1990; Skocpol 2003). Any given history may provide detailed descriptions of particular internal dynamics. Coding schemes designed for highly varied histories, however, collapse those details into overall CSO descriptions. As with contemporary databases, analysts must assume that general descriptions consistently reflect internal dynamics.
Surveys of members offer participant-level data that, when aggregated, describe CSOs overall. For example, surveys about member demographics of all (or most) members of a set of associations are useful for establishing homophily in association joining (McPherson and Smith-Lovin 1986, 1987; McPherson, Smith-Lovin, and Cook 2001). Such data do not, however, reveal who attends convenings and who interacts with whom. Demographic and joining data from general population surveys can be aggregated up to the level of association types (e.g., political, recreational), which reveals demographic heterogeneity by type (Coffe and Geys 2007). Such aggregations, however, may not accurately reflect organization-level demographics (Firat and Glanville 2017). Demographic surveys, therefore, force potentially inaccurate assumptions about interactions.
We have not identified any commonly used data sources produced by external observers who provide characterizations of internal dynamics in general. Overviews of internal dynamics provided by organizational elites, however, are quite common. Many studies survey or interview key informants who are typically organizational elites (e.g., Andrews and Edwards 2005; Berry 1977; Schlozman and Tierney 1986; Walker 1991). Chaves (2004), for example, surveyed key informants from a representative sample of U.S. religious congregations to establish how often members experienced worship, service, politics, and art. Key informant reports are accurate on some, but not all, dimensions of member experience (Chaves et al. 1999; Frenk et al. 2011). Public-facing organizational documents like newsletters or websites can also provide an elite perspective on internal dynamics (e.g., Earl and Kimport 2011), particularly when scholars wish to understand the interpretive contexts within associations (e.g., Schnable 2016). Public-facing documents, however, only feature ideas that won internal debates about what to include; the extensiveness of debate remains hidden. Internal organizational documents, like memos, notes, meeting minutes, or membership records, can be coded for membership numbers, budget allocations, topics discussed, or activities undertaken (e.g., Andrews 2001; McAdam 1988; Skocpol 2003), although their scope is limited to what documents were produced and survived to be coded. In all these methods, limited perspectives stand in for all internal dynamics.
Overviews of internal contexts provided by participants commonly come from surveys. Surveys of the general population ask respondents about the types of organizations (e.g., political, recreational) they have joined (e.g., Almond and Verba 1963; Coffe and Geys 2007; Stolle and Rochon 1998) and occasionally about organizational contexts and member experiences within specific organizations (e.g., Verba et al. 1995). Such data provide more specific characterizations of internal dynamics but from just one participant per organization. Other studies sample survey respondents from organizations’ membership lists (e.g., Baggetta 2009; Knoke 1990; Stolle 2001). 2 These surveys feature many respondents per organization, whose responses can be aggregated, but they capture perceptions of dynamics in general, losing convening-level variation.
Data that characterize specific convenings often come from external observers. An extensive body of research in social movement studies codes news media reports of public protests for event and organization characteristics (Earl et al. 2004; Ortiz et al. 2005). Media reports are especially useful for capturing the frequency and location of events, which can be connected to participating organizations (e.g., Amenta et al. 2009; Sampson et al. 2005) or to the statements of particular actors and organizations in an evolving discursive field (e.g., Leifeld and Haunss 2012). Media, however, typically report on the most public, dramatic, and extreme forms of CSO activity (Amenta et al. 2009; Myers and Caniglia 2004; Oliver and Myers 1999). Government records (like police reports) can be used as an alternative, but they are subject to similar biases (Oliver and Myers 1999). Even if event data were unbiased, they still only capture public activities—and many CSO convenings are routine, internally focused affairs (Oliver and Marwell 1992).
Recently, scholars have identified specific convening data from the perspective of organizational elites. Bail (2012) compiled and analyzed press releases from organizations, almost all of which describe an event. Similarly, Caren and Gaby (2011) described local affiliates of the Occupy Wall Street movement by coding data from their Facebook pages. 3 Although these sources may be more comprehensive than media coverage, press releases and social media postings likely overrepresent public activity (like protests) and underrepresent routine, less-public activity (like meetings).
Convening data from participants’ perspectives are captured through field surveys. Fisher et al. (2005), for example, surveyed environmental protesters at protests. Respondents reported their CSO affiliations allowing for the creation of an organization-level data set that included participant perceptions of a convening. Because field surveys are administered in the midst of events, however, they must be brief and, therefore, limited in scope and, as with other convening data, they only capture public events.
A notable absence from Table 1 is a final row capturing data at each scale level from the vantage point of an internal observer—someone situated physically within the organization who can watch and record what transpires. This is, of course, the terrain of ethnographers, who place themselves in organizational contexts, assemble field notes on convening dynamics, and build up descriptions of internal contexts in general and organizations overall. Ethnographies of CSOs have noted the shortcomings of CSO research that misses convening-level dynamics (e.g., Eliasoph 1998, 2011; Lichterman 2005; Lichterman and Eliasoph 2014). The scope of ethnography, however, prevents the technique from revealing how often quantitative studies mischaracterize CSOs. To address distributional questions, data are needed from convening-level observers at larger scales.
Theory
Before introducing new data from observations of CSO convenings, we briefly present two streams of theoretically driven research that could be improved or extended through the use of SSO data. These are not the only areas where SSO could contribute, but they serve as illustrations of how SSO can refine prior quantitative findings and supply quantitative data to emerging areas of research. The first is research on bridging social capital (Putnam 2000) including connections made among individuals (e.g., Coffe and Geys 2007) and among organizations (e.g., Paxton 2002). The second is work concerning the role of physical space in shaping interactions (Gieryn 2000). In our subsequent analyses, we use our SSO pilot data to evaluate the validity of proxy-based claims about bridging social capital and to demonstrate the potential of SSO for uncovering effects of physical spaces.
Bridging Social Capital
Bridging social capital (Putnam 2000), network ties that knit together communities and societies across salient lines of difference, can be formed between individuals or organizations (Schneider 2009). At the individual level, bridging social capital is often studied as interpersonal connections across salient sociodemographic lines like race (e.g., Baggetta 2016; Weisinger and Salipante 2005). People from diverse backgrounds are more likely to build ties across lines of difference in organized settings where participants pursue common goals (Pettigrew 1998). How often do CSOs play this role? Research findings are inconclusive. Some work identifies cases where cross-cutting ties are formed (Braunstein, Fulton, and Wood 2014), but other research reveals barriers to connections at the micro-interaction level (Eliasoph 1998, 2011; Lichterman 2005; Weisinger and Salipante 2005). Quantitative studies demonstrate that CSO memberships are often homogenous, limiting the opportunities members have to form cross-cutting ties (McPherson et al. 2001).
Prior works are limited, however, by their vantage point and context scale. Data from external and elite perspectives often force analysts to assume homogeneity of memberships and interactions based only on organization names or general descriptions (e.g., Kaufman 2002; Minkoff 1995). General population surveys have identified types of associations that appear more or less diverse (e.g., Coffe and Geys 2007), but such studies may mischaracterize the level of diversity within particular organizations (Firat and Glanville 2017). Organization-based surveys that establish the relative homogeneity of CSO memberships cannot identify how often members interact across lines of difference in the settings where some diversity exists (Baggetta 2016; Frenk et al. 2011; McPherson and Rotolo 1995, 1996; McPherson and Smith-Lovin 1986, 1987; Popielarz 1999). In short, current quantitative techniques allow us to identify the potential for cross-cutting interactions within CSOs; that potential is then used as a proxy for actual cross-cutting interactions. Our first substantive research question, then, is: How often do CSO participants interact across lines of sociodemographic difference? The related methodological question is: Do quantitative proxies for participant interactions across lines of difference reflect actual patterns of interaction?
At the organization level, bridging social capital is often conceived as ties between organizations with different purposes (Schneider 2009). One line of work in this tradition conceptualizes ties between organizations through overlapping memberships (e.g., Paxton 2002, 2007). An individual who participates in two different groups serves as a bridge between them. Organizations with members who hold other memberships are expected to serve as hubs connecting communities together.
We do not know, however, how often those connections are actually made salient within CSOs. Qualitative research suggests that some CSOs engage in conversations about interorganizational ties and which of their members have valuable other memberships (Ganz 2009; Warren 2001). In other CSOs, however, conversation rarely focuses on other organizations and external connections (Eliasoph 1998, 2011). Quantitative analyses must assume that overlapping memberships are salient. Paxton (2002), for example, uses general population survey data on overlapping memberships to categorize CSOs into two types: “isolated” CSOs, like sports leagues, whose participants are members of relatively few other CSOs and “connected” CSOs, like arts groups, whose participants have relatively more memberships. This categorization assumes that the salience of outside memberships is higher within CSOs whose members have more outside ties. If this assumption is warranted, we would expect to hear discussion of connections to outside organizations more often in convenings of connected associations. Quantitative data have not established this, raising our second substantive question: How often are connections to other organizations discussed in CSOs? The related methodological question is: Do quantitative proxies for salience of organizational connections reflect actual patterns of discussion?
Physical Space
An emerging line of research that could also benefit from SSO data investigates the effects of physical environments on interaction (Gieryn 2000; Logan 2012; Parkinson 2012; Taylor and Spicer 2007). Buildings and rooms influence relations within them (Fayard and Weeks 2007; Gieryn 2002). For example, work environments that allow for informal, semiprivate interaction enhance social capital (Fayard and Weeks 2007; Zagenczyk, Murrell, and Gibney 2007), the shapes of legislative assembly chambers affect relationships among representatives (Parkinson 2012), and outdoor environments can facilitate (or discourage) collective action (Gould 1995; Scott 1998; Zhao 1998). While scholars have qualitatively examined the effect of spaces on civic dynamics (e.g., Walsh 2004) and quantitatively analyzed the distributions of civic activity across geographic territory (e.g., Sampson et al. 2005), we are not aware of any efforts to quantitatively track physical space dynamics within routine CSO convenings. Without an established set of CSO-based findings to evaluate, we instead ask an initial, exploratory question: How often does the size of a CSO convening match the size of the space where it occurs?
Methods and Data
To answer our distributional questions about CSO convening dynamics, we analyze SSO data from a pilot study of collegiate CSOs. In this section, we summarize the history of SSO use in sociology, noting our extensions to prior approaches. We then lay out the basic units of observation for SSO in CSOs—which we term convenings. Finally, we describe the pilot study we conducted to develop an SSO tool suitable for CSOs and the data we collected.
SSO
SSO uses trained independent observers and carefully constructed protocols and forms to capture detailed, standardized, quantitative data from many settings. Sociologists’ use of the approach dates to the 1920s, with a wave of popularity in the 1970s (McCall 1984). Reiss (1971) utilized SSO to study actions and interactions of police officers on the beat. Whyte (1980) sent observers to New York City’s parks and plazas at various days, times, and seasons to code who was there, what they did, with whom they interacted, and how long they stayed. More recently, Underhill (1999) studied retail shopping, with observers surreptitiously following shoppers in stores, malls, and concourses. Sampson and Raudenbush (1999) used researchers walking through Chicago’s neighborhoods to catalog urban disorder. Extensions of their technique had observers code videos taken from slow-moving vehicles (Sampson 2012) or images appearing in Google Street View (Odgers et al. 2012). To varying degrees, these projects captured social interactions and the contexts where they occurred. We do the same while also focusing on what can be heard at convenings: talk. Talk is of core theoretical interest in CSOs, as ethnographers have highlighted. We extend our SSO approach to capture it. To be clear, we do not argue that SSO can replicate the richness and depth of ethnographic work on talk (or other dynamics). We do, however, think that SSO provides an opportunity to uncover distributions of important, observable phenomena.
Units of Observation: Convenings
Our SSO technique is designed to collect data at CSO meetings, events, and activities—what we collectively call convenings. More formally, convenings are the intentional assembly of two or more people for some public purpose under the auspices of an at least quasi-formal organization. Participants need not know each other or be affiliated with the same organization, but at least one of the participants must intend for the interaction to occur (e.g., a nonprofit organization staffer at an information booth counts as a convening so long as at least one other person interacts with the staffer).
Most CSO-related activities fit within this definition, such as organization meetings for information sharing, recruiting, planning, and decision-making. Activities intended to build and maintain an organization all fit, including work sessions, practices, rehearsals, and trainings. Activities intended to connect an organization to the outside world also fit, including relatively small interactions like canvassing, tabling, teaching, tutoring, and ministering and larger events like rallies, concerts, worship services, games, performances, speeches, and protests. The definition excludes purely individual actions, even on behalf of a CSO, such as an advocacy group member writing a newsletter column or a theater troupe member studying lines for a play. Also excluded are unintentional assemblies of people, even if participants are CSO members interacting with a public purpose. These are better understood as encounters—like the unplanned interactions between police and citizens captured by Reiss’s (1971) observers.
The Pilot Study
We conducted an 18-month pilot study to develop an SSO tool to capture internal CSO dynamics. We studied college student groups at a large, Midwestern state university. While no single campus represents the variety of civic cultures at U.S. colleges (Binder and Wood 2013), the school shares many characteristics with other large state universities. In particular, the large number of students leads to a large and diverse set of campus CSOs. Our search of university listings and other online sources suggested a population of nearly 600 on-campus groups. We selected a theoretically informed sample of CSOs whose convenings we would observe. The goal was to maximize the range of possible convening-level dynamics. The types of CSOs observed across all waves of the pilot study were arts groups (2 organizations), campus governance (2), cultural identity (1), environment and animals (3), health (2), hobby (1), politics (1), professional (1), religion (2), and sport (2).
Why study collegiate groups? While convenience was a consideration, it was not the foremost one. Student groups are often inconvenient; the CSOs we studied changed plans often, suddenly, and with little communication. 4 Of greater importance was the scope of the CSO field. If America is a nation of joiners (Schlesinger 1944) and organizers (Skocpol et al. 2000), then college students are embodiments of that spirit. The number and variety of CSOs increased our ability to observe a wide range of associational features, allowing us to design a more comprehensive tool.
While the pilot study’s focus was on creating the SSO tool, the data captured also have substantive importance. First, scholars view students as civically malleable; student experiences often affect civic engagement years later (Campbell 2006; Levine 2007; McFarland and Thomas 2006). Learning the distribution of student civic experiences may therefore help us understand the ways colleges do (and do not) shape the political, civic, professional, and personal lives of college graduates (Armstrong and Hamilton 2013; Chambliss and Takacs 2014; McCartney, Bennion, and Simpson 2013). Second, we know that participation by college students in 1960s social movement organizations dramatically shaped the lives of those activists (Fendrich 1993; McAdam 1988). Changing patterns of civic and political engagement for recent generations raise the question of whether findings from 1960s student activism apply to today’s youth (Dalton 2009; McAdam and Brandt 2009; Zukin et al. 2006). Our pilot data offer a first quantitative look at campus-based civic life today—with implications for students’ futures.
We began the pilot by recruiting three undergraduate research assistants who would not seem out of place at collegiate CSO convenings. We trained them in the basics of ethnographic field observation and deployed them to conduct fully qualitative observations of convenings. We provided six focusing questions of theoretical interest: (1) What occurs at convenings? (2) What interactions take place among participants? (3) What observable power or leadership dynamics appear? (4) What activities do participants discuss, choose, plan, and execute? (5) What types of boundaries do groups draw? (6) What (if any) politically inclined discussion or activity occurs? Over four months, our observers visited convenings organized by eight different CSOs and produced 25 sets of detailed field notes.
Working inductively from those field notes and the research literature, we drafted a form that included an initial set of closed-ended items and several open-ended items where we recognized a need for thematic content but had not settled on measures and categorizations. Our observers returned to the field with paper copies of the draft form. Over the following four months, they attended convenings organized by 13 organizations (7 repeats plus 6 new). During observations, they coded using the closed- and open-ended items on the form; they wrote field notes following each observation. Observers also noted which closed-ended items on the form were not useful or relevant during their observations in order to distinguish between the nonappearance of a phenomenon and a poorly performing item. Approximately every two weeks, we reviewed all the data collected, discussed the experiences of the observers, and revised the form to increase the number and quality of quantitative items. As the form became more comprehensive, we reduced emphasis on field notes and focused on finer-grained revisions. Over a final four months in the field (including convenings of three additional organizations), we completed two more major revisions to the form. In the end, 126 observations of performances, rehearsals, practices, games, board meetings, business meetings, planning meetings, activity meetings, social gatherings, and cooperative endeavors between two or more groups went into the iterative process of designing an SSO approach to studying CSOs.
The primary output from the pilot study is a 9-page coding form comprising 102 items capturing 760 substantive variables that could be coded at any CSO convening and an accompanying a 40-page codebook. While space does not permit us to introduce all the measures here, we briefly introduce the six themes that emerged from our qualitative observations and have been codified into stable sections of our SSO form: (1) format and content, (2) time culture, (3) organizational maintenance, (4) strategy, (5) participants and interaction, and (6) physical space. We also note several areas that, after initial efforts, were removed from consideration and two areas that are still in development.
Format and content
We identify 15 types of activities that could take place (e.g., presentation/lecture, team building/bonding, marching/walking/parading) and further identify who does each activity (participants or audience), whether each set of actors are organization members, and how the activity is executed (as a full group, in subgroups, or individually). Our qualitative observations revealed that food and drink played substantial (and sometimes detrimental) roles in shaping convening dynamics. In response, we now code for 16 consumption characteristics ranging from food sales to alcohol consumption. We track 13 major categories (and 64 subcategories) of convening content—the substantive topics addressed during a convening (e.g., “hardship” issues, like homelessness; “environment” issues, like natural resources; “political” issues, like elections).
Time culture
Our qualitative observations revealed the importance of time culture—the pacing and formality of convenings. To capture variation, we include six sets of items: (1) the agenda or program for the convening; (2) levels, timing, and tracking of attendance; (3) the use of rituals or schemes for structuring participation and transitioning between activities; (4) the pace at which planned activities are completed; (5) reference to written organizational documents during convenings; and (6) the presence and nature of side conversations among participants. Relatedly, we track 10 forms of technology or media use during convenings, tallying the frequency of both official (e.g., laptops used by presenters) and unofficial uses (e.g., cell phone texting by audience members).
Organizational maintenance
Organizational maintenance includes selecting activities, recruiting members and participants, and initiating and training members. These appear as gated items on the form: observers initially code for any discussion of these topics at a convening, then, if present, observers code for the nature (consensual, debated, or contentious) and content of that discussion. For example, under recruiting for members, we code for four types of people that the organization might seek out (likely supporters, open/undecided people, likely nonsupporters, or anyone) and seven techniques they might use to find such people (e.g., personal face-to-face contact with people already known to current members). For each option, we also code for whether the option was considered during discussion, chosen to be pursued, or had already been completed.
Strategy
Organizational strategies we capture include collaboration, brokerage, influence-seeking, and individual and team dynamics. As with organizational maintenance, each of these batteries is gated. Collaboration and influence categorizations capture discussions of which types of individuals or organizations the group should work with or seek to influence—and how they should do so (e.g., influence people we know one by one; influence specific decision-makers). Individual and team dynamics capture who should be allowed to participate (e.g., previous participants), what participants are expected to do for a group (e.g., change one’s own behavior, teach skills to others, hold others accountable), and how participants should execute the work that needs doing (e.g., each person works alone on tasks, people work together on tasks). Brokerage discussions were more prominent than anticipated. Building on our qualitative observations, we added eight categories of resources/opportunities that organizations could connect members to (e.g., employment, advising, material goods).
Participants and interactions
We tally the number of participants present across five different categorizations: group status (e.g., regular members, formal leaders, invited guests), gender, race/ethnicity, nationality, and age. As each of these characteristics is captured through observation, we have adopted a conservative approach to coding. Whenever possible, we use spoken self-identifications by participants in order to categorize them. If no such indication is given, the observer codes for a specific category only when highly confident of that designation, choosing the included “unobservable” category when any doubt is present or the appropriate category does not appear on the form. Observers also code for any direct interaction among participants across categories (including the unobservable category) within each categorization. While gender and race/ethnicity categorizations were planned from the start, nationality and age emerged as salient lines of difference requiring tracking. Group status was an initial focus, but the categorization evolved substantially in response to observations. In particular, we observed that at some convenings, the presence of informal leaders—people without official organizational roles but whose voices were clearly respected—was an important subcategory to track. We now distinguish between formal leaders, informal authorities, and other influential figures at convenings.
Physical space
Space characteristics were not prominent at the outset of the pilot study but rapidly became a focal point. In early sets of field notes and debriefing sessions, our observers noted the “interaction between people and the room” as a crucial dynamic. The size of spaces was particularly noteworthy because organizers rarely had full control over the spaces they used. This led to convenings being shoehorned into too small spaces or swallowed by too large ones. Our observations also revealed that organizers typically wanted to defend spaces for the sole use by recognized participants, but many spaces did not easily allow for this delineation. As a result, organizers used signs or people to either greet and welcome or fend off possible entrants to a space. Based on our observations, we now code for features of space location, size, accessibility, and design, for the suitability of a space to the convening, and for efforts by organizers to bend a space’s features to their needs.
Rejected themes
The SSO approach did not prove suitable for all concepts we attempted to capture. Early in the process, we attempted to code values expressed at a convening or suggested by convening content. The set of values proved too vast and varied to be sufficiently distilled into categories and consistently captured by observers. As such, we removed values sections from early drafts of the form. We also faced surprising difficulty in identifying who at a convening was a veteran member of an organization and who was a relative newcomer. The distinction was clear when convening participants were talking about such categories of participants—and this distinction appears when we collect data on initiation and training—but our inability to identify this characteristic among convening participants resulted in cessation of attempts to tally veteran versus rookie participants and interactions across that line.
Emerging themes
We are actively developing two additional modules for the form. First, we are attempting to track cultural boundaries drawn by groups between insiders and outsiders. We currently focus on distinctions based on demographics, substantive field (e.g., business, law, health), political orientation (left, right, center, independent), religious beliefs (e.g., Christian, Hindu, atheist), and other elements of style (e.g., attire, tastes). Observers code for whether any given category is considered an “in” or “out” group, if the category’s in or out status is “contested,” or if it is unclear whether a category is in or out. Observers also indicate whether the evidence for the in or out classification came from something that was said (e.g., a joke made at the expense of some category, suggesting the group is an out group), done (e.g., a motion passed to bar people of some category from participating), or by the presence of participants from a category at the convening (suggesting “in” or “contested” status). Second, our qualitative data suggest dramatic differences across organizations relative to the style of speech accepted within a group. We currently include a battery of 20 paired terms representing polar opposites on style dimensions (e.g., lighthearted/serious, secular/religious, plain/jargon). Coders place each convening on a five-point scale between the poles on all 20 measures. Upcoming applications of the technique will determine whether these approaches are effective.
Quantitative data outputs
As our thematic summary suggests, many of the SSO form items evolved substantially over the course of the pilot study. Some, however, were settled early in the process. For such items, we have accumulated a critical mass of observations. We entered data from settled items from the paper forms into an electronic database using Qualtrics online survey software (version: February 2016) designed to visually approximate the paper form (to reduce data entry errors). We can use data from the long-standing items to conduct descriptive analyses that illustrate the usefulness of this approach. We do so relative to the questions we earlier articulated regarding bridging social capital and physical space.
Results
We marshal our SSO data to offer preliminary answers to the theoretical and methodological questions posed in the theory section. We conducted 126 total observations, of which 104 captured some usable quantitative data (effective Ns vary by item because many items changed over time). Given the small number of organizations (17), we limit ourselves to convening-level analyses (i.e., we do not aggregate to the organization level). Because our sample is small, our results are suggestive and primarily intended to illustrate the viability of SSO for establishing distributions of internal CSO dynamics across theoretically important categories.
Bridging Social Capital
Our investigation of bridging social capital uses convening-level data to test assumptions inherent in prior quantitative research on individual- and organization-level bridging. Table 2 connects the organizations in our sample to typologies used by previous studies as proxies for bridging dynamics. The first column of Table 2 lists simple pseudonyms for the organizations in our study. The second column codes our organizations into the typology used by Coffe and Geys (2007) in their study of bridging across individuals within CSOs. The third column codes our organizations into the typology used by Paxton (2002) in her study of bridging across CSOs within countries. In our analyses, we examine actual interaction patterns within these typologies to see whether proxy-based findings hold up to direct observation.
Pilot Study Organizations by Proxy Typology.
aCoffe and Geys exclude religious and political groups from their analysis. bBoth sports groups are student-run club teams, not university-run intercollegiate teams.
We begin with individual-level bridging social capital. Coffe and Geys (2007) used demographic data on survey respondents to establish how demographically diverse the membership pools were for each type of association. They found that some types (e.g., hobby groups) had much more diverse membership pools than others (e.g., sociocultural groups). Participants in the more diverse types of associations, they argue, should develop more bridging social capital ties.
To investigate this claim, our observers counted the number of individuals at each convening who fell into gender, race/ethnicity, age, and nationality categories. While observational data on demographic characteristics are not perfect, research shows that observers can consistently categorize observed subjects in the same way that subjects self-categorize (Hahn, Truman, and Barker 1996). The coding categories were, for gender: women, men, unobservable; for race/ethnicity: white, black, Hispanic/Latino, Asian, unobservable; for age: youth, traditional college age, young adult, older, unobservable; for nationality: United States, non-United States, unobservable. Our observers also coded for whether they saw any direct interaction between two individuals from different categories in each categorization (i.e., for gender, did at least one woman interact with at least one man?). We use these data to calculate the percentage of convenings where no interactions occurred across lines of demographic difference for each characteristic.
Table 3 compares our observations to the Coffe and Geys’s (2007) expectations. The columns show the Coffe and Geys categories running from least expected bridging on the left to most expected bridging on the right. The rows show the percentages of convenings in each category that had no cross-category interactions on that characteristic (e.g., the percentage of convenings where men interacted only with men or women only with women). Higher percentage values indicate more homophily (exclusively within-category interaction) and less bridging at those types of convenings. A value of zero means all convenings of that type include at least one cross-category interaction (i.e., no entirely homophilous convenings); a value of 100 means all convenings of that type included no cross-category interaction (i.e., all entirely homophilous convenings).
Percentage of Convenings with Entirely Homophilous Interaction by Demographic Characteristic and Coffe and Geys’s (2007) Organization Type.
Note: Cell values are the percentage of convenings held by organizations of that type where participants only interacted within demographic categories (i.e., homophily). Larger percentage values indicate more homophily (i.e., less bridging). Numbers in parentheses are the percentage of convenings where cross-category interaction could have occurred (i.e., participants from more than one category were present), but only within-category interaction occurred (i.e., single-demographic cliques at diverse convenings).
If Coffe and Geys’s (2007) expectations are correct, percentages should decrease from left to right. Our data do not bear out this expectation. None of the demographic characteristics show steadily decreasing rates of cross-category interaction from left to right across types. Rather, the trends are inconsistent. Cross-gender interactions were extremely common across all types. No more than 8 percent of convenings within any type include only same-gender interaction. Racial/ethnic interaction varied more across types, with one of the two types showing the most bridging (community advisory) appearing in the middle of the expected bridging range. Sports convenings, which also appear in the middle of the expected typology range, have the lowest rate of racial/ethnic bridging: 42 percent of sports convenings had exclusively within-race/within-ethnicity interactions. Patterns for age and nationality cut strongly against expectations. Hobby and arts groups are expected to include the most bridging, but more than 80 percent of those convenings had no cross-age or cross-nationality interactions. On the other hand, sociocultural group convenings—expected to be the least bridging—were the most likely to include cross-nationality interaction and third most likely to include cross-age interactions. In short, bridging interactions vary substantially across convening types and in ways that do not align with expectations derived from other methods.
Our data allow us to further investigate the mechanisms underlying patterns of homophilic interaction. Cross-category interactions may be absent for two reasons: (1) convening participants are all from the same demographic category or (2) in a diverse setting, participants interacted only in demographically homogenous cliques. In Table 3, values in parentheses indicate the second mechanism, showing the percentage of convenings of that type where participants from more than one category were present, yet participants only interacted in demographically homogeneous cliques. A value of zero in parentheses means all the convenings with entirely homophilous interactions were due to a lack of diversity among participants. A value in parentheses that equals the primary cell value means all of the entirely homophilous convenings had the possibility for cross-category interaction, but participants declined to interact outside their single-category cliques.
Survey-based studies of CSOs show that members are often demographically homogenous (Baggetta 2016; Frenk et al. 2011; McPherson and Rotolo 1995, 1996; McPherson and Smith-Lovin 1986, 1987; Popielarz 1999), suggesting that convenings should have similarly homogenous participants. Some of the types in Table 3 reflect this tendency. Most convenings of arts and hobby groups that were homophilous included only a single-demographic category; while 100 percent of arts group convenings included no cross-nationality interaction, there were no arts group convenings where there were multiple nationalities present. In other settings, however, the pattern is different. One third of all sociocultural group convenings included no cross-race or cross-age interactions, and in all of those cases, participants represented multiple age and race/ethnicity categories. While homogeneous pools of participants is the more common homophily mechanism throughout the table, there are 11 cells where at least one convening could have included some bridging interactions yet did not.
We next assess organization-level bridging social capital. Paxton (2002) used survey data on the types of associations individuals join to create type-level indicators of connectedness. She argues that “associations whose members have many ties to other types of associations are more likely to cross-cut social boundaries and promote contact with diverse others” (Paxton 2002:269-70). If this assumption is correct, connected organizations should consider collaborating with other individuals, organizations, and groups more often than isolated associations, as members with multiple organizational ties promote additional contact.
To investigate this assumption, our observers coded for the presence of discussion about collaboration and the types of individuals or groups considered as possible collaborators (other students or student groups, university administration, companies/businesses, nonbusiness organizations like nonprofits or clubs, elected politicians or candidates, other government officials or agencies, or explicit arguments that the organization should not collaborate). Table 4 presents our data by Paxton’s (2002) connected/isolated typology. The results strongly support expectations. The first row of Table 4 shows that discussion of collaboration was far more likely to appear in convenings of connected associations than isolated associations (64 percent vs. 19 percent). Subsequent rows show that among the possible collaborators discussed, other student groups were by far the most commonly discussed collaborators (discussed in 48 percent of connected association convenings) with other nonprofits and clubs the second most common (21 percent). Even among the relatively rare discussions about collaboration in isolated associations, collaborations with on-campus membership groups were most likely (16 percent). These patterns suggest that Paxton’s proxy measures are likely capturing the organization-level phenomenon of collaborative ties between groups facilitated by overlapping memberships.
Percentage of Convenings with Discussion of Collaboration by Type of Collaborator and Paxton (2002) Organization Type.
Physical Space
While our analyses of bridging social capital examined assumptions embedded in other quantitative methods, our analysis of physical space extends research in a new direction previously unaddressed by quantitative techniques. We examine how often the size of a CSO convening matches the size of the space where it occurs. We coded whether a convening occurred outside or in one of the four indoor categories: large (suitable for very large gatherings, like an auditorium or gymnasium), medium (not suitable for very large gatherings, but still capable of holding a substantial number of participants, like a lecture hall or a racquetball court), small (suitable for a modest number of participants, like a classroom), or intimate (suitable for no more than about 10 people, like a living room or a dormitory room). We also counted the number of participants at each convening. We collapsed the number of participants into categories that should closely map to our space size categories (2–10, 11–25, 26–99, and 100+). Table 5 shows the distribution of convening sizes across space sizes (i.e., column percentages). If convening sizes match space sizes, most convenings should appear along the diagonal in the table. Convenings above the diagonal are situations where participants were tightly packed; convenings below the diagonal are settings where participants had empty space around them.
Percentage of Convenings in Spaces of Varying Sizes by Number of Participants.
We found that a large proportion of convenings were mismatched to spaces. Sometimes this resulted in crowded situations: medium-sized convenings (26–99 participants) occurred in small rooms 19 percent of the time. Much more common, however, were convenings where spaces substantially outsized the assembled participants. More than half of intimately sized gatherings (2–10 participants) occurred in medium-sized spaces (like lecture halls), and more than 10 percent of intimate and small (2–25 participants) convenings occurred in large spaces (like auditoriums). Occasionally, oversized spaces were a function of the activity (e.g., a dress rehearsal for a play held in an auditorium), but more often small membership meetings took place in large lecture halls or event spaces. While the small number of cases in our data prevents us from extending this analysis to see whether size match is statistically related to patterns of interaction, our qualitative observations suggest a relationship. Participants in small convenings housed in large spaces spread themselves out in ways that appeared to reduce interaction, especially among participants who did not previously know each other. Future research using the SSO technique will allow us to further specify and test such claims.
Conclusion
Scholarship on CSOs has made great strides since Tocqueville’s observations. Innovations in survey methodology, ethnographic technique, document analysis, and other methods have produced important findings. To make the next advances in understanding CSOs, we need a method that brings the advantages of independent observation into a larger-N framework. Our adaptation of the SSO approach to CSO convenings is a first step in meeting this need.
The SSO observation form we developed through our pilot study currently contains 102 items encompassing 760 variables. Rather than attempt to introduce them all in this article, we have used slices of our data to illustrate the value of the SSO technique for evaluating, refining, or extending previous lines of research. Our pilot data suggest that survey-based proxies for individual-level bridging social capital may be poorly capturing actual patterns of interactions, but that proxies for organization-level ties based on overlapping memberships may be functioning as expected. Our data also suggest that some CSOs are conducting their activities in ill-suited physical spaces, which may have implications for other internal dynamics.
There are, of course, limitations to this approach. SSO will not replicate or supplant the work of ethnographers. SSO observers will not develop deep rapport with research subjects and will miss many “backstage” conversations. SSO forms are grounded in inductive work, but when induction is set aside for consistency and breadth, nuance is lost. Scaling up to large numbers of observations will also be logistically challenging. Time and resource constraints will likely prevent SSO studies from reaching the tens-of-thousands scale of large survey studies.
Despite these limitations, we expect the SSO technique will be effective in testing other methodologies’ conclusions and achieving new insights, as our pilot data suggest. With SSO data collected at convening levels and then aggregated to higher units of analysis, scholars can compare dynamics across organizations and institutions. Given the many similarities between on- and off-campus CSOs, we speculate that with minor adjustments to categories, data can be collected from broader CSO populations, allowing for aggregation and comparison across communities and sectors. 5 Blended with conventional survey data, SSO may reduce problems of common method bias (Podsakoff, MacKenzie, and Podsakoff 2012). When geographically referenced, SSO will help answer questions about where civic activity occurs, how much it clusters in space, and how dynamics vary by location. Combining SSO with panel surveys should help resolve questions about causality that permeate this field (Quintelier 2013). For example, a longer-term study of collegiate CSOs could connect attitudes and behaviors of student participants (collected through panel surveys) to SSO data on the collegiate CSOs they join, allowing for analyses of selection into and out of CSOs and individual-level civic changes after joining.
In addition to introducing our SSO tool, this article has outlined an inductive process for developing new SSO items. Scholars using this process could develop new observation modules that supplement or replace those we have developed or extend the technique into different substantive terrain. As the history of the technique suggests, SSO is a dynamic methodology that can be adapted to many questions and settings. We hope it will move beyond episodic use to become a standard component of the sociological tool kit.
Expanding the tool kit is a crucial step toward answering outstanding questions about civil society. In a recent assessment of trends in civic engagement, Walker, McQuarrie, and Lee (2015:18) argued that “to understand the challenges of democratic life today, we need a broader picture of how participation works now, the settings in which it operates, and what it is used for.” We agree. While some scholars have called for better theoretical conceptualizations of civic activity in order to clarify claims (Alexander 2006; Edwards 2014; Fine 2012; Lichterman and Eliasoph 2014; Schudson 1998), we submit that methodological innovation is equally important. Common approaches to collecting data on CSOs have reinforced—and perhaps even created—the conceptual challenges the field face. The lack of detailed data on the internal dynamics of large samples of CSOs has forced scholars into adopting proxies that may not reflect the realities of civic life (Lichterman and Eliasoph 2014). The SSO method we have introduced can reduce the need for such assumptions while answering fundamental empirical questions and opening opportunities for new analyses.
Footnotes
Acknowledgments
The authors would like to thank Andrew Cloran, Alyse Conn-Powers, Polly Fairfield, Emily Hunnicutt, Emily Jackson, Rebecca Kemp, Krista Mantsch, and Anna McQuere for assistance with data collection and coding and Elisabeth Andrews, Kathryn Edin, Sarah Gaby, Kirsten Grønbjerg, Hahrie Han, Tina Nabatchi, Mildred Schwartz, and Mary Tschirhart for their feedback on prior drafts.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a grant from the Spencer Foundation (Grant #201400116), principal investigator: Matthew Baggetta.