Sustainability Factors of Emergent Civic News Websites

Introduction

It is well documented that the crisis in commercial journalism, particularly in the United States, has led to dramatic cuts and the emergence of a vibrant sector of civic and community journalism websites (Remez, 2012). Many of these sites are embedded in their communities; others have come and gone as journalists, entrepreneurs, and academics work to build a new, sustainable model for journalism (Downie & Schudson, 2009; Friedland & Konieczna, 2010; McChesney & Nichols, 2010). A local information system can empower citizens (Knight Commission on the Information Needs of Communities in a Democracy, 2009); several studies have linked local news with voter turnout (Gentzkow, Shapiro, & Sinkinson, 2011; Schulhofer-Wohl & Garrido, 2009), and communication infrastructure theory claims storytelling encourages individuals to participate in their community and develop a collective efficacy (Kim & Ball-Rokeach, 2006). The importance of local information is further demonstrated by the impact of community communication projects such as Metamorphosis and the local news experiment Madison Commons (Robinson, DeShano, Kim, & Friedland, 2010).

Although the sector of civic and community websites is relatively new and in flux, enough data exist to ask what contributes to success and longevity of these organizations. Indeed, this question is imperative, given the sector’s fragility, the importance of carefully allocating scarce resources, and the potential that local communities and foundations will tire of experiments.

This article addresses that question by modeling emergent ecologies of civic news websites in four cities that have embraced the emergence of those sites—albeit in their own diverse ways. To better understand the factors that lead sustainability of civic news websites, this article aims to

By now, our field has many studies of emergent news organizations focusing on individual cases (e.g., Schaffer, 2012). From there, scholars and media critics have moved on to studies of single ecologies (e.g., Anderson, 2013; Culbertson, 2012; Gordon & Johnson, 2012; Project for Excellence in Journalism, 2010), as well as comparative studies (Knight Foundation, n.d.; Naldi & Picard, 2012; Sirkkunen & Cook, 2012). To this point, though, even large comparative studies (Sirkkunen & Cook, 2012) have been based on a narrative understanding of how the organizations in question operate. A major aim of this article is to test our ability to push beyond descriptive narrative, to more systematically sketch out the nature of several ecologies at a time, comparing them with one another in a way that, we hope, will offer suggestions about what leads to sustainable civic news. To that end, we are adopting a method used for qualitative comparisons of social systems. The Qualitative Comparative Analysis (QCA) method (first described in Ragin, 1987; see also Ragin, 2008) allows scholars to systematically compare possible causal relationships of various qualitative variables. It is devised to work with small data sets of qualitative variables, on which regression-based statistical analysis would not be appropriate. In this case, it enables us to model and compare news ecologies to identify combinations of factors contributing to sustainability. More detail of this method, and why we chose it, is offered below.

Our second major contribution is to give analytical precision to the common sense observation that a news organization cannot focus on building readership while ignoring context. To do this, we build a model of these contexts that can be further refined and used as an analytical frame for other ethnographic and comparative case work. To that end, we examine the role of a variety of contextual factors in sustainability: vibrant entrepreneurship, support from foundations, partnerships with educational institutions, networks with other local media, social capital, and civic tradition. Because those variables are difficult to quantify formally, we chose to qualitatively assess them based on their presence and relative strength.

Our article proceeds as follows. First, we explain why we chose news ecologies as our unit of analysis. Then we discuss our method and introduce our four regions (with more detail in the appendix). We present our variables and analysis, followed by a discussion of our findings—specifically the important finding that organizations existing in networks are more sustainable—and finally deal with limitations of our study.

News Ecologies

There is no clear consensus on the definition of “news ecologies.” Scholars have called them “aspects of news production not under the direct control of any one institution” (Anderson, Bell, & Shirky, 2013); exogenous and endogenous environments of news media entities (Lowrey, 2012); and “complex interactions of agents in both the news and civic sectors, affecting the communicative integration of the community they serve” (Friedland & Kim, 2012). For this study, we use “news ecology” to denote the web of emergent online news websites that form the interactions referred to by Friedland and Kim. Using the ecological approach allows us to look at news organizations in concert (Cottle, 2000), as the dynamic interaction of production and distribution of news in particular geographic spaces.

Method

Quantitative general linear modeling is problematic because it “assumes away causal complexity”; qualitative, case-based methods, however, “capture complexity but lack formal tools for the necessary task of generalizing across cases” (Vaisey, 2009, p. 308). QCA is an attempt to fill this void. It allows us to develop typological generalization about the key elements of news ecologies in a comparative framework, despite not having a generalizable sample. The strength of QCA is its logical rather than statistical framework of inference. That is, the purpose of QCA is to find the combination of conditions necessary and sufficient to produce an outcome, in this case sustainable local news sites. While use of QCA in journalism studies is rare, our use of the method comes from the tradition of using it to compare social systems that are better described with qualitative variables. QCA is a well-established research method in comparative political science, particularly in the international realm, where “small- or medium N” research (e.g., whole countries as units of analysis) predominates. Past research using this method has focused on topics ranging from international welfare (Hudson & Kuhner, 2009) and patterns of urban governance (Kilburn, 2004) to personalization of political communication (Downey & Stanyer, 2010). These studies use limited number of cases with a diverse range of possible variables and combinations of variables that may affect the outcome. Moreover, the variables are often qualitative in nature such as the presence or absence of a certain trait or ordinal ranking, rather than an explicitly measurable quantity. These characteristics are similar to the challenges that our research faces.

Emergent online news ecologies are difficult to study because news sites are easy to start but have a high “mortality rate” (Hannan & Freeman, 1977), which makes it hard to collect a representative sample, let alone create an exhaustive database. Unfortunately, such data present two major problems for regression-based statistical analysis. First, they may not provide a sufficient number of cases along every categorical variable (the “small-N problem”). Second, several variables of interest may not be clearly quantifiable, or if quantifiable in principle, data may not yet exist. However, it may be possible to render these variables into qualitatively formed judgments, for example, their presence or absence in a given case, or their ordinal degree of presence. For example, developing a comprehensive quantitative scale reflecting a region’s historic traditions of civic engagement is outside of the scope of this project (and indeed has not been done on a national or regional scale). However, if the goal of the research is to compare a smaller number of regions with distinct characteristics, the problem can still be approached by qualitatively assessing and ranking them. Many qualitative studies, such as those described above, focus on individual or a small number of news organizations. These are of course useful in sketching out the emerging field, but are limited to examining news organizations one at a time. To address these issues, we use QCA to examine groups of news organizations systematically and in context.

QCA utilizes Boolean logic to systematically demonstrate which combinations of variables are necessary and sufficient for a given outcome. It maximizes the number of comparisons by logically laying out all possible combinations and comparing them against cases present in the data. It has been further developed into fuzzy-set QCA (fsQCA), which allows for non-binary attributes such as ordinal values by assigning a value between one and zero based on Ragin’s (2008) consistency score concept (2008). A traditional QCA study is limited to values of “0” and “1”; in other words, it is useful in black and white situations, while fuzzy-set QCA allows for shades of gray. For instance, a news organization either exists after 3 years or does not, so this variable can be described through a “0” or a “1.” However, the degree of social capital in a given region may not be strictly speaking quantifiable, but could be ranked by region into ordinal variables based on qualitative and historical knowledge.

The QCA method involves starting with an outcome variable of interest (roughly corresponding to the statistical concept of the dependent variable) and then enumerating a set of factors plausibly responsible for the outcome in some combination, corresponding to independent variables. Researchers use a software package to create a “truth table,” which starts out listing each case (in this study, cases are the news websites) along with all possible factors and outcomes. The software uses Boolean logic to simplify the truth table, removing redundant columns and logically reducing the individual or combined factors that might influence the outcome variable. The software package we used for our study is fsQCA2.0, developed by Charles Ragin, Kriss Drass, and Sean Davey (2006).

Along with the simplified truth table, the QCA software creates consistency and coverage measures for each solution. The consistency of a variable or combination of variables resembles the notion of statistical significance. It specifies the percentage of cases with that variable (or combination of variables) that yield a specified outcome. For each result, coverage is the ratio of cases that exhibit that result, computed by examining the original fuzzy data set in light of the solution.

Let’s look at a simple example of how QCA works. Below is a simplified version of the truth table we used in our analysis. It contains all the configurations of two variables: entrepreneurial funding of the site in question, and social capital of the region. The last column shows how many of the sites we looked at had survived after 3 years, given the particular configuration of entrepreneurial funding and social capital.

OPEN IN VIEWER

Outcome	Cases with entrepreneurial funding	Cases with social capital in the region	Both conditions present
Survived	6	4	2
Died	1	3	0

The method further enables us to determine how strongly, for instance, entrepreneurial support explained survival—a calculation that is called consistency—by adding up the value of the entrepreneurial variable in all cases that survived, and dividing by the sum of the entrepreneurial variable in all cases (survived and died). In this case, there were six cases that had entrepreneurial funding and survived, and 6 + 1 = 7 cases with entrepreneurial funding, so the consistency of entrepreneurial funding was 6/7 = 0.857. In other words, entrepreneurial funding went 85% of the way toward explaining survival. (Note that, as with p values in statistical methods, scholars can assign a consistency-level cutoff, below which the finding is not considered valid.)

We can also calculate the raw coverage of the entrepreneurship variable, which shows how empirically relevant a particular solution is—that is, what portion of cases with the desired outcome are explained by the given variable, or, in this case, how many of the sites that survived had entrepreneurial funding. Given the truth table above, that would be 6 / (6 + 4) = 0.6. Unique coverage calculates the results that are explained exclusively by a particular factor. In this case, entrepreneurship is the exclusive explanation for survival of those survived cases that have entrepreneurial funding, minus the survived cases with both entrepreneurial funding and social capital, divided by the total number of survived cases. That is, (6 − 2) / 10 = 0.4.

The QCA analysis can produce results on two levels, which are called the more rigorous “parsimonious” solution and the more inclusive “intermediate solution,” distinguished by the exclusion of contradictory combinations suggesting outlier cases. In our discussion, we focus mainly on the parsimonious solution because the scope of this research does not involve examining outlier cases.

The Data

Our data are based on the Community News Sites directory built by the Knight Community News Network (KCNN; 2007), the most comprehensive directory of American community news media sites. Although it can be difficult to specify definitions in a dynamic field such as emergent media, we follow Knight’s definition of community news media as “sites with any original (emphasis added) reporting, analysis, commentary, reviews, photos, audio, video or other content on local news, events or issues, created by individuals not employed by professional news organizations.” We note that we excluded ethnic news outlets because they are, in essence, very different entities—they play a different role in their communities, and often rely on a different funding mix, making them fertile ground for a separate investigation. The first 457 entries were collected in 2007, and were expanded to 1,278 sites in 2012. KCNN’s data collection was based not on controlled representative sampling, but rather on secondary sources and surveys, and through snowballing—meaning that statistical analysis would not be appropriate because the presence of cases is not independent.

This study uses the portions of the directory covering our four regions. Missing data were crosschecked with a directory produced by the Columbia Journalism Review (CJR; http://cjr.org/news_startups_guide) and with direct observations. Prominent new sites were added when needed. For inactive cases, Archive.org was used to look up the date of their last recorded activity. Cases that did not meet our criteria of being an emergent, civic news website functioning within its local news ecology were filtered out (e.g., sites with only national-scale news, or hobby blogs without a local public issue angle). Other sites were dropped if they were launched too recently to meet our test threshold on sustainability (see below). In sum, we analyzed 128 sites across our four regions. Three of the four authors were involved in this process, and we checked for validity to ensure that all of the authors were using the same process to define whether sites were contributing to the local civic news ecology.

Variables

Variables were chosen to capture the organizations’ business model, as well as factors that past studies have suggested are relevant to sustainability. Variables were qualitatively assessed and coded into binary entries, with 1 for full membership in a category and 0 for non-membership, or into numerical entries, using ordinal ranking of the regions. Ordinal data were coded using indirect calibration, by qualitatively assigning equal intervals for our four regions as 1, 0.66, 0.33, and 0 (Ragin, 2008). The data consist of two classes: region-wide and case-specific, with the former describing the situation in the entire area, and the latter focusing on each news site. For example, case-specific data for entrepreneurship refer to whether the individual site was an entrepreneurial effort; region-wide entrepreneurship data refer to the assessed level of overall startup activity in the area.

It is important to note that putting rank order to complex ideas such as civic tradition and social capital flattens reality and runs the risk of losing colorful information. At the same time, this is the trade-off of social science research—the ability to compare across regions must involve some standardization. We accept that trade-off here and acknowledge the limitations it places on our findings.

Analysis

Analysis was conducted with the official Fuzzy-Set/Qualitative Comparative Analysis 2.0 package using the Quine–McCluskey Fuzzy Truth Table algorithm.

The consistency-level cutoff for the outcome was set to “higher than 0.9,” as recommended by Ragin (2008) for a small number of cases.

In the 3-year model (Table 1: Model 1), affiliation with a news network and high entrepreneurship level in the region contributed to sustainability. This result was accompanied by non-membership in high civic tradition, which is counter-intuitive from a normative theoretical viewpoint. However, the unique coverage of this variable is almost zero, which means that, although it is a possible outcome, it was not proven to be significant because there were very few cases in our data that lacked a strong civic and yet survived. To prove this further, an alternative analysis was done without the civic tradition variable, and the results were the same as the first result (see Table 1: Model 2).

OPEN IN VIEWER

Table 1.

QCA Analysis Result With Parsimonious Solution (Three-Year Model).

	Coverage
Assumptions	Raw	Unique	Consistency
1. Model 1
Networked	0.255	0.149	0.970
Entrepreneurship (region)	0.626	0.134	0.952
No Civic Tradition	0.427	0.000	0.952
2. Model 2
Networked	0.255	0.149	0.970
Entrepreneurship (region)	0.626	0.134	0.952

Note. QCA = Qualitative Comparative Analysis.

Although our study uses only the parsimonious solution, the analysis produces the less rigid “intermediate solution” as well. Unlike the parsimonious solution, the intermediate solution does not fully remove conditions that are not sufficiently backed up by actual cases from the combination, to make it possible to further examine individual outlier cases. Regions with foundations supporting meta-studies and regions with foundations supporting individual projects and regions with stronger social capital were in the solution as interacting variables.

A second set of analyses was done with the sustainability threshold at 4 years (Table 2).

OPEN IN VIEWER

Table 2.

QCA Analysis Result With Parsimonious Solution (Four-Year Model).

	Coverage
Assumptions	Raw	Unique	Consistency
Networked	0.223	0.119	0.928
Entrepreneurship (region)	0.030	0.014	1.000
University affiliation	0.435	0.000	0.948
No Civic Tradition	0.625	0.129	0.935

Note. QCA = Qualitative Comparative Analysis.

In this analysis, being networked and having strong local entrepreneurship were again relevant. Non-membership in the civic tradition variable again came up with zero unique coverage. Being formally affiliated with an educational institution emerged as having an impact, with a small coverage but robust consistency.

The variables contributing to sustainability in the intermediate solution were regional and individual foundation support, and social capital.

For both the 3-year and 4-year thresholds, the analysis of necessary conditions yielded high coverage scores and moderate consistency scores (Table 3).

OPEN IN VIEWER

Table 3.

Necessary Condition Test.

Assumptions	Coverage	Consistency
1. Model I (3 years)
Networked + Entrepreneurship (region)	0.956	0.775
2. Model II (4 years)
Networked + Entrepreneurship (region) + University affiliation	0.937	0.770

Although QCA derives logical implications instead of statistical inference, it remains necessary to test whether the results are simply a matter of statistical skew. Because the variables such as being networked and having a university affiliation were case-specific and not region-wide, we need to verify that they are not simply reflections of other region-wide values; in other words, we need to check that the difference we are seeing between the regions is in fact attributable to the variables we examined, and not to some other characteristics of the region—the QCA equivalent of a confounding variable. To test this, we examined the region-wide distribution of surviving civic news websites as shown in Tables 4 and 5. The distribution shows it is unlikely that the effect of being networked is a result of simply belonging to a specific region, in which case it could be an effect a confounding characteristic of the regions. For example, sites from New York and Chicago comprised a larger portion of surviving news media, but those regions had in fact a higher rate of survival for projects without networks.

OPEN IN VIEWER

Table 4.

Region-Wide Distribution I (Three-Year Model).

	Total	Seattle	NYC	Minneapolis	Chicago
1. Sustainability
Survived	127	27	47	15	38
Closed	10	2	1	3	4
Total	137	29	48	18	42
2. Networked (survived only)
Yes	35	15	3	11	6
No	92	12	44	4	32
Total	127	27	47	15	38

OPEN IN VIEWER

Table 5.

Region-Wide Distribution II (Four-Year Model).

	Total	Seattle	NYC	Minneapolis	Chicago
1. Sustainability
Survived	116	22	44	14	35
Closed	12	4	1	3	4
Total	128	26	45	18	39
2. Networked (survived only)
Yes	31	12	22	11	6
No	85	10	42	4	29
Total	116	22	44	15	35
3. University Support
Survived
With Univ. Support	7	2	2	1	2
Without Univ. Support	109	20	42	14	33
Total Survived	116	22	44	15	35
Closed
With Univ. Support	0	0	0	0	0
Without Univ. Support	12	4	1	3	4
Total Closed	12	4	1	3	4

Note that the distribution of the network and educational institution variables (Tables 4 and 5) is not clear or could even seem contradictory, although the truth table analysis finds those variables both have a positive impact on survival. The reason for this apparent discrepancy is that the distribution shows frequencies not controlled for by other variables, whereas, in the truth table analysis, contradicting variables and the combinations among them eliminate each other.

Discussion

The results of the parsimonious analysis show that organizations that are networked and exist in regions with high entrepreneurship are more likely to survive for 3 years. For those surviving 4 years or more, a connection to a postsecondary institution also contributed. QCA also searches for effects of combined interacting variables; our results did not yield such interactions. These findings provide initial glimpses of three conditions that may improve the likelihood of sustainability of civic news sites.

The contribution of membership in networks to sustainability is this article’s key finding. The argument that news networking is a good way to enhance local journalism goes back to the organizational ecology studies of newspaper mortality rates by Carroll and Hannan (1989) and was recently exemplified by a J-Lab report (Schaffer, 2012). Recall that our article is a comparative case study; as such, this is an initial finding, and we are cautious about making recommendations until these findings have been replicated. However, this study suggests that funders may do better to spend their limited resources on network-building projects rather than primarily funding individual projects.

Research conducted by the authors (citation withheld) on the Madison Commons project points to one working form of news network. That organization produces content that it gives away for free, to be republished by other news organizations, positioning itself at the hub of a distribution network born without the effort of the institutional news industry, but which could add significantly to the news ecology within which that industry is embedded.

As for the contributions of entrepreneurship and of affiliation with educational institutions, one could argue that the largest factor is financial. A region with a history of entrepreneurship could offer greater funding opportunities. Affiliations with educational institutions provide resources such as free or subsidized office space and equipment, plus access to a cheap labor force of student interns, but, more importantly, also provide institutional credibility and a (generally) more explicit long-term commitment to projects. In other words, public institutional commitments matter.

However, these findings could make other contributions as well. Urban studies theorists have asked why geography matters to entrepreneurship in the Internet age. Pratt (2000) argues that marketing and coordinating employees give a geographical bias to even the production of software, which leads to the clustering of new media practitioners in New York’s “Silicon Alley.” Zook (2005) adds that “The Internet made standardized information and data widely available but this did not negate the value of face-to-face interactions, local norms, and local institutions in capturing and sharing non-ubiquitous knowledge” (p. 40). Beyond this concentration effect, high levels of tech entrepreneurship provide a base for entrepreneurial startups and a supply of potential users who are habitual consumers of online information and online network participants. This was the case in Seattle (Friedland, 2014) where former Microsoft employees formed many sustainable neighborhood news sites.

The influence of educational institutions could also extend beyond financial. The “teaching hospital model” of journalism is one way journalism schools can get involved in local news ecologies (Lemann, 2009). Journalism schools have the opportunity to become “‘anchor institutions’ in the emerging informational ecosystem” (Anderson, Glaisyer, & Smith, 2011), and journalism programs must “begin to think of themselves as more than simply just the teachers and trainers of journalists, but rather as the anchor institutions involved in the production of community-relevant news that will benefit the entire local news ecosystem” (p. 29). As some universities go down this road, they may be adopting a social responsibility model of journalism (Peterson, 1963), which, combined with a responsibility to teach students, could insulate the organizations from market pressures, perhaps even allowing them to survive if they are losing money. It is important to ask whether sites with college affiliations are vital in community involvement beyond supplying trained journalists and educated consumers. However, as long as the site produces relevant local news read by community members, there is little reason to ignore it.