Open source and journalism: toward new frameworks for imagining news innovation

Journalism and technology

There is a long history of journalists working with digital technology, often with the primary goal of improving the news story. This fusion of computing power and news reporting, sometimes called computational journalism (Flew et al., 2012), is designed to bring ‘algorithms, data and social science to supplement the accountability functions of journalism’ (Hamilton and Turner, 2009: 2). Cox (2000: 1) argues that the first U.S. instance of computer-assisted reporting (CAR) was in 1952, when CBS used the Remington Rand UNIVAC to predict the outcome of the U.S. presidential race between Eisenhower and Stevenson. Reavy (1996) also points to innovators such as Philip Meyer, who in 1967 used an IBM 360 mainframe to analyze survey data about the Detroit riots. Other early-adopter journalists began using computers in their work to analyze data.

In 1973, Meyer wrote Precision Journalism, advocating for the greater integration of computers, data, and social science methods in journalism practice. That same year, in perhaps an early iteration of open-source ethics in journalism, the New York Times made public an interactive system with information about New York City police statistics. Possibly the most visible moment for computational journalism came in 1989 with Atlanta Journal Constitution reporter Bill Dedman’s Pulitzer Prize-winning report on unequal housing practices and red-lining. The U.S.-based National Institute for Computer-Assisted Reporting was also formed that year (see Cox, 2000, for an excellent summary).

By 1991, Meyer argued in the second edition of his book that journalists interested in CAR were their own breed of reporters, who often bought their own computers in advance of newsroom technology (Meyer, 1991). Journalism scholars began writing about the CAR movement around this time, and in 1996 Houston identified three key aspects of technological innovation and journalism production: database reporting, spreadsheets, and online reporting. Other articles in the late 1990s and early 2000s function as scholarly discoveries of the increasing prevalence of the CAR journalist (Gamson, 1998; Maier, 2000). However, the computer-assisted reporter was still primarily a journalist rather than a technologist; the underlying goal was to produce a better story. Today, however, the programmer-journalist differs from this CAR reporter in seeing the end product not as a story but instead as a ‘productive artifact’ of ‘information filtering’ (Jonathan Stray, personal communication, 27 October 2011).

Royal (2012) may have been the first scholar to look at the programmer-journalist in this new iteration: as separate from the CAR journalist. She interviewed and observed interactive news journalists at the New York Times, and tried to assess how these technologists were integrating into the newsroom. Her understanding was that these technologists simply loved and appreciated the values of journalism. And, when Nikki Usher talked to lead technologists at the New York Times, the sentiment was more about the ideals and ethics of journalism than it was about the technology involved. As one technologist put it: ‘I could work for Google and make a whole lot of money, but I just like journalism’ (field notes, 14 April 2010). However, what this literature and these observations leave missing is how technology—its practices and philosophies—influence newswork.

The articles that specifically deal with journalism and open source point to promises and potential without delving into issues of philosophy, culture, or conflict. For example, Witt (2006) offers an ‘annotated checklist to guide those interested in reinventing journalism.’ We agree with Anderson (2012), whose sobering assessment of the majority of literature on computational journalism (which comes from technologists) is that it is simply far too concerned with output, too focused on journalism norms without question, and overly utopian. Consider, for example, the optimism of Cohen et al. (2011: 1): ‘There is a real danger that the proud tradition of original, in-depth investigative reporting will fade away with the ailing traditional news media.… Luckily, a second trend is on our side: the continuing advances in computing.’

Scholars write about journalism as needing an upgrade to the digital age through computer programming. Many argue that computational journalism will both lead to better investigative journalism and create new forms of engagement with audiences, as Flew et al. (2012) note. Hermida’s (2010) write-up of a workshop with IBM addresses how data visualization technologies can be used to enhance journalism and public knowledge. Diakopolous et al. (2010, 2012) similarly suggest that computational journalism can help journalists with their work—for example, by providing tools for visualizing social media data and filtering it for source information. All of these are worthy goals, of course, but this journalism-focused approach to technological innovation might limit the ability of open source, as a structural and cultural dynamic, to inform new normative behaviors for journalism.

We acknowledge that one theoretical lens to explore the interaction of technology and open source is through science and technology studies (STS), particularly its emphasis on the social construction of technology. However, we have chosen to adhere more closely to the texts that speak about the nature of open source as well as to those texts that speak to journalism and innovation. By analyzing journalism norms directly through the language of open source, we seek to achieve a more coherent assessment of how these two value systems intersect.

Open source as architecture and culture

Open source—sometimes referred to in the literature as ‘free and open-source software,’ or F/OSS (e.g. Coleman, 2004)—can be understood as a practical desire for open programming and a philosophical belief in social responsibility through freedom and openness (Coleman, 2012; Coleman and Golub, 2008). In this sense, open source is about ‘operating systems and social systems’ (Kelty, 2008: 57), or both an architectural framework and a cultural context. It is this dual emphasis on the materiality of open source and the ethos of open source that is important for understanding and assessing how journalists and programmers have worked together to build open-source solutions for news. Both architecturally and culturally speaking, open source is:

characterized by a non-market, non-contractual transfer of knowledge among actors, sharing relevant information with a non-definite set of other actors without any immediate recompense. Actors share their ideas with the clear purpose of contributing to a joint development. (Balka et al., 2009: n.p.; cf. Benkler, 2006: 60)

Embedded in that definition are the key structural characteristics of open-source projects (Weber, 2004): they are motivated less by proprietary, profit-driven control, and more by a communal interest in the greater good (Turner, 2005); they rely on contributions from a distributed set of actors/experts, typically outside the boundaries of conventional organizations; they offer non-monetary forms of reward, such as reputation, play, and sense of belonging (Himanen, 2001); and they are premised on an underlying assumption of open, collective intelligence (Lévy, 1997). In short, the assumption is that software works best when it is freely available and jointly programmed. ⁴

At the center of open source is the source code itself: a recipe of computer programming language that structures the parameters of a software application’s functionality (Raymond, 2001). The source code is the kernel from which projects are developed, line upon line of programming script. In open-source projects, users can access, modify, and freely distribute the source code, thus allowing for a successful idea to scale quickly as it is copied elsewhere or built upon by others. As such, open source thus renders transparent the DNA of the technology, encourages users to make their own contributions to the code, and potentially facilitates greater speed and efficiency.

The architecture of open source is about both the collaborative nature of open software coding (the process) and the substance of what such software can produce because it is collaboratively programmed (the product). Wikipedia is often heralded as one of the quintessential open-source projects because of its free-editing structure and its higher-than-expected quality (Bruns, 2008; Reagle, 2010). Open source thus takes as given that the whole is greater than the sum of individual programming parts (Raymond, 2001). This is not to say that open-source coding invariably produces successful sites or applications—for the vast majority of these efforts fail to garner a sustainable community of interested developers (Hindman, 2007)—but rather that open source carries normative assumptions about the proper architecture for software systems.

This architecture motif can be extended to thinking about how software gets built. Raymond (2001) famously outlined two general approaches to software development: the ‘cathedral’ model, which represents the traditional method of engineering software through centralized control, clearly defined roles, and top-down direction by a select few designers; and the ‘bazaar’ model, which represents the (now common) open-source method of assembling software through a bustling hive of collective, incremental contributions. This latter method emphasizes treating users as co-developers; encourages their participation in reporting errors (or bugs) in the source code and working to fix those bugs; allows them to participate in the iterative testing of software throughout; and invites them to annotate their contribution in a running log of the software’s progress.

Like the architecture of open source, this open-source culture carries with it certain assumptions about technology’s normative values: namely, transparency (coding in the open, with bug-tracking); iteration (continuously releasing unfinished code for beta-testing); tinkering (privileging play and experimentation, focusing on the process of work more than its outcome); and participation (encouraging input from the widest possible group of collaborators). This open-source culture can be understood as part of the larger ‘hacker culture’ intertwined with the historical development of the internet itself (Kelty, 2008). With this in mind, we take these normative values and see how they might work within the context of journalism.

Journalism as transparency. Hackers are ‘frequently committed to an ethical version of information,’ believing that information ought to be shared for the greater good (Coleman, 2012: 3). Open-source culture assumes a transparent rubric whereby changes to the source code are shared, accompanied by excessive documentation. Journalism itself strives for transparency as an ideal (Singer, 2007), and digital journalism has enhanced opportunities for such transparency (Phillips, 2010). But open-source transparency can push journalism further. Instead of seeing news as the end product, users would be able to see journalism as a more fluid set of interactions—a process to which they can meaningfully contribute (Robinson, 2011). The Guardian revealing its newslist of the day’s stories, ⁵ begins to shed light on how media organizations give shape to the news. Yet the transition to such openness, because of the challenge it poses to journalistic autonomy, ‘has yet to make the kind of impression forecast by many scholars’ (Karlsson, 2010: 543).

Journalism as tinkering. Open-source culture also advocates an ethic of tinkering, in line with burgeoning ‘maker’ communities of DIY hobbyists (Sivek, 2011). This ethic privileges playfulness, remixing, and radical experimenting, with a pro-social interest in doing good. Tinkering—whether it’s simply toying with a product that may already seem to be working fine, in hopes of making even small improvements, or seeking to make something more finely suited to one’s needs—is a core part of the hacker ethic, with hackers having an ‘inquisitive passion for tinkering’ (Coleman, 2012: 3). Journalism could use a little more tinkering. Too often, newsrooms get swept up in attempting big change (as in the push for convergence documented by Singer, 2004), when, instead, newsrooms might tinker with their existing product: perhaps recalibrating small pieces of the organization, or making incremental improvements to the workflow. Tinkering also means building, suggesting that journalists could begin rethinking how to play with their existing products (e.g. blogs and social media) and recharge them. Nonetheless, we can see journalists as tinkerers already, working to refine and perfect a story; yet, it remains possible to push journalism beyond this form of tinkering.

Journalism as iteration. Along with tinkering comes the freedom to fail. Open-source software privileges iterative forms of development. In contrast to the traditional ‘waterfall’ method of software development—where a product only appears after a step-by-step cascade of formalized and centralized planning, design, testing, and implementation—an iterative method focuses on process rather than product: on releasing beta versions early, engaging users in collaborative design, and being agile in responding to user needs as they are discovered (Chocano, 2004). In open-source development especially, feedback from the community is crucially important in the rapid progress of new ideas (Wusterman, 2009). Traditional journalism faces many challenges as it attempts rapid change that might be unsuccessful, in part because of the economic pressure on many legacy news organizations, leaving little room for failure. But not every change need be a ‘reinvention of the newsroom,’ as the New York Times described its 2007 efforts (Susan Edgerly, personal communication, 2 April, 2010). Newsrooms can take small chances, make incremental steps toward change, and ultimately overcome the fear of failure. Some are starting to do so, and out in the open, such as the Journal Register Company, a U.S. newspaper firm that encourages open news meetings, suggesting that ‘the process of journalism matters as much as the product’ (Garber, 2011).

Journalism as participation. Similarly, open source hinges on distributed participation. The concept, though not historically part of journalism’s normative framework, is emerging as a growing fact of life for news organizations (Singer et al., 2011) and a ‘founding doctrine’ of news innovation—a journalistic ethic for the digital age (Lewis, 2012b). Open source reinforces this ethic with the assumption that ‘given enough eyeballs all bugs are shallow’ (Raymond, 2001), that collective intelligence and wide-scale participation inexorably lead to better, faster, and more efficient development of software. In peer-to-peer (P2P) projects like open source, even though claims to power and authority are unequal, the rhetoric of equal contribution allows participants to construct a cultural framework in which ‘they can be imagined as peers devoted to a collective mission’ (Turner, 2009: 77). An open-source approach to news thus continues to build on what is already happening in journalism writ large (for examples, see Robinson, 2011; Singer et al., 2011), but suggests that users take on a more active, interlinked, and monitorial kind of role—helping to supervise the ‘software’ of news, rather than merely adding comments post-publication, as too often is the case.

In sum, open source as architecture and open source as culture together represent a platform for and ethic of transparency, tinkering, iteration, and participation. The next two sections extend this focus on architecture and culture, examining first how they have been applied (or not) in institutional newsrooms and, second, how they might lead to a fundamental rethinking of journalism. Notably, these values do (and in some ways are intended to) clash with existing practices of newswork, as we note later.

Open source and the newsroom

Though journalists and technologists are increasingly working together to bring open-source tools into the newsroom, they have largely failed to interrogate the old processes of newswork. Instead, their collaboration has focused on new tools that help traditional journalists further their goals of doing journalism the way it has always been done. While it is notable that journalists have new ways for achieving the best of traditional journalism, what open source has yet to do is push journalists beyond the newsroom, figuratively and literally. For instance, many of these tools are created mainly for newsrooms, even if the open-source code, by its nature, implies possible applications for anyone online. Furthermore, most tools fail to provide the kind of transparency that open-source journalism could ideally embrace.

A consistent trend has emerged as newsrooms have adopted new technologies: journalism practices, norms, and routines have made ‘ordinary’ the potentially innovative nature of blogging (Singer, 2005), user-generated content (Wardle and Williams, 2010), and social media (Lasorsa et al., 2012). For example, user-generated content has been deemed useful only if it meets the needs and quality of a pre-determined news story (Usher, 2011). Newsrooms have been quick to impose social media ethical guidelines; instead of experimenting with how audience participation might change the journalism conversation, news institutions have tended to retrofit yet another reporting tool (Hermida, 2012).

This approach of tool-driven normalization is apparent in several emerging examples of open source in journalism, all of these much heralded by the journalism community at events such as the Online News Association convention or by institutions such as the Knight Foundation. ⁶ In concert with the literature reviewed that celebrates what open source can do for journalism, these tools signal the acceptance of traditional journalism practice and methods as an inherent good, and aspire to help journalism reach its ideals. Yet open source offers the capacity to rethink some of these practices because of its alternative emphasis: participation from multiple sources, collaborative community building, and increased transparency.

The New York Times developed Fech, ⁷ a tool that helps journalists crawl financial disclosures by political candidates simply by knowing a filing number (Strickland, 2011). Just as the discourse around open-source tools emphasizes their pro-social benefits, Fech’s creators note that more access to these filings will lead to better journalism (personal communication, Derek Willis, 24 October 2012). This is quite in line with open-source thinking that more transparency will produce better results. However, there are barriers to entry that limit wide public participation: while the source code is posted on GitHub for other developers, the tool has been built mainly to help people in the newsroom. Those who do not have the skills cannot build upon the tool.

Some open-source advocates have pushed for greater transparency with news and investigation, suggesting that the public can actually help ongoing investigative work (Meredith and Meinrath, 2011). However, other open-source tools continue to maintain a one-to-many relationship with the public. ProPublica’s TimelineSetter—an interactive HTML timeline tool that turns ‘ugly spreadsheets into pretty timelines’ (Phelps, 2012)—makes it easier for the public to understand the unfolding events of a particular news issue. ⁸ The process of reporting the story is shared with the public—but generally after the fact. The practice of keeping every reader apprised of a new detail is not what we advocate; but we do think that reporters could tell the story of how they do their work as they do it, perhaps inviting the community to participate in the open-sourcing of their news processes.

Tool builders themselves have begun to critique this practice of building applications that assume this normative approach to solving problems. As New York Times programmer-journalist Jacob Harris (2011, para. 2) wrote:

What we’re really looking for is something that will tame the complex work of reporting itself. What we see in each new tool is the frothy hope that it will be, somehow, The One. But it never will be. There is no tool that will make the journalism simple.

Instead, he argues that we ought to relish the complexity of journalism and think more broadly. The best programming happens as a result of complexity and unknowns, and, as he argues: ‘The essential complexity in journalism is there because good journalism is by its definition not something that’s been done before’ (2011: para. 8). Open-source tools are not the answer if they reinforce the status quo; their potential lies in the capacity to challenge the existing norms of traditional journalism.

Some tools do embrace open-source culture, primarily because they create opportunities for collective intelligence and bug-fixing, and because they envision journalism as process rather than product. One such example is DocumentCloud, ⁹ a Knight News Challenge winner led by Aron Pilhofer of the New York Times. The goal of DocumentCloud is to host primary source documents for researchers and journalists, which they can then annotate and publish to the web. ‘Readers will also be able to quickly search, annotate and bookmark documents—and for the first time link directly to specific pages or passages’ (Seward, 2008: para. 4). This collaborative, annotative journalism resembles the kind of documentation that is common to open-source software practices, from the back-end of Wikipedia to Linux kernels.

New frameworks for news

With architecture and culture in mind, we propose a possible intervention by rethinking the framework of traditional news. Going beyond merely swapping tools or tinkering with newsroom culture, a tangible, radical change might be to imagine how to make news structurally different. This process of rebooting journalism might embody values that are different from, but are nonetheless still complementary to institutional journalism. Such a reshaping requires both change in structure (technology architecture, in this case open-source software) and change in milieu (technology culture, in this case the hacker ethic), played out across the profession at large and, in particular, newsroom contexts where innovations get deployed. This rethinking might take up questions such as: how might news production become more participatory, more open, and more iterative and process-based—in the style of open-source software development? Likewise, how might newswork be rethought to engage information in new kinds of ways, reconfiguring the journalist’s role relative to the community to encourage transparency and engagement? How can newswork, both structurally and culturally, become more iterative and open to learning through failure? Herewith we suggest two thought experiments.

The limitations of open source

Open source, like journalism, also requires critique. As such, we find it necessary to raise some of the most common problems with open source, which are also likely to come with open-source journalism. Open source has been criticized for its lack of scalability, its re-creation of hierarchies, and its unclear relationship with corporate entities. Within the context of combining open source and journalism, we should also question the influence of major institutions like Knight, Mozilla, and Google in this integration. These problems, however, do not overshadow the promise of the underlying ethos of open source as a starting point to rethink some of the normative assumptions about journalism; rather, examining the concerns avoids making open-source journalism sound overly utopian.

As Raymond (2001) notes, every good open-source project starts with a developer’s itch. But open source requires that people build and contribute (Kelty, 2008)—and many such projects fail when a critical mass of participation never appears or quickly falls off (Hindman, 2007). Newsrooms face two challenges in this regard: the first is attracting top programming talent to news projects, beyond the informal get-togethers of Hacks/Hackers and the contest-style collaborations of Mozilla and Google. The second is attracting people to projects once they are built: for example, getting citizen contributors to put the dots on the map to show where an event has taken place. Hackers need to continue to feel that working with journalism is indeed a project that helps further the freedom of information; if newsrooms are too reluctant to innovate with greater transparency and iterative development, hackers may be less likely to choose to spend their time working with journalism. Similarly, users need to feel that their contribution to an open-source project makes a real difference to the process, not merely as an afterthought to the published product.

Additionally, while open source suggests equality of contribution, rooted in the rhetoric of technology culture (Turner, 2009), good open-source projects require leadership. Open source can be led by a ‘benevolent dictator,’ as in the case of Torvalds’ steering of the Linux operating system (Raymond, 2001), or it can function as heterarchy (Bruns, 2008), with distributed authority. But someone must be in charge of standards. If the open source community becomes too concerned with standards, innovation and participation may be hampered—as can be seen in the case of Debian Linux kernel, which requires credentialing and testing for potential programmers to become certified. Ultimately, many open-source projects have drifted toward centralization and bureaucracy; this is true even of Wikipedia, the widely heralded open-source success (Reagle, 2010).

We are also concerned with claims that journalism should become more like Silicon Valley, as news prognosticator Alan Mutter (2011) put it. Silicon Valley, as a culture ‘promiscuously combin[ing] the free-wheeling spirit of the hippies and the entrepreneurial zeal of the yuppies’ (Barbrook and Cameron, 1996; see also Turner, 2005), inspires the freedom to fail, but Silicon Valley also represents a culture of backstabbing profiteering by venture capitalists and bursting bubbles. And, like corporate culture generally, it’s driven by a profit motive that doesn’t always sync with the public service orientation of journalism’s ideals. Open source, decoupled from its roots in free software (Stallman, 2002), in many instances has been co-opted in this environment; major corporations such as IBM have been among the largest investors in open-source development (Hindman, 2007). While it’s true that open-source software continues to shift away from purely ‘free’ projects, it nevertheless can and should preserve its origins in the hacking spirit of transparency.

Finally, these problems with open source also point to the need to question its aggressive promotion by the likes of Knight, Mozilla, Google, and other institutions seeking to shape the future of journalism and technology. Issues of power, ideology, and control ought to be part of future studies of this emerging connection between the journalism field, tech communities, and open source.

Conclusion

With the emergence of programmer-journalists in the newsroom, the energy around open source and journalism is more than a passing fancy of the hacker community. What hackers bring to journalism has the potential to reinvigorate newswork in a way that moves beyond merely thinking about newsroom economics and gets to the heart of newsroom philosophy. Open source offers the opportunity for technologists and journalists together to think about traditional journalism as espousing new values—not diminishing long-standing virtues, but instead providing a new framework that makes journalism more relevant in a participatory, digital culture. These values of iteration, tinkering, transparency, and participation, each embedded in the open-source ethic, can be brought into the newsroom as architecture and culture—as a structural retooling of news technologies and user interfaces, and as a normative re-articulation of what journalism means in a networked media setting. While acknowledging that our repositioning of new frameworks for journalism may be radical, we hope that they also offer a chance to think expansively about the new intersection between open source and journalism in the newsroom.

Thus, we suggest an important warning: just as journalism and its professionalism can become calcified ideology (Deuze, 2005), creating barriers against needed innovation, open source too can become ideology (Kreiss, 2011). The culture, ethic, and practices of open source should remain open to critique. Open source should not be seen as the end-all answer but rather as a strategy among many for accomplishing innovation. Codifying what it means to do things in an ‘open-source’ way necessarily limits the expansive opportunities for unexpected creation that might occur outside such a framework. Ultimately, we should be careful not to fetishize this concept, or any other, as a panacea, particularly at a time when the latest technology invention is too readily seen as the salvation for journalism’s troubled model in the 21st century.