Abstract
Civilian victims of aerial warfare too often go uncounted and unrecognised by the belligerents. Myriad images and video of attacks against Syrian civilians did little to end their suffering, for example. The UK-based not-for-profit Airwars has had tangible impact on civilian harm disclosures and reparations because they have been able to shape such representations in a form that will be recognised by those with the power to enact change. Building on established theories of media witnessing and their extension to what Gray calls ‘data witnessing’, we argue that Airwars reveals the operative role of framing in open-source investigation and the forms of it witnessing it produces. Through interviews with key team members and detailed analysis of Airwars published methodology and other materials, this article shows how open-source investigations broadens the frame for witnessing civilian harm and in doing so generates relational, multi-scalar accounts of state violence that remain open to contestation and confirmation. In doing so, Airwars claims an epistemic authority via its distinctive framing of emergent practices of witnessing that depend upon the assembling of roles, standards, spatialities and techniques.
Over the last decade, digital open-source investigation has become an increasingly widespread practice. Often traced back to the Arab Spring and incidents such as Russia’s downing of Malaysian Airlines flight MH17 over Ukraine, open-source investigation describes the use of publicly available information – documents, statistics, social media posts, YouTube videos, media reports, satellite images, online databases, discussion forums, etc. – to investigate disputed events (Beauman, 2018). In its more ad hoc forms, open-source investigations can be found in the work of freewheeling online collectives on platforms such as Reddit, most infamously in the aftermath of the Boston Marathon bombing. But its more consequential manifestations can be found in a diverse collection of professional organisations that includes Syrian Archive, Bellingcat, Forensic Architecture, Amnesty International and Airwars, the UK-based not-for-profit that is the subject of the present article (Higgins, 2016). At the same time, experts such as Bellingcat’s Elliot Higgins became an ‘interpreter tier’ between on-the-ground citizen journalists and mainstream media organisations, working to collate, analyse and mobilise diverse sources (Sienkiewicz, 2014). But as our study shows, institutions like Airwars have developed a much wider role and purpose.
While human rights organisations, governments, companies and individuals have conducted research using such information for decades, ‘the volume of content available and the speed of its transmission and relay’ has transformed ‘human rights organizations’ ability to use open-source content for advocacy and accountability, ushering in a new era of human rights investigation’ (Dubberley et al., 2019: 5). 1 With a growing acceptance of the reconstruction of events through open-source information by media organisations and legal entities such as the International Criminal Court, open-source investigation has become a key form of witnessing as both a socio-political and juridical practice. This article uses Airwars as a case study to show how one open-source investigation agency catalyses on-the-ground testimony to produce authoritative accounts of civilian harm that can contribute to and even spur public debate while also being legible and credible to militaries and other government actors.
Based at Goldsmiths College, University of London, Airwars has operated since 2014 as an independent monitor of civilian harm by militaries, providing journalists, among other stakeholders, with alternative sources of information to contest or confirm government and military claims. Airwars operates as a not-for-profit company registered in England and Wales. It is funded by philanthropic organisations (such as the Joseph Rowntree Charitable Trust) and public donations, as well as volunteer contributions. The organisation considered financial support from organisations and individuals on a series of conditions, including that they ‘will not normally accept financial support from organisations’ involved in the ‘manufacture, sale or distribution of arms/weapons’ (Airwars, n.d.). In an interview in 2020, Director Chris Woods told us the origin story of Airwars, which he co-founded with French data journalist Basile Simon, who remains on its board. As a reporter with The Bureau of Investigative Journalism, Woods became deeply aware of a central problem in modern conflicts: militaries weren’t admitting to civilian casualties, even when they knew they took lives.
NATO’s intervention in Libya had troubled me because NATO had killed probably several hundred civilians in that war, [which was a] relatively low number of civilian deaths, but. . . five years after that war, NATO’s official position was still that it hadn’t killed any civilians, not because it didn’t believe it hadn’t killed any civilians, but because it didn’t have any way of understanding where it had killed civilians. . .
Targeted communities knew that civilians were dying: they were losing family, friends, neighbours. But states inflicting violence from the air could all too easily deny civilian death, pretend it wasn’t happening, or simply insist that everyone killed by its bombs was an enemy combatant. With the widespread adoption of social media, particularly Facebook and Twitter, citizens on the ground were able to document and, crucially, share imagery and eyewitness accounts of airstrikes and other military actions producing civilian harm, enabling non-local reporters and conflict monitors to more easily collect and collate data (Andén-Papadopoulos, 2013a; Wu and Montgomery, 2020). As Woods told us, ‘when we began Airwars, it was a premise. It was a question. Is it possible to track civilian harm from the perspective of affected communities in real time and leverage that information?’
To date, Airwars has tracked around 60,000 cases of civilian harm, established processes for relaying information to journalists, elected representatives and militaries and even developed a working relationship with the US military around disclosure of civilian harm events and, to a lesser extent, mechanisms for restitution. While Airwars partners with news organisations such as The Guardian, New York Times and Washington Post, it also assembles an heterogeneous array of eyewitness testimony, visual evidence and other documentation to frame authoritative accounts of civilian harm events through its own online database and published reports. As staff at Airwars explained, military assessments tend to over-rely on electronic and aerial surveillance: Militaries observe the bodies of civilians from the air, but if you bomb urban areas, you will not see civilians killed. They will be rumbled under the building collapsing on top of them and only people who are in the stage of strike can capture the evidence from the ground. Then they share it on different channels and Airwars carefully pick up those claims. Therefore, only gathering the information from the sky will never present a real picture of civilian toll.
In this context, Airwars sees its role as both monitoring conflicts and advocating for the validity of their alternative representations. For its advocacy to gain an audience and have impact, Airwars believes it needs to ensure its methods and results present as legitimate, and to build relationships of trust with its stakeholders, despite potential antagonism with state actors. As one assessor and advocacy officer told us: ‘to make [a] change, it must be influential people . . . sitting with us, talking to us and be really willing to make a change in their work’. Public awareness, care and concern are important goals, but Airwars’ larger aim is to change military practices and help bring justice to civilians harmed by conflict through the identification of potentially unlawful violence, including war crimes. How, then, does the production of authority take place at Airwars? What socio-technical processes and practices enable the formation and standardisation of their accounts of civilian harm? In this article, we argue that Airwars broadens the frame for witnessing civilian harm and in doing so generates relational, multi-scalar accounts of state violence that remain open to contestation and confirmation. In doing so, Airwars claims an epistemic authority via its distinctive framing of emergent practices of witnessing that depend upon the assembling of roles, standards, spatialities and techniques.
For this research, we conducted an empirical study of Airwars involving interviews and close reading of organisational documentation. Eight in-depth, semi-structured conversations were carried out with eight key staff across various roles: researchers, assessors, geolocators, advocacy specialists and managers. Interviews were conducted online via Zoom, between August and September, 2020 and they lasted between 45 and 80 minutes. Participants were informed about the research and its aims and gave their informed consent to participate. We offered the participants the choice of being identified or anonymised. A member check was also conducted after the interviews. We provided the participants with their interview transcripts, along with an overview of the preliminary results of the analysis. We also conducted a close reading of organisational documentation (Prior, 2008: 479–492) such as incident reports, manuals, policies and reporting standards. Interview data and documentation were thematically coded using an iterative process of identifying emerging themes and topics and establishing relationships between themes.
The resulting study contributes to understanding how open-source investigations function with a particular emphasis on how they might produce authority. Building on established theories of media witnessing (Frosh and Pinchevski, 2009) and their extension to what Gray (2019) calls ‘data witnessing’, we argue that Airwars reveals the operative role of framing. Our argument is that open-source conflict monitors like Airwars act as both agents of witnessing in their own right and as intermediaries for on-the-ground eyewitnesses and journalists, but what is most significant is their assembling of numerous disaggregated accounts into a relational and open-ended database through processes of standardisation and verification. While data witnessing (Gray, 2019) provides a generative analytical lens, it needs to be augmented with a deeper account of what is left in and out in the aggregation, analysis and circulation of data. The framing of data witnessing at multiple perspectives and scales enables the production of authority. Asking how data witnessing frames and is framed, we connect the digital and media witnessing practices of open-source investigation to the political question of whose lives count, how they are counted, and how responsibility can be generated at political as well as ethical level. For Airwars, this frame is constituted by the interplay of expert and institutional roles, the implementation of data and research standards and the marshalling of distributed spatialities via digital platforms. Crucial to this is the published Airwars methodology, a technique of research and assessment which insists on keeping the frame of every event it investigates open to be contested, confirmed or supplemented by new information. Airwars ensures that its database operates as an active witnessing apparatus, since contestation constitutes a crucial component of witnessing in public fora (Schuppli, 2020). While there are limitations and risks to the Airwars approach, we argue that framing data witnessing enables political and material claims to be made for the harm done to individuals and communities that are often outside the frame of Western media and political debate.
Framing data witnessing
Open-source investigation can be understood under the broad umbrella of media witnessing, defined by Frosh and Pinchevski (2009) as ‘witnessing performed in, by and through media’ (p. 1, emphasis in original). Media witnessing builds on an established body of research on witnessing and testimony in response to violence, atrocity and trauma, which emphasises the impossibility of fully representing trauma yet the necessity of seeking to give account (Felman and Laub, 1992; LaCapra, 2001). As Boltanski (1999), Chouliaraki (2006) and others have pointed out, the distance between the viewership of reportage of atrocity raises fundamental questions about the distinction between witnessing and spectatorship, even when images of events and voices of survivors and eyewitnesses are included. Shared knowledge might well be produced, but it is an open question whether this produces in any meaningful way the ‘responsibility to the event’ that Peters (2001) influentially defined as the essential element of witnessing. Rentschler (2004) argues that these questions of what can be witnessed and whose suffering counts speak to fundamental hierarchies of valuation inscribed within Western media institutions and cultures. Media witnessing, then, is always political, as it both maintains and attempts to negotiate whose lives count as grievable (Butler, 2004). Distance makes witnessing ambivalent (Kozol, 2014), not least because it becomes ‘a ground of distrust and doubt’ in the ‘veracity gap’ that applies to all witnessing (Peters, 2001: 717). For Peters, this gap can be addressed in different ways, but most notably through the assertion of presence – of the body in the time and place of the event – which can be captured by media technologies.
While broadcast media was the focus of much early research into media witnessing (Chouliaraki, 2006; Ellis, 2000), more recent scholarship has shifted focus to the impact of digital technologies. These technologies and associated practices of witnessing have generated novel responses to the ‘veracity gap’. Smartphones have produced the potential for ‘mobile witnessing’ (Reading, 2009), ‘crowd-sourced’ evidence (Andén-Papadopoulos, 2013a), ‘digital witnessing’ (Chouliaraki, 2015) and ‘witnessing databases’ (Papailias, 2016), while their convergence with social media has enabled affected individuals and communities to narrate crises in culturally distinctive ways (Wu and Montgomery, 2020) and to self-represent their witnessing (Rae et al., 2018). These technologies have made media witnessing more mobile, intimate and shareable, enabling it to capture and communicate the intense affectivity of events (Andén-Papadopoulos, 2013b; Hjorth and Cumiskey, 2018; Richardson and Schankweiler, 2019).
At the same time, visual human rights organisations and practices have emerged that seek to harness the communicative efficacy of imagery for public and institutional advocacy (Ristovska and Price, 2018). Ristovska’s (2019) research into the Berlin-based video advocacy agency Syrian Archive reveals how ‘human rights collectives are positioning themselves as visual experts that both mimic established institutional modalities and help offset the lack of replicable workflows and clear visual standards for eyewitness video across journalism, the law and political advocacy’ (p. 333). While open-source initiatives such as WITNESS and Syrian Archive (also an Airwars partner) have had impacts on human rights investigation and advocacy, they also introduce a kind of proxy professionalism that mediates on-the-ground testimony through entities situated (even if by necessity) in the Global North and thus at a remove from the conflict itself (Ristovska, 2021). When coupled with the affordances and constraints of distribution platforms such as YouTube, the transformative potential of visual imagery can be complicated by the affective economies of the platforms themselves (Andén-Papadopoulos, 2020).
Examining Airwars in this context of changing practices, we build on the existing scholarship on witnessing to extend and develop the concept of ‘data witnessing’, which Gray (2019) defines as witnessing that is ‘collective, mediated, distributed across space and time, and accomplished with the involvement of a plethora of both human and non-human actors’ (p. 986). Coining the term to describe ‘how situations can be accounted for and responded to with data’, Gray (2019) examines Amnesty International’s Decoders projects of translating human rights archives into structured data to show how the collective analysis of data through microtasking, design workshops and custom-built user interfaces constitutes a distinct approach to witnessing injustice (p. 974). Reflecting on those practices, Gray shows how witnessing can become ‘a collective accomplishment which enables concern and solidarity to be extended across space and time, as opposed to the “thereness” of singular personal experience’ (p. 987). In his account, data witnessing produces diverse kinds of media objects – databases, maps, visualisations, algorithms – but also enables a variety of different approaches to injustice. While Gray’s conception of data witnessing is powerful, it does not explain how particular witnessing accounts might obtain and sustain authority.
In this article, we argue that certain data witnessing accounts can have greater purchase than other accounts because of the ways in which they are framed within information ecosystems dominated by data logics. Tkacz (2015) applies frame theory to explain how Wikipedia limits what can be included in the encyclopedia despite its stated universal, open and inclusive goals. Tkacz articulates the ways in which frames are both communicative and material. Frames communicate ‘more than the message itself. . . (providing) signals about signals’ (p. 70). Frames are also material because they are inscribed in rules relating to conventions, the roles of actants and rules about spatialities within the frame. Goffman’s (1974) theatrical frame, for example, ‘marks all that takes place within it, from the different roles of people (e.g., performer, audience), differing expressive and material conventions (applause, the red curtain) and different spatialities (seating area vs. stage area, backstage, cloakroom)’. The role of frames (consisting of these rules and their application) ‘organizes matter for the interpreter’ (Goffman, 1974: 245). Frames, in other words, do the work of sorting, ordering and marking distinctions between things. ‘The frame sorts the outside from the inside, but also orders the inside’ writes Tkacz (p. 75). Actors can disagree about frames and framing activities, hence the occurrence of framing disputes and political struggles over the frames. As Butler (2009) writes, ‘the frame does not simply exhibit reality, but actively participates in a strategy of containment, selectively producing and enforcing what will count as reality’ (p. xiii). While Butler’s project principally concerns the destructive power of frames of war, frames also constitute sites of political possibility because the struggle over framing is a question of whose lives (voices, experiences, knowledges) matter and whose suffering can be witnessed.
How Airwars frames witnessing
Applied to conflict monitoring and the calculation of civilian casualties, our research proposes the material aspects of the frame to include (a) the roles of actants within the civilian casualty reporting assemblage and the institutional networks to which they are attached, (b) the data standards that characterise conflict events, (c) the spatialities of distributed digital platforms and extended geographic networks and (d) the techniques of research, analysis and communication. The frame defines what counts not only in the instance of the particular event but what counts in this type of event. The choices that are made by Airwars (to involve all stakeholders, to apply verifiable and externally accountable data standards, to commandeer but not be subsumed by common digital infrastructure) constitute a frame that seeks to potentially encompass all relevant perspectives and yet do so through an open-ended methodology of event analysis and articulation, which insists on the potential within the frame for new evidence to reshape how events are understood.
Roles: experts, institutions, networks
Decisions by witnessing organisations about how to count civilian casualties requires decisions about whose expertise is recognised in the midst of competing accounts. There are multiple civilian harm monitoring organisations in Syria, for example, including groups such as Raqqa is Being Slaughtered Silently, the Syrian Network of Human Rights and the Violation Documentation Centre. For Airwars, those groups constitute sources that they consult and represent in their civilian harm reports. As with civil society organisations, the sharing of data between journalists and Airwars sometimes involves intense collaboration close to the epicentre of the event. During the Battle of Mosul in 2016, Woods recalls: [W]e were in touch with a dozen journalists on the ground in Mosul and the information was passing backwards and forwards. You know, we were helping them to triangulate allegations. They were feeding back very detailed information to us. We were engaging with the coalition about trying to get them to reduce the number of strikes in particular neighbourhoods where we were seeing these brutal spikes and civilian deaths and it was a symbiotic engagement, I would say, between ourselves and journalists. . .
Airwars also reports military and government data about incidents in its reports. But to those militaries, Airwars plays a more antagonistic role as a kind of reliable dissenter. There is an inherent tension in Airwars’ relationship to militaries as it works to advocate against many of their civilian harm calculations while also being open to collaboration. In one example, Airwars worked with the US Pentagon over a period of 2 years (in partnership with other NGOs) on a new US policy for civilian harm reduction. The Pentagon measured Airwars data against their own data to understand the reasons for the vast gap. That triggered an internal investigation which, in turn, helped lead to a 2-year process and an eventual commitment to produce a Department of Defense Instruction (DOD-I) on Minimizing and Responding to Civilian Harm in Military Operations (Anderson, 2020).
Airwars plays a significant role in the context of the civil society, military and journalistic networks, then, both because of its position in the network as a database of databases, but also because of the ways in which it is able to manage information access about incidents for particular groups. Airwars can only perform the latter function because of the expertise of its researchers, assessors and geolocators who act as important intermediators compiling information from multiple sources and making critical decisions to assess them. Airwars’ practice is predominantly manual (rather than enhanced by significant automated or artificial intelligence technologies) and primarily performed by a small team of paid employees (rather than being distributed among volunteers). Core to Airwars’ practice is the knitting together of multiple, competing representations of the same event and the (re-) assessment of that event’s likelihood of civilian harm. In addition to assessment, advocating for such re-assessments among military and government personnel is an important practice. Airwars’ small team is divided into personnel who engage with a spectrum of casualty reporting practices: researchers, assessors, geolocators, advocacy personnel, management. Such practices require expertise that is not limited to individual, technical adroitness of Web searching or an individual applying rules from a rulebook. Airwars acknowledges both local expertise and expertise that arises through a tight knit community of practice as critical to their operation. Three themes emerge when highlighting the expertise demonstrated both individually among team members and as a highly functional working body as an organisation: local knowledge, trusted judgement and translation.
Airwars staff and volunteers come from the US, UK, Europe and the Middle East, with local language researchers primarily drawn from the conflict countries they monitor. The ability to speak the language of local sources from eyewitness accounts and local media is critical to this choice. According to Woods, ‘Arabic is the first spoken language of Airwars’. But it is not only the relevant languages that key personnel possess. It is also crucial that they understand the nuances of a conflict that is ‘endlessly moving and shifting from village to village, town to town, community to community’. While some staff in its UK offices lack localised expertise, Airwars has staff located outside the European metropolitan and long-term relationships with reporters and civil society actors based in conflict zones. These measures aim to limit the potential for extractive relationships between Airwars and on the ground sources, but that potential does remain – an issue that applies to most open-source investigative organisations.
Sometimes an investigation starts with a fragment of information. Researchers, says Woods, ‘will then track down, they will go through their sources, they will proactively reach out to people in the field, trying to boil down this allegation and this claim to sort of understand it and they will piece together those fragments and build that picture’. Other times, researchers can be dealing with an abundance of information. With research in place, Airwars explicitly relies on the trusted judgement of the assessors – not only as individuals, but as a team that continuously deliberates with one another about the work that they are doing. Individually, assessment and geo-location requires stitching together a wide variety of sources and resolving the inherent ambiguity between them. Judgement can be arrived at through deliberation. Airwars has developed an Assessors Manual with rules on how to grade incidents but there are always cases outside the rulebook. In those cases, assessors discuss the question internally in order to reach agreement.
There’s always a margin of disagreement. . . Sometimes some of them are in the manual, but the thing is we sometimes disagree with the manual. . . the incident might be very complicated that the manual doesn’t necessarily give the answer. We usually discuss it with the management or sometimes in the team and we come to an agreement on one of the grading, but it’s always open for disagreements.
Airwars assessors, then, are constantly making decisions that require contextual knowledge in order to make a determination among the multiple alternative readings of the same event by different stakeholders. For those readings to be effective, they also need to be translated to very different communities of practice in military and government circles. Typically, the changes to policy that Airwars is advocating for take many years and require significant personal connections with key personnel. The process of translating assessments into the language and practice of militaries and governments requires knowledge not only of the technical process of reparation and policy development but also acquaintance with and knowledge of individuals key to decision-making in those groups. While Airwars tends to lack traction with non-Western governments, such as Russia, Woods described building effective relationships with democratic state institutions. For example, developing connections and credibility at Pentagon enabled Airwars to help change policies about how civilian casualties are defined, recorded and made public, which in turn increases the potential for victims, families and communities to successfully seek reparations. Similarly, advocacy by local staff and submissions to the Dutch Parliament contributed to an eventual admission of civilian by the Netherlands over its airstrikes in Iraq. There are larger issues here, too, around legal and political process, but they are outside the scope of what we can address in this article.
In sum, then, Airwars asserts the importance of paid, professional intermediaries in the representation of conflict events and the advocacy necessary to ensure reparation. Instead of featuring only its own calculations of civilian harm in its reports, it also provides the calculations of other stakeholders including civil society groups, journalists and militaries. This constitutes a wide frame of investigation to include all stakeholders by translating and interlinking different frames, and coalescing those varied frames within its witnessing database (Papailias, 2016). Doing so allows Airwars to produce higher order frames that include what are positioned as lower order frames (by the accounts of other stakeholders), thus establishing Airwars as an authority.
Data standards
The conventions of the Airwars frame are constituted by data standards in two domains: sectoral standards for casualty and conflict monitoring and internal standards for information collection, verification, retention and review, including those for the use of geographic information systems (GIS). The initial design of the organisation and its methodology was conducted in consultation with not-for-profit standards body Every Casualty Counts (ECC). According to ECC, ‘casualty recording is a process of systematically and continuously attempting to document and record incident or individual-level information about direct deaths from armed violence’ (Every Casualty Counts [ECC], 2020: 12). Their Standards for Casualty Reporting document (v1.01, 2020, which updates English language expression from v1.0, 2016) aims to establish an ‘agreed baseline for the practice of casualty recording’ that will encourage ‘more actors to access, trust, use and/or share the data produced by casualty recorders’ (ECC, 2020: 12). Rather than a set of prescriptions, their objective is to produce ‘publicly accessible casualty records that are transparent, detailed, and reliable in order to ensure accountability, and positively contribute to post-conflict reconstruction and stability’ (ECC, 2020: 12).
While global standards provide guidance for internal operational processes, Airwars also maintains its own internal standards that accord with those in the Standards for Casualty Reporting but are translated into the specific context and processes of Airwars itself. These internal standards feed into the expert know-how required by providing mechanisms for checking data, undertaking geolocation and other organisation-specific operations. For example, Wood notes the importance of sampling across conflicts and staff: I’ll sample and then engage with people on work that they’re doing to make sure that there’s consistency across our projects and so on, because people can be a little bit siloed and we just want to be certain that they’re not sort of drifting in the way that we’re covering the conflicts because it’s important to us from a methodological point of view that we don’t deviate between conflicts and belligerents. So we treat Russia and Syria the same as we treat the [Libyan National Army] and the CIA and Pakistan.
This is important because it is often in the minutiae of the application of standards within the Airwars methodology (see next section) that tensions arise. For example, as one assessor noted, the documented method is very detailed but even with its multiple options for grading an incident, ‘these options are not enough because some of the incidents are very complicated’. There is also an inherent ambiguity to the counting of civilian casualties that stems from conflicting reports about who was killed – men, women, children – and how many of each. Maintaining internal standards – whether through managerial guidance, peer checking or updates to the methodology manual – involves significant labour. For assessors, standards are in part implemented through the structured information required to complete an assessment and the informational processes afforded by the content management system of the Airwars site. For one assessor, the process evolves through the ‘filling in’ of information fields (date, location, code number, sources, summary, pull quotes, casualty tallies) and finally ‘tick[ing] the boxes for the assessing of civilian harm status and the strike status’.
Geolocation practices also have specific processes that establish standards of accuracy and exactitude. For example, an incident might have no visual evidence and thus determining with any certainty the village in which it took place would be some degree of success. In other cases, detailed text might be accompanied by a significant amount of visual imagery, allowing the geolocator, in their words, to go into ‘a separate process in which I’m doing panoramic shots, I’m making a shadow analysis to see which direction everything is. I’m tagging photos and usually that would produce more imagery’. In such cases, location might be determined to a scale of 1, meaning exact location. Taken together, then, standards are an essential component of the witnessing frame, but also highlight the tensions within this kind of specific and high-skilled work, as well as the incapacity of standards alone to grant authority and legitimacy to the media events within the Airwars database.
Spatiality: distributed sources and digital platforms
Airwars’ work takes place predominantly online using digital platforms that operate as infrastructure at three distinct functions within the witnessing frame: as sources of information, as compositional tools for the construction of reports and as disseminators. These functions work to counteract difficulties of information moving across disparate geographic sites, but also serve to distribute the witnessing frame across geographic, material and institutional contexts.
In the investigation process, researchers ‘source information primarily from social media, Twitter and Facebook and local news sources’ on incidents of potential civilian harm. Twitter is the main source, with researchers checking in regularly with specific accounts (of local journalists, activists or active social media users) and also undertaking regular searches using strings of key terms. Once a strike is identified, more tailored searching can be undertaken via Twitter, Google and Facebook, as well as through local and regional news sites. Google is valuable because it can broaden the range of potential sources, while Facebook is difficult to use due to its limited search functionality, which lacks strong filtering and other advanced search capabilities. By contrast, Twitter’s more refined advanced search tools enable contemporary and historical research: I’ve kind of created a kind of catch-all search term which looks for any reference of Yemen or strike Yemen or kind of raid Yemen and then any combination of like 20 key words that I’ve been, which are various spellings of Al-Qaida, US, different provinces. That kind of thing. . . I’ll do that in 14 day chunks. So I’ll go from the 1st of February 2011 to 14th of February 2011 and I’ll go through all of that. Instances or unique instances of strikes, which are alleged to be American. . . so that’s what I’m pretty much doing all day today.
While effective, this manual approach is labour intensive and indicative of the limitations of such platforms, as we describe in more detail below. Occasionally, sources will appear via Telegram, and some regular collaborators can be contacted through Signal, the encrypted messaging service. Google Maps and Google Earth help researchers identify potential strike sites, while reverse image search can be used to work out whether images on social media are original to an event or repeated from previous strikes. Geolocators primarily use Google Maps, Google Earth, GeoName and Wikimapia, particularly since satellite services such as TerraServer have walled off more functionality and priced out small organisations. For all these platforms, there are limits on what English-only researchers can achieve, which means that Arabic speakers (‘the first language of Airwars’ as Woods put it) are essential to the research process.
Other platforms are used as compositional tools within the witnessing frame. Google Docs is the primary workspace for researchers, with different documents existing for time periods within conflicts. English speaking researchers also use Google tools such as Translate to swiftly check meanings or provide basic translations. Web sources are always saved to archive.is, which protects against deletion. For the assessors that analyse collated research material, the Airwars backend database is also a critical platform because it structures the required information and predetermines certain requirements such as casualty estimates, degree of certainty and so on. This is then reflected in the public database itself, and carried through into the weekly, monthly and targeted topical reports produced and distributed on Twitter and Facebook, as well as to an email list of around 600 reporters. In this sense, Airwars commandeers private platforms to create its own public database of civilian harm events.
This reliance on third-party platforms and on the walled gardens of social media creates both limitations and risks for Airwars. As one researcher pointed out (prior to Twitter broadening access), it would be ‘very useful to have access to have the Twitter API [application programming interface] and be able to much more kind of effectively scrape like information’. Commercial changes to Google News have resulted in less media organisations included in its searches and this also present challenges. While flexible and accessible, Google Docs does have downsides: it hogs computer resources, slows down once larger than 500 pages, and can be difficult to navigate. These limitations highlight familiar tensions in the way platforms serve as ‘public infrastructure’ but remain private, corporate and ad-revenue dependent entities that will always act in the interests of preserving audience engagement and data ownership. For Airwars, this reliance on (mostly) freely available but privately controlled platforms is a risk: rules and affordances of access can change, content can disappear and particular sources can be banned or cut off for various reasons. From friction in the collection of data to the usability and reliability of tools, there are thus in-built limitations on the robustness of digital platforms jerry-rigged into witnessing infrastructures. Navigating those limitations and shifts in their nature is one of the key tasks of maintaining the effectiveness of Airwars. The case of Airwars demonstrates how some social actors commandeer digital infrastructure (Fotopoulou and Couldry, 2015) for the purpose of asserting alternative witness accounts that, in some cases, come to replace official accounts but that, as a result, they are often vulnerable to the lack of control that they have over such infrastructure. This reliance on digital platforms combined with the geographic distribution of Airwars staff and sources also means that the witnessing assemblage might take on, even inadvertently, the extractive logics of the platforms themselves. A crucial question, then, is how Airwars makes this assemblage operational via its defined methodology and principle of always-open events, a practice that seeks to make witnessing contestable.
Technique: the Airwars methodology
For expertise, standards and spatialities to cohere, techniques of research and assessment are necessary to cross-cut and crystallise the diverse framing elements of the witnessing assemblage. Airwars applies an established, publicly available methodology to its research and assessment of civilian harm events. It makes the local global, prioritising the experiences, voices and knowledge of those proximate to or directly injured or affected by civilian harm events. It supports analysis using an ‘all source’ approach, capturing digital representations of events from multiple perspectives, languages, locations and media. And most importantly, it is open-ended, capturing all available information about the event but leaving assessments open to be re-evaluated when/if more information becomes available.
Airwars’ framing emphasises eyewitness accounts reported across social media channels in local languages. Initially, Airwars had expected to be primarily collecting local traditional media reports of civilian casualties. But by 2015, when Airwars received their first major grant to track civilian harm in Iraq and Syria, social media had taken over as the main place where harm was being reported by civilians. Much of this was not outward facing, but rather people sharing information within their communities. As one staff member told us, ‘the problem with war reporting has been for a while that western media, it doesn’t really look into what local sources are saying. . . Airwars digs into that and we’re trying to find local sources from social media, from local press and conflict zones and we record them’. But this localism is about much more than simply relying on social media sources because they are readily available thanks to the affordances of the major platforms. Rather, it is supported by an organisational commitment towards an epistemological reorientation: shifting the vantage point of the event from one that was primarily visualised from the air to one that grounds multiple spatialities in the lives of those at the event’s epicentre. At the same time, Airwars seeks to remain legible to organisations that might otherwise dismiss the local. Instead of anonymising victims of attacks, Airwars’ policy is to ‘name wherever possible civilian non-combatants reportedly killed’, often with accompanying ‘photographs of victims in association with specific events’ and ‘images of the locale’. The decision was made because they believed that it was important to ‘publicly [mark] those killed’, especially given ‘repeated denials of responsibility by belligerents’ (Airwars, n.d.).
Airwars’ reporting methodology is expansive. Airwars sees itself as a sensing body, open to reports of events from everywhere. Personnel call their methodology an ‘all source’ approach (Airwars, n.d.). Even though the all-source approach refers to only digital representations of events, there are clearly additional sources of knowledge in the flow of practice between collaborating institutions and the expertise of the Airwars assessors (who make assessments based on multiple competing accounts). Crucially, this expansive sourcing is closely tied to the open-ended nature of Airwars’ assessment of events. Airwars presents an analysis of events in terms of the likeliness of their occurrence but they note that this is a ‘provisional grading system’ that can change as new information becomes available. According to Woods, ‘we never close an assessment’. It remains ‘permanently open. . . for review and change. The coalition or the US or Russia may bring forward new evidence that radically changes our understanding of an event that happened years ago and we’re not precious about that at all. We’re not wedded to our assessment. Our assessment simply represents where we are based holistically on the available evidence’. This open-ness positions all events as contestable and confirmable, such that the frame does not foreclose the transformation of its witnessing.
In practice, according to the published methodology, an event is graded as ‘weak’ when it is supported by ‘single source claims. . . from a reputable source - and with international strikes confirmed in the near vicinity for the date in question’ (Airwars, n.d.). The ‘contested’ tag is used when no combatant confirmation has been determined (see Figure 1). An event is only confirmed when ‘a specific belligerent has accepted responsibility for the killing or injuring of non-combatants or allied forces in a particular incident’ (Airwars, n.d.). Below the grade for each event is a list of sources and alternative evaluations, and they ‘urge those using the site to make their own judgement based on available sources’ (Airwars, n.d.). Woods believes that making sources available underneath each assessment (even those that might contradict one another) leads to greater trust in the organisation.
They needed to know that the data they were getting from Airwars was fair and transparent and they could recreate it themselves critically, so even though we might make a determination, they could see the original sources, make their own determinations and I think we’ve been very successful in then actually building an organisation where our findings and data are trusted, whether that’s by journalists or governments or militaries. They may disagree with that – findings and data – and that’s okay, but they completely understand how we reach our determinations and that those determinations are accurate based on our methodology. . .
Example assessment of incident on July 24, 2021 (https://airwars.org/civilian-casualties/r4353-july-24-2021/).
In other words, keeping the frame of data witnessing open means that Airwars never precludes an alternative conclusion or the refinement of an existing assessment. A discursive authority resides in this strategy, enabling Airwars to demonstrate rigour in its research and assessment and commit to the potential for its own assessments to change. This very potential for change – the open-ness to contestation or confirmation – is critical to its own claims of rigour and thus to the authority of its witnessing database. Airwars’ open-ended methodology is precisely what crystallises its distinctive witnessing assemblage.
Conclusion
The principles for sorting and classifying events determines not only how they are seen, but shapes their political effects. There are multiple (often contradictory) witnessing reports competing for authority in the context of a proliferation of networked data witnessing projects. In order to be effective, reports need to be authoritative, especially to those organisations (namely militaries) that they contradict if they are to affect policy change and contribute to justice for victims. In this case study, we examined the strategies of Airwars, an organisation that has had some success in marshalling such authority. Investigating how Airwars works, we realised that the ways in which it frames witnessing accounts is central to its authority. Airwars’ frames operate at multiple scales and contain multiple perspectives. This high level framing powers their authority. Airwars garners authority by enclosing seemingly contradictory views within a single frame, but without foreclosing diverse viewpoints, incomplete evidence and contradictory evidence. It does this by listing the body counts of military belligerents alongside their own (often contradictory) counts as determined by alternative evidence (with an emphasis on local accounts). It insists on keeping every investigation open. This constitutive openness means that authority, almost paradoxically, depends on its own refusal to claim definitive veracity: truth is always partial, contingent and able to be contested.
Aside from the necessity of protecting the identity of vulnerable sources and staff, Airwars operates in keeping with the epistemology and ideology of the open web, refusing to blackbox or make proprietary its data or methods of collection, even though it is itself dependent on private platforms to perform its work. Like Wikipedia, Airwars assessments are never complete. They show their workings by making available the sources that led to their evaluation. They claim to capture sources from ‘everywhere’. In addition to the methodology published on the website and summarised in its various reports, Airwars makes its approach visible in the database itself by ‘showing the working’ through the presentation and verification of sources, notations regarding conflicting information and accounts of the process itself. Crucially, Airwars also shows its working through assessment narratives that seek to capture uncertainty and which are updated when new information comes to light.
With networked media democratising the capacity to document conflict and produce accessible archives, understanding how such sites and institutions identify, verify and assess their own claims and those of others is increasingly important. As a contribution to witnessing theory, the framing of data witnessing offers a conceptual lens for understanding the distributed and data-driven production of media events, particularly by actors outside the mainstream of media organisations. Witnessing frames provides a compelling analytic for dealing with epistemological open-ness and the restless mutability of events as they enter into and are amplified, consolidated, contested and shared via digital media. In an era of informational uncertainty and instability, Airwars’ witnessing frames suggest a path forward that leverages the restlessness of events by turning a lack of fixity into a virtue that can force even the most powerful militaries on the planet to change how they reckon with the harm they inflict on civilians.
