Abstract
Societal impact is an increasingly important imperative of academic funding. However, there is little research to date documenting how impact is accomplished in practice. Drawing on insights from Actor–Network Theory, we explore the research–policy interface within an interdisciplinary research project on the relationships between air pollution and human health. Health policy impact was important to the researchers for moral as well as pragmatic reasons but it was a goal that was seen as potentially in tension with that of doing science. In fields such as air pollution and health, networks of policymakers and researchers are inevitably entangled, and we found that processes of engagement operated to delineate science from policy. Health was initially black-boxed and under-explicated, used as a signifier in itself for societal impact. By mobilising networks of policy actors, brought together in workshops to rank the importance of policy scenarios for the research team, the connections between air pollution and health were materialised and made actionable. This was achieved by framing existing data sets, emission technologies, policy expertise, pollutant species and human health in particular ways and, in doing so, excluding others. The process of linking air pollution and health research to achieve societal impact not only influenced how these phenomena were known but, critically, enabled and constrained potential policy responses. Tracing these research arrangements made the material discursive processes of ‘impact’ visible and analysable as objects of social science scholarship, and therefore generated a productive site for critically engaging with processes of environment and health science and policy.
Introduction
There are increasingly explicit imperatives to achieve societal, particularly policy, impact from academic research. Funders such as the European Commission and UK research councils require applicants to detail how they will develop ‘pathways to impact’, and UK university research rankings (which determine funding allocations) include a component on the impact research has achieved outside academia. Internationally, knowledge economies have been reconfigured such that societal outputs are expected, and must be planned for, within research programmes (Bozeman and Youtie, 2017; De Jong et al., 2016). This article explores how one research project, on the relationships between air pollution and health, accomplished impact in practice and, in doing so, made air pollution a tangible and actionable public health concern.
Pollution of the ambient air has become a leading environmental concern for public health. The World Health Organization (WHO, 2018) estimates that globally, it is responsible for 4.3 million premature deaths every year. In the United Kingdom, fine particle pollution may cause a loss of life expectancy from birth of around 6 months or an annual total of 29,000 to 40,000 premature deaths (Committee on the Medical Effects of Air Pollutants (COMEAP), 2010; Royal College of Physicians (RCP) and Royal College of Paediatrics and Child Health, 2016). There is no established safe level of air pollution below which health effects do not occur, so there are expected health benefits from reduction of pollution to as low a level as possible. Currently, the United Kingdom is required to report air quality compliance under two European Directives (The Council Directive on ambient air quality and cleaner air for Europe (2008/50/EC) and The Fourth Daughter Directive (2004/107/EC) under the Air Quality Framework Directive (1996/62/EC); Department for Environment, Food and Rural Affairs (DEFRA), 2017). Understanding of the effect of air pollution control strategies on air quality, and the consequent impact on health, is a key area for environment and health research.
These policy imperatives aimed at controlling and combating air pollution have encouraged new alliances between the environmental and health sciences, brought together through funding calls for interdisciplinary collaboration. These research arrangements raise political and empirical questions about the practices of science and policy, such as how knowledge is co-produced (Jasanoff, 2004) across different disciplines, but also across interfaces of research and policy.
A well-cited field in social studies of science now addresses the flows of knowledge between research and policy communities, which critiques rational, linear and one-dimensional models of impact (Jasanoff, 1987; Nowotny et al., 2001; Wynne, 1992). The limitations of linear models in health research have certainly been well documented. Berridge and Thom (1996), for instance, outlined the role of medical doctors as conduits of research evidence within policy communities in the field of drug policy. Stevens (2011), in an ethnographic study of how policy actors utilise research findings, suggests that in the face of uncertainty and an ‘oversaturated’ body of evidence, civil servants craft the ‘policy stories’ that are the most likely to recruit other actors into a credible narrative. On politically contentious areas, he argues, bureaucratic disincentives and systematic communicative distortions ensure that evidence around some issues (such as health inequalities) is systematically elided. Other studies have identified the role of local (Phillips and Green, 2015) and national (Qureshi, 2013) political considerations in shaping how academic evidence is utilised in policy practice. A recognition of the need to capture more complex paths of translation, and the multiplicity of impacts, particularly in domains such as health, has incentivised a plethora of models and approaches to impact assessment (Milat et al., 2015). These include more theoretical models, which recognise that knowledge production and societal impact are co-produced. Matt et al. (2017), for instance, delineate four ‘ideal types’ of pathway, noting the different configurations of actors and ‘chains of translation’ that lead to different forms of societal transformation.
Yet, how this impact is achieved in practice is often black-boxed. That is, most empirical research on evidence flows has focused on what happens to the outputs of academic knowledge: how findings, once in the public domain, are then translated, incorporated, cited, ignored or utilised. There has been relatively little attention paid to how evidence flows are managed by researchers within the lives of projects, despite increased incentives for researchers to include policy stakeholders as actors from the outset. In contrast, there is a proliferation of research on interdisciplinary science in action, which does focus on how shared knowledge is achieved within collaborative projects despite disciplinary and cultural differences (Edwards et al., 2011; Galison, 1996; Star and Griesemer, 1989). This research on interdisciplinarity has relevance to studying the science–policy interface in practice, given the attention to process rather than outcome, and material practice rather than abstract knowledge.
We draw on research of science in action, and particularly approaches from within Actor–Network Theory (ANT), to explore evidence flows across the research–policy interface within one interdisciplinary project on air pollution and health. ANT is a methodological and theoretical sensibility that emerged from science studies, which orientates researchers towards the relations between objects, things and processes that are part of the mutual construction of science and policy (Callon, 1999; Latour, 1999, 2005; Law, 1992). ANT follows the networks of actors and specifically the circulations within networks rather than attempting to define entities, their essence or provenance (Latour, 1999: 20). In his study of economic markets, for instance, Callon (1999) argues that entities assumed to be singular, like money, can only act because of their particular attachments. For Callon, measurement instruments, material investments and economic theory are agents that do not simply describe the economy but perform it. Employing Callon’s concept of framing, this article considers the science–policy interface as a space for jointly producing research evidence on air pollution and for making scientific findings impactful. For calculations, such as the generation of policy evidence, to be performed and completed, the agents and goods involved in these calculations have to be disentangled and framed (Callon, 1999: 186). The interface provided the conditions for framing, entangling and disentangling the goods of research, specifically data, in order to achieve science–policy translation. Like Callon’s description of the economy, different people, things and processes at the science–policy interface produce situations that make ‘actionable knowledge’ possible (Matt et al., 2017).
The case study
The Weather, Health and Air Pollution (WHAP) 1 project provides a useful case study for exploring these processes of science–policy translation and the generation of impactful knowledge. An interdisciplinary project, WHAP included senior investigators and research staff from five different UK universities, from disciplines including public health, epidemiology, risk assessment, environmental measurement, air pollution and climate modelling, built environment modelling, decision-analysis, sociology and environmental policy engagement. It was funded to investigate the relationships between air pollution and human health. WHAP was, by most measures, a successful project from a team experienced in, and with past successes in, interdisciplinary work. The collaborators had a number of jointly funded projects, published together across a range of journals, and included senior investigators who were extremely highly cited in their own fields. They were willing to include an ethnographic component in the original WHAP proposal to conduct a study of the research team itself, focusing on knowledge generation and translation within the project.
This article draws on this ethnographic component, which is premised on an understanding of knowledge translation as material and performative as well as epistemic and discursive. That is, as well as documenting participants’ accounts of their interest in, for instance, policy impact, our focus was on how the linear model of science gets taken up as a normative ideal: as something that participates in knowledge translation rather than simply guiding it. Other elements of the actor network include non-human forms, such as technologies and data. We consider these kinds of ‘attachments’ and examine how they transform scientific data relating to air pollution and health into evidence of air pollution and health. Tracing these attachments, we propose, is a starting point for addressing an empirical gap in the literature on research–policy networks in action. Instead of focusing on how scientists produce knowledge that policymakers can use to act in the world, we demonstrate how action, specifically in this case, of ‘impact’, emerges through particular relational networks, and the ways in which different actors participate in its performance.
Methods
Ethnographic research was conducted between 2010 and 2014, largely by the first author (E.G.), who was engaged as an ethnographer on the project. These data include fieldnotes and transcripts from weekly project team meetings, larger in-person 1-day collaborator meetings and stakeholder workshops; formal in-depth interviews; informal in-person and email interviews; and project documentation (proposals, email threads, presentations and publications). What counted as relevant to the ethnographer was not conceptualised a priori but emerged through the tracing of the social and material practices of science. The ethnographic accounts drawn upon have been divided into three inter-weaving sections to show the different ways in which ‘impact’ was framed and how different actors contributed to and composed the ‘science–policy interface’.
Researching science in action is challenging. Much work done by researchers is individual, private and largely invisible to observers. Tasks such as computer coding, running computer model simulations, drafting papers or cleaning data were, for the most part, done alone at a desk, and there are a limited number of times an ethnographer can ask for the work to be explicated without disrupting progress and risking severe irritation. Furthermore, a large amount of research work is also routinely disavowed as work by the researchers actually doing it. Any research project entails a considerable number of administrative tasks: setting up facilities for phone conferencing; completing data requests; drafting job descriptions and testing shared online documenting spaces. In the early stages, meetings around such tasks were difficult to access simply because investigators would say ‘this won’t be interesting for you – we haven’t really started the work yet’. Accessing the more collaborative practices (team meetings, email exchanges) is practically easier, but raises ethical dilemmas. Although the majority of the project collaborators were positive about the ethnographic component of the project – and indeed generous with their time and knowledge – it becomes difficult, in a team meeting, or when accessing an email thread, to remove data from those who preferred not to be researched.
Our ethnographic data have, then, some inevitable limitations: we have minimal access to ‘off-line’ work that has not been materialised in documents, emails, interviews or at meetings. However, we benefitted from the interdisciplinary nature of the project, as researchers from different disciplines had to explain (and justify) their assumptions, methods and skills to each other. In doing this, they made explicit (and available to an ethnographer) what can otherwise be taken for granted rationales for practice. Studying, analysing and reflecting on this for the duration of the project was an opportunity to participate in the rhythms of research in action, while experiencing the spatial limits and temporal constraints that comprise interdisciplinary relations. As such, analysis was iterative, shaping the fieldwork process as well as its interpretation.
Findings
Producing policy relevant science
The translation of science to policy was an explicit WHAP aim, with plans to engage stakeholders from various policy sectors through workshops, and more formally through their participation in designing a decision aid for assessing future emission reduction policy options against criteria such as mortality, health inequality and greenhouse gas emissions: Our previous experience shows a clear need for engagement with stakeholders and policy organisations at the start and at regular intervals through the project. The first workshop will […] allow us to shape the detail of our work plan at the outset to be of most benefit to end users; for example, discussions at this workshop will identify aspects of air quality policy likely to be most important in the UK context from which we can develop appropriate air quality policy scenario assessments. We will also present the elements of our [decision-making framework] for comment and refinement to be most useful to policy makers. (Pathway to Impact Plan, WHAP funding bid 2010)
A computer-based decision aid was anticipated to be the primary method for materialising findings from the academic work such that they would be available to policymakers: a way of framing the links between air pollution and health and making the effects visible and modifiable, thus (in principle) allowing impact to be achieved.
For both senior investigators and researchers on the project team, achieving impact was reported as an incentive for doing research and a satisfaction of working on a topic such as air pollution. Responding to the health effects of air pollution was widely reported as a moral driver for doing science. Generating meaning was understood as a process of making a situated scientific claim speak to policy, in terms of, for example, potential interventions to improve health and reduce health inequality. For researchers in fields outside public health, the fact that the project included partners from health sciences, such as epidemiologists, was in and of itself the means by which the project might make an impact. As Chris, an environmental chemist, suggested, making the links between air pollution and health visible was a tangible output of their work because it obligates a policy response: If you are demonstrating there is a health impact then it is logical to say what we are going to try and do to ameliorate that health impact. So, yes, I think there is a strong angle to look at policy interventions, which therefore means sort of gazing into future about what you could do to ameliorate it. (Chris, Interview, October 2011)
He later elaborated this as a motivator for getting involved in this kind of interdisciplinary research, with what was seen as ‘clear’ health policy outcomes: I would say that one of the things I like about this part of my work and this project is that it has a very clear public end point in terms of public health, and trying to […] derive policy actions that will improve public health. (Chris, Interview, November 2011)
Another investigator, an air chemistry transport modeller, described the appeal of WHAP as doing research on ‘something we can do something about’ (Sam, Interview, November 2011). At the first in-person collaborators meeting, the air pollution modellers presented maps showing distributions of air pollution species such as ozone and PM2.5 (fine particulate matter) and discussed their complexities: interactions between species, the trends of increasing or decreasing concentrations over time and the different information conveyed by mapping at different spatial resolutions. Given the very wide range of pollution data that have been and can be generated and the multiple questions that could be asked, the health researchers at the meeting asked, ‘what are the key species for health?’ Further discussion was directed by one investigator prompting, ‘we have a vast data set – the question is how to focus’ (Fieldnotes, Collaborator meeting, June 2011). As one senior modeller explained, in relation to her colourfully plotted maps on the computer screen, ‘what does that mean and is it important?’ The movement of her (modelled) data to the field of policy, she anticipates, is what is generative of potential health impacts because it ‘adds meaning’ beyond the situated practice of producing and interpreting (scientific) data.
A first step in making research relevant was narrowing down potential questions to ask of the data, and health correlates provided the initial funnel. Of course, these too were complex, with multiple data sets of indicators of morbidity, mortality and health service use available at different (temporal and spatial) scales. Policy stakeholders were therefore described as vital for helping to finesse core research questions for the team, because they could identify the most relevant associations to explore in terms of which most clearly linked to specific sources of pollutants that could (at least in theory) be affected by policy change. As one noted, ‘we shouldn’t be looking at sulphur but at sulphur producing processes’ (Fieldnotes, Collaborator meeting, June 2011). Thus, actionable policy concerns were enrolled early on to help make selections from a potentially overwhelming number of possible relations for study.
However, environmental science has been subject to considerable controversy (Lӧvbrand and Oberg, 2005; Oreskes, 2004; Reay, 2016), and the WHAP project team was well aware of many of the key points summarised above from the research impact literature that policymakers operate within complex power relations; that policy and science are inevitably intertwined and that ‘work’ was needed on their outputs to maximise the likelihood that they would travel across different fields of practice. Making scientific data relevant for policy action was, therefore, recognised by many investigators as a goal that was potentially in tension with the requirements of ‘good science’. The Principal Investigator (PI) of the team described engagement more like a transaction between science, as the producer of knowledge, and policymakers, as the consumers. This was framed as a productive relationship because knowledge could potentially do something else once in the wider world, albeit how to achieve this was not so well articulated. As he put it, when describing the need to achieve a balance between being useful for policymakers and being useful for academia: ‘The project is not just about serving needs of policy makers, advancing science for its own sake is important: it is an academic project’ (Interview, October 2011). Researchers were attuned to the potential for academic researchers and policymakers to have different needs: at the first collaborators’ meeting, one of the co-investigators interrupted the discussion about which pollution species were to be included in the atmospheric chemistry model to ask, ‘What are we most interested in? That might be different from what policy makers are interested in’ (Fieldnotes, June 2011). The border between science and policy sometimes required careful policing to maintain the integrity of scientific knowledge. The WHAP team nonetheless considered it vital to allow data to travel to policymakers. Not only did such translation fulfil the duties of scientific work according to those that fund it, but also cohered with their social and moral orientations and provided an anticipated set of actors (potential policy interventions) which could help refine an otherwise unmanageable potential research agenda.
In managing the interplay between maintaining scientific purity and fostering translation, one distinction drawn on at many points throughout the research project by researchers was that between ‘data’ and ‘evidence’. These were delineated by distance from production, with data positioned closer to the phenomena studied, and evidence more stable and independent of the research context. Yet, in practice, what distinguished data from evidence was not so clear cut. The logic of moving data along the science–policy network in order to make ‘evidence’ was governed explicitly by a concern of linking scientific work to real problems in the world, rather than its remaining in a vacuum. Intervening with the material form of data was a necessary requirement for enabling them to travel into policy fields (Leonelli, 2009).
In its application for funding, WHAP anticipated a range of actions that would help with the transformation of data into evidence and thus enhance their utility. One was preparing data to reflect levels of (statistical) uncertainty which might influence policy decisions: An important element will be the use of ‘post-processing’ of the exposure validation data to examine the uncertainty in the evidence and its potential bearing on policy recommendations. (Project Protocol)
Here, post-processing involved making sure various levels of uncertainty are shown in the data produced. Given the inevitable ‘frictions’ (Edwards et al., 2011) that emerge between data (and metadata) produced by different disciplines, at different sites, on different platforms and for different purposes, these processes entailed considerable translation work for the WHAP team to ensure that data travelled across disciplines. This work has been described in detail elsewhere (Garnett, 2016, 2017). In short, disciplines were delineated by their particular craft skills of data production: environmental chemists employed experiential knowledge to ‘feel’ for error in readings on monitors, for instance, and modellers described their ‘eye’ for checking modelling runs are credible (Garnett, 2016). Despite a high degree of commitment to interdisciplinary work, there was also a desire not to display these craft skills, and uncertainties about disciplinary knowledge, outside the disciplinary team, of ‘not washing your dirty linen’ as one investigator put it. These processes of ‘off line’ or back-stage work were also at play in effecting the science–policy exchanges necessary to achieve health impact. Similar processes of smoothing the friction between different data sets and metadata were also burdensome in interactions with policymakers. In the next section, we examine how before data could become evidence they had to transcend their disciplinary and situated processes of production.
Mobilising the policy network
One example of ‘off-line’ work was the process of inviting stakeholders to workshops. In the research proposal, policy engagement was planned through ‘a set of three major workshops open to all stakeholders in air quality and health measurement, modelling regulation and policy making’ (WHAP Project Protocol). The first of these workshops was held at the end of the first year of the project. Discussions in team meetings about who to invite were brief and consisted largely of reminders about people and organisations, using first names that were well known to other team members (‘Jackie from the (government) department – and either Vikram or Sandy from PHE’, Fieldnotes, Team Meeting, October 2011). The topic of ‘who should be invited’ remained a regular item on the agenda of the research team meetings running up to the workshop, although never discussed explicitly. Yet work got done and stakeholders were invited. These workshops brought together an ongoing community of known others, as suggested in these fieldnotes: More stakeholders arriving. People seem to know one other when they greet. No name badges are required either […]. Team members also seem to know many of the invited stakeholders by face […] Little effort needed to encourage conversation, the hoped for ‘dialogue’. Tim starts to ask everyone to quieten and settle to get the meeting going. (Fieldnotes, Stakeholder workshop, November 2013)
The formal work package on stakeholder engagement was led by Francis, a mathematical modeller, who was designing a computer-based decision aid that could be used by policymakers to inform decisions about policy initiatives, to assess their likely impact on pollution, health and other criteria. He explained that for WHAP, the terms ‘stakeholder’ and ‘policymaker’ were being used interchangeably, as policymakers were the key stakeholders. On being asked how the lists of invitees had emerged, seemingly without much wider discussion, Francis later explained the considerable ‘off-line’ processes involved: … we had a stakeholder meeting for [a preceding project]. … [we] started with the list of stakeholders/policy makers who attended [and] added/replaced some names from the same organisations. In addition [a policy collaborator] suggested his contacts from the [government departments] … [so] there was no discussion dedicated to this topic. Most of the correspondence about names was done off-line … the fact that we had a contact list from the previous collaborative project helped a lot. (Francis, personal correspondence, January 2014)
Organising and carrying out the stakeholder workshops brought together already established networks of contacts and colleagues, rather than invited unknown external actors. These workshops maintained and sustained existing relationships, which spanned science–policy networks. Participants included ex-academic colleagues of the research team who had retained honorary appointments at universities, while also being affiliated to organisations which use research to shape policy by, for example, contributing to policy reports for the EU and Intergovernmental Panel on Climate Change (IPCC). Stakeholders also included representatives from government departments, including DEFRA, the UK Committee on Climate Change and Public Health England. These representatives were very often firmly established contacts of the research team.
As experts on air pollution, these policy stakeholders were also often data producers and therefore played a key scientific role in the production of data. At various points during the stakeholder workshops, attendees were, for example, asked to help provide more detailed data on particular emissions and ended up suggesting avenues for accessing key scientific sources:
We are starting to go over time now. So, we have a number of suggestions that we should focus on when it comes to PM2.5: transport, agriculture and a combination of policies/pollutants.
[Nitrogen] fertilisation rates decrease, as do ammonia rates.[…]
Climate Change Committee can tell you information of base range on those.
Other policies which effect NOX [Nitrogen oxides] and SOX [Sulphur oxides] are secondary organics, [I suggest modelling these] together not individually because of cross-pollutant interference
Look at [another UK based air pollution project] because their scenarios are modelled according to pollutant effects.
And look at the recent Climate Change Committee publication which will be in print and out on 11th December and discusses co-benefits. (Fieldnotes, Stakeholder workshop, November 2013)
In the preparatory team meetings, it became clear that other candidate stakeholders, such as members of civil society organisations or public representatives would not be invited to these workshops because representing the public’s view or translating science to the public was not the kind of impact the project was aiming for during these meetings. What was also absent in early exchanges in the project was any discussion of stakeholders from health domains. Indeed, there was little discussion at this point around what was meant by health. It could be argued that, for WHAP, health stood in for making science that is relevant for ‘the public good’ (Interview with environmental chemist, November 2011) and the project was about how this could be achieved at a distance. Yet, what counted as health-relevant remained undetermined. Physiological health effects from named pollutant species were to a large extent black-boxed by stakeholders during the process of shaping the research agenda. By that we mean health effects were referenced, but as generic outcomes that would flow from (potential) policy interventions: there was little discussion until later (as we go on to show) of what was meant by health, nor were there any representatives from health or public domains within the first stakeholder workshop who might have opened this up.
Describing this process of mobilising stakeholders and harnessing resources suggests that evidence of the effects of air pollution on health is produced through specific networks of people and data. Those networks composed the environmental health phenomenon air pollution through the generation and sharing of knowledge and data about particular pollutants and their effects.
Making data for impact
Having mobilised the science–policy network in WHAP, a key aim of the stakeholder workshop was to decide which policies and future scenarios the WHAP project should focus their production of data on, and which criteria to choose for comparing the likely effectiveness of policy options. Much of the data produced by WHAP comprised computer simulated pollutant concentrations. These computer models can simulate different future scenarios by assuming changes in emissions from different economic and societal sectors. Emissions were classified and mapped into different emission sectors according to the Selected Nomenclature for sources of Air Pollutants (SNAP sectors) that included, for instance, waste, agriculture and transport. Changes to SNAP sector emissions modified inputs for the atmospheric chemistry model, which could be run to predict how air pollution levels might change in consequence, and in turn (by integrating data from epidemiological components of the project) affect human exposure and health outcomes. By shaping the production of data through the influencing of emissions input into the model, the SNAP sector classifications were also a way to ensure data carried meaning in terms of potential policy action, as these were, by definition, organised around pollutant sources. The simulated pollution data were then inputted to other models for health impact assessment.
A subsequent computer-based decision aid was a mechanism for explicitly incorporating stakeholder perspectives into the generation and interpretation of modelling results in ways relevant to policy choices: a tool that would model the pollution, health and other impacts of policy choices, allowing policymakers to weigh outcomes (criteria) in terms of their importance. At team meetings in the run up to the stakeholder workshop, Francis, the co-investigator leading on the policy engagement component, explained that the decision aid worked by enabling policymakers to model the impact of changes in the SNAP sectors themselves. Here, the work of translation suddenly became rather tangible, materialised as a series of classifications that the decision-aid technology could then perform in speculative ways: [Francis moves the cursor] ‘Well, what I was going to show is, for example, how if you move this, and then these weights move this [as the weightings change the orange bars at the top of the screen the scores change] … so you can see how one changes things [the scores]…when you change the evidence, and/or you change the weights, the policy option which scores the highest changes… and the ratings are what the modellers bring through impact calculations, and the weightings are what the stakeholders in the second workshop decide on’. (Fieldnotes, Team Meeting, November 2013)
The stakeholder workshop began by sharing details of the perturbation of the SNAP sectors and related scenario run on a computer model. A series of graphs were used to orientate discussions. The impacts of policies on emission levels and thereby air pollution concentrations were visualised, with colour-coded maps making the effects of air pollution literally visible for those present at the meeting. The results from these future scenario runs could then be used to rate which air pollution policies should be prioritised. These acknowledged ‘value-based’ decisions were then made more scientific (and thereby more useful) by putting them in the mathematically based decision-making model.
As one senior modeller explained, which simulations to run is dependent on the pollutants considered as a health risk, and therefore of interest to the policymakers. ‘Passing a law on cars’ was one example given for how the project anticipated the use of their data in policy making. This led onto a wider discussion in a team meeting about the role of the decision-making tool and what the project could actually offer the stakeholders in terms of policy-relevant science:
Say they [the stakeholders] are interested in reducing transport emissions by 15%, and we say to them you tell us what it costs. And if they can provide that, are we going to incorporate this?
I don’t know if they can provide the costs, but [rather] whether, how they think we could achieve the 15% reduction in emissions. Say, the person in DEFRA could suggest a policy they are interested in.
Is that then compatible with what we are giving them [in terms of the scenario outputs of general emissions changes and pollution changes]?
I would hope we get a list of criteria from stakeholder and umm we could map them on to something sensible and do the best job we [can]. (Fieldnotes, Team meeting, November 2013)
During a preparatory team meeting, one modeller explicated the limitations of making policy claims from modelled data, given that the relationship between emission changes and pollution concentrations may be non-linear. This non-linearity limits the model’s capacity to estimate the impact of policies that achieve different levels of emissions reduction given that, for practical reasons, the national models could only be run for a limited number of scenarios. Thus, model outputs indicate the consequence of one or two selected (percentage) reductions in emissions and not those that would correspond to a wider array of policy interventions. This hints at the perceived tension between representing scientific uncertainty and producing evidence-based claims, while trying to demonstrate the relevance and applicability of findings.
Francis explained that if all information is available then an evidence-based decision is possible because the model can numerically account for uncertainty in the available data. This means that uncertain data could then be used as (certain) evidence: keeping uncertainties in the foreground while also making data a function of the network. It was the stakeholders’ role to judge the appropriateness of the available information in terms of the quantification of uncertainty and in terms of which emission sectors to prioritise over others, yet these prescribed roles and responsibilities were not immeditately obvious for participating stakeholders: So, different policy makers have different views on ‘criteria’ [for judging policies] and it [the decision-making tool] is sensitive to these in ranking the policy options. The policy options are also to be discussed, so the aim of this meeting is to select the criteria which compare policy options. The impacts will be measured by models and experts and the ratings [of each policy option on each criterion] are the underpinning science, the contributions of the project. [One stakeholder interrupts asking if the weightings are subjective] ‘the weightings are produced by you the experts [stakeholders]’, explains Francis. (Fieldnotes, Stakeholder workshop, November 2013)
Developing the decision-making tool was a technical process that involved scaling the health (and non-health) evidence to numerical values between 0 and 1 and then valuing a range of policy outcomes-the criteria, such as mortality, health inequality or biodiversity. The latter process was subsequently done by email and involved assigning weights to the criteria so that policy outcomes were prioritised according to their deemed policy relevance and need.
At the workshop, Francis described how what he referred to as ‘the science’ was shaped by the tool: [T]he science evidence base needs to be normalised, for example, say there is a maximum of 20,000 [attributable] deaths, this would be assigned the value 1 and anything under 20,000 would be normalized between 0-1 [there could be different normalization units for each criterion] … and in terms of the weights for assigning importance to the criteria, you ask people, decide on weighting and [then] look at their impact on policy scores. (Fieldnotes, Stakeholder workshop November 2013)
Unlike ‘the science’, assessing the importance of particular health impacts was conceptualised as a subjective process involving the making of decisions about which air pollutants to prioritise and which criteria to use for comparing policy options. This was a process referred to at times by WHAP members as seeking the stakeholders’ ‘qualitative input’. Francis described the process as a way to account for their different interests and viewpoints, in part ‘because health is not the only criteria by which to judge scientific evidence’ (Interview, February 2014). The decision-making tool meant that the role of the policymakers could be accounted for and made transparent. In doing so, the science was implicitly disentangled from political and pragmatic decisions and then framed and built into the model as ‘technical’ input (in contrast to the ‘subjective’ views of policy stakeholders about the selection of criteria and assigning of their weightings). Scientific integrity was ensured and preserved through the particular affordances of the decision aid that rendered subjective decisions visible and values explicit.
Re-making air pollution at the science–policy interface
Towards the end of the stakeholder meeting, as these choices about what pollutants or pollutant sources to concentrate on were being finalised, statements were less attuned to complexity and uncertainty and became more fixed and certain:
In terms of environmental impacts, the one metric, nitrogen deposition, is the one to go for.
In terms of the UK, NO2 is of relevance in terms of policy.
Put it [social acceptability] in, but rate it low. (Fieldnotes, Stakeholder workshop, November 2013)
Invited policymakers suggested ways to shape the research process of WHAP, pinpointing realistic potential policy actions through which emissions sectors could be influenced or by detailing the interactions by which a more finely grained understanding of health impacts could be sought: [mentions include] Impact of new technology, cars electric, the only way is to change the fleet, in terms of modelling ‘change the fleet’; [look at] fleet age. (Fieldnotes, Stakeholder workshop, November 2013) Should we explore further the interactions between socioeconomic class and health impacts? (mapping Census data) (Minutes from stakeholder meeting November 2013)
Potential policy modifications such as congestion zoning, differential taxes on cars and expansion of hybrid buses were considered as actions which could be simulated through perturbed runs of the model. This also opened-up what had initially been black-boxed: health. Stakeholders pushed researchers on WHAP to consider more specifically what they meant by health: ‘is it just number of bodies, is it life years lost, or is it some amount of morbidity or mortality?’ (Fieldnotes, Stakeholder workshop, November 2013). As the PI went on to reflect at the team meeting following the stakeholder workshop, ‘they also wanted to know what measures we will be using of inequality’ (Fieldnotes, team meeting, December 2013), a component of the project that was going to be conducted but had not been completed at the time of stakeholder engagement.
These decision-making processes defined the relationships between pollutant source, pollutant type, its effects on the atmosphere and its effects on indicators of human health in a way which ultimately relied on their ability to construct a convincing relationship between air quality and human health: in essence, one that was amenable to manipulation in the model and in the ‘real world’. Given the known uncertainties, making compelling narratives entailed not just recruiting ‘thinkable’ policy options (such as congestion charging), but also scientific actors who could defend against known uncertainties. One stakeholder alluded to the uncertainties involved in determining health effects of pollutant species from the transport sector, such as NO2 and PM2.5, and suggested drawing on findings from larger, international research projects to aid credibility:
There is evidence which relates to NO2 and O3 and mortality […] but with NO2, it is highly correlated with PM2.5 […] and O3 is all over the place … My suggestion is that you should at least start with what has come out of the larger international groups […] there is a co-efficient there, but there are different grades of uncertainty. (Fieldnotes, Stakeholder workshop, November 2013)
Despite various sources of uncertainty, ‘impact was achieved here by including findings from other projects on as-yet unknown and potentially non-linear health effects. For the policymakers to use data as evidence, a narrative about environmental health as made up of linear relationships between emission sources – pollutants – health effects had to be achieved, even if these were partial and variable. Air pollution became both stabilised and multiple through the situated framing practices of decision making.
Indeed, a number of different source-pollutant-effect relationships were decided upon through engagement with policymakers, which resulted from the delineation of certain from uncertain air pollution relations, and pollutants of concern from those considered less of an environmental health risk. Scientific data became evidence through their attachment to anticipatory, situated policy interventions. As the gap between what scientists find interesting and what policymakers need weres reconstructed through processes of translation and iteration, the objects of interest (relations between air pollution and health) were also re-framed, by generating new attachments between air pollution and health and grounding them in existing or potential future policy narratives. For example, although policy options were only hypothetically materialised, they resulted from anticipating how to manipulate relations and enact specific socio-technical changes (in, say, transport or agriculture). Furthermore, ‘decision-making’ was not only a process of choosing some relations over others because what makes up causal relations was actively composed at the science–policy interface. In foregrounding work with data, we have examined the shifting meanings, forms and arrangements of people, data and scientific claims that rendered the material discursive processes of impact visible and analysable.
Discussion
Environmental health science is a complex field in which the relationships between researchers, their science, policy advocacy and policy decisions have come under particular scrutiny (Lӧvbrand and Oberg, 2005; Pielke, 2006; Reay, 2016). Pielke (2006) urges researchers to abandon the ‘futile effort to keep science and politics separate’ and instead act as ‘honest brokers’ in delivering evidence on a range of policy options, their uncertainties and their relative advantages. The case described here is perhaps one attempt to act as honest brokers in policy advocacy, as stakeholder engagement was set up deliberately to do this. There are of course a number of potential models of policy–science engagement (Hoppe, 2005; Matt et al., 2017), but in many, like our case study, research and policy are inevitably entangled from the outset, precluding any simplistic questions around the ways in which academic research has societal relevance. Further, linear models are rhetorical devices and normative ideals (Pielke, 2006) that may limit our capacity to understand and intervene in science–policy interfaces. Although the stakeholder meeting did adhere in many ways to a normative construct of knowledge translation, the bringing together of environmental and health research, data and expertise also contributed to developing new ways of framing air pollution and its relations to health.
The different actors and agents involved in the stakeholder workshop moved between academic and policy institutions and some shared roles across these institutions; those invited as ‘policymakers’ were also experts on various aspects of scientific research. These identities and responsibilities were not stable but emerged through socio-technical data and evidence flows, demanding pragmatic, local and mundane tinkerings (Knorr-Cetina, 1981): detailed discussions, for instance, in which researchable constellations of data relations can be identified. Navigating the tensions between commitments to health policy relevance and the material obligations of research phenomena was managed by arranging and organising science and policy in ways that enabled both to co-exist, for instance, by orchestrating space for ‘backstage work’. Such ‘offline work’ was largely invisible (to ethnographic researchers as well as to other actors in the field) yet essential to the conditions that make doing impact possible. This invisible work demonstrates how making research impact has organisational effects, orientating scientific work in particular ways and providing an overarching narrative of ‘public good’. The social relations and identities of knowledge also shift. Because collegiality was already assumed within a network of known others, stakeholder workshops were spaces in which differences between science and policy could be materially enacted and thereby impact achieved.
We have detailed the processes by which data was turned into evidence in the WHAP project to argue that research impact is enacted through situated material practices. First, the decision aid and SNAP sectors articulated and temporarily stabilised the separation between data and evidence and air pollution and health. Second, the SNAP sectors arranged people and scientific data in ways that contributed to the material and discursive transformation of scientific research into policy evidence, by, for example, linking data to information on source emissions or current policy initiatives. Third, health impact was performed through the modelling of air pollution scenarios that anticipated certain kinds of environmental health futures. The scenarios mediated the complexity of air pollution and science–policy differences while performing particular kinds of air pollution and health relations. For instance, focussing on the specific pollutants of concern and their emission sources, other dimensions of health, beyond mortality and morbidity statistics, were not included in the workshop outputs. Ultimately, what air pollution–health relations to prioritise were framed by three criteria: the capacity to measure and therefore accurately manipulate emissions; an understanding of and ability to model the air pollutant being studied; and the facility to construct a causal emission–pollutant–health effect relationship.
Yet, like other policy issues, air pollution is complex, heteoregenous and multiple (Law and Singleton, 2014). Framing air pollution and health in through stakeholder engagement meant some health dimensions, emission sources and pollutants species were not included in the generation of impact. Socioeconomic inequalities and domestic fuel use were not prioritised because of the lack of available evidence and the expertise of those present at the meeting. Studying the process of framing actions towards air pollution and health meant the inclusions and exclusions effected through the situated articulation of policy relevant science were made explicit. In general, we found that decisions centred on data and technical knowledge rather than other candidate actors in a network of air pollution and health, such as local emission sources or the public’s perspective. The workshop affected scientific work too, informing what kinds of model simulations were to be run, which measurement instruments to use and ultimately how air pollution and health were materialised and made prescient. As such, the interface re-framed air pollution and health in ways that also mapped out particular pathways for action while closing down other possibilities.
Conclusion
Air pollution is a concern with many of the characteristics of related environment and health challenges: it is complex, multi-scalar, interdisciplinary and uncertain. As such, this account of generating impact in action is relevant to other contexts of environmental health research where multiple actors and data sets are brought together. Impact generation can be achieved in different ways and engaging policymakers is just one way of doing impact in contemporary science. For example, public understanding of science is a value achieved by mobilising a different kind of network, where different scientific and non-scientific identities are performed in order to imbue scientific knowledge with new value. In our study, researchers’ engagements with scientific actors and those supposedly ‘outside’ of science were empirical opportunities for exploring how practices of science can be opened-up and questioned. The processes involved in generating scientific impact are, then, potentially productive sites for critically engaging with environmental health research and policy more broadly.
By focusing on the material work and performance of the science–policy interface we have explicated the particular ways in which air pollution and its effects on health were made visible. We found that tracing this process was helpful for considering what aspects of air pollution and health are made invisible as a result. Attending to performances of impact drew our attention towards excluded pollutants or pollutant sources that were less easy to evidence and thereby led us to consider other relations outside of science, such as with non-governmental actors or publics. We show that if research networks are emergent processes then how we participate in these networks cannot be pre-determined. For instance, researchers’ deliberations on how to measure and conceptualise health as a result of the stakeholder meeting also led to attempts to unpack its black-boxing. This was a creative process. By expanding health outcomes to include other indexes of value to those of mortality or morbidity statistics the initial meaning of impact was challenged. In a context where policy impacts are increasingly demanded of scientific research, there is a need to explore how interfaces between research and policy shape environmental health concerns like air pollution. Tracing who and what is not included in accounts of impact, and the people, processes and things that lie outside of each performance of impact (what is not explicitly valued), can also point to other practical ways of achieving impact, and thereby offer opportunities to challenge and re-figure dominant models of science and science translation.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Ethics statement
This project had ethical approval from the LSHTM ethics committee (approval number 6051).
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors received funding for this research from the Natural Environment Research Council (Grant No. NE/I007938/1).
